Projects

The contents of this document are influenced by 12factor.net, a sre zen put together by some of the folks at Heroku.

Manifest

A convention makes its easier for both developers and system adminstrators to test, execute, configure, and deploy the project. For system administrators, this convention will make it much easier to understand the application code and allow for a more standard way to execute and monitor the application in production environments. When a crisis occurs, both developers and system administrators will be able to operate more efficiently by having a clear understanding of this contract.

Project directory structure

If all of the conventions above are followed, an engineer should see something similar to the directory structure below.

.
├── .env              # Foreman .env file contains and documents all of the ENV variables.
├── .ruby-version     # Supporting dotfiles (What version of Ruby are we using?)
├── Gemfile           # Foreman, etc should be in here.
├── Procfile          # Foreman file that describes how the app is executed.
├── README.md         # Documentation that describes the app's purpose and how to run, test, and deploy.
├── app|lib           # The app or lib code should live in this folder.
├── bin               # Binaries (or supporting binaries) for the app.
├── docker            # A subdirectory for each docker container this project uses.
│   ├── container_a   # The PollEv CLI tool hooks into this directory while building and deploying the project.
│   ├── container_b   # Generally we have one main process per container. But often that process will spawn sub processes which it manages.
├── doc               # Additional supporting documentation.
├── spec|test         # Describes app's intent progrmatically and prevents regression issues.

But what if I’m working on a node.js app?

Well, this should all be the same except you won’t need a .ruby-version or Gemfile. Instead you’ll want to create a package.json file, which serves much the same purpose. Here’s a cheatsheet for the package.json format, and don’t forget to include "private": true unless you intend to open source the code!

Execution convention: foreman start

A Procfile describes to the development and operations team how an application is run. A developer should be able to run foreman start from the root of any project and access it through their browser on 127.0.0.1:PORT, assuming a web application.

# An Example Procfile. Note the use of PORT in the app role.
web:    bundle exec thin -p $PORT -R config.ru start
stream: bundle exec firehose server
worker: bundle exec quebert

An antipattern is to boot “commonly available” resources in the Procfile.

# Do NOT boot commonly available resources in our environment
db: mysqld_safe
postfix: /etc/init.d/postfix
redis: /use/bin/local redis

These resources should be stored as URI’s in the .env file, and the application should bind to these resources when it boots. Read more about resource binding at 12factor.

Deployment convention: pollev application deploy

Our Site Reliability team supports a CLI tool which standardizes deployment commands across projects. This tool handles opening SSH connections to production and staging servers to instruct them to pull run the approriate Docker containers.

A developer should be able to simply run pollev deploy -a my-app -e staging to push code to a server. Not only will this help our team push code faster to production, but each project’s docker directory helps document all the containers required for a project and how they interact, which is invaluable during a crisis.

Configuration convention: .env files

The .env file should have all available variables defined that the application reads. If the variables are not being used, they should be commented out.

DATABASE_URI=mysql://u:p@localhost:3306/db_dev  # Location of MySQL resource
REDIS_URI=redis://localhost:6890/0    # Location of Redis resource
SMTP_URI=smtp://localhost             # Location of SMTP resource
# HOPTOAD_API_KEY=                    # When this API key is present, the app will report exceptions to Hoptoad

A developer and operations engineer should be able to look inside of this .env file to understand what features can be turned on/off, the resources that need to be bound to, etc.

The purpose of storing all of our configuration variables in a .env file makes sure that we’re not checking production credentials into the git repository (this is the problem with bin/production wrapper scripts or rails-like configuration/production.rb files). Consider it an anti-pattern if you have a file or class, other than a .env file, named production, staging, development. The difference between environments should only be credentials, resource bindings, and enabled features.

If you’d like to express the differences between environments, you may create .test.env, .production.env files respectively; however, please don’t check credential into the repository.

Logging convention: STDOUT

Application log output should be sent to standard out so the operations engineer can redirect this output to wherever log files are to be stored in the production environment.

Its an antipattern to store log output in actual files from project code because it creates the possibility that an engineer doesn’t see that, the rogue log file fills up the disk, and the box crashes.