Storing secrets for Symfony applications – some ideas how to approach the topic

During my first day at the Symfony EU-FOSSA 2 Hackathon, I spent some time discussing with other attendees how application secrets can be stored and provided to Symfony applications. This blog post tries to explore and document the options already on the table today in Symfony 4.3.

Written by: Matthias
Published on: 2019-04-06

In this post

Before going into the details, a disclaimer: I haven’t taken this to production or integrated it into a CI/CD pipeline. This post is basically a write-up of basic ideas and how things might possibly work. So please, consider it a first sketch and challenge the approach – just drop me a line.

Many thanks to jdreusse, tobion and nicolas-grekas for their time and for discussing the topic in an open-minded way. The PR we were initially discussing is symfony/issues/27351. Later on, @dunglas also pointed me to an earlier PR.

Why it is convenient to do it wrong

What you might have seen already is that secrets like database passwords, API tokens and such are added to configuration files and committed into code repositories:

# config/services.yaml
    mysql.password: nobody-knows-this
    some_api.token: this-is-a-secret

It doesn’t matter if it is the config/services.yaml file, or maybe different files like parameters_prod.yml, parameters_dev.yml which are included depending on the environment ( dev, prod) that you’re in.

Storing plain-text secrets in your code repository is a bad practice. That way, they are distributed to everyone and everywhere the repo is used, like development laptops, CI servers and production systems.

You might be tempted to do it this way because it is simple and convenient. At least for two reasons:

  • You have the necessary parameters right alongside your code. A team member can just clone the repo and they’re ready to go. No need to figure out the right MySQL password, tokens and stuff before you can start working on a project.

  • Imagine you start using another API in your application. If you commit the code that requires the API token in one repo and the token itself somewhere else, then you’d have to make sure that during deployment both repos/versions will be aligned. You don’t want to run a version of the code when the token is not yet in place.

Storing secrets safely

Obviously, it would be better if we had a way to store secrets in a „safe“ way. You can, for example, keep all your secrets in a secrets.json file and use off-the-shelf tools to encrypt this file before you commit it into version control.

The Puppet ecosystem provides a neat tool for this called hiera-eyaml. It’s written in Ruby, and if you’re lucky, gem install hiera-eyaml might be all you need to get it installed. There’s also a Docker image available, if you're more comfortable with that.

To prepare your repo, run eyaml createkeys. That will create a private-public key pair and store it a ./keys directory. Grab the private_key.pkcs7.pem file from there and stow it away in a safe place – not in your repo. Commit the public_key.pkcs7.pem file and keep it in your repository.

With this in place, the eyaml command has some operating modes to encrypt and/or decrypt strings or files:

# Encrypt "bar" and print as YAML for the "foo" key
eyaml encrypt -l foo -s bar 
# Ask for the secret value as a hidden password
eyaml encrypt -l foo -p 
# Encrypt the value "bar" and print as a plain string
eyaml encrypt -s bar -o string

All of these work with just the public key being available. Make sure to check the documentation or --help output for more details.

Also, eyaml edit -d secrets.json will open secrets.json in your $EDITOR. Try putting the following lines into that file:

    'mysql_password': 'DEC::PKCS7[dont-tell-mum]!',
    'another_token': 'DEC::PKCS7[another-secret]!'

Once you exit the editors, all DEC::PKCS7[…]! sections will be replaced with encrypted values.

Don’t let the eyaml name misguide you. In fact, the tool does not care about the file format. As you will have noticed, I’ve used JSON in the example above.

secrets.json will be checked into version control. This way of keeping encrypted values in a file that is itself unencrypted is useful because version control commands like blame or diff still work as one would expect. If the entire file were encrypted, every single change would result in a completely different file being committed.

Decrypting secrets in the runtime environment

At runtime, you need to decrypt the secrets so that Symfony can work with the plain values. Let's try to keep that decryption step out of Symfony and solve it with the tools we already have.

So, in the case of eyaml, the following command will read the secrets.json file and write it decrypted to .secrets.json. Obviously, this time you need to provide the private key.

eyaml decrypt --pkcs7-private-key=/path/to/private_key.pkcs7.pem -f secrets.json > .secrets.json

Honestly, I am not sure how you would get the private key safely in place. Maybe you can distribute it to your production machines using some kind of infrastructure management or tooling, or mount it into the container when running your application in Docker.

The point is that this command can be run at a very late stage, during the actual deployment.

When using Docker, you don’t have to bake the decrypted secrets into the image, which also avoids pushing them into the registry. You can instead defer this step until you actually start the container.

Even more, you can remove the private key again after this file has been dumped but before actually start the application.

(To-Do: Explain in more detail how this could work. Use an entrypoint script? create the container and extract the secrets file before starting it?)

Reading secrets at runtime

With the decrypted .secrets.json file is in place, you can use Symfony’s EnvVarProcessors to read it. Consider this:

    env(SECRETS_FILE): '.secrets.json'
    mysql.password: '%env(key:mysql_password:json:file:SECRETS_FILE)%'
    some_api.token: '%env(key:another_token:json:file:SECRETS_FILE)%'

This gives you two parameters %mysql.password% and %some_api.token% that you can use in your application and container definition, just like in the opening example.

Since we’re using the %env(…)% syntax, the values are not compiled into the Dependency Injection container. Yet, you can still warm the Symfony Cache during CI and ship Docker images with it.

More sophisticated secrets management

In fact, the second half of this approach – accessing the secrets at runtime through %env()% in Symfony – is independent from how you get the secrets there.

The eyaml-based approach outlined above works nicely for the use case we initially discussed. It's a low-barrier approach when you want to keep the secrets in the code repository itself and trust your production machines enough to have the plain secrets lying around in files.

If you want to take a more comprehensive approach, tools like Docker Secrets, Hashicorp's Vault or something from your PaaS provider are definitely the way to go.

Docker Secrets will provide decrypted secrets as files in a memory-based filesystem mounted into a particular container. So, in the case of crashes, there is not even a file left on the disk. I am sure Vault can do something similar.

As  %env()% is evaluated at runtime, you can also do fancy things like automatically rotating your secrets every day. Whatever tool you need for this, it does not need to do anything special with your already-running application.

You might want to note that the above approach is not limited to a single file. Use several files, for example when secrets need to be updated by different tools.

By using an environment variable that points to the secrets file, you can change the path to that file at deployment time. So, you could also come up with hybrid approaches: For example, have a plain text file that contains "default" database credentials on your development machine. On production, switch to a Vault-backed location with credentials that are rotated automatically.

Why not use env variables?

When we first engaged in the discussion this morning, my first question was „why not simply use environment variables for that“? After all, env vars are the recommended way of configuration for 12 Factor Apps.

However, as others noted already, env vars have a few downsides that make them less suitable for sensitive information. The main arguments brought are:

  • Environment variables may easily be leaked. For example, they are contained in the phpinfo() output or can be part of ps process tables.

  • Also, in the case of failures, environment variables are often included in dumps, reports or logs.

  • Environment variables are inherited to child processes, for example when using the Symfony Process component. This violates the principle of least privilege. Also, it makes the previous point more important – for example when running other tools, you’d have to make sure that they don’t „leak“ the environment either.

  • (New) developers might not even be aware that the environment contains sensitive data and thus not treat it with the necessary care.

Exposing secrets through files or, even better, through files in per-process memory filesystems does not have any of these drawbacks.

Closing remarks

I have a pending Symfony PR that adds a new require EnvVarProcessor type. With that, you can have a PHP script that will be executed at runtime every time the container is booted.

This PHP script can then return the value that will be used in the configuration.

You can use that to return array structures that will even be cached in the PHP opcache. Should your secrets file end up in the document root, chances are that fetching a .php file does not reveal anything to the outside.

On top of that, the fact that it’s just plain PHP opens up new ways of creatively fetching API tokens and secrets from somewhere .

Erfahren Sie mehr über unsere Leistungen als Symfony Agentur.

Did we spark your interest?

If so, we'd love to hear from you. Don't hesitate to get in touch with us, if you have any questions or further remarks. And if you want to discuss any type of project, product, problem or idea with us, we would like to even more!