SSL: Regenerating All Certificates in a Puppet Deployment

Included in Puppet Enterprise 3.7. A newer version is available; see the version menu above for details.

Note: If you’re visiting this page to remediate your Puppet Enterprise deployment due to CVE-2014-0160, a.k.a. “Heartbleed,” please see this announcement for additional information and links to more resources before using this guide. Before applying these instructions, please bear in mind that this is a non-trivial operation that contains some manual steps and will require you to replace certificates on every agent node managed by your Puppet master.

IMPORTANT: This page describes the steps for regenerating certs in an open source Puppet deployment. If you use Puppet Enterprise you should NOT use this page since doing so will leave you with an incomplete replacement and non-functional deployment. Instead, PE customers must refer to one of the following pages:

In some cases, you may find that you need to regenerate the certificates and security credentials (private and public keys) that are generated by Puppet’s built-in certificate authority (CA). For example, you may have a Puppet master you need to move to a different network in your infrastructure, or you may find that you need to regenerate all the certificates and security credentials in your infrastructure due to an unforeseen security vulnerability.

Regardless of your situation, regenerating your certs involves the following three steps (complete procedures follow below):

  1. On your master, you’ll clear the certs and security credentials, regenerate the CA, and then regenerate the certs and security credentials.
  2. Next, you’ll clear and regenerate certs and security credentials for any extensions.
  3. Lastly, you’ll clear and regenerate certs and security credentials for all agent nodes.

Note that this process destroys the certificate authority and all other certificates. It is meant for use in the event of a total compromise of your site, or some other unusual circumstance. If you just need to replace a few agent certificates, you can use the puppet cert clean command on your Puppet master and then follow step 3 for any agents that need to be replaced.

Step 1: Clear and Regenerate Certs on Your Puppet Master

On the Puppet master hosting the CA:

  1. Back up the /etc/puppetlabs/puppet/ssl/ directory. If something goes wrong, you may need to restore this directory so your deployment can stay functional. However, if you needed to regenerate your certs for security reasons and couldn’t, you should get some assistance as soon as possible so you can keep your site secure.
  2. Stop the Puppet agent service with sudo puppet resource service puppet ensure=stopped.
  3. Stop the Puppet master service. Your Puppet master is probably managed by a Rack-supporting web server, such as Apache with Passenger; you’ll need to stop that web server. (For example, sudo puppet resource service apache2 ensure=stopped.)
  4. Locate Puppet’s ssldir by [reading the setting from Puppet][config_print]; run either sudo puppet config print ssldir --section master or sudo puppet master --configprint ssldir.
  5. Delete all files in the ssldir with sudo rm -r <PATH TO SSLDIR>.
  6. Regenerate the CA by running sudo puppet cert list -a. You should see this message: Notice: Signed certificate request for ca.

  7. Generate the Puppet master’s new certs with sudo puppet master --no-daemonize --verbose.
  8. When you see Notice: Starting Puppet master <your Puppet version>, type CTRL + C.
  9. Start the Puppet master service by reversing whatever you did in step 3. (E.g., sudo puppet resource service apache2 ensure=running.)
  10. Start the Puppet agent service with sudo puppet resource service puppet ensure=running.

At this point:

  • You have a brand new CA certificate and key.
  • Your Puppet master has a certificate from the new CA, and it can once again field new certificate requests.
  • The Puppet master will reject any requests for configuration catalogs from nodes that haven’t replaced their certificates (which, at this point, will be all of them except itself).
  • If you are using any extensions that rely on Puppet certificates, like PuppetDB or Puppet Dashboard, the Puppet master won’t be able to communicate with them. Consequently, it may not be able to serve catalogs, even to agents that do have new certificates.

Step 2: Clear and Regenerate Certs for any Extensions

You may be using extensions like PuppetDB or Puppet Dashboard to enhance Puppet. These extensions probably use certificates from Puppet’s CA in order to communicate securely with the Puppet master.

For each extension like this, you’ll need to regenerate the certificate(s) it uses. Many tools have scripts or documentation to help you set up SSL, and you can often just re-run the setup instructions.

Step 3: Clear and Regenerate Certs for Puppet Agents

To replace the certs on agents, you’ll need to log into each agent node and do the following:

  1. Stop the Puppet agent service. On *nix nodes, run sudo puppet resource service puppet ensure=stopped. On Windows nodes, run the same command (minus sudo) with Administrator privileges.
  2. Locate Puppet’s ssldir and delete everything in it. The ssldir can be located by running puppet config print ssldir --section agent or puppet agent --configprint ssldir — be sure to run that command with sudo on *nix nodes and with Administrator privileges on Windows nodes.
  3. Re-start the Puppet agent service. On *nix nodes, run sudo puppet resource service pe-puppet ensure=running. On Windows nodes, run the same command (minus sudo) with Administrator privileges.

    Once the Puppet agent starts, it will automatically generate keys and request a new certificate from the CA Puppet master.

  4. If you are not using autosigning, you will need to sign each agent node’s certificate request. You can do this by logging into the CA Puppet master server, running sudo puppet cert list to see pending requests, and running sudo puppet cert sign <NAME> to sign requests.

Once an agent node’s new certificate is signed, it will fetch it automatically within a few minutes and begin a Puppet run.

Once you have regenerated all agents’ certificates, everything should now be back to normal and fully functional under the new CA.

↑ Back to top