Agent-master HTTPS communications

The Puppet agent and master communicate via host-verified HTTPS.

Note: Before the agent has a certificate, it makes unverified requests. During these requests, the agent does not identify itself to the master, and does not check the master’s certificate against the Certificate Authority (CA). After the agent gets a certificate, requests are host-verified unless stated otherwise.

The HTTPS endpoints that Puppet uses are documented in the HTTP API reference. Access to each endpoint is controlled by auth.conf settings. For more information, see Puppet Server configuration files: auth.conf.

Persistent connections and Keep-Alive

When acting as an HTTPS client, Puppet reuses connections by sending Connection: Keep-Alive in HTTP requests. This reduces transport layer security (TLS) overhead, improving performance for runs with dozens of HTTPS requests.

You can configure the Keep-Alive duration using the http_keepalive_timeout setting, but it must be shorter than the maximum keepalive allowed by the master's web server.

Puppet caches only verified HTTPS connections, so it does not cache the unverified connections a new agent makes to request a new certificate. It also does not cache connections when a custom HTTP connection class has been specified.

When an HTTP server disables persistent connections, Puppet requests that the connection is kept open, but the server declines by sending Connection: close in the HTTP response. Puppet starts a new connection for its next request.

For more information about the http_keepalive_timeout setting, see the Configuration reference.

For an example of a server disabling persistent connections, see the Apache documentation on KeepAlive.

The process of Agent-side checks and HTTPS requests during a single Puppet run.

  1. Check for keys and certificates:
    • If the agent doesn't have a private key at $ssldir/private_keys/<NAME>.pem, one is generated.

    • If the agent doesn't have a copy of the CA certificate at $ssldir/certs/ca.pem, it is fetched using an unverified GET request to /certificate/ca.
      Note: This could be vulnerable to Man-In-The-Middle attacks. You can make this step unnecessary by distributing the CA cert as part of your server provisioning process, so that agents never ask for a CA cert over the network. If you do this, an attacker could temporarily deny Puppet service to brand new nodes, but would be unable to take control of them with a rogue Puppet master.
    • If the agent has a signed certificate at $ssldir/certs/<NAME>.pem, skip to Request node object and switch environments. If the agent has a cert but it doesn’t match the private key, an error is generated.

  2. Get a certificate, if needed. If the agent has submitted a certificate signing request (CSR), and if autosign is not enabled, an admin user will need to run puppet cert sign <NAME> on the CA Puppet master before the agent can fetch a signed certificate. Because incoming CSRs are unverified, use fingerprints to prove them, by comparing puppet agent --fingerprint on the agent to puppet cert list on the CA master.
    • Try to fetch a signed certificate from the master using an unverified GET request to /puppet-ca/v1/certificate/<NAME> . If the request is successful, skip the rest of this section and continue to Request node object and switch environments. If the cert doesn’t match the private key, exit with an error.

    • Check if there's already a CSR for the agent.

      • Check in $ssldir/certificate_requests/<NAME>.pem.

      • Check with the master whether the agent has already requested a certificate signing. The agent might have lost the local copy of the request. Make an unverified GET request to /puppet-ca/v1/certificate_request/<NAME>.

      • If the CSR exists, the agent exits the process because user intervention is needed on the master. If waitforcert is enabled, the agent waits a few seconds and returns to the first step of this section to request a cert.

    • If no cert request was made, return to the first step of this section to request a cert.

  3. Request a node object and switch environments:
    • Do a GET request to /puppet/v3/node/<NAME> .

      • If the request is successful, read the environment from the node object. If the node object has an environment, use that environment instead of the one in the agent’s config file in all subsequent requests during this run.

      • If the request is unsuccessful, or if the node object had no environment set, use the environment from the agent’s config file.

  4. If pluginsync is enabled on the agent, fetch plugins from a file server mountpoint that scans the lib directory of every module:
    • Do a GET request to /puppet/v3/file_metadatas/plugins with recurse=true and links=manage.

    • Check whether any of the discovered plugins need to be downloaded. If so, do a GET request to /puppet/v3/file_content/plugins/<FILE> for each one.

  5. Request catalog while submitting facts:
    • Do a POST request to /puppet/v3/catalog/<NAME>, where the post data is all of the node’s facts encoded as JSON. Receive a compiled catalog in return.
      Note: Submitting facts isn't logically bound to requesting a catalog. For more information about facts, see Language: Facts and built-in variables
  6. Make file source requests while applying the catalog:

    File resources can specify file contents as either a content or source attribute. Content attributes go into the catalog, and the agent needs no additional data. Source attributes put only references into the catalog and might require additional HTTPS requests.
    • If you are using the normal compiler, then for each file source, the agent makes a GET request to /puppet/v3/file_metadata/<SOMETHING> and compares the metadata to the state of the file on disk.

      • If it is in sync, it continues on to the next file source.

      • If it is out of sync, it does a GET request to /puppet/v3/file_content/<SOMETHING> for the content.

    • If you are using the static compiler, all file metadata is embedded in the catalog. For each file source, the agent compares the embedded metadata to the state of the file on disk.

      • If it is in sync, it continues on to the next file source.

      • If it is out of sync, it does a GET request to /puppet/v3/file_bucket_file/md5/<CHECKSUM> for the content.
        Note: Using a static compiler is more efficient with network traffic than using the normal (dynamic) compiler. Using the dynamic compiler is less efficient during catalog compilation. Large amounts of files, especially recursive directories, will amplify either issue.
  7. If report is enabled on the agent, submit the report:
    • Do a PUT request to /puppet/v3/report/<NAME>. The content of the PUT should be a Puppet report object in YAML format.