Puppet Server: Running From Source
Included in Puppet Enterprise 2016.2.
So you’d like to run Puppet Server from source?
Before we get started, let’s highlight a few gotchas that might save you some time. The things that you are most likely to run into trouble with are pretty much the same things that were tricky about running the Webrick server from source:
- Knowing where your confdir/vardir/codedir are
- Knowing what your certname is
- Making sure that your certname is resolvable
In Puppet Server, we always specify /vardir/codedir explicitly in the Puppet Server config. This way, they are never dynamically determined based on your user or anything magical like that. So, if you need to know where they are, look at your config file(s).
Your certname comes from
facter fqdn or puppet.conf, just like it does in Webrick.
facter fqdn returns something that is not resolvable, you’re gonna have a bad
time. Either set your certname explicitly in
The following steps will help you get Puppet Server up and running from source.
Step 0: Quick Start for Developers
# clone git repository and initialize submodules $ git clone --recursive git://github.com/puppetlabs/puppetserver $ cd puppetserver # Remove any old config if you want to make sure you're using the latest # defaults $ rm -rf ~/.puppetserver # Copy the sample repl utilities namespace $ cp dev/user.clj.sample dev/user.clj # Launch the clojure REPL $ lein repl # Run Puppet Server user=> (go) user=> (help)
You should now have a running server. Your confdir is set to
see ~/.puppetserver/puppetserver.conf to examine this and other default paths.
In another shell, you can run the agent:
# Go to the directory where you checked out puppetserver $ cd puppetserver # Set ruby and bin paths $ export RUBYLIB=./ruby/puppet/lib:./ruby/facter/lib $ export PATH=./ruby/puppet/bin:./ruby/facter/bin:$PATH # Create the modules directory for the production environment, to avoid the # annoying module error during pluginsync $ mkdir -p ./target/master-code-dir/environments/production/modules # Run the agent $ puppet agent --no-daemonize --debug --trace --verbose \ --confdir ./target/master-conf-dir \ --vardir=./target/master-var-dir \ --server `facter fqdn` --onetime
NOTE that the default config is not ideal for long-lived dev environments,
because it sets codedir, confdir, and many other things to live inside of
./target. This directory WILL BE COMPLETELY REMOVED any time you run
lein clean, so if you’ve been setting up any modules / manifests in there,
you’ll lose your work. It’s much better to create a longer-lived home for
these files. To do that, you just need to edit the settings in your
file. More details below.
Also, I generally prefer to use separate confdir and vardir settings for my master and agent, because otherwise it’s super confusing to try to guess which one of them touches which files otherwise. To use separate conf/vardirs between master and agent on Puppet Server, you just need to:
- Choose one set of directories for the master via
- Choose a different set of directories for the agent via CLI args or
- Use a different certname for the master than for the agent. (I usually put
certname=localhostinto my master’s
puppet.conffile, use a different confdir for the agent, and do
--server localhoston the CLI for my agent runs. This also has the pleasant side effect of not needing to jack with
facter fqdnfor the master.
More detailed instructions follow.
Step 1: Install Prerequisites
Use your system’s package tools to ensure that the following prerequisites are installed:
- JDK 1.7 or higher
- Git (for checking out the source code)
Step 2: Clone Git Repo and Set Up Working Tree
$ git clone git://github.com/puppetlabs/puppetserver $ cd puppetserver # initialize git submodules (which are located in ./ruby, and contain the # puppet/facter ruby source code) $ git submodule init $ git submodule update
Step 3: Set up Config Files
Choose a directory outside of the git working copy, e.g.
~/.puppetserver, where you’d
like to maintain local config files for development. Then, copy the file
dev/puppetserver.conf.sample to that directory, and name it
Edit it to suit your needs. A few notes:
- Relative paths in the config file refer to the root directory of the puppetserver git working copy.
- Puppet Server reads much of its configuration data from
puppet.confand Puppet’s usual
confdir. The setting
master-conf-dirmay be used to modify the location of the Puppet
confdir. Otherwise, it will use the same default locations as Puppet normally uses (
~/.puppetfor non-root users,
- Similar to
confdir, you will likely want to specify the other top-level Puppet directories
You probably don’t need to make any changes to the sample config for your first run, but the settings that I edit the most frequently are:
jruby-puppet.master-conf-dir: the puppet master confdir (where
manifests, etc. should be located).
jruby-puppet.master-code-dir: the puppet master codedir
jruby-puppet.master-var-dir: the puppet master vardir
jruby-puppet.master-run-dir: the puppet master rundir
jruby-puppet.master-log-dir: the puppet master logdir
jruby-puppet.max-active-instances: the number of JRuby instances to put into the pool. This can usually be set to 1 for dev purposes, unless you’re working on something that involves concurrency.
If you prefer, you may break the individual configuration sections from
.conf files, and place them in a
Step 4a: Run the server from the clojure REPL
The preferred way of running the server for development purposes is to run it from
inside the clojure REPL. The git repo includes some files in the
that are intended to make this process easier.
When running a clojure REPL via the
lein repl command-line command, lein will look
for a namespace called
user.clj on the classpath, and if one is found, it will
load that namespace by default. We provide a sample version of this namespace that
is tailored for use with Puppet Server. To use it, simply do:
$ cp dev/user.clj.sample dev/user.clj
The git repo contains a
dev/user.clj, so each user can customize
their user namespace to their liking.
Once you’ve made a copy of the
user.clj file you’ll want to edit it. It contains
some comments explaining the contents of the file. The main change you’ll want to
make is to edit the
puppetserver-conf function to your liking; you’ll probably just
want to change it so that it points to the path of the
puppetserver.conf file that
you created in step 3.
Running the server inside of the clojure REPL allows you to make changes to the
source code and reload the server without having to restart the entire JVM. It
can be much faster than running from the command line, when you are doing iterative
development. We are also starting to build up a library of utility functions that
can be used to inspect and modify the state of the running server; see
for more info.
(NOTE: many of the developers of this project are using a more full-featured IDE called
Cursive Clojure, built on the IntelliJ IDEA platform, for
our daily development. It contains an integrated REPL that can be used in place of
lein repl command-line command, and works great with all of the functions described
in this document.)
To start the server from the REPL, run the following:
$ lein repl nREPL server started on port 47631 on host 127.0.0.1 user=> (go) user=> (help)
Then, if you make changes to the source code, all you need to do in order to restart the server with the latest changes is:
Restarting the server this way should be significantly faster than restarting the entire JVM process.
You can also run the utility functions to inspect the state of the server, e.g.:
Have a look at
user_repl.clj if you’re interested in seeing what other utility
functions are available.
Step 4b: Run the server from the command line
If you prefer not to run the server interactively in the REPL, you can launch it as a normal process. To start the Puppet Server when running from source, simply run the following:
$ lein run -c /path/to/puppetserver.conf
Other useful commands for developers:
lein testto run the clojure test suite
rake specto run the jruby test suite
Installing Ruby Gems for Development
Please see the Gems document for detailed information on installing Ruby Gems for use in development and testing contexts.
For more information about debugging both Clojure and JRuby code, please see Puppet Server: Debugging documentation.