librarian & r10k

Check your environment

Dominik Richter / @arlimus

This is about managing your Puppet


Assuming your puppet is managed via git,
this may look familiar:
├── .git/
├── .gitmodules
└── modules
    ├── stdlib
    ├── apache
    ├── ...

Most modules these days are on

git & Forge

Ask yourself

  • How do I get my modules?
    From git or forge? or custom?
  • How about dependencies?
    Who takes care of missing links?
  • How do I tackle changes?
    hotfixes, new featues, adjustments, ...
  • One repo to rule them all?
    having all features/fixes tracked in one giant repo?
We'll answer these by looking at git submodules, librarian-puppet, and r10k.

#1 Git Submodules

Having all modules in one giant git repo makes tracking changes and updates difficult
├── .git/
└── modules
    ├── a  (where is this module compared to upstream?)
    ├── b  (changes done here mix with...)
    └── c  (... changes here, which is difficult to re-use)
It works as long as your project is fairly small.

Giant-repo issues

  • All change history is mixed together More difficult to separate and track what happened.
  • Missing ACLs per folder Either your users get too many permissions or too few.
    The first may result in GAUs/catastrophy, the second in a tedious update process.
  • You have to check upstream modules into your repo This may or may not be something you want.
    Checking it in offers stability at the cost of flexibility
  • Additional work to publish a module At least if you want to offer the DVCS history to contributors to join.
Create submodules
├── .git/
├── .gitmodules
└── modules
    ├── a (via submodule)
    ├── b (via submodule)
    └── c (via submodule)
New features / fixes are tracked in each module separately.

Great because...

  • No extra tools to install; just git

    Though just a minor bonus, as Puppet requires Ruby

  • Easy to move pinned commit

    Just change into the folder, look at git tracks, check out what you need

  • Simple

    Get all modules with a simple command


  • Static

    (only commit-pinning; not branch- or ref-based)

  • Complex process to create new features

    I.e. create feature at root repo, create feature at submodule, add contents, commit to submodule, commit to root

  • Only works with git

    (which is pretty much the default)

  • No dependency resolution

    you have to track it manually

#2 librarian-puppet

Ruby's Bundler coming to Puppet
A shiny new Puppetfile
forge ""


mod 'puppetlabs-stdlib', '4.3.2'
mod 'puppetlabs-apache',
  git: 'git://'
Install via
librarian-puppet install

Librarian will take care of pulling all required modules

Configure git branch/tag/commit with ref
mod 'apache',
  git: '',
  ref: '1.1.x'

mod 'stdlib',
  git: '',
  ref: '4.3.2'

mod 'mysql',
  git: '',
  ref: 'f9dcac055baef105f57fdf63e26a3268a1d77cfd'
... which will result in
├── modules
│   ├── apache/
│   ├── concat/
│   ├── mysql/
│   └── stdlib/
├── .librarian        (local librarian config)
├── .tmp              (local cache)
├── Puppetfile
└── Puppetfile.lock   (pinned configuration)
This includes dependencies, git history, and a Puppetfile.lock:
    puppetlabs-concat (1.1.0)
      puppetlabs-stdlib (>= 4.0.0)
    puppetlabs-stdlib (4.3.2)

  ref: 1.1.x
  sha: 212e09d383c7382aa269e8c00e5c20c1c3808b2d
    apache (1.1.1)
      puppetlabs-concat (>= 1.0.0)
      puppetlabs-stdlib (>= 2.4.0)
For all your custom modules, add dependencies to your metadata.json
  "name": "hardening/ssh_hardening",
  "version": "1.0.2",
  "dependencies": [
      "name": "saz/ssh",
      "version_requirement": ">= 2.3.6"
      "name": "puppetlabs/stdlib",
      "version_requirement": ">= 4.2.0"
Available commands
  librarian-puppet clean           # Cleans out the cache
  librarian-puppet config          # Show or edit the config.
  librarian-puppet help [COMMAND]  # Describe commands
  librarian-puppet init            # Initializes the directory.
  librarian-puppet install         # Install modules
  librarian-puppet outdated        # Lists outdated dependencies.
  librarian-puppet package         # Cache to vendor/puppet/cache.
  librarian-puppet show            # Shows dependencies
  librarian-puppet update          # Updates and install
  librarian-puppet version         # Displays the version.

Great because...

  • Supports Git and Puppet Forge

    also supports internal Forges like Pulp

  • Track via Commit, Tag, Branch

    gives more flexibility and removes a few error-cases

  • Dependency resolution

    so you don't forget new modules and versions

  • Version pinning

    Your current installation is written to "Puppetfile.lock"


  • Caching only once per instance

    Which is tricky if you deploy from git for different environments and commits

  • Doesn't handle failures gracefully

    It may remove local modules and doesn't load from cache

#3 r10k

Use it to

Manage modules via Puppetfile


Dynamic environments via Git

Puppetfile extended with git branch, tag, and commit
mod 'apache',
  git: '',
  branch: '1.1.x'

mod 'stdlib',
  git: '',
  tag: '4.3.2'

mod 'mysql',
  git: '',
  commit: '4890ddb2f0c65c40101683c5577f943a7594b509'

Puppet originally introduced static environments

# Environment independent settings
vardir = '/var/lib/puppet'

modulepath = '/etc/puppet/environments/production/modules'

modulepath = '/etc/puppet/environments/testing/modules'

This was troublesome for development Imagine 2+ features being developed at the same time
=> code pollution in development environment

Use dynamic environments

vardir = '/var/lib/puppet'
modulepath = '/etc/puppet/environments/$environment/modules'

$environment will get resolved once you run

puppet agent -t --environment myenv

r10k creates environments from branch names
E.g. a production branch will result in an environment by that name.

Configure r10k
Point it to your repository (and included branches) and environment folder:

    remote: 'git://'
    basedir: '/etc/puppet/environments'

To deploy
To run the deployment with Puppetfile synchronized and verbose output

r10k deploy environment -pv

Which will create

├── puppetlabs-apache (v1.1.1)
├── zack-r10k (v2.2.8)
├── puppetlabs-apt (v1.1.0)
├── puppetlabs-stdlib (v4.3.2)
└── ...

Great because...

  • Optimized caching of Git checkouts

    With cache reuse when possible

  • Supports Puppetfile-Syntax

    with some adjustments

  • Graceful with failures

    Uses cache whenever possible

  • Conceived for changing environments

    Especially well fit for large projects and quick development cycles


  • No dependency resolution

    At the moment you have to do it manually

  • Some hidden failures

    This is rare but may happen with some Puppetfile configurations


Hidden failures

Modules may not get installed. In most cases you will get an error or at least warning when something went wrong. In some rare cases this also happens without any errors due to to internal inconsistencies.

Check your modules/ folder to verify.

Flexibility at a price

With git branch dependencies, commits may vary New commits to the branch will move it along. Two deployments may not be on the same commit-ID.

Librarian-puppet provides Puppetfile.lock In this file commit-IDs are pinned for all references. Use it to gain consistency. Alternatively pin to tags or IDs.

Missing commit

When pinning commits, they may be missing e.g. if they haven't been pushed yet
git history rewrites may alltogether wreck your debugging history.


Finch's musings on Puppet/r10k motivation [1][2][3]

Gary's Puppet R10K environment workflow [1]