Oct 28 2021

Debian: PuppetServer Cluster

Puppet LogoPuppet is the most powerful configuration management tool in the solar system. It’s the engine that drives your compliance, baseline, drift remediation, and deployment needs. It has always been and always will be open source, with freely downloadable operating system-specific agent packages, a massively scalable server, and data warehousing capabilities via PuppetDB.

This instruction will use the following configuration, there are various paths to a high-available Puppet solution, this is the simple one I chose for my setup. Review the official puppet site for reference architectures where you can offload the various components.

  • Primary Puppet Server (which will also act as a CA)
  • Secondary Puppet Server
  • PuppetDB Server
  • LoadBalancer (Based on NSX-T)

 


pupp

Step 1: Install Packages

Debian does include native pupet-agent packages, however for latest and greatest include the official Puppet deb community package repository to your server and clients.

Puppet Agents: Use Bullseye Packages

# wget https://apt.puppet.com/puppet7-release-bullseye.deb
# dpkg -i puppet7-release-bullseye.deb
# apt update
# apt install puppet-agent
# systemctl start puppet
# systemctl enable puppet

Puppet Server  (Bullseye not available yet)

# wget https://apt.puppet.com/puppet7-release-buster.deb
# dpkg -i puppet7-release-buster.deb
# apt update
# apt install puppetserver
# systemctl start puppetserver
# systemctl enable puppetserver

PuppetDB Server  (Bullseye not available yet)

# wget https://apt.puppet.com/puppet7-release-buster.deb
# dpkg -i puppet7-release-buster.deb
# apt update
# apt install puppetdb puppetdb-termini puppet-agent
# systemctl start puppetdb
# systemctl enable puppetdb
# systemctl start puppet-agent
# systemctl enable puppet-agent

Don’t forget that PuppetDB needs SSL configuration so run the configuration 

# puppetdb ssl-setup

Step 2: PuppetServer #1 Configuration

Define the main puppet agent configuration file, edit /etc/puppetlabs/puppet/puppet.confand define the various PuppetDB storage locations and the aliases for the PuppetServer (dna_alt_names):

[master]
vardir = /opt/puppetlabs/server/data/puppetserver
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
codedir = /etc/puppetlabs/code
certname = puppet-01.mydomain.org

# PuppetDB Configuration
storeconfigs = true
storeconfigs_backend = puppetdb
reports = store,puppetdb

[main]
dns_alt_names = puppet.mydomain.org,puppet,puppet-01.mydomain.org,puppet-01
server=puppet-01.mydomain.org
ca_server=puppet-01.mydomain.org
archive_file_server=puppet-db.mydomain.org

Edit the puppetserver configuration file and define the specific cipher suite to allow /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf:

http-client: {
    # A list of acceptable protocols for making HTTPS requests
    ssl-protocols: [TLSv1.2]

    # A list of acceptable cipher suites for making HTTPS requests
    cipher-suites: [TLS_RSA_WITH_AES_256_CBC_SHA256,
                    TLS_RSA_WITH_AES_256_CBC_SHA,
                    TLS_RSA_WITH_AES_128_CBC_SHA256,
                    TLS_RSA_WITH_AES_128_CBC_SHA]

    # Whether to enable http-client metrics; defaults to 'true'.
    metrics-enabled: true
}

Step 3: PuppetServer #2 Configuration

Define the main puppet agent configuration file, edit /etc/puppetlabs/puppet/puppet.confand define the various PuppetDB storage locations and the aliases for the PuppetServer (dna_alt_names). Also make sure to point this server to the primary puppet server as a puppet server (no chicken and the egg).

[master]
vardir = /opt/puppetlabs/server/data/puppetserver
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
codedir = /etc/puppetlabs/code
certname = puppet-02.mydomain.org

# PuppetDB Configuration
storeconfigs = true
storeconfigs_backend = puppetdb
reports = store,puppetdb

[main]
dns_alt_names = puppet.mydomain.org,puppet,puppet-02.mydomain.org,puppet-02
server=puppet-01.mydomain.org
ca_server=puppet-01.mydomain.org
archive_file_server=puppet-db.mydomain.org

Edit the puppetserver configuration file and define the specific cipher suite to allow /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf:

http-client: {
    # A list of acceptable protocols for making HTTPS requests
    ssl-protocols: [TLSv1.2]

    # A list of acceptable cipher suites for making HTTPS requests
    cipher-suites: [TLS_RSA_WITH_AES_256_CBC_SHA256,
                    TLS_RSA_WITH_AES_256_CBC_SHA,
                    TLS_RSA_WITH_AES_128_CBC_SHA256,
                    TLS_RSA_WITH_AES_128_CBC_SHA]

    # Whether to enable http-client metrics; defaults to 'true'.
    metrics-enabled: true
}

Step 4: PuppetDB Configuration

Define the main puppet agent configuration file, edit /etc/puppetlabs/puppet/puppet.confand define the various PuppetDB storage locations and the aliases for the PuppetServer (dna_alt_names). Also make sure to point this server to the primary puppet server as a puppet server (no chicken and the egg),

# This file can be used to override the default puppet settings.
# See the following links for more details on what settings are available:
# - https://docs.puppetlabs.com/puppet/latest/reference/config_important_settings.html
# - https://docs.puppetlabs.com/puppet/latest/reference/config_about_settings.html
# - https://docs.puppetlabs.com/puppet/latest/reference/config_file_main.html
# - https://docs.puppetlabs.com/puppet/latest/reference/configuration.html
#
[main]
server=puppet-01.mydomain.org
ca_server=puppet-01.mydomain.org
archive_file_server=puppet-db.mydomain.org

[agent]
environment = production
runinterval = 2h
[master]
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

Step 5: Sign Certificate – PuppetDB

Once the puppetdb server is enabled, you must sign its certificate else it will not function as a puppetdb backend for the puppetservers. Make sure the agent is enabled, started and has a certificate pushed to the server:

# cd /etc/puppetlabs/puppet/ssl/
# find . -type f
./certs/ca.pem
./certs/puppet-db.mydomain.org.pem
./private_keys/puppet-db.mydomain.org.pem
./crl.pem

If the certificates have been made, logon to the primary (CA) puppet server and sign the puppetdb certificate

# puppetserver ca sign --certname puppet-db.mydomain.org

Step 6: Sign Certificate – Secondary Puppet Server

Perform the same action as Step 5, but then for your secondary puppet server.

Step 7: NSX-T LoadBalancer

To create a NSX-T LoadBalancer Virtual Server entry for the PuppetServers, perform the following steps.

Application Profile

  • Logon to the NSX-T Manager and go to > Networking > Load Balancing  > Profiles
  • Create Application Profile  as ‘Fast TCP’ call it e.g. ‘app-profile-puppet’

 

Monitor Profile

  • Logon to the NSX-T Manager and go to > Networking > Load Balancing  > Monitors
  • Create Monitor Profile  as ‘HTTPS’
    • Name: monitor-puppet
    • Monitoring Port: 8140
    • HTTP Request:
      • HTTP Method: Get
      • HTTP URL: /status/v1/simple
      • HTTP version : 1.1
    • HTTP Response:
      • Response Code: 200
      • Response Body: running

 

Server Pool

  • Logon to the NSX-T Manager and go to > Networking > Load Balancing  > Server Pools
  • Create Server Pool
    • Name: pool-puppet
    • Algorithm: round-robin
    • Members:
      • Puppet Server #1 + IP Port 8140
      • Puppet Server #2 + IP Port 8140
    • SNAT Translation: Disabled
    • Active Monitor : Your monitor defined above – monitor-puppet

 

Virtual Server

  • Logon to the NSX-T Manager and go to > Networking > Load Balancing  > Virtual Servers
  • Create Virtual Server
    • Name: vs-puppet
    • Address: VIP (Virtual IP Address)
    • Ports: 8140
    • application Profile: Your application profile defined above – profile-puppet
    • Persistence: Source IP
    • Server Pool: Your server profile pool defined above – pool-puppet

 

Once created, attach this virtual server to your T1 router and both Puppet Server should become available!

Step 8: Puppet Clients

Each puppet client will need to have its configuration file properly defined to point to the primary CA and the loadbalancer as the puppet master entry. For each client edit /etc/puppetlabs/puppet/puppet.conf

  • puppet.mydomain.org = LoadBalancer VIP
  • puppet-01.mydomain.org = CA Server
  • puppet-db.mydomain.org = Puppet-DB Server

 

[main]
server=puppet.mydomain.org
ca_server=puppet-01.mydomain.org
archive_file_server=puppet-db.mydomain.org

[agent]
environment = production
runinterval = 2h

[master]
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

Additionals: Syslog Forwarding

If you need your Puppet components to forward log files to your centralized logging environment you can do this by adding an additional log appender to Puppet Masters and PuppetDB:

Puppet Master Server

Edit /etc/puppetlabs/puppetserver/logback.xml

.....
    
 <appender name="RSYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
      <syslogHost>loginsight.mydomain.org</syslogHost>
      <facility>LOCAL1</facility>
      <suffixPattern>pe-puppetdb: %msg%n</suffixPattern>
</appender>

......

PuppetDB Server

Edit /etc/puppetlabs/puppetdb/logback.xml

.....
    
 <appender name="RSYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
      <syslogHost>loginsight.mydomain.org</syslogHost>
      <facility>LOCAL1</facility>
      <suffixPattern>pe-puppetdb: %msg%n</suffixPattern>
</appender>

......