Learn how to install and configure PuppetDB with a PostgreSQL database as the backend for better scalability in the MPLI Productions environment.
(silence) - [Instructor] In this video we talking about PuppetDB and PuppetDB is what is used for exported resources. It's basically a data store for Puppet. You can use it to store your facts, your reports and your exported resources. And expected resources we're going to cover in a later section. But, to get started with it, we're going to configure a machine as our PuppetDB. We're going to use the puppetca that we configured previously, but we're going to build a new machine that will house our puppetdb and our postgresql database.
PostgreSQL is basically the only database that is supported by PuppetDB at this point. Future versions may support other databases, but at this point Postgres is probably your best choice. The quickest way to get started with this is to actually use the puppetlabs/puppetdb module which you can download from the forge. Takes care of the entire configuration of PostgreSQL and PuppetDB for you. The only real problem with that is that it does obscure the details of how everything works. So we're going to do it from scratch ourselves.
So the first thing we want to do is bring up our database. So we're going to install Postgresql. What we're going to use a later release than is included with our operating system. So I believe our operating system has 9.2 and we're going to use 9.5. So what we want to do is come into the yum postgres repository and then find CentOS and grab the latest of the 9.5 series. The 9.6 is still beta and, obviously, the 9.2 is what's included with our OS. So we're going to do 9.5.
And first thing will do is just install that rpm. And with that rpm we're then going to install the server. So that's postgresql95-server. Say Yes. That should download the requirements that we need to get this going. And once that's installed, we're going to have to create an initial database.
So we'll just make sure that postgres is in my PATH. I believe I already put it in my PATH earlier, yeah. This is where those utilities will be located in this directory here. So you just add that to your PATH. With that in my PATH, I'm going to initialize the database. (keys clicking) I had a previous incarnation of the data scenario so let's just remove that.
All right, so we got a new database up. (keys clicking) Now we're going to start the server. Oh, 9.5. So that's now running, so we'll just do a status on that. So, that's running. The port for postgres is 5432 so let's just make sure something's listening there. All right, so postgres is listening. The next thing to do is become the postgres user and create the puppetdb user for postgres.
So will do that with sudo. We'll become this user. And now we want to use the createuser command. And we're going to use the password of PacktPub. And I entered it wrong. There we go. All right, so that database is ... That user's now created. Now we got to create the database. We'll tell it to use UTF8 as the language.
All right, so the database is now created. Now the next thing is we've got to exit out of the postgres user. Let's get back into root and let's go to var/lib/pgsql/9.5/ and in the data directory. So this was just created by what we just did. We want to look at the pg_hba file. And we'll see that there's not a lot of content here for access to things. This is all by ident and whatnot. What we want to use actually is the password that we just created.
So we're just going to add these three lines. Had these ready to go earlier so this is basically saying that the user ... Access to the database puppetdb by the user puppetdb is by an md5 password from the local host or from the IPv6 local host. Now, with that change in place, we now need to restart postgres because we've changed the access rules. So we'll do a systemctl restart postgresql-9.5.
And while we wait for that, the next thing we want to do is come back and do a psql. We're going to connect to the localhost. We're going to connect to the database puppetdb and we want to be the puppetdb user. And we just want to see ... Now it's going to ask for that password and I said it was PacktPub, and we're in! And there should be nothing in here right now. There's no database. So, now that the database is there, the next thing to do is install puppetdb.
So let's just yum install that. So, puppetdb will be installed on this machine. Cool, all right, so, puppetdb is now installed. The next thing we want to do is just run puppet on here. And because I have puppet running on a different port, just going to get it to generate a new key for us.
(keys clicking) I'll get puppet to generate a new key for us. So here's puppet generating a new key. And then we'll go back to our puppet server here. We should have puppetdb waiting to be signed. (keys clicking) Puppetdb should be signed at this point.
Great. So now we just want to run it one more time. There shouldn't be actually much that has to be done right now. We're still trying to send puppetdb facts which we don't want to do yet. So that give us the certificates. So the next thing we want to do is tell puppet to use those, puppetdb to use those. So we're going to do puppetdb and we're going to do an ssl-setup command. And this is a shorthand command that will go and copy the ssl certificates into the right directories for us.
So, basically, what it did was copy these over. It's telling me that ... Because I have used this machine in the past and didn't wipe the old certificates, it's just letting me know that this is wrong. But it did copy the certificates over now for me. So, if I go to the etc/puppetlabs/puppetdb directory and the ssl directory, I can see that it's created these files for me. And the next thing to do is then go on to the puppet server and tell it where to find puppetdb. (keys clicking) So if we're back on our puppet server, we have to tell our puppet server where to find puppetdb.
So let's go over to the puppet directory now. We don't have a file yet, but what we do is, we'll call ... we'll make one. I'm going to copy one that I had from before into here. And all this basically says is to go to the puppetdb machine on port 8081. The next thing I want to do is copy the routes, the yaml file to here. And this is just a configuration file that allows us to use puppetdb for ...
tells the puppet server, basically, about where to find the node terminus. This file, you really shouldn't have to play with very much. Now back on our puppetdb machine, we're not quite done yet because we have to go in and tell puppetdb where to find the database that we just configured. So let's go to the conf.d directory and let's look at the database, that ini file, and we'll see that it isn't configured at this point. And we want to make sure that it's configured to use our postgresql server that we created earlier.
So all of these things have been commented out. So we want to make sure that the classname is the postgresql.Driver, the Subprotocol is postgresql. The database address is this machine on port 5432 in the database called puppetdb. The username we used was puppetdb. And the password, which I said, was PacktPub. So we've got the username, the password, the subname and the protocol.
That should be enough to get puppetdb started. So what we'll do is we'll do systemctl, tell puppetdb to restart. While that's restarting, you might take a few seconds ... While we're still in this directory we do have to actually tell puppet to use that stored config backend. So what we'll do is we'll tell it that storeconfigs = true and that the storeconfig_backend that we wish to use = puppetdb.
So with those two things in place, we should be able to start working again. And we do have to wait for this to restart. Okay. Now we want to make sure that the 8081 port is listening. That is correct. And the 8080, yes. So, puppetdb is listening on those two ports. We can verify this with systemctl status puppetdb. So puppetdb is running.
It's just sitting there waiting. We can try running our puppet agent again. (keys clicking) And puppet should try to store its information in puppetdb. Now puppetdb's showing it has the wrong certificates. So we've obviously screwed up with our ssl-setup. So let's go back to puppetdb.
These certificates are too old. Let's remove them. And do puppetdb ssl-setup. That's probably what that error was telling me before, but I wasn't listening to it. (keys clicking) It created the directory out underneath me. All right, those look a little more up-to-date. We do have to restart because the certificates changed out from underneath puppetdb it'll have to be reloaded. And once that's reloaded, we'll be able to run again and just see if we can connect.
If our facts can be sent to puppetdb. This time to restart can take a varying amount of time depending on how fast your VMs are. For me it usually takes 15 to 20 seconds. All right, let's verify that that's listening. All right. Let's just do the same puppet agent run against our puppetdb. All right, so you can see that it now was able to submit our facts into puppetdb.
So, now we can go back with psql, go to localhost, go to the puppetdb ... As the puppetdb user to the puppetdb database and we'll enter that password. And now we should see a bunch of relationships. So let's make an extended display. (keys clicking) So, if we look at all the entries in the catalogs we see that there's nothing there yet.
(keys clicking) Catalog_resources. You can see there's nothing in there right now. We can quit. And we will go and run puppet again. Great, so we got a good run on here. Puppet was told to run and if we look back on our puppet machine, let's just verify that ...
All right, so we've got ... Our puppet machine is configured to use the storeconfigs of puppetdb. Storeconfigs is set to true. So now what we're going to do is restart puppetserver to tell it to use that setting. And, again, 'cause this is a nice Java service, we're going to have to wait a little while while this restarts. And once this restarts, we will have to run puppet again on puppetdb machine jus to get a catalog into our puppetdb because the last time we ran it we weren't actually storing the con ...
we weren't using storeconfigs yet. All right, so that's been restarted. We're going to run one more time. This time we should see that the information gets sent to puppetdb. So let's do a psql. (keys clicking) Do an x. We could see that this is the catalog from the machine that we just created.
So this was sent in here and now if we look at catalog_resources as well we should see that Notify that we created earlier. So here's the entry for that Notify that we saw and then these are just records that are in every catalog. But this right here is the one that we created. The one that said, "Created a standalone puppet server." And if we scroll back, we can see that that is the message we got from our Notify. So our puppetdb is working and that was not in exported resources and what we'll cover in the next section is what exported resource actually means.
But as we can see right here, that exported is false. So, we'll cover that next.
This course was created by Packt Publishing. We are honored to host this training in our library.
- Puppet servers and environments
- Creating a Puppet server machine
- Performance tuning
- Using PuppetDB
- Extending Puppet with custom facts and types
- Using Hiera
- Generating reports
- Testing and troubleshooting Puppet environments