2024 0611 LDAP migrations

My homelab Kubernetes cluster requires an identity management system. After surveying some options, I decided that lldap wouldn’t be enough work, so I deployed 389 Directory Server instead.

Something I knew intellectually about LDAP, but didn’t truly appreciate, was how little comes with it out of the box².

This meant I needed to make structural changes to the LDAP server during initialization. However, there is nothing as nice as SQL’s CREATE TABLE IF NOT EXISTS or INSERT INTO ... ON DUPLICATE KEY UPDATE. Instead, you must check whether the change you want to make has already been made. But LDAP is itself a database. Why not have it keep track itself?

I create an OU called ldifMarkers containing objects which represent what LDIF files have been applied to the server. A script checks if an object under ldifMarkers has a cn with the LDIF’s filename, and if so, skips it. I keep all my LDIF files in Git (actually in a Kubernetes ConfigMap) and I can only expect to add to them – once an LDIF file is created and applied to the server, any changes made to the file in Git will not be applied the next time the server starts, unless the entire LDAP database is reset to empty first.

Zooming out a bit, this looks a lot like database migrations, and in fact, is inspired by a database migrations system built in Powershell for Microsoft SQL Server that I maintained for several years at a previous job.

A system like this is pretty easy to understand, and is suitable for large structural changes and maybe some initial administrative users, but you probably wouldn’t use it to manage all your users and groups.

The code

The canonical version of this stuff is in my IaC repo. I’ll provide links to the repository as of ~now, as this repo is used for all my infrastructure and is constantly in flux; by the time you read this I may have changed everything and I still want the links to make sense.

• The StatefulSet deploys the 389 Directory Server.
• The initsetup ConfigMap contains scripts that run in a sidecar when 389 Directory Server is deployed.
• The initldifs ConfigMap contains the actual LDIFs that are applied.
• The initsecrets Secret contains passwords for services and users. Secret objects are encrypted with sops and applied with flux, so you can’t see their contents from the link above; see this example file instead.
• If you really want to see how the whole thing gets deployed, see the kustomization, but that includes a lot of details irrelevant for this discussion.
• These ConfigMap and Secret objects are mounted as files inside the running containers.

When the app gets deployed, a configurator sidecar runs the initial configuration script, which configures the backend and some other stuff that has to be done very early. It also creates the ldifMarkers OU. This can only be done once, but if any of these items fail, the LDAP database will be empty so it can just be deleted and run again. Future container restarts will see that the backend has already been created and skip all of this.

In the LDIF application script, we get to applying the LDIFs. We do this in a specific order:

1. service.*.ldif files contain service accounts under the services OU.
2. user.*.ldif files contain (person) user accounts under the people OU.
3. group.*.ldif files contain LDAP groups under the groups OU.
4. membership.*.ldif files add users to groups. Membership is an attribute of a group, not a service or user, and there is no attribute we can set at account creation time that would set group membership, so we use membership files istead.

The magic is in apply_marked_ldif.sh, which is called for each LDIF. That script first adds a marker object under ldifMarkers with the filename, then tries to apply the LDIF file, and removes the marker object if the LDIF file fails to apply. Marker objects are organizationalRole because those objects support a cn (common name) attribute where we track the LDIF filename and a description attribute where we record the date it was applied, and don’t require any other attributes. It would be more semantically correct to create a custom object class like ldifMarker or something, but that would require a schema change and I had gotten side tracked enough already.

Finally, we can set passwords in a single idempotent operation. Accounts are created without a password, and a password change is a modify operation on the existing account object, and that works whether the password is already set or not. This lets us use a few very simple password files from initsecrets to create small change LDIFs in the passwords script. Because these LDIFs apply idempotently, we don’t track these in markedLdifs. Also, this means that the passwords are always set to a known value when the LDAP server is redeployed. You would obviously not want this for user passwords, but it can be pretty convenient for service account passwords and break-glass administrator passwords which should not be changed outside of a controlled process. Only passwords found in the initsecrets files are set this way.

That’s basically the whole system. The initsetup ConfigMap has more stuff in it, like scripts that allow modifying the ldifMarker OU and applying marked LDIF files for convenience when debugging, but that’s incidental.