/blog/

0001 0101 Pants Build for me.micahrl.com

A Pants translation of the project’s vars.mk Makefile.

Quick Start

# Install the Pants launcher
curl --proto '=https' --tlsv1.2 -fsSL https://static.pantsbuild.org/setup/get-pants.sh | bash

# Run from the repo root (copy pants.toml and BUILD here first):
./pants run //:git-info         # write build/git-info.json
./pants run //:build-prod-obverse
./pants run //:spellcheck-prod-obverse
./pants run //:deploy-prod-obverse

./pants is a small shell script that downloads the Pants version pinned in pants.toml. The version is content-addressed; teammates get the same binary.

Available Targets

Run ./pants list :: to see all targets. Run a target with ./pants run //:NAME.

Target Description
build-dev-obverse Build dev site (sandboxed)
build-prod-obverse Build production site (sandboxed)
build-prod-reverse Build production mirror (sandboxed)
build-test-obverse Build test site (sandboxed)
build-test-reverse Build test mirror (sandboxed)
build-onion-obverse Build onion site (sandboxed)
build-onion-reverse Build onion mirror (sandboxed)
git-info Write build/git-info.json
check-symlinks Check for symlinks in content/
check-exifstripped Verify/strip EXIF from images
spellcheck-prod-obverse Spellcheck HTML output
go-mod Update go modules
cfn-base-test-obverse Deploy test-obverse CFN base stack
cfn-base-test-reverse Deploy test-reverse CFN base stack
cfn-distrib-test-obverse Deploy test-obverse CFN distribution stack
cfn-distrib-test-reverse Deploy test-reverse CFN distribution stack
deploy-prod-obverse Deploy to Netlify
deploy-prod-reverse Deploy mirror to Netlify
deploy-test-obverse-hugo Upload test-obverse site to S3
deploy-test-reverse-hugo Upload test-reverse site to S3
deploy-onion-obverse Deploy via rsync
deploy-onion-reverse Deploy mirror via rsync
deploy-clean Clean all generated files

Note: there is no aggregate deploy-test-obverse or deploy-all target. See No sequential dep chaining below.

How This Compares to Make

What Works Well

1. BUILD files are Python

The biggest syntactic win over Make. The define/endef macro becomes an ordinary Python function:

# Pants: regular Python function, env captured by closure
def _hugobuild(env):
    shell_command(
        name=f"build-{env}",
        command=f"hugo ... --environment {env}",
        sources=[":hugo-sources"],
        ...
    )

_hugobuild("prod-obverse")
_hugobuild("prod-reverse")
# Make: define/endef macro, called with $(call ...)
define hugobuild
    hugo ... --environment $(1)
endef

public/prod-obverse/.sentinel: ...
    $(call hugobuild,prod-obverse)

No special DSL to learn. Constants, list comprehensions, helper functions — all available because BUILD files are real Python modules evaluated at analysis time.

2. Content hashing, not timestamps

shell_command targets track sources by content hash (SHA-256), not mtime. Checking out a branch and reverting it won’t trigger a rebuild. Touching a file without changing it won’t either.

This is the same advantage Task has over Make, and it’s the default in Pants.

3. Remote caching

Pants has first-class remote cache support via the Remote Execution API (REAPI). Add a [GLOBAL] remote_cache_address to pants.toml and Pants will check the cache before rebuilding. A Hugo build that produces the same output as a previous run (same sources, same Hugo version) will be served from cache.

Make has no remote cache support. pydoit has none either.

4. Parallelism by default

Pants schedules independent shell_command targets in parallel automatically. All seven Hugo builds run concurrently without any -j flag or deps: config.

5. Pants version pinned in config

pants_version = "2.23.0" in pants.toml means every developer and CI run uses exactly the same Pants binary, downloaded and verified automatically. Compare to Make: the system make version varies across machines.

6. system_binary declarations

system_binary(name="hugo-bin", binary_name="hugo")
system_binary(name="aws-bin",  binary_name="aws")

These declare what executables a target needs, and Pants includes them in the sandbox. The declaration serves as documentation of external tool dependencies, and Pants can warn if a required binary isn’t found at analysis time.

What Doesn’t Work Well / Trade-offs

.git/HEAD is not a source file

The vars.mk pattern’s key trick is:

build/git-info.vars.mk: .git/HEAD .git/index
    scripts/emit-git-vars.sh $@

Pants tracks source files for content hashing. .git/HEAD and .git/index are VCS metadata — Pants deliberately ignores the .git directory. There is no shell_command equivalent that re-runs only when the commit changes.

The workaround: use run_shell_command (real environment, no sandbox, no caching). The git-info step always re-runs when invoked; Pants provides no up-to-date checking for it. Make’s build/git-info.vars.mk: .git/HEAD .git/index is actually cleaner here.

shell_command cannot depend on run_shell_command output

This is a fundamental Pants constraint. shell_command runs in a hermetic sandbox with only declared inputs. run_shell_command writes to the real filesystem. These two execution models don’t compose.

In this BUILD file, _hugobuild reads build/git-info.json via a relative path outside the sandbox (../build/git-info.json). This is a workaround that breaks the hermetic guarantee — Pants can’t track this dependency.

The vars.mk pattern, pydoit, and Task all handle this cleanly because they have a single execution environment. Pants’s sandbox/real split creates an impedance mismatch for workflows where a non-source input (git state, API output) feeds a sandboxed build step.

No sequential dep chaining

run_shell_command has no sequential dep chaining.

The CFN deploy workflow requires strict ordering:

  1. cfn-base-test-obverse — deploy base stack, write JSON
  2. deploy-test-obverse-hugo — upload Hugo output to S3 (bucket must exist)
  3. cfn-distrib-test-obverse — deploy distribution (reads base JSON)

run_shell_command targets have no dependencies field that enforces execution order. Pants is a build tool: it runs tasks that produce artifacts, not a task runner that sequences side effects. There is no deploy-test-obverse aggregate target — the user must invoke these three targets in order.

Compare:

# Make: left-to-right prereq ordering (implicit but correct)
deploy-test-obverse: build/cfn-test-obverse-base.vars.mk \
    deploy-test-obverse-hugo \
    build/cfn-test-obverse-distrib.vars.mk

# pydoit: explicit task_dep chain, executed in order
def task_deploy_test_obverse():
    return {"task_dep": ["cfn_base:test-obverse", "deploy_test_obverse_hugo", "cfn_distrib:test-obverse"]}

Both Make and pydoit can sequence these steps from one invocation. Pants cannot.

Sandboxing and deployment are fundamentally incompatible

shell_command runs in an isolated chroot. AWS credentials, network access, SSH keys, and ~/.netrc are not available in the sandbox. Every deployment step (aws cloudformation deploy, netlify deploy, rsync) must use run_shell_command, which provides none of Pants’s caching or reproducibility guarantees.

Pants is excellent for building artifacts reproducibly. Deploying them is a separate concern that Pants doesn’t address — you need a CI system (GitHub Actions, Buildkite) or a separate deployment tool.

Heavy bootstrap

Pants downloads itself (~50 MB Rust binary) plus a dedicated Python interpreter (pants.toml can pin a Python version). First run on a clean machine involves significant download time. Make is pre-installed on every Unix; pip install doit is fast; even go install task@latest is faster.

For a personal site, the bootstrap cost is a one-time nuisance. For a team with standardized toolchains, it’s invisible.

No pattern rules

Like every alternative investigated, Pants has no equivalent to Make’s:

%.exifstripped: | %
    scripts/exif smartstrip "$<"

The check-exifstripped target uses a shell loop — same trade-off as pydoit and Task.

run_shell_command has no up-to-date checking

For shell_command targets, Pants checks whether the inputs have changed (by content hash) and skips the build if the cached output is still valid. For run_shell_command, Pants always re-runs the command when invoked. There is no sources: or generates: equivalent.

This means every ./pants run //:deploy-prod-obverse re-deploys unconditionally. Make’s file-based up-to-date check (public/prod-obverse/.sentinel exists and is newer than sources) would skip the deploy if nothing changed.

Targets Not Translated

  • dev: Interactive hugo server; not a build artifact.
  • deploy-all / deploy-test-obverse: Require sequential side-effect ordering that Pants does not support for run_shell_command targets.
  • vars-clean: No vars.mk files in this implementation.
  • macos-dev-*: macOS-specific; not relevant.

Structural Differences Summary

Aspect Make (vars.mk) Pants
Syntax Makefile DSL Python (BUILD files)
File tracking Timestamps Content hash (SHA-256) for shell_command
Remote caching None First-class REAPI support
Parallel execution make -j flag Default for independent shell_command targets
Sandboxing No isolation Hermetic chroot for shell_command
Deploy steps Same model as build steps run_shell_command — no sandbox, no caching
Dynamic file deps -include + two-phase re-read Not supported for run_shell_command
VCS state as input build/git-info.vars.mk: .git/HEAD No equivalent; must use run_shell_command
Sequential side effects Left-to-right prereq ordering Not supported; targets are independent
Macros define ... endef Python helper functions
Pattern rules %.exifstripped: | % Not supported
Installation Universal (pre-installed) Downloads ~50 MB launcher + Rust binary
Config location Repo root (Makefile) pants.toml + BUILD files

Verdict

Pants is the wrong tool for this workflow — not because it’s a bad build system, but because this isn’t a build problem. It’s a deployment orchestration problem.

For the Hugo build stages specifically, Pants is excellent: content hashing, remote caching, hermetic sandboxes, and parallel execution are all meaningful wins over Make. If the goal were to build Hugo outputs reproducibly and share build artifacts across CI runners, Pants would be a strong choice.

But the workflow is dominated by deployment steps: CFN deploys, S3 syncs, Netlify uploads. These are side effects that need network access, AWS credentials, and sequential ordering. Pants’s execution model (sandboxed shell_command for reproducible builds; unsandboxed run_shell_command for everything else) splits these two concerns — and the deployment half gets nothing from Pants that Make doesn’t already provide.

The fundamental gap:

  1. .git/HEAD can’t be a Pants source file — git info must use run_shell_command
  2. run_shell_command output can’t feed shell_command input — the git-info → hugo-build dependency chain breaks the hermetic model
  3. run_shell_command has no sequential ordering — the CFN base → Hugo upload → CFN distribution chain can’t be expressed as a single Pants target

pydoit and Task both handle this workflow more naturally because they have a single execution model (run commands, track files) rather than Pants’s two-tier model (sandboxed artifacts vs. real-environment side effects).

Pants belongs in a monorepo building Python packages, Go services, and Docker images — where its caching and reproducibility guarantees apply end-to-end.

Responses

Webmentions

Hosted on remote sites, and collected here via Webmention.io (thanks!).

Comments

Comments are hosted on this site and powered by Remark42 (thanks!).