/blog/

0001 0101 Buck2 Build for me.micahrl.com

A Buck2 translation of the project’s vars.mk Makefile.

Quick Start

# Install Buck2
cargo install buck2
# or download a prebuilt: https://github.com/facebook/buck2/releases

# Requires .buckconfig at repo root (marks the project root for Buck2):
cat > .buckconfig <<'EOF'
[repositories]
    root = .
EOF

# Build a site:
buck2 build //:build-prod-obverse

# Build all production sites:
buck2 build //:all-prod-builds

# Deploy a CFN base stack (side-effectful; see caveats below):
buck2 build //:cfn-test-obverse-base

Available Targets

Target Description
//:build-dev-obverse Build dev site
//:build-prod-obverse Build production site
//:build-prod-reverse Build production mirror
//:build-test-obverse Build test site
//:build-test-reverse Build test mirror
//:build-onion-obverse Build onion site
//:build-onion-reverse Build onion mirror
//:all-prod-builds Build all four production sites
//:git-info Capture git commit info → git-info.json
//:cfn-test-obverse-base Deploy test-obverse CFN base stack
//:cfn-test-reverse-base Deploy test-reverse CFN base stack
//:cfn-test-obverse-distrib Deploy test-obverse CFN distribution stack
//:cfn-test-reverse-distrib Deploy test-reverse CFN distribution stack

Note: deploy targets (Netlify, rsync, S3 hugo deploy) are not translated — see Targets Not Translated.

How This Compares to Make

What Works Well

1. glob() replaces both $(shell find ...) and the -include dance

The vars.mk Makefile solves parse-time overhead with a two-phase trick:

  1. build/hugo-sources.vars.mk is a target that runs find and writes HUGOSOURCES := ...
  2. -include build/hugo-sources.vars.mk causes make to re-read the Makefile
  3. Hugo targets depend on $(HUGOSOURCES)

Buck2 solves this at the language level: glob() in a BUCK file runs at analysis time, once, with results cached. No re-read needed:

# Buck2: evaluated once at analysis time, tracked by Buck2
_HUGO_SOURCES = glob(["archetypes/**", "config/**", "content/**", ...])

# Make (vars.mk): runs find on every invocation, or...
HUGOSOURCES = $(shell find archetypes config content ... -type f)

# Make (plain): ...requires the two-phase -include workaround
build/hugo-sources.vars.mk: archetypes config content ...
    scripts/emit-hugosources-vars.sh $@
-include build/hugo-sources.vars.mk

2. Content-hash caching instead of mtime

Make uses filesystem timestamps. Buck2 hashes file contents. This means:

  • Reverting a file to a previous state doesn’t trigger a rebuild
  • Moving files between machines and building works correctly
  • The build cache can be shared across machines (remote cache)

3. Starlark functions replace define/endef

Make’s macro syntax is positional and hard to debug. Starlark is a real language with named arguments:

# Buck2: named arguments, readable at the call site
def _hugobuild(env):
    genrule(
        name = "build-" + env,
        srcs = _HUGO_SOURCES + [":git-info"],
        cmd = "... --environment {env} ...".format(env = env),
        out = env,
    )

_hugobuild("prod-obverse")
# Make: positional $(1), called with $(call ...)
define hugobuild
    @echo "Building for $(1)"
    hugo --destination public/$(1) --environment $(1)
endef

public/prod-obverse/.sentinel: ...
    $(call hugobuild,prod-obverse)

4. JSON replaces .vars.mk fragments

CFN outputs become a JSON file. Downstream targets read it with jq — no make variable expansion, no namespace prefixing to avoid collisions:

# Buck2: read JSON inline; no global variable namespace
cmd = "B=$(cat $(location :cfn-test-obverse-base)) && ... $(echo $B | jq -r .ContentBucketName) ..."
# Make (vars.mk): -include expands into global make variables
-include build/cfn-test-obverse-base.vars.mk
# Now TEST_OBVERSE_ContentBucketName is a make variable; use as $(TEST_OBVERSE_ContentBucketName)

5. Explicit dependency graph

Buck2 sandboxes actions: a target that doesn’t declare a file as srcs cannot read it. This makes undeclared dependencies a build error rather than a silent correctness bug. In Make, accidentally depending on a file that isn’t listed as a prerequisite works fine until it doesn’t.

6. Parallel by default

Buck2 runs independent targets in parallel automatically. Make requires -j.

What Doesn’t Work Well / Trade-offs

1. Side-effectful deploys don’t fit the model

Buck2 is designed for building artifacts — hermetic, reproducible, cacheable. Targets that have external side effects (AWS CloudFormation, netlify deploy, rsync) are awkward:

  • Buck2 will cache a target’s output and skip re-running it if inputs haven’t changed
  • But buck2 build //:cfn-test-obverse-base might skip the CloudFormation deploy if the template hasn’t changed, even if the stack was deleted
  • There’s no make deploy-prod-obverse equivalent that forces re-execution

The CFN targets in this translation are a best effort — they work on a clean build but may not behave correctly for re-deploys. pydoit and plain Make handle side-effectful targets more naturally.

2. No pattern rules

Make’s %.exifstripped: | % handles per-image EXIF stripping elegantly. Buck2 (like every alternative in this series) has no pattern rule equivalent. The list comprehension generates one genrule per image, which is correct but verbose and requires enumerating all images at analysis time:

# Buck2: one target per image
[genrule(name = "exif-" + src.replace("/", "-").replace(".", "-"), ...)
 for src in _IMAGES]
# Make: one rule covers all images
%.exifstripped: | %
    scripts/exif smartstrip "$<"

3. Hermeticity friction with .git/

Buck2 sandboxes actions; accessing .git/HEAD requires declaring it as a src. But git commands like git rev-parse HEAD also read .git/packed-refs, .git/refs/, etc. A truly hermetic git-info target would need to declare all of these — or accept that the sandbox is partially violated.

The BUCK file here declares .git/HEAD and .git/index, which covers the common case, but isn’t fully hermetic.

4. genrule outputs a single path

Each genrule produces one output (file or directory). Make targets can produce multiple outputs. This is a minor constraint in practice but occasionally requires splitting a single make target into two Buck2 targets.

5. Steep setup and learning curve

Buck2 requires:

  • .buckconfig at the repo root
  • Understanding target labels (//:name, //path/to:name)
  • Starlark syntax (close to Python, but not Python)
  • Buck2’s build model (analysis phase, execution phase, caching)

Make is pre-installed everywhere. Buck2 must be installed, and the project must be configured as a Buck2 project.

6. Over-engineered for this use case

Buck2 was built for monorepos with millions of files and hundreds of engineers. Its strengths — remote caching, distributed builds, strict hermeticity — don’t matter for a personal website. The complexity cost isn’t paid back.

Targets Not Translated

  • deploy-prod-obverse, deploy-prod-reverse: netlify deploy is a side-effectful command. Buck2 would cache it and skip re-running, which is wrong for a deploy.
  • deploy-test-obverse-hugo: hugo deploy ... --target s3-* is similarly side-effectful.
  • deploy-onion-*: rsync over SSH is a side effect.
  • deploy-clean, vars-clean: Buck2 uses buck2 clean instead.
  • dev: Interactive hugo server; not a build artifact.

Structural Differences Summary

Aspect Make (vars.mk) Buck2
Syntax Makefile DSL Starlark (subset of Python)
File listing -include + two-phase re-read glob() at analysis time
Stage outputs .vars.mk fragments included into Makefile JSON files read by downstream genrule cmds
Macros define ... endef + $(call ...) Starlark functions returning genrule() calls
Change tracking Filesystem mtimes Content hashes
Remote caching No Yes (built-in)
Parallel tasks make -j flag Automatic
Pattern rules %.exifstripped: | % Not supported; list comprehension per file
Side-effectful targets Natural (make deploy) Awkward; caching fights re-deploy semantics
Hermeticity None (anything on PATH is accessible) Sandboxed; undeclared deps are errors
Installation Universal (pre-installed) Must install; requires .buckconfig
Target naming Dashes (deploy-prod-obverse) Dashes ok (//:build-prod-obverse)
Re-run semantics Always runs if deps are newer Skips if inputs (by hash) haven’t changed

Verdict

Buck2 solves the vars.mk problem at the language level: glob() at analysis time cleanly replaces both $(shell find ...) and the -include two-phase dance, with no extra mechanism needed.

The key wins:

  1. glob() replaces parse-time $(shell ...) and the -include workaround — no two-phase re-read
  2. Content-hash caching is more correct than mtime-based tracking
  3. Starlark macros are cleaner than define/endef
  4. JSON for stage outputs is cleaner than .vars.mk fragments (same win as pydoit)

The main losses:

  1. Side-effectful deploy targets don’t fit Buck2’s build model — this is the biggest problem for this use case
  2. Hermeticity requires declaring all inputs; .git/ access is partially non-hermetic
  3. No pattern rules (same problem as all other alternatives)
  4. Significant setup overhead for a personal website

Buck2 is the right tool if you have a monorepo and care deeply about build correctness, caching, and reproducibility. For a personal website with frequent deploy targets that have external side effects, it’s the wrong level of abstraction. Make (with or without vars.mk) or pydoit are better fits.

Responses

Webmentions

Hosted on remote sites, and collected here via Webmention.io (thanks!).

Comments

Comments are hosted on this site and powered by Remark42 (thanks!).