Case study

Reviving Watchtower — Fixing an Abandoned Docker Auto-Updater

Go 1.22 Docker SDK v26 Multi-stage build Alpine 3.21 Docker Compose

3

Bugs fixed in lifecycle.go

~30

Dependencies updated

1.20 → 1.22

Go toolchain

1.25 → 1.41

Minimum Docker API

The problem

Thousands of homelab operators and small teams rely on Watchtower as a “set it and forget it” way to keep containers updated. It watches running containers, pulls updated images when a registry has a new version, and recreates each container with the options it was started with.

In late 2024 the maintainers posted a deprecation notice and walked away. Without upstream work the codebase drifted: newer Docker SDK releases reorganised their types, modern daemons rejected the ancient API version the tool asked for, and the Dockerfiles still assumed a pre-compiled binary already existed on disk. A tool meant to reduce babysitting had become something you had to babysit.

Goal: turn the abandoned project back into a working, production-ready, drop-in replacement with no configuration changes for existing users, and document every change transparently.

Phase 1 — Code audit and bug fixes

The first pass was a full read of the container lifecycle and registry code. Three distinct defects lived in a single file.

lifecycle.go

Nil pointer dereference
Under certain conditions a pointer was dereferenced without a nil check, crashing the whole updater.

lifecycle.go

Silently swallowed errors
Failures from container stop operations were dropped instead of logged or surfaced, turning production debugging into guesswork.

lifecycle.go

Out-of-order log messages
Events were logged in the wrong sequence, so the timeline read backwards during post-incident review.

registry.go

Log ordering
Update checks appeared to finish before they began. Reordering the log calls restored a chronological story.

go.mod

Stale dependency graph
Roughly 30 packages were years behind. Go moved from 1.20 to 1.22 and every dependency was refreshed.

The major-version trap

The dependency refresh exposed a subtle mismatch: go.mod referenced ginkgo/v2 and robfig/cron/v3 while the source still imported the v1 paths. Go treats major versions as different modules, so the build simply failed. Pinning both back to their v1-compatible releases, ginkgo v1.16.5 and robfig/cron v1.2.0, aligned the manifest with what the code actually imports.

Phase 2 — Building from source inside Docker

The original Dockerfiles copied an already-compiled binary into a scratch image. There was no way to build without a full local Go toolchain. A new multi-stage Dockerfile compiles entirely inside Docker:

# Build stage
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN GOFLAGS="-mod=mod" go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux \
    go build \
    -ldflags="-X github.com/X4Applegate/watchtower/internal/meta.Version=${VERSION}" \
    -o watchtower ./cmd/watchtower/

# Final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /
COPY --from=builder /app/watchtower .
ENTRYPOINT ["/watchtower"]

GOFLAGS="-mod=mod" is doing real work here. The go.sum checksums were stale after the dependency update and could not be regenerated through GitHub’s web editor, so the flag lets Go resolve module requirements during the build instead of failing on the old checksums.

Phase 3 — Docker SDK v26 migration

Building against a current Docker SDK failed immediately with undefined: types.ContainerListOptions and friends. SDK v26 moved these types out of the top-level types package into types/container under new names.

Old (SDK ≤ v25)New (SDK v26+)
types.ContainerListOptionscontainer.ListOptions
types.ContainerRemoveOptionscontainer.RemoveOptions
types.ContainerStartOptionscontainer.StartOptions

Adding the github.com/docker/docker/api/types/container import and updating the three call sites in pkg/container/client.go got the build green.

Phase 4 — The one-line fix that mattered most

The first test run of the fresh binary died with: “client version 1.25 is too old. Minimum supported API version is 1.40.” A constant in internal/flags/flags.go had not been touched since 2017.

Before

const DockerAPIMinVersion string = "1.25"

After

const DockerAPIMinVersion string = "1.41"

Docker 20.10 and later require API 1.40 at minimum. 1.41 puts the fork in a safe range for any daemon shipped in the last several years.

v1.0.1 — Hardening the release

The first tagged release of the fork focused on the things that silently rot in a forked repository.

  • Base images unpinned and staleAll four Dockerfiles pinned alpine:3.19.0, a December 2023 patch release. They now use alpine:3.21 for current CA certificates and time-zone data, and the unpinned golang:alpine is fixed to golang:1.22-alpine.
  • Version string never injectedDockerfile.dev-self-contained still pointed its ldflags at the old module path, so builds silently carried no version. Fixed, with a git describe fallback for untagged builds.
  • Building the wrong repositoryDockerfile.self-contained was cloning the abandoned upstream repo instead of the fork. It now builds this codebase.
  • Dead flags removedGO111MODULE=on has been a no-op since Go 1.17 and is gone; the Compose file and environment variables were simplified.
  • Transparent historyA CHANGELOG.md documents every change from upstream, and the README carries a clear fork notice in place of the old deprecation warning.

Using the fork

It is a drop-in replacement: point your existing Compose file at the new image and keep your configuration.

services:
  watchtower:
    image: x4applegate/watchtower:latest
    container_name: watchtower
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_POLL_INTERVAL=3600
      - TZ=America/Los_Angeles

Prefer to build it yourself? No local Go toolchain is needed any more:

git clone https://github.com/X4Applegate/watchtower.git
cd watchtower
docker build -t x4applegate/watchtower:latest .

What’s next

GitHub Actions for automated builds and Docker Hub pushes, migrating the test suite from Ginkgo v1 to v2, and finishing the module path rename across every source file so the fork stands fully on its own.