OPERATIONS · v0.4.2.14

Deployment guide

Source: modules/anthill/docs/DEPLOYMENT.md, a path from the root of the product's repository — versioned with the code, rendered here at every release.

Status: Docker, LXC, and Windows deployments are all ready to use today. Like the rest of ANTHILL they remain under continuous development and improvement — the current refinement target is deeper Windows SCM integration (see the roadmap at the bottom). This doc is the living reference for all three; it grows as each improvement lands instead of being rewritten from scratch each time.

Goal: run ANTHILL the way most home-lab/production setups actually deploy things — a standalone LXC container, a Docker container, or a Windows service — reachable at the IP of the machine it's running on, the same way any other appliance/container on your network would be. No cloud dependency, no separate reverse proxy required to make it reachable.

1. What changed to make this possible

Two small architectural changes underpin every deployment target in this doc, not just Docker:

  • ANTHILL now binds all interfaces (0.0.0.0) by default, in every safety profile. Previously the default (and every safety profile's forced override) was 127.0.0.1, which meant a fresh container/service install was unreachable from the network until someone edited config.json. The security boundary is the operator login (password auth, PBKDF2-SHA256, role-based sessions — see Security Model) rather than network isolation. Set api_host to 127.0.0.1 (or ANTHILL_HOST=127.0.0.1) if you specifically want localhost-only.

    Corrected at v0.3.8.91. This paragraph used to end "so this changes a cosmetic default, not the real safety rail". That reasoning holds from the second account onward and fails for exactly the window it was describing: on a fresh install there is no operator login yet to be the boundary, so /auth/setup — unauthenticated by necessity while no account exists — belonged to whoever reached the port first. Combined with operator_shell_enabled, which shipped true and is host command execution for administrators, the wildcard bind was a real exposure and not a cosmetic one.

    Both halves are fixed. The operator terminal now ships off. And on any non-loopback bind the process mints a single-use first-run setup token at startup, prints it to the service log, writes it to SETUP-TOKEN.txt under the workspace directory, and requires it on /auth/setup. Setup spends it permanently.

    Behind a reverse proxy? The token rule reads the BIND, not the caller's address — otherwise a proxy, whose requests all arrive from localhost, would authorise the whole internet through one hop. A proxy in front of a loopback bind is the one shape this process cannot see: set setup_token_required: true (or ANTHILL_REQUIRE_SETUP_TOKEN=1) so the token is required anyway.

  • ANTHILL_HOST / ANTHILL_PORT / ANTHILL_OLLAMA_HOST / ANTHILL_OLLAMA_MODEL environment variables now override config.json, with the highest precedence of any config source. This is what makes container/LXC/service deployment clean: you configure the deployment (docker-compose environment:, an LXC profile, a Windows Service's env block) without baking settings into an image or editing a file inside a running container. It's also what makes the CLI's --host/--port/--ollama-host/--ollama-model flags actually take effect — they set these same env vars under the hood now (previously they wrote a static field that AnthillRuntime.Initialize() silently overwrote a moment later from config.json; this was a latent bug, now fixed as a side effect).

  • A reachable LAN IP is auto-detected and printed at startup (NetworkUtil.GetLikelyLanIPv4, Anthill.Core/Common/NetworkUtil.cs) — a local UDP "connect" that never sends a packet, just asks the OS which interface/address it would route through. 0.0.0.0 isn't a URL you can open in a browser; this is what lets the console banner and GET /status (reachable_ip field) show you the actual address to use instead, on both Linux and Windows, without you having to run ip addr/ipconfig yourself.

Auth is always on (see the config-level comments in AnthillConfig.cs/config.example.json), and from v0.3.8.91 the first-run window is covered too — see the correction above. What these changes alter is what a fresh install looks like on first boot.

2. Docker (implemented)

Ships at the repo root: Dockerfile, docker-compose.yml, .dockerignore.

Quick start (Linux Docker host)

docker compose up -d --build
docker compose logs -f anthill

Watch the logs for a line like:

ANTHILL v1.8.6 API listening on http://0.0.0.0:8713
Open the colony console at http://192.168.1.50:8713/ui  (or http://localhost:8713/ui on this machine)
Listening on all network interfaces — protected by the operator login, not network isolation.

Open that URL and create your admin account. That's it — no config file to prepare first.

Why network_mode: host by default

The shipped docker-compose.yml uses network_mode: "host", which makes the container share the host's network stack directly instead of Docker's usual NAT'd bridge network. Two consequences, both desirable here:

  1. ANTHILL's auto-detected reachable IP genuinely is your host machine's LAN IP — not a Docker bridge address like 172.17.0.2 that nothing else on your network can reach without a port publish.
  2. A locally-running Ollama at http://localhost:11434 (the common home-lab setup: Ollama and ANTHILL on the same box) just works, with no host.docker.internal plumbing.

This only works on Linux Docker hosts — it's the standard setup for Proxmox/LXC-hosted Docker, bare Debian/Ubuntu, TrueNAS SCALE, Unraid, etc. If that's not you, use bridge mode instead:

Bridge mode (Windows/macOS Docker Desktop, or if you just prefer explicit port publishing)

services:
  anthill:
    build: .
    image: anthill:latest
    container_name: anthill
    ports:
      - "8713:8713"
    environment:
      - ANTHILL_OLLAMA_HOST=http://host.docker.internal:11434
    extra_hosts:
      - "host.docker.internal:host-gateway"   # Linux Docker Desktop / Docker Engine also needs this line;
                                                # Docker Desktop for Mac/Windows provides host.docker.internal automatically
    volumes:
      - anthill-data:/app/.anthill
    restart: unless-stopped
volumes:
  anthill-data:

With bridge mode, ANTHILL is reachable at http://<docker-host-ip>:8713/ui — the port-published address — rather than whatever IP NetworkUtil detects inside the container's own network namespace (that detected value is only meaningful in host-network mode).

Persistence

Everything that needs to survive a container recreate lives under /app/.anthill inside the container: the SQLite database, config.json (auto-seeded with container-safe defaults on first boot if not already present), logs, backups, exports, and the AES-256-GCM field-encryption key. The compose file mounts this as the named volume anthill-data. Back that volume up like you would any other stateful container volume; there's nothing else to persist.

Reaching a project directory for self-modification missions

If you want the file ant to read/propose patches against a real project on disk, bind-mount it and point agent_workspace_dir in .anthill/config.json at the same path:

    volumes:
      - anthill-data:/app/.anthill
      - /path/on/host/to/your/project:/workspace
// .anthill/config.json (inside the anthill-data volume)
"agent_workspace_dir": "/workspace"

Upgrading

git pull
docker compose up -d --build

The named volume persists across this; only the image layers change.

Optional: a static API token for scripts/CI

Not required for normal use (the web UI creates an admin account on first launch), but if you want one for programmatic/CI access, uncomment ANTHILL_API_TOKEN in docker-compose.yml — it must be at least 32 characters, and a value with a space, tab or newline before or after it refuses the start ([ANTHILL SECURITY] API refused to start.) rather than being trimmed into another token:

openssl rand -hex 24

The token is a service identity of the installation's organization, registered as svc_api_token at every start: it holds every permission except managing accounts and the host shell, reaches every project in that organization and nothing in any other, and never reaches unfiled missions. Changing the variable and restarting rotates it; removing it and restarting ends it. For anything narrower — one project, read-only, an expiry — an owner mints a service identity of its own with POST /organizations/{id}/service-identities and the token can go.

No native kernel in the image (by design)

The Dockerfile does not build the optional C++20 native compute kernel — ANTHILL falls back to a bit-identical managed implementation automatically when the native library is absent (see src/Anthill.Core/Native/NativeKernel.cs, and native_kernel: managed-fallback in /status). This keeps the image free of a cmake/g++ build stage. If you want native-kernel acceleration in-container, add a cmake stage before the build stage in the Dockerfile and copy the resulting shared library into native/anthill_kernel/ before dotnet publish runs.

3. LXC (implemented)

Ships at deploy/lxc/: setup.sh (unattended installer/upgrader) and formicaria.service.template (the systemd unit it installs). Targets a fresh Debian 12+ or Ubuntu 22.04+ LXC container — Proxmox-created or otherwise, anything with systemd and apt.

Unlike Docker, an LXC container gets a real network interface directly (no bridge/NAT layer to work around), so NetworkUtil's auto-detected reachable IP just works here with zero extra networking config — this is the simplest of the three targets to actually use once it's running.

Creating the container (Proxmox)

From the Proxmox host shell (or Datacenter → node → Create CT in the web UI):

# Download a template if you don't already have one (Debian 12 shown; Ubuntu 22.04/24.04 works too)
pveam update
pveam available --section system | grep debian-12
pveam download local debian-12-standard_12.*_amd64.tar.zst

# Create an unprivileged container — 2 vCPU / 4 GB RAM / 16 GB disk is plenty for ANTHILL itself
# (Ollama, if it runs on the same box, wants a lot more — see README's Resource Requirements)
pct create 200 local:vztmpl/debian-12-standard_12.*_amd64.tar.zst \
  --hostname anthill \
  --cores 2 --memory 4096 --swap 512 \
  --rootfs local-lvm:16 \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --unprivileged 1 \
  --features nesting=0 \
  --onboot 1

pct start 200
pct enter 200

No special LXC features are required — this is a plain systemd service running a normal .NET binary, not Docker-in-LXC, so nesting, keyctl, or privileged mode aren't needed. (nesting only matters if you're planning to run Docker or another container runtime inside this same LXC container — unrelated to ANTHILL itself.)

Using plain LXD/incus instead of Proxmox? lxc launch images:debian/12 anthill (or the equivalent incus launch), then lxc exec anthill -- bash gets you to the same starting point — everything below is identical.

Installing ANTHILL

Inside the container, with a deploy key that can read the repository (it is private; the script's header says how to give the box one), clone it to /opt/anthill/src and run the installer from the checkout:

apt-get update && apt-get install -y curl ca-certificates git
git clone git@github.com:Formicaria/formicaria.git /opt/anthill/src
bash /opt/anthill/src/modules/anthill/deploy/lxc/setup.sh

That one script: installs the .NET 10 SDK (via Microsoft's apt repo, falling back to dotnet-install.sh on distros/versions Microsoft's repo doesn't have an entry for yet), pulls the checkout, publishes a self-contained linux-x64 binary, creates a dedicated unprivileged anthill system user, installs and enables the systemd unit, and starts the service. A box without repository access installs from the release tarball instead (README, "Linux quick start") and uses the systemd notes below.

Check the result:

systemctl status formicaria --no-pager
journalctl -u formicaria -n 20 --no-pager   # look for the "Open the colony console at http://..." line

Open the printed URL from another machine on your network and create your admin account.

Upgrading

cd /opt/anthill/src   # setup.sh's default checkout location
git pull
bash /opt/anthill/src/deploy/lxc/setup.sh

Or just re-run the same one-liner from the install step — setup.sh detects the existing checkout, pulls latest, republishes, and restarts the service. The named data directory (/opt/anthill/.anthill by default: DB, config.json, logs, backups, exports, encryption key) is untouched by an upgrade.

Customizing the install location / service user

ANTHILL_INSTALL_DIR=/srv/anthill ANTHILL_SERVICE_USER=anthill-svc bash setup.sh

Uninstalling

systemctl disable --now formicaria
rm -f /etc/systemd/system/formicaria.service
systemctl daemon-reload
rm -rf /opt/anthill /etc/anthill   # or just delete/destroy the whole LXC container instead

4. Releases

.github/workflows/anthill-release.yml, at the repository root, builds a tagged release: self-contained linux-x64/win-x64 archives, the Windows installer, a versioned Docker image pushed to ghcr.io/formicaria/formicaria, and a GitHub Release with notes pulled straight from the matching ## vX.Y.Z section of CHANGELOG.md. Every archive carries LICENSE, README.md, CHANGELOG.md and config.example.json beside the binaries; every asset has a .sha256 beside it, written in the same job from the bytes that were archived (§5a); and the release carries formicaria-<version>.manifest.json, written after the build legs from the digests they attached (never from a second build), listing every asset with its size and SHA-256 beside the version, the source tag, the commit and the component versions this build agrees with.

Tags are namespaced per module in this repository — formicaria/vX.Y.Z[.W] — because FORAGER and MICROMOUND release from the same tag list. The gate is on the release box, before the tag: dotnet test Anthill.sln -c Release on Windows — green, or no tag. That run is why CI's own Windows leg (anthill-ci.yml) is on demand rather than on every push: a private repository bills Windows minutes at twice the rate, and the box already runs the suite where it matters. To cut a release once a version bump is committed to main:

git tag formicaria/v0.3.13.0
git push origin formicaria/v0.3.13.0

The workflow's first job (verify-version) fails loudly if the tag doesn't match AnthillRuntime.Version in the pushed commit — this guards against tagging before the version bump actually landed. If it fails: bump the version, commit, delete the bad tag (git tag -d formicaria/v0.3.13.0 && git push origin :refs/tags/formicaria/v0.3.13.0), and re-tag.

The release is published automatically on tag push — no manual "Publish" step. The draft is opened right after the version check, the build jobs upload straight onto it (release assets are not metered; workflow artifacts are), build-manifest writes the manifest from the digests on the draft, and publish-release publishes it (marked "latest") only when exactly the seven expected assets are on it. Four-part maintenance tags (vX.Y.Z.W) are handled the same as three-part releases.

Then the release is copied where anyone can read it. This repository is private, and a private repository's releases cannot be read without a token — not by an installed colony's update check, not by the site's download buttons, not by a browser. So publish-public copies the seven assets and the notes to the public release repository named by AnthillRuntime.ReleaseRepo (Formicaria/formicaria-releases, plain vX.Y.Z[.W] tags, nothing in it but releases). That is the repository UpdateChecker, UpdateStager, the desktop updater and formicaria.us read. The job needs RELEASES_TOKEN, a fine-grained token with Contents read/write on that one repository; without it the job says so and the release stays private.

The Docker image is the exception: GHCR creates the package as private by default, and a private package is invisible to docker pull and counts against this organisation's metered package storage. Make it public once, when the first image is pushed there, at github.com/orgs/Formicaria/packages/container/formicaria/settings → Change visibility; a public package is neither metered nor gated.

Releases without Actions (by hand)

The workflow is the normal path. When it cannot run — the month's metered minutes spent, which is how this path came to exist at v0.3.13.0 — the same release is built from the same tag by three scripts under deploy/release/, each the workflow job it is named after, on the tagged commit in a scratch worktree so the checkout's own state never leaks into a release:

  1. build-archives.sh on any Linux with the .NET 10 SDK (the anthill LXC has one): publishes linux-x64 and cross-publishes win-x64 exactly as build-binaries does, packages the tarball and the zip with LICENSE, README.md, CHANGELOG.md and config.example.json, writes the two .sha256 sidecars, and checks that the linux binary reports the version it was built as. Or build-archives.ps1 on the release box itself, when no Linux box can compile — the same three publishes, linux-x64 cross-published from Windows, the zip written with root entries and forward slashes, and the tarball written by MakeTar.cs (a .NET 10 file-based program, dotnet run compiles it) so anthill carries the executable bit a tarball made with Windows tools would lose. The LXC could not compile the first hand-cut release; this is how it shipped.
  2. build-installer.ps1 on the Windows release box (Inno Setup 6 for the machine: winget install --id JRSoftware.InnoSetup -e --scope machine, since the script looks only under ${env:ProgramFiles(x86)}): publishes the server and the desktop shell for win-x64, compiles deploy/windows/formicaria-setup.iss against that output, writes the installer's sidecar.
  3. publish.ps1 from wherever the six files have gathered (scp the archives to the box): writes formicaria-<version>.manifest.json from the six files' digests and sizes — never from a second build — verifies every sidecar against its file, and creates or refreshes the release in this repository under formicaria/v<version> and in AnthillRuntime.ReleaseRepo under v<version>, notes cut from CHANGELOG.md at the tag, all seven assets on each, both marked latest. It needs FORMICARIA_RELEASE_TOKEN in the environment — a fine-grained token with Contents read/write on both repositories — and the tag already pushed: it refuses to create a release on a tag GitHub has not seen, because GitHub would mint that tag at the default branch.

Which root of trust (roadmap C-1). All three take -TrustRoot development|production (--trust-root for build-archives.sh), development by default, which is what every release so far has been. After the root ceremony every release is production on all three: the switch passes -p:FormicariaProductionRoot=true to every publish and bundles FORAGER with --trust-root production, and publish.ps1 records it in the manifest as components.trust_root, read out of the engine the zip carries. Before the ceremony a production build is refused at the tag, before anything is built. publish.ps1 -DryRun writes the manifest and the notes and sends nothing. deploy/release/README.md has the detail.

git tag -a formicaria/v0.3.13.0 -m "v0.3.13.0 - ..." && git push origin formicaria/v0.3.13.0
# archives: on a Linux box ...
#   bash /opt/anthill/src/modules/anthill/deploy/release/build-archives.sh formicaria/v0.3.13.0 /root/release-assets
#   scp -r root@<lxc>:/root/release-assets/0.3.13.0 .\release-assets\
# ... or here:
powershell -ExecutionPolicy Bypass -File deploy\release\build-archives.ps1 -Tag formicaria/v0.3.13.0
powershell -ExecutionPolicy Bypass -File deploy\release\build-installer.ps1 -Tag formicaria/v0.3.13.0
$env:FORMICARIA_RELEASE_TOKEN = '<token>'
powershell -ExecutionPolicy Bypass -File deploy\release\publish.ps1 -Tag formicaria/v0.3.13.0

What this path does not do: build the Docker image (build-docker logs into GHCR with the workflow's own token; a hand-cut release's notes say the image was not built). No image has ever been pushed as ghcr.io/formicaria/formicaria: the last Actions release run, v0.3.12.0, pushed ghcr.io/formicaria/anthill, whose latest is still that build and the only image docker pull can get; deploy/release/README.md says what pushing one by hand needs. Nor does it deploy the site (that is site-deploy.yml; by hand it is pnpm run test then wrangler deploy per app with the Cloudflare token in the environment). Everything the channel's consumers read — the seven assets, the sidecars' shape, the manifest — is the same either way, which is the point.

CI artifacts (on demand, no tag needed)

Since v1.8.23.3, plain CI (.github/workflows/anthill-ci.yml, at the repository root) can also produce a package: after the linux-x64 publish + --selftest job succeeds, the publish output is archived as anthill-linux-x64-v<version>.tar.gz and uploaded as a workflow artifact. The publish and the self-test run on every push; the tarball is kept only for a run started by hand (Actions → ANTHILL CI → Run workflow), and for three days — the repository is private now and artifact storage is metered, so a package per push is not affordable. Grab it from Actions → (the run) → Artifacts. Tagged Releases remain the permanent distribution channel.

Operator-shell service control (polkit)

The systemd unit runs with NoNewPrivileges=true, which blocks sudo/setuid escalation — so the admin-only operator Shell console (running as the anthill service user) can't sudo systemctl restart. setup.sh therefore installs a scoped polkit rule (/etc/polkit-1/rules.d/49-formicaria.rules, from deploy/lxc/formicaria-polkit.rules.template) that authorizes the service user to manage only the formicaria.service unit (restart/stop/start/ status) over D-Bus — systemd performs the action, so no privilege escalation is needed and the unit's hardening is untouched. The Shell tab's "Restart service", "Service status", and "Recent logs" buttons use it. Nothing else is granted — no other units, no package or system management; upgrades still run bash deploy/lxc/setup.sh from a root shell. If polkit isn't installed the rule is skipped (the installer says so) and service control from the console won't be available.

5. Windows Service — NOT SHIPPED

Corrected at v0.3.8.149. This section previously said a Windows Service path was "ready to use today" and pointed at a README.md section — "Deploy on Windows → Option C" — that has never existed. Nothing in this repository registers a Windows service, and ShellQuickActionTests actively guards against one appearing by accident: the console's Windows quick actions target the FormicariaDesktop process precisely because no service exists to target.

That guard is not an oversight to be fixed later. A service running as LocalSystem is the standard way to give a program the right to update itself without prompting, and it is a permanent privileged attack surface. v0.3.8.149 reached the same goal from the other side — the desktop app installs per-user, owns its own directory, and therefore replaces its own files with no elevation and no service. See §7.

The supported Windows shapes are the installer (per-user, self-updating) and the portable zip.

5a. Updating (v0.3.8.149)

Every release publishes a .sha256 beside each artifact, generated in the same CI job from the bytes it archived. Anything that installs an update — the desktop app, the systemd pre-start hook, the background stager — verifies the download against it and deletes a payload that does not match. Verification is not a setting.

A hash fetched over the same channel as the file is not a signature: anyone who can replace the asset can replace the sidecar. It defeats corruption, a bad mirror, a truncated download and a tampered CDN object; it does not defeat a compromised GitHub account. Authenticode signing of formicaria-setup-*.exe is the control that would, and it needs a certificate this project does not yet have. Verify by hand with sha256sum -c formicaria-<version>-linux-x64.tar.gz.sha256.

Since v0.3.11.0 every release also carries formicaria-<version>.manifest.json: one document naming the version, the source tag and commit, every asset with its size and SHA-256, and the component versions the build agrees with (the FORAGER engine floor, the MICROMOUND contract). The background stager verifies a download against the manifest when the release has one and refuses a manifest that names another version, so six correct-looking sidecars from one release beside archives from another no longer verify; a release without a manifest is judged by its sidecars exactly as before, and the desktop updater still reads the sidecars, so nothing shipped earlier reads differently. The manifest is not signed — signature is null and signing says why — because a signature needs a key with an owner and a custody procedure this project has not decided (W1-06). The field is there so signing becomes a value, not a schema change.

auto_update (Settings → Colony) decides what happens when a newer release exists:

Value Behaviour
silent (default) Download, verify, install at the next start. No prompt, no elevation.
notify Check and report it. Install nothing.
off Do not check.

Updates always apply at the next start, never to a running process — a program cannot replace its own files while executing, and an update that interrupts a mission is worse than the prompt it replaced.

Shape How an update lands
Windows installer formicaria-setup-<v>.exe runs /VERYSILENT at next launch. Per-user, so no UAC.
Windows portable Only the paths the new archive contains are replaced. .anthill beside the binary is never touched; an archive reaching into it is refused whole.
LXC / systemd Staged into .anthill/updates, swapped in by ExecStartPre=anthill --apply-staged-update. Needs the ReadWritePaths=<install>/bin line in the unit — delete it and the ExecStartPre line, and set auto_update=notify, to keep bin/ read-only.
Docker Never self-updates. docker compose pull && docker compose up -d.

6. Roadmap

Target Status Key files
Container-style IP binding + env var overrides ✅ DONE. Underpins all three targets below. NetworkUtil.cs, AnthillConfig.cs, AnthillRuntime.cs, Anthill.Cli/Program.cs, Anthill.Api/ApiHost.cs
Docker ✅ DONE. Dockerfile, docker-compose.yml, .dockerignore at repo root. see §2 above
LXC ✅ DONE. deploy/lxc/setup.sh + formicaria.service.template. see §3 above
Tagged releases ✅ DONE. Binaries + Docker image (GHCR) + published GitHub Release, all automatic on tag push. .github/workflows/anthill-release.yml, see §4 above
Windows desktop app ✅ DONE. Per-user installer, self-updating (see §5a). deploy/windows/formicaria-setup.iss, src/Anthill.Desktop/UpdateService.cs
Silent updates, all shapes ✅ DONE at v0.3.8.149. SHA-256 verified, applied at next start. src/Anthill.Core/Updates/, src/Anthill.Api/UpdateStager.cs, see §5a
Windows Service ❌ NOT SHIPPED, and not planned. This row said READY for 75 releases against a README section that never existed (corrected in §5). A LocalSystem service is the usual way to buy unprompted self-update; v0.3.8.149 bought it with a per-user install instead, which needs no privilege at all. ShellQuickActionTests guards against one appearing by accident. tests/Anthill.Tests/ShellQuickActionTests.cs
Code signing (Authenticode) ⬜ OPEN. The checksums shipped at v0.3.8.149 defeat corruption and a tampered object; only a signature defeats a compromised release account, and it needs a purchased certificate. .github/workflows/anthill-release.yml

Implementation order: container-style networking (done) → Docker (done) → LXC (done) → Windows desktop app + silent updates (done). Code signing is the open item.