Skip to main content

ADR 005 — Infrastructure as Code: OpenTofu + Helm + Ansible (+ Docker Compose, Air-gap)

Status: Accepted Date: 2026-04-23 Deciders: Jerome Related: prd-detection-tiering, detection-multitier-architecture, brainstorm-deployment-models, saas-security-legal-framework

Context​

Noxys sells into three very different customer topologies:

  1. Cloud SaaS (Noxys-managed) — we own Kubernetes clusters on EU cloud providers
  2. Self-hosted K8s — regulated customer with their own cluster (OpenShift / Rancher / vanilla EKS/GKE/AKS)
  3. Self-hosted VM / bare-metal — smaller customer, one or two Linux VMs, no Kubernetes
  4. Air-gap — government / defence, no internet access, offline install

No single IaC tool covers all four. We must pick a stack that is sovereign-compatible (part of our positioning) and gives us first-class ergonomics in each topology without forcing customers into our tooling choices.

Decision​

Three first-class IaC tools, plus Docker Compose and an offline bundle.

TopologyPrimary tool
Cloud SaaS (Noxys clusters)OpenTofu provisions cloud resources → Helm deploys workloads
Self-hosted K8s (customer cluster)Helm chart
Self-hosted VM / bare-metalAnsible role
Self-hosted single-node POC / devDocker Compose
Air-gapTar bundle with cosign-signed image + pre-downloaded models + install script

Single source of truth for application configuration lives in the Helm values schema. OpenTofu and Ansible consume the same config surface where relevant; air-gap bundle embeds a pre-rendered Compose file or Helm template.

Consequences​

Positive​

  • Customer choice: we meet them where they already are (K8s shop → Helm, Ansible shop → Ansible, no-ops shop → Compose)
  • Sovereign-first: OpenTofu (not Terraform) aligns with our open-source EU positioning
  • Air-gap story is a natural extension of the signed image pipeline
  • Helm values remain the config contract; reduces drift across deployment modes

Negative​

  • Three IaC skill sets in the team. Mitigated by: only ops team touches OpenTofu; Ansible role is thin (it wraps Docker, not bespoke config management); Helm values documented thoroughly.
  • CI must smoke-test all four modes (Kind for K8s, Vagrant/Docker-in-Docker for Ansible role, dry-run for OpenTofu, compose up for Compose).
  • Air-gap bundle rebuild is a separate release artifact — adds ~10 min to the release pipeline.

Alternatives Considered​

Alternative A — Terraform instead of OpenTofu​

  • Pros: larger ecosystem, most customer ops teams know it
  • Cons: Terraform moved to BSL (Business Source License) in Aug 2023. Using BSL in our own infra while selling a "sovereign open" pitch is a narrative contradiction customers in sovereign sales notice. OpenTofu (Linux Foundation stewardship, MPL-2.0) resolves this.
  • Note: OpenTofu is drop-in compatible with Terraform providers; our customers can still use Terraform on their side.
  • Rejected: license alignment with brand positioning is load-bearing

Alternative B — Pulumi (TypeScript / Python / Go)​

  • Pros: real language, test-friendly, good abstraction
  • Cons: team is not Pulumi-native; state management (SaaS backend or self-hosted) adds complexity; language lock-in per project; smaller community in EU sovereign ops; customer ops teams rarely run it
  • Rejected: learning curve + ecosystem mismatch outweighs the ergonomic gains at our stage

Alternative C — Pure Kubernetes manifests (no Helm)​

  • Pros: minimal tooling, gitops-friendly (Kustomize + ArgoCD)
  • Cons: config variance per tier / per customer exploded quickly in our prototypes; Helm's value templating + lint + chart repo is exactly what we need; Kustomize overlays become unreadable at > 3 variants
  • Rejected: Helm is the K8s packaging lingua franca; we can layer Kustomize later if needed

Alternative D — Ansible only (no K8s, no OpenTofu)​

  • Pros: one tool, everywhere
  • Cons: Ansible at K8s scale is painful; cloud SaaS economics demand K8s-native autoscaling; OpenTofu much better for cloud-resource lifecycle
  • Rejected: wrong tool for cloud SaaS tier

Alternative E — Crossplane or other K8s-native IaC​

  • Pros: one control plane
  • Cons: bleeding-edge, non-standard customer skill, doesn't solve the bare-metal or air-gap problems
  • Rejected: mismatch with customer reality

Enforcement​

  • Helm chart is the canonical manifest; OpenTofu module composes it (helm_release)
  • Ansible role's job is only to install Docker + pull container + systemd unit + TLS; it does not re-implement app config
  • Docker Compose file is generated from a reduced Helm template (chart helper)
  • CI matrix: kind, vagrant (or skip if too heavy), tofu plan -target, docker-compose up
  • Air-gap bundle generation is a Makefile target; release pipeline publishes the tarball alongside the Helm chart

Follow-up Actions​

  • DT-5 delivers all five modes
  • Document trade-offs per mode in docs/ops/sidecar-deployment.md
  • First-customer pilot (Business tier self-host) validates Helm path
  • First air-gap pilot validates offline bundle

References​