Skip to content

Releasing

Overview

The controller and the Helm chart are released together: one version number, one v<version> git tag, one changelog, one release pipeline. The VERSION file is the single source of truth; charts/haptic/Chart.yaml carries the same value in both version and appVersion, and CI refuses to tag a release when they disagree.

Versions follow Semantic Versioning with support for pre-release suffixes.

Version numbering

Format: MAJOR.MINOR.PATCH[-PRERELEASE]

Type Example Description
Stable release 0.1.0, 1.0.0 Production-ready version
Alpha 0.1.0-alpha.1 Early testing, APIs may change
Beta 0.1.0-beta.1 Feature complete, needs testing
Release candidate 0.1.0-rc.1 Final testing before release

CHANGELOG conventions

There is one changelog file, CHANGELOG.md at the repository root, covering the controller and the Helm chart. It follows the Keep a Changelog format with an ## [Unreleased] header at the top.

  • Controller changes go in a release's top-level ### Added/### Changed/### Fixed/… sections.
  • Helm chart changes (values, templates, chart defaults, values migrations) go under the release's ### Helm chart subsection, with #### Added/#### Changed/… sub-headings.

charts/haptic/CHANGELOG.md is a pointer to the root changelog and doesn't take entries.

Prerequisites

Before releasing:

  1. Clean working directory - All changes committed
  2. All tests passing - CI pipeline green on main branch
  3. Documentation updated - Any new features documented

Publish a main snapshot

Ordinary main pipelines build pipeline-scoped test images but don't publish the multi-architecture main-<sha> controller, SPOA-hub, and chart snapshot.

  1. Open Build > Pipelines > New pipeline in GitLab.
  2. Select the main branch.
  3. Add the variable PUBLISH_MAIN_SNAPSHOT with value true.
  4. Run the pipeline.

The three snapshot publishers run as one dependency chain, so the chart is published only after both matching images succeed.

Release process

The main branch is protected, so releases are made via merge requests. CI automatically creates the tag when the VERSION file changes on main.

Step 1: Curate the changelog

Rewrite the accumulated ## [Unreleased] entries in CHANGELOG.md into user-facing release notes: merge related entries, and drop items that never shipped in a release. Update the hand-curated artifacthub.io/changes annotation in charts/haptic/Chart.yaml with summary bullets for the new release.

Don't rename [Unreleased] yourself — the release script does the mechanical promotion.

Step 2: Cut a release branch

main is protected, so the release commit lands via merge request. Branch first; the script commits to whatever branch is currently checked out.

git checkout -b release/v<version>

Step 3: Run the release script

make release RELEASE_VERSION=<version>
# or directly: ./scripts/release.sh <version>

The script:

  • Validates the version format (X.Y.Z or X.Y.Z-suffix.N)
  • Promotes CHANGELOG.md [Unreleased] to [<version>] - <date> and leaves a fresh empty [Unreleased] above it (skipped when a [<version>] section already exists)
  • Writes <version> to the VERSION file
  • Updates Chart.yaml version, appVersion, and the artifacthub.io/images annotation (controller and spoa-hub image tags)
  • Updates the helm install ... --version <version> examples in the READMEs and docs, and the landing page's fallback version
  • Keeps commands under Upgrading to <version> headings within that version line; older migration steps retain their target version
  • Re-pins the controller image tag in the docs' migrate-check docker one-liner (haptic:<version>-haproxy<series>, with the series taken from versions.env's DEFAULT_HAPROXY)
  • Rewrites hosted-docs links (/docs/dev/ → /docs/<version>/) and repo-blob links (/-/blob/main/ → /-/blob/v<version>/) so a released page links to the matching sources
  • Stages and commits everything as release: haptic v<version>

The docs-site changelog page needs no release-time sync: it's generated from CHANGELOG.md on every mkdocs build (docs/site/hooks/changelog.py).

The script doesn't create a tag — that happens in CI after the MR merges.

Step 4: Push and open the MR

git push -u origin release/v<version>

glab mr create --title "release: haptic v<version>" \
  --description "Release haptic v<version>" \
  --target-branch main

Step 5: Prepare the SPOA release image

Wait for the MR preflight pipeline and code review to pass, then enable auto-merge to enter the merge train. Run the manual prepare-spoa-release job in the train pipeline. It builds all three architectures under the chart's release tag, before the chart's install tests need that image. Install-test jobs wait for this job to succeed before starting. The chart itself remains unpublished.

The job records the upstream image digest, Docker build inputs, and verified plugin bytes in the image's input identity. An existing release image must match those inputs and contain all three architectures; it's never rebuilt or overwritten. If the bundle changes after preparation, choose a new release version.

Temporary preparation tags use ci-spoa-prepare-*, covered by the project's configured ^ci-.* registry cleanup policy. The versioned release tag is retained.

The train merges only after image preparation and the full pipeline pass. The final tag pipeline verifies and reuses the prepared image, then signs it, attaches its software bill of materials, and smoke-tests amd64 and arm64 before publishing the chart.

Automatic tag creation

After the MR is merged, CI automatically:

  1. Verifies the merged tree against its successful merge train; during the initial rollout, also repeats the full test matrix
  2. Verifies VERSION, Chart.yaml version, and appVersion agree
  3. Creates and pushes the v<version> tag
  4. Triggers the release pipeline
Manual Tagging (Fallback)

If automatic tagging fails, you can create the tag manually:

git checkout main
git pull origin main
git tag -a v<version> -m "Release <version>"
git push origin v<version>

What CI does automatically

When a v* tag is pushed, CI does the following:

  1. Build binaries for linux/amd64, linux/arm64, linux/arm/v7
  2. Create the GitLab release with:
    • Signed binaries
    • SHA256 checksums
    • Release notes from the version's CHANGELOG.md section (controller + chart)
    • Pre-release flag (for alpha/beta/rc versions)
  3. Build Docker images for every supported HAProxy series (3.0-3.4)
  4. Verify the prepared spoa-hub image (spoa-hub:<version>) without rebuilding it
  5. Package the Helm chart and push it as a signed OCI artifact to registry.gitlab.com/haproxy-haptic/haptic/charts
  6. Sign all artifacts with Cosign using keyless OpenID Connect (OIDC)
  7. Generate a Software Bill of Materials (SBOM) for each image and attach it as a Cosign attestation
  8. Trigger the versioned documentation build (the /docs/ site)
  9. Run release conformance against the published image and retain the report and provenance without scheduled expiration

The conformance job runs after publication. A failed job marks the release pipeline failed but doesn't retract published artifacts; inspect its report and fix the failure before claiming conformance for that release.

Add the release to the upgrade test

Once the chart is published, add the new stable version to test-chart-upgrade.parallel.matrix in .gitlab-ci.yml. Each entry runs the upgrade test from that release in its own job. Until you add it, check-chart-upgrade-baselines fails on every pipeline that runs the upgrade tests, so the new release can't go untested. Pre-releases aren't required. The check covers published stable versions at or below VERSION; a matrix entry above VERSION also fails it.

Patch releases for an older release line

A patch release for an older minor version (for example 0.2.2 while main already carries 0.3.0 work) is cut from a maintenance branch, maint/<major>.<minor>, instead of main.

When to use a maintenance branch

Port only security fixes and severe regressions, and only to the latest released minor version. Never port a feature: a patch release must not change behaviour beyond the fix. If main holds nothing unreleased beyond fixes, release from main instead.

The fix lands on main first, or in parallel; the port is a separate merge request.

Step 1: Create the maintenance branch

Branch from the last tag of the release line. maint/* is a protected branch pattern: only maintainers push to or merge into it, and nobody can force-push.

git fetch origin --tags
git push origin "v<major>.<minor>.<patch>^{commit}:refs/heads/maint/<major>.<minor>"

Skip this step if the branch already exists.

Step 2: Port the fix

Branch off the maintenance branch and open a merge request into it:

git fetch origin
git checkout -b fix/<issue>-<slug>-<major>.<minor> origin/maint/<major>.<minor>

Adapt the fix to the older code instead of cherry-picking it blindly: helpers, file layout, and tests may differ. Port the tests that prove the fix, and add the CHANGELOG.md entry under ## [Unreleased] on the maintenance branch.

glab mr create --target-branch maint/<major>.<minor> \
  --title "fix: <summary> (<major>.<minor> backport)"

Step 3: Prepare the release

Once the fix is merged, prepare the release on a branch named exactly release/v<version>. The prepare-spoa-release job only runs for that name.

git fetch origin
git checkout -b release/v<version> origin/maint/<major>.<minor>
./scripts/release.sh <version>

Update the artifacthub.io/changes annotation in charts/haptic/Chart.yaml for the new version, then commit, push, and open the merge request into the maintenance branch:

git push -u origin release/v<version>
glab mr create --target-branch maint/<major>.<minor> \
  --title "release: haptic v<version>" \
  --description "Release haptic v<version>"

Run the manual prepare-spoa-release job in the merge train pipeline, as in Step 5 of the regular release.

Step 4: Merge and let CI tag

Maintenance merge requests run the same full merge-train matrix as main, including every supported HAProxy version and both conformance suites. The post-merge pipeline verifies the tested tree before create-release-tag creates v<version>, with the same VERSION/Chart.yaml check as on main.

test-chart-upgrade upgrades only from published stable versions at or below the branch's VERSION, so a release from a newer line, such as 0.3.0 on maint/0.2, is never used as a baseline.

A maintenance branch never publishes main-only artifacts: no main-<sha> snapshots, no dev docs, no landing page, and no /playground/dev/.

What moves to the new version

Artifact Behaviour for a maintenance release
latest-haproxy<series> and spoa-hub:latest image tags Move only when the version is the highest stable v* tag (scripts/release-is-latest.sh)
/docs/<version>/ and /playground/<version>/ Added; no other version directory changes
latest docs alias Points at the highest stable version by semver, so it stays on the newer line
dev docs and landing page Unchanged; only main rebuilds them

Step 5: Follow up on main

After the chart is published, open one merge request into main that:

  1. Adds the ## [<version>] section to CHANGELOG.md, in version order below ## [Unreleased].
  2. Adds <version> to test-chart-upgrade.parallel.matrix in .gitlab-ci.yml (see Add the release to the upgrade test).
  3. Runs ./scripts/release.sh <version> if the release line is the newest one, so VERSION, Chart.yaml, and the install examples on main point at the patched version. The script skips the changelog promotion because the section from item 1 already exists.

Don't open it before the chart is published: check-chart-upgrade-baselines fails on a matrix entry that isn't published yet.

Security fixes

Keep the issue confidential until the release is published, and describe the fix in merge requests without an exploit recipe. List it under ### Security (or #### Security in the Helm chart subsection) in CHANGELOG.md, which becomes the GitLab release notes. After publication, request a CVE ID from the confidential issue (Create CVE ID Request in the issue sidebar; GitLab is the CVE Numbering Authority), then make the issue public.

Documentation versioning

Each release creates a versioned snapshot under /docs/<version>/. The version menu keeps older stable releases available and marks the highest stable version as latest.

Release type Documentation behavior
Stable (0.3.0) Creates a snapshot; updates latest if this is the newest stable release
Prerelease (0.3.0-alpha.1) Creates a snapshot without the latest alias
Final after pre-releases Removes alpha, beta, and rc snapshots for that exact version

For example, publishing 0.3.0 creates /docs/0.3.0/ and removes /docs/0.3.0-alpha.1/. It keeps 0.2.0, 0.1.0, dev, and any pre-releases of later versions. Cleanup covers the documentation, playground, and their version menus on every publication. Retrying an old prerelease can't bring it back after its final release exists.

The 0.1.0 entry links to its original, separate controller and Helm chart docs. The playground menu only includes releases with a published playground bundle.

Pre-release vs final release

Pre-releases

Pre-releases (alpha, beta, rc) have these differences:

  • Docker images built but don't get latest tag
  • Documentation created but not marked as latest
  • GitLab release marked as pre-release
  • Not recommended for production use

Final releases

Final releases (no suffix):

  • Docker images get latest tags if the version is the highest stable release
  • Documentation gets latest alias if the version is the highest stable release
  • Pre-release documentation versions are removed
  • Recommended for production use

Troubleshooting

Release script fails

Error Solution
Warning: Working directory is not clean — an interactive Continue anyway? [y/N] prompt, not a hard failure Answer N, commit or stash your changes, and re-run
CHANGELOG.md has no [Unreleased] section Restore the ## [Unreleased] header
Invalid version format Use X.Y.Z or X.Y.Z-suffix.N

CI pipeline fails

  1. Check GitLab CI logs for specific error
  2. Verify tests pass locally with make test
  3. Check Docker builds work locally

Docker image missing

If images don't appear after release:

  1. Check the release-controller job completed
  2. Verify registry authentication succeeded
  3. Check for build errors in job logs

Supply chain security

All release artifacts are signed and include security metadata:

Artifact signing

All artifacts are signed with Cosign using keyless OIDC:

  • Binaries: Checksums file signed with detached signature
  • Docker images: Each image tag signed
  • Helm chart: OCI artifact signed

Verify image signature:

Release tags produce one image per supported HAProxy series (<version>-haproxy<series>). The v prefix from the git tag is stripped, so the git tag v0.1.0 yields image tags 0.1.0-haproxy3.0, 0.1.0-haproxy3.1, 0.1.0-haproxy3.2, 0.1.0-haproxy3.3, 0.1.0-haproxy3.4:

cosign verify \
  --certificate-identity-regexp='https://gitlab.com/haproxy-haptic/.*' \
  --certificate-oidc-issuer='https://gitlab.com' \
  registry.gitlab.com/haproxy-haptic/haptic:0.1.0-haproxy3.4

SBOM (software bill of materials)

Each controller image includes an SBOM attestation in Software Package Data Exchange (SPDX) format (the spoa-hub image uses CycloneDX — see Stream Processing Offload Agent (SPOA) Hub):

View SBOM:

cosign verify-attestation \
  --type spdxjson \
  --certificate-identity-regexp='https://gitlab.com/haproxy-haptic/.*' \
  --certificate-oidc-issuer='https://gitlab.com' \
  registry.gitlab.com/haproxy-haptic/haptic:0.1.0-haproxy3.4 \
  | jq -r '.payload' | base64 -d | jq '.predicate'

The SBOM lists all packages, libraries, and dependencies in the container image.

Version files reference

File Content Updated By
VERSION Release version (single source of truth) Release script
Chart.yaml:version Same value as VERSION Release script
Chart.yaml:appVersion Same value as VERSION Release script
Chart.yaml artifacthub.io/images Controller and spoa-hub image tags Release script
Chart.yaml artifacthub.io/changes Release summary bullets By hand (Step 1)
Found a problem on this page? Report it