Skip to content

Changelog

All notable changes to HAPTIC — the controller and its Helm chart — are documented in this file. Controller changes are listed first; chart changes (values, templates, chart defaults) follow under each release's "Helm chart" subsection.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Added

  • On HAProxy 3.4, adding or removing a route — an Ingress, an HTTPRoute, or a custom-CRD route — whose backend is dynamic-shaped (an empty body, a dynamic-capable balance) and whose per-route logic is map-driven now propagates with no HAProxy reload: the backend is created, populated and published, or drained and deleted, over the runtime API. On 3.0–3.3 the route still reloads (no runtime add backend), but its servers are always added and removed at runtime. A route whose backend carries a filter (for example response compression, on by default for Ingress via haproxy-haptic.org/compress-enable) stays structural and reloads on add or remove.

  • New spec.maps.<name>.ordered (default true) declares whether the position of an entry inside a map file changes what HAProxy does with it. Set it to false for a map read with map_str, map_beg, map_ip or map_str_int — those find a key by its own value, so a new entry can be appended over the runtime API instead of rewriting the file and reloading. Keep the default for map_reg, map_sub, map_dom, map_dir and map_end, which HAProxy evaluates as a first-match-wins list.

  • HAProxyCfg pod status reports the applied and running render plan id, the apply mode and its reasons in status.deployedToPods[].
  • The HAProxyCfg carries the render gate's verdict as ConfigValidated (HAProxy's own message on a refusal) and ConfigPinned (renders are held because it refused two in a row), plus a new haptic_config_pinned gauge.
  • New haptic agent subcommand: the HAPTIC agent, which the chart now deploys as the agent container of every HAProxy pod. It owns that pod's file tree and runtime sockets, applies the desired file set in one transaction, runs the runtime commands the controller composed or reloads (paced by --reload-interval-min), restores the last known good set when HAProxy rejects the result, and exports haptic_agent_* metrics on its own port (agent page).
  • New haptic diff answers "will this change reload?" before the change is applied: it renders a candidate configuration, or reads a running pod's applied plan with --from pod://<namespace>/<pod>, and compares the two with the decision the controller makes per pod. The first line is the verdict — runtime, file_only or reload — followed by a reason for every change that cannot run at runtime and the runtime commands it composed. --test renders both sides with a validationTest's fixtures, --output json gates a pipeline on the result.
  • New haptic agent state prints one HAProxy pod's agent state: the applied, running, worker-ops and last-known-good plan ids, the runtime inventory, the deletes still outstanding, and the last apply's mode, error stage and reload outcome. Run it with kubectl exec … -c agent -- haptic agent state; --verify re-hashes the tree first so the digests are observations, --files lists every file with its digest and size, and --output json prints the raw /v1/state.
  • New /debug/heapdump endpoint on the debug port writes a runtime/debug heap dump: every heap object, the pointer edges between them, and the roots. pprof reports where memory was allocated but not what still holds it, so this is what identifies a retainer. The heap is collected before the dump is written, since an uncollected dump is dominated by unreachable objects that by definition have no retainer. Read it with a heap-dump reader such as heapspurs. Writing the dump stops the world for its duration — health checks and admission both stall — so a concurrent second request is refused with 409, and the endpoint answers on loopback only, like /debug/pprof. The dump is roughly heap-sized and is staged in $TMPDIR, which counts against the pod's ephemeral-storage; it refuses with 507 rather than filling the filesystem, and HAPTIC_HEAPDUMP_DIR redirects it to a mounted volume. The written dump is checked for completeness, since the Go runtime ignores write errors while dumping and would otherwise return a truncated object graph with a 200.
  • Type-preserving collection pipeline for templates: map, filter, reject, flat_map, unique, unique_by and group_by, chained with |. Each stage keeps its input's element type, so typed field access still works at the end of a chain and a renamed field fails the config load instead of silently emptying a map file. See ADR-0018.
  • Arrow closures x => expr with parameter and result types inferred, accepted anywhere a function is expected.
  • Collection pipelines with inline stage functions compile to a single fused loop, so a chain costs what the equivalent hand-written loop costs.
  • sort_by accepts a func(a, b T) int comparator alongside its JSONPath criteria, and accepts typed slices.
  • Nested shapes inside a watched resource are nameable as type expressions (resources.endpoints.Endpoints, resources.gateways.SpecListeners), so a helper or closure can declare types below the resource root.
  • A struct whose every field is its zero value is falsy, so an optional object is tested with if ingress.Spec.DefaultBackend.Service instead of a dig() probe. Use not/and/or rather than !/&&/|| on struct operands.
  • append is Go's own builtin, so append(dst, src...) works and a widening spread is a compile error instead of a render-time panic. The engine's replacement native is gone rather than renamed: a slice reached through any is asserted at the boundary, append(gf["hosts"].([]any), h). Templates appending to an any-typed value must add that assertion, and append(nil, x) must start from a typed slice — the engine names each such call site at config load.
  • New untar_gz(archive) expands a gzip-compressed tar into a map of entry path to content, for templates that build configuration out of a fetched archive. Expansion is all-or-nothing and bounded by entry count and size, path traversal is rejected, and a corrupt archive returns an error instead of failing the render.
  • New recordEvent(resource, reason, message) records a Kubernetes Warning Event against any watched resource; the leader forwards them to the API server, so they surface under kubectl describe.
  • New currentFiles global exposes the accepted map files, general files and crt-lists to reconciliation renders and the published output snapshot to admission renders. Paired with the new randBytes(n), a template can self-rotate on-disk state with no external component. SSL certificates and CA files are excluded.
  • New renderMode global ("admission" or "reconcile") lets a check fail loud under the admission webhook while only warning during a live reconcile.
  • New admissionSubject global names the watched object and every watchedResources alias affected by its admission request, so a route-scoped check denies only the violating resource.
  • Admission responses surface template-recorded Warning events as AdmissionReview warnings (capped at 10 plus a suppressed count), so kubectl apply prints the consequences of a change.
  • A configuration is now one HAProxyTemplateConfig plus any number of HAProxyTemplateLibrary objects, referenced in order through spec.libraryRefs (ADR-0017). The config carries what an operator writes — podSelector, watchedResources, templatingSettings, validators — and libraries carry the bulk (templateSnippets, validationTests, maps, files, sslCertificates, k8sResources, haproxyConfig, templatingSettings), leaving the config at 1.3% of etcd's per-object limit. A library carries no podSelector, watchedResources or dataplane, so it cannot redefine the controller's operational identity. The config merges last, so its inline content is always the override point.
  • Each libraryRefs entry names a revision the referenced object must report; the controller compares the two strings and never recomputes either, so a half-applied set is visible as a mismatch and holds the last-good configuration. The config's ownerReference is stamped onto every library it references.
  • --crd-name / CRD_NAME name a single HAProxyTemplateConfig; which libraries it pulls in, and in what order, comes from its spec.libraryRefs. Startup waits for the config and every reference to resolve.
  • New haptic config view --input prints the merged input configuration — the config plus every library it references — as opposed to the rendered HAProxy output config view shows.
  • haptic validate -f is repeatable and accepts multi-document files, assembling every config and library it finds by spec.libraryRefs, so helm template … > all.yaml && haptic validate -f all.yaml validates exactly what the controller assembles. New --dump-merged prints the merged spec and exits.
  • New haptic preflight -f values.yaml runs a configuration through the controller's own checks before it is deployed: it renders the image-embedded chart with your values and runs the load gate (structural validation plus every validationTest, including haproxy -c), then vector validate on the sidecar config and varnishd -C on the Varnish configuration. Schemas come from the cluster you deploy to; --schema-dir runs it fully offline. --expect-chart-version / HAPTIC_EXPECT_CHART_VERSION hard-fails when the embedded chart is not the chart being installed.
  • New haptic apply-crds server-side applies the CRDs bundled in the image-embedded chart (or --chart / $HAPTIC_CHART_DIR), closing the gap Helm leaves for CRDs installed from a chart's crds/ directory. It never deletes and never touches CRD .status.
  • HAProxyTemplateConfig.status reports observedGeneration and a standard Validated condition, stamped on every object of the merged set with that object's own generation, plus an Observed printer column — so GitOps health checks work on whichever object was applied. A startup load-gate rejection is recorded as Validated=False with reason LoadGateFailed and the failing tests before the controller crash-loops.
  • Every validation test is checked for render determinism: the runner renders twice and compares the config and every auxiliary file, so a template whose output depends on map-iteration order fails its own suite. Six such sites in the bundled chart were fixed.
  • New validationTests[*].currentServers declares a previous deployment's servers as data (backend → server → {address, port}), which templates read as currentConfig.ServerIndex. It replaces currentConfig, whose raw HAProxy text needed a config parser to read the same server index out of.
  • New haptic validate --snapshot-dir <dir> writes every validation test's rendered haproxy.cfg, map files, general files and certificates to <dir>/<test>/, so the output of two checkouts can be compared with diff -r.
  • New spec.validators[i].dataFiles sends a validator the files a validated file references, alongside it in the same request and marked kind: "data". The result cache keys on their content too.
  • Every changed render is judged by haproxy -c and by the configured pluggable validators. Leader reconciliation, watched-resource admission, and HTTP-store promotion share this gate; identical output returns from content caches. See ADR-0020.
  • --dump-rendered includes k8sResources and status patches, not just haproxy.cfg, maps, files and certificates.
  • New haptic_runtime_map_divergence_total{map} counts runtime maps whose post-apply read-back disagreed with the desired content and forced a reload fallback. Steady growth means endpoint churn is quietly reloading HAProxy.
  • New haptic_runtime_backend_fallback_total{reason} counts runtime backend batches a pod reloaded instead of running. reason="name_collision" is a fresh backend whose name a not-yet-deleted one still holds; steady growth means deferred backend deletes are lagging behind routes reusing their names.
  • New fleet-convergence and staleness metrics: haptic_haproxy_fleet_size, haptic_haproxy_fleet_converged, haptic_last_full_sync_timestamp_seconds and haptic_deployment_consecutive_failures.
  • A browser playground for HAProxyTemplateConfig runs the controller's production render path client-side in WebAssembly — nothing is uploaded. It shows the haproxy.cfg, map files, decoded certificates, status patches, recorded events and rendered Kubernetes resources a config produces, with per-line provenance, a reload-vs-runtime impact verdict, and an annotation-migration report. Published per version at /playground/<version>/.
  • The playground runs a config's spec.validationTests in a tests tab. A browser has no haproxy binary, so haproxy_valid assertions fall back to the pure-Go syntax and schema check, labelled syntax + schema.
  • The user docs and landing page embed the playground inline: runnable, editable examples — including challenges with revealable solutions — each with an "open in the full playground" link. Deep-link an example with /playground/?preset=<id>.
  • Measured render cost against object count is documented in Performance: roughly 0.11–0.15 ms per Ingress, with the path maps growing faster than the object count and reaching about 40% of the render at 5,000 Ingresses. The same figure is the per-admission cost, since the webhook renders the whole configuration once per admitted object.
  • The docs site serves every page as raw Markdown at the page URL plus index.md, with llms.txt and llms-full.txt indexes. Every page carries an edit link and a report-a-problem link that prefills a GitLab issue.

Changed

  • The config-load validation gate runs its bundled validationTests' haproxy -c checks concurrently (across the worker pool, bounded by the pod's CPU allocation) instead of serializing them behind a single slot. On a multi-core pod this cuts controller startup and config-change reinitialization time noticeably when the merged configuration carries many haproxy_valid tests — measured ~5.6× faster (12.8s → 2.3s) on a 411-check suite on 16 cores.
  • Losing the leader Lease no longer reinitializes the controller replica. Election is re-entered in place: the leader-only components stop and the same instances restart on re-acquisition, while the replica's admission webhook and resource stores keep serving throughout. Previously a leadership transition tore the whole iteration down, which retired the webhook's validators for the length of a full resync and denied admission requests routed to that replica with "retry after controller initialization".
  • HAProxy's haproxy -c no longer runs on the reconcile wall clock. A render is dispatched to the fleet immediately and the same check runs concurrently, on a semaphore slot of its own so the admission webhook never queues behind it (controller.config.controller.renderGateInterval caps its duty cycle). Coverage is unchanged — the webhook and the configuration-load gate still check synchronously, and every reconcile render is still checked. A refusal is closed on both ends: every pod that took the plan without its own HAProxy loading it is asked to restore its last known good file set, and every later render is held until one passes. A pod whose own binary reloaded the plan is left alone. The verdict surfaces as the HAProxyCfg's ConfigValidated / ConfigPinned conditions and as a RenderRefusedByHAProxy Kubernetes Event on the config. A render that accepts HTTP-store content for the first time still takes the check synchronously, before that content becomes the accepted version. See ADR-0022.
  • BREAKING: the HAPTIC agent replaces the HAProxy Data Plane API. The controller compares the render with what each pod reports and sends that pod either runtime commands or a file set plus a reload; map entries, certificate and CA content, crt-list entries and server address, weight or state now reach the running worker without a reload, and on HAProxy 3.4 so does adding or removing a route. A pod's HAProxy binary is what judges a configuration, so any directive it accepts can be deployed. See ADR-0022; the chart migration is below.
  • BREAKING: readiness semantics on the HAProxy pod. HAProxy's /ready on the stats port stays the pod's readiness probe, and it answers 200 only once a rendered configuration is running. The agent adds a startupProbe on /readyz and a livenessProbe on /healthz; /readyz reports that the agent can accept applies and stays true after one it rejected, so a fleet-wide rejection can never drain the Service or fence off the repair. A pod whose first apply fails keeps answering 503 and never joins the Service.
  • BREAKING: currentConfig exposes only currentConfig.ServerIndex[<backend>][<server>].Address/.Port — the parsed sections (Backends, Frontends, Global, …) are gone. It is no longer a parse of the deployed configuration but the servers the render itself declared, so a template that read another field fails at the configuration-load gate; run haptic preflight before upgrading to see it beforehand.
  • BREAKING: the controller binary is now haptic (was haptic-controller): haptic run|validate|preflight|apply-crds|config|benchmark|version, installed at /usr/local/bin/haptic in the image and published as haptic-<version>-<os>-<arch> on the releases page. Kubernetes object names (haptic-controller Deployment, ServiceAccount, labels) are unchanged. CI jobs and scripts that invoke the binary directly must use the new name.
  • The controller-wide watcher debounce default is 100ms (was 2s), for every watched kind including an operator's own resources: bursts are already coalesced while a render is in flight and reload pacing lives in the agent, so the longer refractory window only added propagation latency. The config-reinitialisation debounce stays 2s; per-resource debounceInterval overrides are unchanged.
  • The controller no longer throttles its own apiserver requests by default: client-side rate limiting is disabled (--kube-client-qps / KUBE_CLIENT_QPS default -1), so it relies on the apiserver's API Priority & Fairness (APF) — the posture controller-runtime and kgateway ship. Previously each of the controller's clients carried a private 50-QPS bucket, so under a large rolling fleet or bulk apply the dynamic client's status writes queued behind client-side throttling into multi-second latency while its other buckets sat idle; and because a client-side-throttled request blocks rather than returning 429, client-go's automatic retry never engaged. APF returns 429 + Retry-After, which client-go retries. Set --kube-client-qps above 0 to reinstate a client-side cap, applied as one budget shared across all the controller's clients.
  • The bundled SPOA hub is v0.13.0. A hub config change now reloads only the plugins whose library or params changed and carries the rest over untouched, so a WAF- or schema-annotated Ingress edit no longer resets the rate-limit plugin's store connection and circuit breaker; and plugin log records reach the hub log ("target":"plugin", plugin=<name>, span.req_id) for plugins built against plugin-api 0.8 — the bundled rate-limit plugin (v0.4.1) is, and it now logs one warning per lost store connection. v0.4.1 also fixes the plugin staying in the local fallback for good after a shared-store outage: the periodic lease refresh could take the breaker's single half-open probe without reporting it, so no exact check or borrow ever reached the recovered store until a reload.
  • A watch event that only echoes the controller's own status write no longer triggers a reconciliation: the watcher refreshes its store from the event but recognises the write by the resourceVersion the API server returned for it. Each render's status patches came back as three to four watch events, and every one of them re-rendered the whole configuration and queued behind the pipeline; under sequential Gateway route creation that was ~200 ms of the per-route propagation time and most of the controller's render CPU. External status writes still trigger as before.
  • regex_search reuses a bounded set of compiled patterns per engine instead of recompiling repeated patterns on every call.
  • The template engine releases what a pooled VM held from its previous render instead of truncating around it, so a render's object graph is no longer pinned for the process lifetime. On a 1000-route churn this cuts peak RSS from 2603-3527 MiB to 1476 MiB and retained heap from 600-1096 MiB to 78 MiB, while rendering 7.5% faster — fewer live references is less for the collector to mark.
  • Watched-resource admission reuses the reconciliation pipeline's compiled templates and caches only its validation verdict instead of retaining a second parsed HAProxy configuration.
  • Endpoint caches share one parsed desired HAProxy configuration after post-reload comparison proves equivalence, instead of retaining an equivalent parsed read-back per pod.
  • The event commentator retains scalar correlation metadata instead of complete event payloads, so its 500-entry history no longer pins configuration objects, rendered resources, or status patches.
  • An unparseable or set-based watchedResources[].labelSelector now fails the configuration load, naming the resource and the offending selector, instead of being silently discarded — which widened the watch to every object of that kind cluster-wide with no diagnostic. Set-based syntax (in, notin, !, !=) is rejected rather than supported; == is now accepted as equality instead of being read as the value =nginx. This hard-fails at startup and in haptic validate; live reload keeps its existing fail-open behaviour and logs.
  • http.Fetch's refresh cadence option is now interval; delay keeps working as a deprecated alias, and setting both is an error. The option governs how often content is re-checked, never a wait before the first fetch.
  • A validation test's per-test extraContext deep-merges into the global templatingSettings.extraContext instead of replacing whole top-level subtrees. A map carrying __replace__: true opts back into wholesale replacement.
  • A validation test's shared _global block contributes an isolated extraContext baseline, folded under each test below any per-test override.
  • Changing a general file no longer reloads HAProxy when it carries spec.files.<name>.reloadOnPush: false; templates registering a file at render time pass the same flag as a fourth argument to fileRegistry.Register. The default stays true.
  • Removing a general file reloads HAProxy only when the rendered configuration or a crt-list still names it.

Removed

  • The Data Plane API client, its seven generated per-version clients, the configuration comparator and the sync orchestrator. Nothing has called them since the agent cutover; the controller image is ~86,000 lines smaller and no longer ships an OpenAPI client generator (oapi-codegen, oapi-codegen/runtime and kin-openapi leave go.mod).
  • Syntax and schema validation of the rendered configuration. haproxy -c is a strict superset of both for loadability, and the schema half only checked conformance to the Data Plane API's model, which nothing speaks any more. What is lost is that model conformance and nothing else: the webhook, the configuration-load gate and rendergate all still run haproxy -c, the pluggable output validators still run synchronously on the reconcile path, and the browser playground keeps the syntax + schema check as its haproxy_valid answer because a browser has no HAProxy binary. No production binary parses HAProxy configuration any more, and make lint fails if one starts.
  • The deprecated validationTests[*].currentConfig field. Declare a previous deployment with currentServers instead — it reaches templates as the same currentConfig.ServerIndex.
  • Data Plane API-era metrics that the agent cutover left unwired: haptic_dataplane_api_operations_total, haptic_runtime_fast_path_fires_total, haptic_runtime_fast_path_applies_total, haptic_runtime_fast_path_failures_total, haptic_runtime_fast_path_server_updates_total and haptic_deploy_runtime_divergence_total. The "Where the old metrics went" table on the Monitoring page names each replacement.
  • Data Plane API-endpoint capability keys from the template capabilities map: supports_waf, supports_waf_global, supports_waf_profiles, supports_udp_lb_acls, supports_udp_lb_server_switching, supports_keepalived, supports_udp_load_balancing, supports_bot_management, supports_git_integration, supports_dynamic_update, supports_aloha, supports_advanced_logging, supports_ping and is_enterprise. They were always false; a template that reads one is unchanged, since a missing key is falsy. The map gains supports_ssl_ca_files, supports_ssl_crl_files, supports_quic_initial_rules, supports_log_profiles, supports_traces and supports_acme_providers, which the fleet's HAProxy version does decide.
  • Per-pod server-side configuration validation. The Data Plane API's validate_cmd ran haproxy -c on each pod before activating a push; the agent has no equivalent, because the pod's own binary rejects a configuration it can't parse at reload and the agent then restores the last known good file set. The webhook and the configuration-load gate keep the full haproxy -c, and rendergate runs it on every render.
  • The standalone deployer.NewDeploymentScheduler constructor. Use NewDeployStack so structural and runtime writes share endpoint state.
  • The four-slot parsed-config cache in the dataplane parser, and the haptic_parser_cache_hits_total, haptic_parser_cache_misses_total, haptic_parser_cache_hits_by_source_total and haptic_parser_cache_misses_by_source_total metrics that reported it. The cache existed to let the desired config be parsed once and reused across HAProxy replicas, but the deploy path already carries that parse by reference (ValidationCompletedEvent.ParsedConfig), so its one beneficiary never consulted it — measured at 122 hits against 1509 misses on a live cluster, while pinning four fully-parsed configs (~900 MB of live heap at 1000 HTTPRoutes, since a parsed config is 35-90x its text size). Parsing is now always direct.

Fixed

  • The controller no longer rebuilds the watched-resource reflect types on every render. buildResourcesValue and BuildPerResourceStoreType reconstructed the resources struct type and each per-resource store type via reflect.StructOf per render, though those types are fixed after startup — the dominant allocation under render churn (measured 69% of allocation on a live controller; total render allocation dropped 62% once cached). This transient churn was the memory pressure behind controller OOMKills under a high-concurrency admission burst; caching the types restores headroom without changing rendered output.
  • On a reload-gated fleet (HAProxy < 3.4) under rapid add/remove churn, a config change could silently stop propagating: a route deletion was skipped as a no-op and the fleet stayed on the previous config. Content-addressed renders recur — an add and its delete hash to the same plan every cycle — and the skip-unchanged gate compared each render to the last fully deployed config, so a recurring render matching that value was dismissed even after a paced deploy had moved the fleet past it. The gate now skips only when the fleet is actually settled on that config, never while a later render is still converging.
  • A backend's per-server weights are now applied on the runtime add-server path. A server added over the runtime API carried no weight, so any backend that distributes traffic by per-server weight — weighted TCPRoute backendRefs, which become one balance roundrobin backend — collapsed to an equal split, and a weight-0 server, which must receive no traffic, was added with HAProxy's default weight 1 and took traffic. The add server command now carries the server's weight, including weight 0. (HTTPRoute and GRPCRoute weighting is unaffected: it distributes across separate backends through a weighted map, not per-server weights.)
  • The leader no longer crashes (nil dereference in the config publisher) when leadership is lost while a publish is pending: retry.OnError reports an interrupting context cancellation as the last retriable error, which is nil when none preceded it, so an interrupted HAProxyCfg or auxiliary-file write came back as a success without a result. It's now an error naming the cancellation.
  • Status writes to auxiliary-file CRs (HAProxyMapFile, HAProxyGeneralFile, HAProxyCRTListFile) are no longer re-stamped when unchanged, eliminating a client rate-limiter PATCH storm under endpoint churn that could starve ingress-status writes. Their content-hashed names make every re-stamp byte-identical; status.deployedToPods is unaffected — every value change is still written, the periodic drift-prevention re-sync still re-stamps each live pod authoritatively, and pod departure, an aux-file CR deletion, and a leadership change each invalidate the cache.
  • Controller memory could grow without bound under sustained route churn. The status applier's mailbox collapses only uninterrupted runs of the same event type, and under churn its three types arrive strictly alternating (render → resources.applied, deploy → deployment.completed), so nothing ever collapsed while the applier was slower than the arrival rate: a 2,048-deep backlog of full status-patch sets held 6 GB of live heap (12 GB RSS) on a 1,900-route churn. The applier now coalesces each declared type across the whole queue, so the backlog is bounded by the number of event types.
  • Status stamping at iteration start no longer targets the referenced HAProxyTemplateLibrary objects as if they were HAProxyTemplateConfigs, which logged a NotFound warning per library on every leader start.
  • Version-cache hits no longer force an unnecessary HAProxy reload, and concurrent runtime updates can no longer leave a reusable stale endpoint observation.
  • A template function that renders more than 256 distinct parallel go render targets now fails to compile instead of silently rendering the wrong target's body. The compiled render operand is a single byte, so target 256 onward wrapped modulo 256 and produced full-length output for the wrong macro with no error. (The bundled chart reuses one macro per runtime loop, so it never approached the limit.)
  • Local Docker builds and tagged-release controller binaries now report their source-input hash instead of unknown.
  • The current-config store no longer re-parses an unchanged HAProxy configuration when only auxiliary files changed. It keyed the decision on spec.checksum, which covers the config and every map file, certificate and CRT-list — so ordinary endpoint churn rewriting a map file re-parsed byte-identical config text. A parsed configuration costs 35–90× its text, which made this one of the larger allocations on the churn path that OOMKilled the controller at its default memory limit. The parse decision now uses a hash of the config text; the checksum remains a fast path for the case where nothing changed at all.
  • Losing leadership now disables the drift-prevention timer without firing and re-arming it on the former leader.
  • Writing a watched resource in any served apiVersion other than the one the controller resolved is no longer denied. The admission webhook's rules are rendered from a watched resource's full apiVersions candidate list, but the controller registered a validator — and keyed its resource lookup — only for the single version it resolved to, so with failurePolicy: Fail every other served version was rejected permanently, advising the operator to "retry after controller initialization" for an initialization that had already finished. On a default install this denied every gateway.networking.k8s.io/v1beta1 HTTPRoute write cluster-wide while the identical v1 object was admitted. Both sides now cover every version the cluster serves; a resource nobody watches is still refused, and objects in the added versions get the same full render + haproxy -c validation.
  • The controller no longer accumulates deployed renders without bound under endpoint churn. Every applied config was queued for publication holding its full rendered bytes, but the queue drained one entry per configPublishInterval (10s) while the runtime-raw lane skips minDeploymentInterval by design — so pod churn filled it far faster than it emptied, at ~687 KB per entry. Queued entries are now dropped once every pod has demonstrably moved past their checksum, which bounds the queue by the number of checksums live across the fleet instead of by the deploy rate. A checksum any pod still reports is never dropped, so status.deployedToPods[].checksum stays resolvable against spec.
  • On-demand exact lookups now return every resource sharing an index key even when only part of that bucket is warm in the cache.
  • Template resource lookups with no keys or more keys than indexBy now fail the render instead of silently omitting resources.
  • Watched-resource index components now preserve their boundaries, so values containing / no longer collide with another key or leak into partial Fetch results; empty and Unicode values use the same semantics in both store modes.
  • Back-to-back renders now read currentFiles from the exact last accepted output instead of an asynchronously updated debug cache, so self-referential files don't rematerialize from stale input.
  • Reconciliation, admission, and all-replica proposal validation now read currentFiles from one completely committed HAProxyCfg auxiliary reference set, including certificate Secret metadata. Partial publications, legacy in-place Secret updates, and a missing set ID after modern publication fail closed instead of freezing or reusing ambiguous input. A replica that saw a pre-set-ID publication change while it was a standby accepts the visible snapshot when it becomes leader instead of failing every render until a restart; during a rolling upgrade from a pre-set-ID controller every new leader used to go 503 and be liveness-killed once.
  • http.Fetch no longer serves a response fetched with old credentials after its authentication changes. Each render must declare one authentication and option set per URL; source changes fence in-flight refreshes and replace, re-arm, or stop that URL's timer. Rejected admission candidates no longer replace the live source or timer.
  • Cold and source-replacement HTTP responses remain render-local until the exact complete output passes every validator, then the whole candidate set is accepted atomically. Failed or canceled renders refetch instead of caching unvalidated bytes, and source replacement immediately revalidates pending content that survives a retired refresh-validation batch.
  • Pluggable-validator responses with a missing, unknown, or contradictory aggregate result now fail the render as protocol errors instead of potentially admitting invalid output; malformed responses are never cached.
  • Cancelling validation now terminates queued or running haproxy -c checks and prevents their result from entering the success cache.
  • Kubernetes read errors, typed watched-resource conversion failures, and ambiguous GetSingle lookups now fail rendering instead of producing a partial configuration from an empty result.
  • Interrupted validation can no longer produce a valid verdict or crash the controller during template cancellation, and a retired leader term no longer publishes reconciliation results after cancellation.
  • Watched-resource admission now overlays configured watchedResources aliases instead of the Kubernetes plural. Multiple aliases for one group, version, and resource tuple and label or field selector transitions render the same proposed state their watchers will store.
  • Deployment timeouts now cancel blocked Dataplane calls out of band and retain scheduler ownership until the exact deployment attempt terminates, so stale completions cannot release or overwrite a newer deployment.
  • Watched resources no longer remain accessible through an old indexBy key after the indexed value changes. Both store: full and store: on-demand move the resource atomically, and remove its old entry if the updated object no longer yields a complete key.
  • On-demand watched-resource reads no longer restore stale cache entries when an informer update or delete races a cache-hit renewal or live API fetch.
  • A superseded configuration validation can no longer restart the controller with an older candidate, discard credential or schema changes, or make the next iteration refetch an unvalidated newer CR. Reloads carry the exact accepted snapshot, failed replacements retry live state, and schema re-resolution passes the complete config gate before activation.
  • Failed controller-iteration retries no longer renew the 90-second reinitialization grace window indefinitely; unresolved failures now make /healthz return 503 after one fixed grace episode.
  • Config publication now retries incomplete output from immutable render snapshots and commits success only while its generation and leader term remain authoritative; superseded retries stop promptly, terminal failures release the queue, and leadership reacquisition starts clean workers.
  • Config publication now heals same-checksum drift, prunes obsolete auxiliary resources, gives long, colliding, or invalid artifacts stable Kubernetes-safe names, and rejects ambiguous file identities before deployment.
  • HAProxy pod discovery now proves both the Dataplane API version and HAProxy binary series, publishes rotated credentials in order, re-probes replacements, container restarts, image changes, or URL changes, preserves Enterprise edition detection, honors per-endpoint retry backoff, formats IPv6 endpoints correctly, retires stale deployments, clients, and caches, and binds deployment status to the pod runtime epoch.
  • An incomplete runtime-only deployment now invalidates its shared baseline and retries structurally, preventing a failed server update from being omitted by the next runtime diff.
  • Rendered-resource apply and orphan-prune failures no longer publish successful infrastructure status. An incomplete desired set also no longer drives pruning, so a transient API or discovery failure cannot delete a resource that is still rendered.
  • A full critical EventBus subscriber buffer now fails and rebuilds the controller iteration instead of logging the lost coordination event and continuing with partial state. Drop counters survive the rebuild; explicitly lossy observability subscribers remain non-fatal.
  • Controller reinitialization and leadership loss now join every owned worker, informer, timer, and admission call before closing dependencies or starting replacement work. Obsolete leader terms can no longer publish configuration after losing authority, and invalid leader-election timings fail the configuration gate before startup.
  • HTTP refresh validation now binds verdicts and timer callbacks to exact content revisions. Late results and retired refreshers can no longer promote or strand newer pending content.
  • Validation-suite deadlines now cancel the active main, auxiliary-file, Kubernetes-resource, and determinism renders instead of taking effect only between tests.
  • Admission no longer accepts changed invalid output merely because the live baseline is also invalid. The recovery exception now requires both renders to complete and produce identical content checksums.
  • Webhook validator registration is atomic and fails controller initialization when any configured kind can't be mapped or the dry-run validator is absent. A request routed to an unregistered kind is denied with HTTP 503 instead of admitted.
  • Pluggable-validator errors now block every render path before publication or deployment, including deletes, HTTP refreshes, startup, and ordinary reconciliation. Malformed file globs reject the merged config, and manager construction failures abort the controller iteration instead of silently disabling the feature.
  • Config and credential updates observed while an iteration starts are replayed after initialization. Exact bootstrap versions remain suppressed, while newer concurrent updates can no longer be lost.
  • /debug/vars/auxfiles, /debug/vars/state and /debug/vars/all no longer serve auxiliary-file contents. They serialized every rendered TLS private key and the tls-ticket-keys session-ticket file to anything that reached the controller's loopback interface — which any sidecar in the controller Pod, or anyone with pods/portforward, can do. Paths, filenames and counts are unchanged, so the endpoints keep their debugging value.
  • pkg/httpstore refuses a redirect that downgrades an https:// source to plaintext, and drops every credential header when a redirect changes host. Fetched content becomes HAProxy configuration and Coraza rules, so a plaintext hop was a config-injection path; and because net/http keys its Authorization strip on the host alone, a same-host downgrade also re-sent the credential in cleartext while API keys set through AuthTypeHeader survived any cross-origin redirect. The store's trust model — TLS posture, size bounds, and the absence of checksum pinning — is now written down in pkg/httpstore/README.md.
  • A configuration change mixing a runtime-eligible server field with a reload-only one is no longer misclassified as runtime-only. The lane classifier counted every modified server as runtime-eligible, so such a diff was written to disk with skip_reload, only partly applied to the live worker, and then recorded as activated — leaving the on-disk and in-memory configurations divergent until an unrelated reload. haptic-controller's reload-impact prediction had the same error.
  • A backend whose only change is an http-error rule is deployed instead of silently dropped. The update was gated on a field list that cannot name fields living outside BackendBase, so the change produced zero operations.
  • The go statement is disabled in templates. It was enabled with a comment claiming the chart's parallel rendering needed it; parallel rendering uses a different construct, and no template in the repo used the statement form — which was the only way to create a goroutine outside the render's control, whose panic took the process down. pkg/templating/README.md now records the sandbox posture.
  • The controller no longer leaks four informers per iteration. Watches on HAPTIC's own output CRDs were started on a channel that was closed only when cache sync failed, so every configuration change left another set running for the process's lifetime.
  • The admission webhook no longer closes idle API-server connections after 10s. net/http falls back to ReadTimeout when no IdleTimeout is set, so the webhook dropped pooled connections well before the API server's 90s client-go default did, and the API server's next request raced the close into an EOF — surfacing as intermittent failed calling webhook errors on kubectl apply. The server now sets a 120s idle timeout, above the client's, so the API server is always the side that closes.
  • Documentation described capabilities HAPTIC doesn't have. The most consequential: an admission webhook for HAProxyTemplateConfig (removed by ADR-0016 — a bad config is caught by the leader's first render, the startup load gate, and preflight, not at apply time); an HAProxyTemplateConfig per template library (they are HAProxyTemplateLibrary objects referenced from one config's spec.libraryRefs); the access log reaching the haproxy container's stdout (it goes to the Vector sidecar by default); and the HTTPS listener inheriting HAProxy's default ciphers (the SSL library ships a TLS policy with a TLSv1.2 floor). Also corrected non-existent chart values (controller.networkPolicy.ingress.dataplaneApi.*, rbac.create), wrong defaults (governance.enabled, leader-election timings, alert and exclusion counts), and template examples that could not compile.
  • The template reference documents all registered helper functions — 17 appeared nowhere in the docs, including dig_string, selectattr, make_guid, sort_ints, and the resource/jsonpathGet/jsonpathSet governance helpers — plus the currentFiles global, sort_by's comparator form, and the collection-pipeline stages. The sort_by pipe rule was documented backwards: a direct call needs two variables, while a pipe stage keeps only the first result.
  • spec.libraryRefs, the HAProxyTemplateLibrary kind, spec.validators[i].dataFiles, validationTests.<name>.currentFiles, the events assertion target, the reserved haproxy-pods and _global test entries, the haptic benchmark subcommand, and four registered metrics (haptic_runtime_map_divergence_total, haptic_deploy_runtime_divergence_total, and the two by-source parser-cache counters) are now documented.
  • An HTTP-sourced map or blocklist no longer freezes at its last accepted content when a validation verdict is lost. A pending validation that produces no verdict within five minutes is abandoned and the URL refreshes again; a panic in the proposal validator now publishes a failure instead of nothing.
  • A new HAProxy pod no longer waits for an unrelated change to receive its configuration when a pod-index update is dropped under load. Pod discovery now re-reads the pod store on every drift-prevention tick.
  • A deployment can no longer push one render's configuration alongside a different render's parsed form, which computed the runtime-server diff against a configuration that was not the one being sent. The scheduler now checks that a validation verdict describes the render it holds.
  • Events discarded because the startup buffer was full are now counted in haptic_events_dropped_critical_total and reach the critical-drops alert. They were previously logged only, so the one drop path that can lose the controller's bootstrap events reported zero drops.
  • Fixed three data races in the event bus, all unsynchronised reads of the subscriber list against leader-election and scatter-gather subscription churn: the subscriber-count gauge, the startup/leadership event replay, and the drop callback. The drop callback also now runs with no bus lock held, so a callback that publishes cannot deadlock its own publisher.
  • Deleting one EndpointSlice no longer evicts every EndpointSlice of the same Service. Resources sharing a non-unique index key — the chart indexes EndpointSlices by namespace plus kubernetes.io/service-name — shared one store bucket, and a delete dropped the whole bucket, emptying the backend until an unrelated event happened to repopulate it. Deletes are now scoped to the deleted resource's namespace and name, for both store: full and store: on-demand.
  • Watched resources are no longer modified while the informer still owns them. Field filtering and the float-to-int conversion moved into the informer's transform hook, which runs before the object is cached and before any handler sees it. On-demand kinds keep their body-stripping projection; full kinds get a normalise-only transform, because for them the stored body is what templates read.
  • The HAProxy config parser cache no longer thrashes under churn. Post-reload read-backs and current-config reads carry a _version header that changes on every push, so they could never hit the cache but still evicted the desired config — a live cluster measured a 2.4% hit rate. Single-use parses now bypass the cache.
  • A runtime-map add that finds its key already present converges it with set map instead of failing.
  • Fixed a controller panic when a watched config or credentials object was absent as the watcher finished its initial sync: the callback received a typed nil pointer in a non-nil interface, so its obj == nil guard did not fire. A component panic now also logs a stack trace.
  • Programmed (and the Ingress loadBalancer.ingress address) is set only when every HAProxy replica has taken the configuration, not when at least one has. A partial deploy surfaces the deployFailed variant instead of advertising an address the fleet does not uniformly serve.
  • Fixed HAProxyCfg.status.deployedToPods[].checksum advertising content spec never carried. Deployed checksums were dropped by a one-slot latest-wins queue (one in 31 on a real run) and could be overtaken by validation publishes (measured 5.2s). Distinct deployed checksums are now queued in arrival order and drained first.
  • Fixed the mid-flight runtime apply corrupting an HAProxy pod's on-disk configuration. It now patches the configuration the in-flight deploy actually wrote, so the only on-disk difference is the runtime-eligible server line, and records that push as proving nothing about the running state — the next sync reloads rather than trusting an empty diff.
  • Routes no longer silently 404 for 15–30 seconds after a deploy reports success under churn (#84). The runtime-bypass fast path now pushes the last reload-activated config patched with only the runtime-eligible server changes, a structural deploy verifies the on-disk config after its reload (counted as haptic_deploy_runtime_divergence_total), a headerless on-disk config always forces a reload, and a superseded render abandons its retry instead of re-pushing a stale body.
  • A failed or timed-out deployment invalidates the deploy scheduler's lane baseline, so a queued runtime-only render re-dispatches as a full structural sync instead of restamping the version header over structural content the workers never loaded (#76).
  • A retryable deploy failure self-reschedules with bounded exponential backoff instead of waiting up to a minute for the drift check, and a fully-failed deploy reports Programmed=False with a reason instead of freezing the last-known status (#72).
  • A runtime server update against a backend the loaded config does not have yet (No such backend/No such server) fails fast to the scheduled structural deploy instead of being retried for up to 2 seconds per apply.
  • The live config gate's validationTests budget scales with suite size (a 25s floor plus ~100ms per test) instead of a fixed 25s cap, so a large all-passing suite is no longer rejected as partially-validated; fail-closed on genuinely incomplete runs is unchanged (#77). HAProxyTemplateConfig admission uses the same budget, capped by the configurable config-admission deadline.
  • HAProxyTemplateConfig admission no longer turns an internal render timeout into a denial that can block the very config update needed for recovery: it has a separate configurable deadline and admits timed-out checks with a warning for the load gate to enforce. Watched-resource admission keeps its tighter fail-closed budget.
  • An admission request for a kind with no registered validator is logged and counted as unregistered on haptic_webhook_validations_total before being denied with HTTP 503. This means the ValidatingWebhookConfiguration and registered validators have diverged.
  • Every HAProxyTemplateConfig admission decision is logged at INFO, including a clean allow — previously indistinguishable from a webhook the API server never reached.
  • The controller honors the chart's WEBHOOK_PORT instead of always binding 9443, and an invalid metrics or webhook port fails startup instead of silently falling back.
  • haptic benchmark gains --schema-dir / HAPTIC_SCHEMA_DIR and resolves the effective spec like validate, so it can benchmark configs that use typed-resource access — as the bundled chart does.
  • Normal configuration reloads no longer emit a false Metrics component failed: context canceled error.
  • The docs site's changelog page is generated from CHANGELOG.md at build time, so /docs/dev/ always shows the current [Unreleased] changes.

Helm chart

Added

  • New controller.kubeClient.qps / controller.kubeClient.burst set the controller's client-side apiserver rate limit. The default (qps: -1) disables client-side throttling and relies on apiserver Priority & Fairness; a positive qps reinstates a shared client-side cap.
  • Gateway API per-route filters are driven by map files instead of per-route directives: header modifiers (rule- and backendRef-level), RequestRedirect, URLRewrite, spec.rules[].timeouts and RequestMirror each become a map entry keyed by the route's rule id, read by one static block. Adding, changing or removing any of them is a runtime map update rather than a change to haproxy.cfg, so it deploys without reloading HAProxy. Two exceptions keep a configuration line: the first route in the cluster to name a given header, and the two shapes one map value cannot describe — a rule matching several path prefixes with ReplacePrefixMatch, and a rule whose mirrors sample at different percentages. See the gateway library page.
  • Gateway route filter values now pass render-time validators: a RequestRedirect statusCode outside {301, 302, 303, 307, 308}, a scheme other than http/https, a hostname or mirror target that is not a DNS name, a port outside 1-65535, a header name outside the HTTP token charset, and a timeouts value that is not a duration are each reported against the route (denied at admission, warned and skipped on reconcile).
  • New base macros for libraries that want the same treatment: RegisterMap(name, lines, opts) writes a map file and declares its entry order, and HeaderModifierRules(direction, keyExpr, mapPath, setNames, addNames, delNames) emits one header line per header name rather than per resource. Both are documented under Map files.
  • The rendered global declares tune.bufsize, and on HAProxy 3.4 tune.cli.max-payload-size — the ceiling on what one runtime CLI batch can carry, which the controller sizes its batches from. Override with controller.config.templatingSettings.extraContext.tune.bufsize / .cliMaxPayloadSize.
  • The rendered global and the HAProxy bootstrap config carry a worker runtime socket, stats socket <baseDir>/haproxy-worker.sock mode 600 level admin. The master socket relays only the first command of a ;-joined line and holds no per-connection session state, so batched commands, wait and payload commands need a socket on the worker.
  • HTTPRoute and GRPCRoute backendRefs[].filters[] of type RequestHeaderModifier and ResponseHeaderModifier are applied per backend, keyed by rule id and backend name.
  • New Vector sidecar on every HAProxy pod, enabled by default (vector.enabled). It receives the HAProxy access log over a mounted UNIX datagram socket (vector.socketPath) and prints it to stdout, so records surface under kubectl logs <pod> -c vector, and re-exports the SPOA hub's metrics alongside its own on one port (vector.metricsPort, default 9598). HAProxy's own exporter is not re-exported: Prometheus scrapes it directly on the stats port, where it applies the chart's exclusion policy itself (see extraContext.prometheusExporter below). Set vector.enabled=false to remove the sidecar.
  • The Vector config follows the SPOA hub's delivery path: HAPTIC renders it and pushes it into general storage, where --watch-config picks it up without a restart. A bootstrap ConfigMap seeds the file before regular containers start, a post-start reload closes the initial read/watch race, and a PID 1 supervisor restarts an exited or unresponsive Vector child without withdrawing healthy HAProxy traffic.
  • New per-request metrics derived from the access log (vector.requestMetrics, on with the sidecar): one counter and six histograms keyed by route, with the upstream call split into connect, response headers and full response. Names and labels match ingress-nginx's, so its dashboards work — set prefix: nginx_ingress_controller for a literal drop-in. Opt out per label (terminationStateLabel, pathLabel, hostLabel) or per family; cardinalityLimit caps label values at 500 per metric.
  • New vector.sizeMetricsPort (9599) exports the two size histograms, since vector's exporter takes one bucket list per sink. haproxy.monitoring.podMonitor declares both endpoints.
  • Metrics can be derived from the access log declaratively in vector.logMetrics, for in-path components with no scrape endpoint of their own. Each entry names a log field and how to project it (kind: enum → a counter tagged with the value, kind: numeric → a counter incremented by it), with an optional requires path switching it off. Four ship for the cache tier: haptic_cache_status_total{status}, haptic_cache_age_seconds_total, haptic_cache_uncacheable_total{reason}, and haptic_degraded_cache_total.
  • New extraContext.prometheusExporter sets the query HAProxy's Prometheus exporter applies to a scrape that sends none, so every scraper gets the chart's exclusion policy — the bundled PodMonitor and a hand-written job alike — and a scraper's own query still wins. excludeMaintServers (default true) passes HAProxy's ?no-maint, dropping the empty reserved-slot servers (67% of the series on a measured fleet, no metric name disappears); excludeMetrics is a map of named exclusions, each with enabled, exact families (sent as metrics=-<name>) and an optional requires extraContext path. The shipped exclusions drop the never-resetting haproxy_*_max_* gauges and four more families — 31 families, roughly a third of a scrape; haproxy_backend_agg_server_status is deliberately kept as the free-slot census, and backendHttpCompression ships off because compression is on by default.
  • New haptic_denied_total{reason} counts every rejection by the control that made it. The *_unavailable reasons are the ones to alert on: they mean a control could not reach its dependency, not that a client hit a limit.
  • New haproxy.monitoring.podMonitor (default off): one PodMonitor for every metrics endpoint on the HAProxy pod — HAProxy's exporter on the stats port, vector's endpoints while the sidecar is on, and the SPOA hub's metrics port when the hub is on and vector is off (with vector on, vector re-exports the hub over loopback). A hub pinned to a loopback bind that would be scraped directly fails the render with guidance rather than emitting a dead target.
  • New HAProxyAccessLogRecordsDropped alert (on by default) fires when HAProxy discards access-log records because the Vector sidecar stopped draining the socket. The socket absorbs ~167 records at the default rmem, about 170ms of stall at 1000 req/s.
  • New HAProxyControllerConfigPinned and HAProxyAgentRecoveryReloadFailed alerts (both on by default). The first fires while HAProxy has refused two renders in a row, so nothing new reaches the pods; the second when a pod's recovery reload failed after a rollback, leaving its worker on a set nobody described. HAProxyAgentInvariantViolated no longer double-fires on the second one.
  • New controller.config.controller.renderGateInterval (default 1s) caps how much CPU the render gate's haproxy -c runs can take from the admission webhook.
  • New HAProxyFleetDiverged alert (toggleable) and a Grafana fleet-convergence panel.
  • Structured JSON access logs. Every frontend emits one JSON object per request (or per connection, on TCP frontends) using HAProxy's native %{+json}o, with the log target on format raw so records carry no syslog prefix. The core record covers request identity, the five HAProxy timers (request_time_ms, queue_time_ms, connect_time_ms, response_time_ms, total_time_ms-1 when a phase never happened), retries, the termination state, resource (the <namespace>/<name> that owns the matched route), and denied_by, which names the gate that blocked a request. HAProxy's own process and health-check lines on that stream are not JSON, so a collector must tolerate them.
  • Template libraries contribute access-log fields for the features you configure, through a new log-fields-* extension point: WAF verdict, rate-limit budget, external-auth status, request-schema outcome, cache status, authenticated consumer, TLS version/SNI/resumption, mTLS result, matched Gateway rule, captured request headers, and the real peer behind a forwarding header. A field is emitted only when its feature is in use.
  • New accessLog.fields adds custom JSON fields from a YAML hash of field name to HAProxy sample expression, and accessLog.maxLineBytes (default 16384) bounds the record. Names and expressions are validated in both Helm and the render.
  • New accessLog.targets routes the access log away from the container's stdout to an access-controlled destination — the record carries client_ip, which is personal data. Each entry takes an address (stdout, stderr, fd@<n>, <host>:<port>, a socket path, or ring@<name>), format, facility and level, or a ring block for a buffered TCP client that queues records while the collector is down. HAProxy's process messages keep a separate stdout target. Combinations HAProxy accepts and then silently gets wrong — a level above info, a Unix-socket ring server, an undeclared ring@<name>, duplicate targets, an empty map, an undersized ring buffer — are rejected at render time.
  • New opt-in accessLog.suppress.successful drops records for 2xx/3xx requests that no gate denied, keeping every denial, 4xx and 5xx.
  • Access-log records omit fields whose value is empty (vector.omitEmptyLogFields, on by default), measured 27% smaller on a real fleet. Set it to false if you feed a strongly typed index or have queries written as field == "".
  • The access log and trace spans name the backend pod (server_pod), its Kubernetes Service and namespace. Servers are named after their pods, so server_pod is the chosen server's name (%s) directly — no address→name map.
  • The access log and trace spans identify which HAPTIC pod served the request (instance_pod, instance_node) and which address it arrived on (destination_ip). The pod identity comes from the downward API, read once at startup.
  • New listener_port access-log field: the port the routing lookup was keyed on. Host and path map keys are scoped by it, so a route that matches nothing is now distinguishable from one that matched a different listener's keys. It is logged as a JSON string on every frontend; remove any accessLog.fields.listener_port you added, since colliding with a built-in name is rejected at render time.
  • New waf_matched_var access-log field names the request fields a WAF rule matched on (ARGS_GET:id,REQUEST_LINE) — never their values. With waf_rule_id it gives both halves of a false positive without Coraza's audit log, which writes client IPs and request bodies.
  • New cache_age and cache_uncacheable_reason access-log fields: how old the served object was, and why a response was not stored (content_type_excluded, too_large, set_cookie, status_not_cacheable, origin_refused_sharing).
  • New route and bytes_in access-log fields, backing the path label and request_size. Each renders only when something reads it.
  • Hub log lines correlate with the access log: every SPOE message carries req_id=unique-id and the bundled hub adopts it, so a hub warning and the access-log record for the request that caused it share one key.
  • New opt-in distributed tracing, owned by HAProxy itself rather than a sidecar (extraContext.tracing). HAProxy adopts a valid inbound W3C traceparent (honouring its sampling decision), mints one otherwise, takes an edge sampling decision (tracing.sampleRate), and sets traceparent before the backend sees the request, so your services join the trace; span_id, parent_span_id and trace_flags join the access log. Set tracing.otlp.endpoint to also export HAPTIC's own spans: the Vector sidecar turns access-log records into OTLP/HTTP spans, which covers every request including HAProxy-generated 502/503/504s, denials, redirects and errorfiles. Off by default.
  • Traces carry a span per request phase: a SERVER span for the request and a Server session CLIENT span for the upstream call. Spans are named {method} {host}{route} and Server session [backend], with http.route carrying the matched path template (* marks a prefix match) and falling back to the owning resource in brackets, or the bare method, when nothing matched. Phase timings ride along as attributes rather than child spans.
  • Trace spans carry the request detail the access log has: protocol version, response body size, server port, TLS version/SNI/resumption, consumer, frontend, idle time, and the HAPTIC decision fields (WAF, external auth, rate limit, schema validation, cache, mTLS, Gateway rule id), each named haptic. + its access-log field name. No client IP is exported — correlate with the access log through haptic.req_id.
  • Exported spans identify which HAPTIC deployment produced them, through the OpenTelemetry resource attributes service.namespace, service.version, k8s.namespace.name and k8s.deployment.name, plus k8s.cluster.name from the new extraContext.tracing.otlp.clusterName (no default; Kubernetes exposes no cluster name to a pod).
  • New extraContext.tls cipher and protocol policy sets a forward-secret default on every HTTPS bind: tls.ciphers, tls.ciphersuites (TLS 1.3) and tls.minVersion (TLS 1.2 floor). The list is cert-agnostic, so it serves RSA-only, ECDSA-only and dual deployments; each sub-key is overridable, and an empty string omits its directive.
  • New extraContext.tls.hsts.enabled (default off) sends Strict-Transport-Security on every TLS response, tunable with maxAge/includeSubdomains/preload and still overridable per host by the hsts annotation. The render warns when it is enabled without an HTTP→HTTPS redirect.
  • New extraContext.tls.sessionTickets.enabled (default off) turns on fleet-wide TLS session resumption: every HAProxy pod shares one session-ticket encryption key, covering TLS 1.2 (RFC 5077) and TLS 1.3 (RFC 8446 PSK). The key is generated in-cluster and self-rotates daily through a 3-key sliding window with one hitless reload.
  • New defaultSSLCertificate.ecdsaSecretName makes the default certificate dual: secretName points at the RSA Secret and ecdsaSecretName at an ECDSA one, so HAProxy serves ECDSA to modern clients and RSA to the rest on the no-SNI path. Per-host dual certificates already worked.
  • New extraContext.ingressDefaultSSLRedirect (default off) redirects every HTTPS-served Ingress host from HTTP to HTTPS with one global toggle; ingressDefaultSSLRedirectCode (default 308) sets the status code. Only hosts actually served over HTTPS are redirected.
  • New extraContext.proxyProtocol (enabled, httpPort 8081, httpsPort 8444) adds binds that require a PROXY protocol header, for HAPTIC behind a layer-4 load balancer that would otherwise hide every client behind its own address. They are additional listeners — haproxy.ports.http/https stay open, because HAProxy has no optional-header bind. Enabling adds the ports to the Service, the container, and the NetworkPolicy.
  • New extraContext.requestBuffering (enabled, default on; waitTimeout, default 10s) makes HAProxy wait for the request body before taking a backend connection — the standard slow-POST defence. Only requests declaring a Content-Length are held, so gRPC and chunked streaming uploads are excluded by construction. The new haproxy-haptic.org/request-buffering annotation (on/off) overrides it per route through a reload-free map.
  • Shared response cache: an opt-in, chart-deployed Varnish tier (cache.varnish.enabled) that HAProxy shards by URL with bounded-load consistent hashing, so the cache is shared across the whole fleet. Deployed through the controller's k8sResources with a PodDisruptionBudget, soft node spreading, an optional HorizontalPodAutoscaler and a release-scoped NetworkPolicy. Per-route control via cache-enable, cache-ttl, cache-negative-ttl (for 404/410), cache-key (consumer/header/cookie/query/src), cache-exclude-content-types, cache-exclude-paths and cache-max-object-size; responses carry X-Cache: HIT/MISS/STALE. The cache key includes the resolved backend, an origin's Cache-Control: no-store/private or Vary: * is honoured, a route whose key cannot be expressed downstream is marked Cache-Control: private, and credential checks run on the client leg so a cached response is never served to a caller who presented none.
  • Staleness and revalidation control for the shared cache: cache-stale-while-revalidate serves a stale response while it refreshes in the background, cache-stale-if-error reaches the stale copy only when a refresh fails, cache-ttl: auto follows the origin's Cache-Control/Expires, cache-revalidate keeps an expired object for a conditional refresh, and cache-strip-set-cookie drops a Set-Cookie that would make a public asset uncacheable.
  • Traffic shaping on the native library: upload-bandwidth-limit caps bytes per second received from the client alongside the existing download-bandwidth-limit, and bandwidth-limit-scope chooses who shares the budget — stream (default), client or service. The shared scopes add a backend stick-table, so combining them with the per-source rate-limit-* caps fails the render with an actionable message instead of a config HAProxy refuses to start.
  • Shared request-rate limiting: opt-in haproxy-haptic.org/rate-limit-requests annotations enforce one budget across the fleet through the bundled rate-limit SPOA plugin. The chart-managed default store is HA Valkey with Sentinel (three pods, failover, PodDisruptionBudget, NetworkPolicy); bring your own with rateLimit.shared.externalStore.urls. downAfterMilliseconds defaults to 5000 and the chart refuses a value at or below 3334, which cannot survive a Sentinel TILT window.
  • JSON request-body validation: opt-in haproxy-haptic.org/request-schema-* annotations validate POST/PUT/PATCH bodies against JSON Schemas from ConfigMaps or Secrets through the bundled api-gateway SPOA plugin. HAProxy rejects oversized bodies before the SPOE round-trip. extraContext.requestBodyInspection.haproxyBuffer configures the shared buffer capacity.
  • A HAProxy PodDisruptionBudget, enabled by default, preserves at least one load balancer during voluntary disruptions and validates against the static or KEDA minimum fleet size.
  • Cache and request-schema dependency failures now have cache_degraded and schema_degraded access-log fields and counters.
  • New haptic-annotations template library exposes HAPTIC's native haproxy-haptic.org/* vocabulary — a superset of the haproxytech, haproxy-ingress and nginx-ingress libraries with one annotation per capability. Enabled by default, and the recommended vocabulary for new configs; the vendor libraries remain for migrating existing annotations.
  • API-gateway annotations on the native library (pure HAProxy config, no plugin): API-key authentication (api-key-secret/api-key-header/api-key-query/api-key-consumer-header) backed by a reload-free key→consumer map; stateless request gating (allowed-methods, require-content-type, require-headers, mock-response, fixed-response); and request correlation IDs (request-id/request-id-header/request-id-accept-inbound).
  • JWT, HMAC and consumer-group annotations on the native library: asymmetric JWT verification (jwt-secret/jwt-algorithm/jwt-issuer/jwt-audience/jwt-required-claims/jwt-forward-claims) with an alg-confusion guard and exp/nbf/iss/aud checks; HMAC request-signature verification (hmac-secret/hmac-algorithm/hmac-header/hmac-signed-string); and consumer-group authorization (consumer-groups-secret/allowed-consumer-groups). All fail closed with 503 when their Secret is missing.
  • Response compression annotations on the native library (compress-enable/compress-algorithm/compress-types); brotli and zstd fail the render, being unavailable in the community HAProxy build.
  • Reusable Coraza WAF policies for native Ingress annotations: administrators define bounded policies in chart values or RBAC-protected cross-namespace ConfigMaps (waf.policies.configMapRefs), and Ingress authors select one with haproxy-haptic.org/waf-policy, optionally overriding enforcement with waf-mode. Mode, vendor rule snippets and raw HAProxy escape hatches are centrally authorized and default-deny.
  • Structured, self-service-safe CRS tuning on WAF policies: allowedMethods, paranoiaLevel, anomalyThreshold.inbound/.outbound, crsSettings (content-type allowlist, maxFileSize, maxNumArgs, totalArgLength) and ruleExclusions (by rule id or tag, optionally scoped to a literal path). They lower to chart-owned SecLang spliced at the crs-setup position, so no tenant-authored regex runs on the request path and enforcement itself cannot be disabled.
  • Namespaced self-service WAF policy authoring (waf.policies.selfService): with enabled: true, every namespace defines policies for its own Ingresses in a well-known waf-policies ConfigMap. Policies are namespace-scoped, trusted-catalog names win and collisions are rejected loudly, a broken policy fails only that namespace's selecting routes closed with 503 plus a Warning Event, secLang needs the separate allowSecLang grant, per-namespace and total budgets cut deterministically, and a self-service detect policy cannot weaken a default-on baseline.
  • New extraContext.waf.crs.url replaces the OWASP CRS compiled into the coraza plugin with one fetched over HTTP, so a CRS release can be adopted without rebuilding the plugin image. HAPTIC writes each rule file to general storage and substitutes only the @crs-setup.conf.example and @owasp_crs/*.conf includes, leaving rule ORDER alone. Refresh is a conditional GET on refreshInterval (default 1h); a fetch failure never fails the render and falls back to the deployed ruleset, then to the plugin's embedded CRS. Adopting a ruleset reloads neither HAProxy nor the hub.
  • Org-wide policy guardrails (extraContext.governance.*): cluster admins declare generic, JSONPath-driven rules that namespace teams cannot omit. rules is a map keyed by a name you choose, so rules a library ships and rules you add merge instead of replacing each other. Each rule targets a watched resource by name and either injects a default when a value is absent — flowing into the same render — or validates the present value with required, min/max (with onViolation: clamp or reject), allowed, pattern, anyOf or satisfiedBy: tls. enforcement: reject denies a violating resource at admission and records a GovernanceViolation Warning Event for existing ones; enforcement: audit warns only. exemptNamespaces skips infra namespaces. Disabled by default.
  • New pre-rollout validation gate (preRolloutValidation.enabled, default on): a pre-install/pre-upgrade Job renders the chart embedded in the controller image with the release's own values and runs the full load gate — including haproxy -c — before any object is applied, so a failing configuration fails the release with the previous one still serving. Argo CD runs it as PreSync. The Job hard-fails on chart/image version drift.
  • A pre-install/pre-upgrade Job applies the bundled CRDs on every install and upgrade (crds.upgradeJob.enabled, default true), running haptic apply-crds under its own scoped RBAC (customresourcedefinitions, never delete) and removed on success. This makes helm upgrade and GitOps sync pick up additive CRD schema changes, which Helm never applies for CRDs in crds/.
  • New haproxy.dataplane.logLevel (default info) replaces a hardcoded trace for the Dataplane API sidecar, which logged ~671 lines for a single startup plus config cycle.
  • New haproxy.org/cors-respond-to-options: when "true", HAProxy answers the CORS preflight with a 204 instead of forwarding it, matching the upstream HAProxy Kubernetes Ingress Controller.
  • New spoaHub.plugins.<name>.adaptiveConcurrency (default false per plugin) lets the hub resize a plugin's admission semaphore from live request latency, with maxConcurrency as the ceiling rather than a fixed limit.
  • The controller webhook timeout is configurable as controller.webhook.timeoutSeconds (default 10); the chart validates Kubernetes' limit and derives a one-second-shorter controller deadline.
  • Each vendor annotation library declares machine-readable migration coverage, used to generate the per-source annotation-support tables in the migration guide.
  • Ingress request/response header modifiers are driven by map files instead of per-backend or per-host directives: the request-set-header / response-set-header (haproxytech, haproxy-haptic.org), custom-request-headers / custom-response-headers (nginx) and headers (haproxy-ingress) annotation values become entries in ing-reqhdr.map / ing-reshdr.map keyed by the resolved backend, read by one static line per header name. Adding or changing a header on a route that already uses that name is a runtime map update, not a change to haproxy.cfg.
  • The settable per-backend timeouts move to a map: timeout-server and timeout-tunnel (all four annotation libraries; nginx's proxy-read-timeout/proxy-send-timeout collapse into the server timeout) become integer-millisecond entries in backend-timeouts.map, read by one uniform http-request set-timeout line every ingress backend carries. Editing a timeout is a map-only change. The non-settable timeouts (connect, queue, http-request, http-keep-alive, check) stay as backend directives. A value that is not an HAProxy duration is reported against the ingress at render time.
  • The new author contract for reload-free routing — the Backend() slots (profile/body/servers/serverLines), the when-does-a-backend-reload table, and where to put a directive — is documented for custom CRDs, Ingress and Gateway alike on the Reload-free routing page.

Changed

  • The controller's RBAC is tightened to least privilege. Grants the controller exercises only in the release namespace move from the cluster-wide ClusterRole to the namespaced Role: pods get/list/watch (the HAProxy-pod watch is pinned to the release namespace); the leader-election leases (the Lease and its fencing epoch live in the controller's own namespace); the haproxytemplateconfigs/haproxytemplatelibraries watch, owner-stamp and status writes; and the output CRDs (haproxycfgs, haproxygeneralfiles, haproxycrtlistfiles, haproxymapfiles) with their status — previously only two of the four were namespaced. The hardcoded cluster-wide namespaces grant is removed — the only namespaces watch is the gateway library's, already granted through the watched-resources rules when that library is enabled. customresourcedefinitions read, cross-namespace events writes, and the gateway library's cross-namespace services/gatewayclasses writes stay cluster-wide because their targets are cluster-scoped or in other namespaces. A golden helm unittest snapshot pins the rendered grants so future drift is a deliberate update.
  • The admission webhook now validates Gateway, BackendTLSPolicy, TLSRoute and TCPRoute (previously stored unchecked and only caught later at the config-load gate), so a malformed one is rejected at kubectl apply instead of poisoning the whole config. The rules cover the object spec on CREATE/UPDATE only, never the status subresource, so the controller's own status writes don't re-enter admission. GatewayClass stays unvalidated by design — the controller emits its own via Server-Side Apply, so an admission rule would intercept that write.
  • The chart-size guard band is raised to 960,000 bytes to accommodate the map-driven gateway filters and their required validation tests; a release-split is mandated above 975,000 (see scripts/check-chart-release-size.py).
  • The nginx (custom-request-headers/custom-response-headers), haproxy-ingress (headers) and haproxy-haptic.org (request-set-header/response-set-header) header annotations now apply to the ingress's own backends rather than to every request for the ingress's hosts. For a host owned by one ingress this is the same set of requests; for a host shared across ingresses it is more precise, and it fixes the case where a wildcard or host-less ingress silently applied no header. haproxytech's request-set-header/response-set-header were already backend-scoped.
  • A header set by two ingresses under differing capitalization is emitted once, spelled the lexicographically smallest of the two, so the config no longer depends on ingress order.
  • A Gateway RequestRedirect preserves the query string the request arrived with. HAProxy's own keep-query does not exist on 3.0, so the query is composed explicitly.
  • A backendRef-level ResponseHeaderModifier now overrides a rule-level one for the same header, as the specification requires. Both run in the frontend, backendRef-level second; the old placement put backendRef-level modifiers in the backend section, where HAProxy evaluates http-response rules before the frontend's, so the rule-level modifier won.
  • Two Gateway routes that spell one header name differently share a single configuration line, spelled the lexicographically smallest of the spellings in use. Header names are case-insensitive, so this is the same header.
  • Every map the bundled libraries write is registered even when it has no entries, and declares whether its entry order matters. A map created on the first entry and deleted with the last costs two reloads; an unordered map can instead take a runtime append.
  • map-path-regex-500-gateway emits its entries in Gateway API precedence order. map_reg returns the first match, and the previous emitter wrote routes in list order.
  • The Gateway advanced-matcher pass skips routes the two route maps already resolve, instead of processing every route and emitting nothing for the map-decidable majority. This cuts the frontend render for the common plain-route case at scale; the emitted config is unchanged.
  • BREAKING: the HAProxy pod's dataplane container is now agent and runs the HAPTIC image, so HAProxy pods pull from the controller's registry; haproxy.podSpec.imagePullSecrets defaults to the controller's. The <release>-haproxy-dataplane Service, its dataplane port name and port 5555 are unchanged — a Deployment's selector can't be changed in place — and the credentials Secret keeps its dataplane_username / dataplane_password keys. The pod's global section gains a worker stats socket, which is what carries every runtime command.
  • The HAProxy pod exposes an agent-metrics port, scraped by the bundled PodMonitor. haptic_agent_* counters report what each pod did with an apply.
  • Four controller.config.dataplane fields changed meaning without changing name: minDeploymentInterval is the shortest interval between two reloads of one pod (the chart passes it to the agent), driftPreventionInterval is how often a pod re-hashes its tree, reloadVerificationTimeout is how long the agent waits for a reload, and syncTimeout is how long the controller waits for a pod to answer.
  • Every backend section the bundled libraries emit is declared through the base Backend() macro (util-backend), which builds the section text from a record it hands the controller. A library writing a backend section by hand keeps working; only what goes through Backend() is described to the controller as data. BackendServers() returns server records instead of server lines, so a library that called it must pass the result to Backend() as servers rather than showing it.
  • Ingress and Gateway API backends are now dynamic-eligible: their defaults-legal directives (default-server, server/tunnel timeouts, retries, session cookie, the BackendTLS fail-closed 503, balance) move into the shared profile, and namespace/service move into a new backend-service.map (read by the access log's namespace/service fields via var(txn.backend_name)), so a plain backend's section is only from/guid/server lines — on HAProxy 3.4 such a route is added or removed at runtime without a reload. Stick-table rate limiting, bandwidth/compression filters and raw operator injections keep their backends structural. The per-installation dynamic-cookie-key (extraContext.dynamicCookieKey, default haptic-dynamic-cookie) replaces the per-backend sha256(<backend>) so cookie backends of the same shape share one profile.
  • Every backend inherits from a content-addressed named defaults haptic-be-<hash> profile (backend <name> from haptic-be-<hash>). Backends of the same shape — mode, balance, hash-type, default-server keywords and shared directive lines — share one profile section, so on HAProxy 3.4 a route of an existing shape can be added at runtime without a reload. The anonymous defaults becomes the rule-free parent defaults haptic-base; a trailing, never-referenced defaults haptic-implicit from haptic-base is what proxies without an explicit from inherit. tune.defaults.purge is never emitted (it would make named defaults un-inheritable by add backend).
  • The SSL passthrough loopback's server line no longer runs into the next snippet's comment (send-proxy-v2# gateway/...).
  • Behaviour change for hash-based load balancing: balance/hashType are structured Backend() arguments (carried by the profile, not a raw balance line), and the chart injects hash-type consistent by default for every hash-family algorithm (source, uri, url_param, hdr(), rdp-cookie, hash <expr>) — including the haproxy.org/load-balance, haproxy-haptic.org/load-balance, haproxy-ingress.github.io/balance-algorithm, nginx.ingress.kubernetes.io/load-balance: ip_hash and upstream-hash-by annotations. Consistent hashing lets a pod be added/removed at runtime without a reload (plain add server is refused on a map-based hash), and only remaps its own share of keys on a pod change instead of rehashing every key. Annotation users who relied on map-based (modulo-N) distribution get consistent hashing instead; static-rr (and an explicit hashType: map-based for a chart author) opt out and reload on pod churn.
  • Backends hold one server per endpoint, named after the pod (server <pod> <ip>:<port>), instead of a fixed pool of SRV_N slots with unroutable placeholders. The rendered file always equals the current pod set (ADR-0011), so show servers state, logs and per-server metrics read the pod name, and a rolling update is an add/remove of named servers over the runtime API. Not-ready and terminating endpoints stay in the config as disabled servers (they take no traffic), so a readiness flip is a runtime set server state rather than a del+add.
  • Gateway API backends are rendered only for resolvable backendRefs; a backendRef whose Service doesn't exist (or isn't permitted by a ReferenceGrant) no longer produces a placeholder-only backend section, halving the config for routes with an invalid sibling ref.
  • BREAKING: the chart renders one HAProxyTemplateLibrary per enabled template library, named <controller.configName>-<library>, plus <controller.configName> for your own controller.config, which references them through spec.libraryRefs. Anything that post-processes helm template output expecting exactly one object, or reads the whole config out of one, must be updated — haptic config view --input prints the merged configuration and validate -f accepts a multi-document stream. Only the config object is yours to edit; override a snippet by name under controller.config.templateSnippets. The split exists because the single merged object had reached 99.4% of Kubernetes' ~1.5 MiB per-object limit with nginx-ingress enabled.
  • BREAKING: controller workload values moved under controller.* (image, replicaCount, probes, resources, serviceAccount, rbac, service, securityContext, extraEnv/volumes/sidecars, autoscaling, podDisruptionBudget, monitoring, networkPolicy, webhook), and flat extraContext keys are restructured into diagnostics/statusPatches/annotationCompatibility/tls trees. The routing diagnostic response headers (formerly debug, on by default) are now opt-in via diagnostics.routingHeaders.enabled. A value left at its old path fails the render with a message naming the new one.
  • BREAKING: one owner per runtime setting. Rename controller.crdNamecontroller.configName, controller.debugPortcontroller.ports.healthz, controller.config.dataplane.porthaproxy.ports.dataplane, controller.config.routing.regexMatchOrdercontroller.config.templatingSettings.extraContext.routing.regexMatchOrder, and controller.defaultSSLCertificate → top-level defaultSSLCertificate. Remove the no-op controller.config.controller.healthzPort and metricsPort. Legacy paths fail with an explicit migration error.
  • BREAKING: haproxy.enterprise.version was removed. haproxyVersion now selects the controller compatibility series, Enterprise image revision and derived binary path together; an empty haproxy.image.repository derives the registry from haproxy.enterprise.enabled.
  • BREAKING: the access log is JSON on every frontend. option httplog/option tcplog output is gone and the log target uses format raw, so any pipeline parsing the previous text shape must be updated; records carry their own ts field (microsecond precision) in place of the syslog timestamp. Override util-log-format-http/util-log-format-tcp via controller.config.templateSnippets to keep a text format.
  • BREAKING: request correlation IDs are opaque RFC 9562 UUIDv7 values, emitted for every request rather than only when an Ingress opts into request-id. The previous format embedded the client IP — personal data — in a value forwarded upstream and copied into application logs. request-id-accept-inbound now preserves a client-supplied id only when it matches ^[A-Za-z0-9._:-]{1,128}$. unique-id-format moved to base.yaml; use a defaults-settings-* snippet above band 150 for a custom format.
  • BREAKING: the vendor annotation libraries (haproxytech, haproxyIngress, nginxIngress) are disabled by default; the native haproxy-haptic.org/* library is the only annotation library on by default. Enable the matching library explicitly if you use those prefixes. The native vocabulary is a superset of all three, and prefixes from different families can coexist on one Ingress as long as each feature comes from a single family.
  • Ingress is served over HTTPS by default: any Ingress binds haproxy.ports.https and terminates TLS with the default certificate — no spec.tls required. Set extraContext.ingressDefaultHTTPS: false to keep Ingress plaintext-only.
  • HAProxy compresses responses by default. A bundled governance rule injects haproxy-haptic.org/compress-enable: "true" on any Ingress that does not set it, so the default is visible in the rendered config; an Ingress setting it either way keeps its own value. Only responses the backend left uncompressed are touched, and HAProxy adds Vary: Accept-Encoding itself. Note the BREACH tradeoff: a route whose HTTPS response carries a secret and reflects attacker-controlled input should opt out with compress-enable: "false", or fleet-wide via governance.rules.haptic-compress-enable.enabled=false.
  • defaults sets retry-on conn-failure empty-response response-timeout, closing the half of the pod-termination race option redispatch could not — clients saw SH-- 502s during every rolling update. Retries that replay the request are limited to idempotent methods (RFC 9110), matching nginx-ingress; this narrows haproxy-haptic.org/retry-on, which previously replayed non-idempotent methods too. Set extraContext.retryNonIdempotent: true to restore it, or extraContext.retryOn to change the condition list.
  • HAProxy nbthread is derived from the CPU limit, not the CPU request. With no limit the directive is omitted, so HAProxy auto-detects all node cores; with haproxy.resources.limits.cpu set it renders nbthread = ceil(limit). Previously the default 250m-request pod was pinned to a single thread. Set haproxy.nbthread to pin a value.
  • The shared Varnish cache uses the canonical HAProxy+Varnish "sandwich" topology: Varnish fetches misses from a dedicated internal backend-fetch frontend (cache.varnish.loopbackPort, default 8090) instead of looping back through the client-facing one, so the WAF, rate limiter, external auth and routing run exactly once per request. The loopback is routed to the client leg's chosen backend through an internal header, so rewritten and weighted routes stay correct. The port is internal-only and restricted to Varnish by the NetworkPolicy.
  • The coraza and api-gateway plugins default to adaptive concurrency with a memory-derived ceiling, so WAF concurrency needs no manual tuning: the hub finds the right concurrency from live request latency and maxConcurrency becomes a ceiling derived from spoaHub.resources (~16 at the 256Mi default). Set adaptiveConcurrency: false or a literal maxConcurrency to opt out.
  • The SPOA hub sidecars set GOGC from the new spoaHub.hub.goGCPercent (default 300) and derive GOMEMLIMIT as a soft cap at 90% of the container memory limit, trimming GC frequency on the request path.
  • The bundled SPOA hub is v0.12.0. It processes pipelined SPOE NOTIFY frames concurrently and bounds in-flight frames with a process-wide backpressure gate, so HAProxy is backpressured instead of the hub shedding 503s under saturation (0% 503 across a c=16→96 WAF saturation sweep); it accepts the files a validated config references, so a ruleset included by path is checked at admission; it adopts the access log's req_id; and it adds SPOE data-path latency metrics (spoa_backpressure_wait_seconds, spoa_ack_flush_duration_seconds, spoa_ack_batch_size, spoa_inflight_frames, spoa_plugin_adaptive_limit).
  • The bundled coraza plugin is v0.10.0. It recompiles its WAF in place when the rule files its Include directives name change (which is what makes a fetched CRS take effect), and adds per-matched-rule metrics (plugin_coraza_rule_hits_total{phase, rule_id, severity, app}) that count every rule that fired, including on allowed traffic — the signal a detect-mode policy needs before flipping to deny.
  • The bundled rate-limit SPOA plugin is v0.3.1. Shared limits fall back from Valkey to bounded per-sidecar enforcement for lease and exact GCRA, then fail open if the plugin can't answer; rateLimit.shared.failClosed=true keeps strict denial, and degraded decisions are observable through rate_limit_degraded and plugin metrics. A shared-store connect that never resolves no longer pins a sidecar in that fallback for good after a Sentinel failover.
  • Request mirroring no longer allocates a per-target SPOE message slot. Each mirror source appends its target to a per-request list the single static mirror message ships to the plugin, so adding or removing a target changes only the HAProxy frontend — never spoe.conf or the hub TOML — and there is no cap on mirror targets. Removed spoaHub.haproxy.mirror.minMessageSlots.
  • Cache-enabled GET/HEAD requests bypass Varnish when no semantically healthy shard is available; other methods always go directly to the application. Local and origin-loop probes distinguish a live cache process from one unable to reach HAProxy, while a two-stage dispatch retries cache failures before response delivery directly. cache.haproxy.responseTimeoutMs sets the Varnish-hop inactivity timeout.
  • Each SPOE message has its own HAProxy engine and processing deadline, and the shared backend performs SPOP health checks, so a slow authentication dependency no longer gives rate limiting, WAF, schema validation, or mirroring its multi-second failure budget.
  • Dataplane API, SPOA hub, and Vector children run under exit and health-check supervisors without sidecar readiness probes. HAProxy's activated-config endpoint is the chart's only configured readiness probe; whole-container failures still affect Kubernetes pod readiness.
  • Managed Valkey uses noeviction, native shared rate-limit annotation rules whose refill horizon exceeds the plugin's one-hour state lifetime fail validation, and external-store sharding is rejected because the bundled plugin shares one circuit breaker across its shards.
  • Mirror requests use their own bounded target budget (2000ms, zero retries by default) instead of inheriting application timeouts and retry counts.
  • Request-schema validation allows a request the plugin could not judge, matching the WAF and rate limiter. Set apiGateway.requestSchemaValidation.defaultFailOpen: false for the old posture.
  • A missing auth or CA Secret no longer denies an Ingress at admission. The webhook renders against the watched-resource store, which lags the API server, so "Secret missing" and "Secret not observed yet" are indistinguishable and applying a Secret with its Ingress could be denied intermittently. The route still fails closed with 503 and recovers once the store catches up. Rejection is kept for cases that cannot race: a malformed value, auth-type: basic with no auth-secret, and an empty group list.
  • A misconfigured routing or presentation annotation on an already-present Ingress warns instead of bricking the fleet: a bad redirect code, malformed rewrite, or invalid CORS/canary/mirror/compression value records a Warning Event and skips just that feature during a live reconcile, while admission still rejects the change. Security features (authentication, mTLS, WAF, rate limiting, body validation, X-Forwarded-For) still hard-fail in every mode.
  • Cross-family annotation collisions are caught: an Ingress configuring the same feature through two annotation families is rejected at admission with a message naming the feature, the families and the annotations, and an existing one records an AnnotationFamilyConflict Warning Event. The check fires even when the values agree, since the result would otherwise depend on library render order. Different features across families stay allowed.
  • The governance rule engine moved to its own governance template library (on by default) from ingress-annotations-compat, where disabling the Ingress annotation scaffold silently took governance with it. Rules and exemptNamespaces are unchanged.
  • cache-enable can be combined with hmac-secret and consumer-groups-secret: both are now enforced on the client leg, before the cache is consulted, as api-key-secret already was. consumer-groups-secret additionally requires api-key-secret on a cached route, since only api-key resolves the consumer identity on the client leg.
  • The chart's cluster-scoped objects — the controller ClusterRole/ClusterRoleBinding, the ValidatingWebhookConfiguration and the CRD hook's RBAC — are named with the release namespace appended, so one release name can be installed into two namespaces. The IngressClass and the CRDs keep their stable names.
  • The HAProxyTemplateConfig admission webhook entry is removed from the chart, and apply-crds deletes it from live webhook configurations before manifests apply. controller.webhook.haproxyTemplateConfig.* values are gone.
  • The template libraries moved from charts/haptic/libraries/ into conditional subcharts under charts/haptic/charts/. Helm does not store subchart source in the release Secret, cutting the estimated payload by 287 KB. Rendered output is byte-identical; no values change.
  • Migration coverage is build-time tooling data instead of a CRD field. The playground loads per-source assets generated from each annotation library, and rendered config and library objects no longer carry the 88,909-byte metadata copy.
  • The Vector sidecar no longer re-exports HAProxy's Prometheus exporter; Prometheus scrapes it directly on the stats port, and HAProxy applies the exclusion policy itself. Measured standalone at 2,500 backends, the re-export was 1.3 GB steady / 2.3 GB peak of vector's memory — 90% of it — and no allocator or scrape-interval setting moved that. vector.excludeMetrics and vector.excludeMaintServerMetrics moved to extraContext.prometheusExporter (pattern is gone: the exporter filters by exact family name), and vector.podMonitor and spoaHub.monitoring.podMonitor merged into haproxy.monitoring.podMonitor; the old keys fail the render with the new location.
  • vector.resources defaults to a 256Mi request and 1Gi limit. Its memory tracks the request-metrics series (traffic shape, bounded by cardinalityLimit), measured at 146 MB idle and 364 MB steady / 698 MB peak with 5,000 distinct routes at 500 records/s. The supervisor restarts an exited or unresponsive child, while a whole-container OOM can still briefly affect pod readiness. The container also sets MALLOC_ARENA_MAX=2 to bound glibc's per-thread arenas.
  • The Vector and SPOA-hub configs no longer reload HAProxy when their content changes. Both are watched by their own sidecar and never opened by HAProxy, so a WAF policy edit or an access-log field change reaches the sidecar without touching HAProxy. spoe.conf still reloads, since HAProxy's filter spoe reads it.
  • The internal h2c demultiplexer frontend sets option dontlog-normal; every connection it accepts is logged by the inner HTTP frontend that handles the request. Abnormal terminations still log.
  • Shared source-IP rate limits run before Coraza and request-schema validation, so rejected floods do not consume API-gateway processing; consumer-keyed quotas run after native API-key/JWT authentication instead of falling back to source IP before the identity existed.
  • spoaHub.plugins.<name>.enabled template strings must resolve to exactly true or false; any other value fails the render instead of being silently coerced to disabled.
  • The chart fails the render when controller.config.dataplane.mapsDir, sslCertsDir or generalStorageDir point somewhere the bundled HAProxy pod cannot honour — a value outside the mounted volumes silently landed rendered files where the sidecars could not read them. Unaffected with haproxy.enabled=false.
  • The chart fails the render when controller.config.dataplane.minDeploymentInterval or reloadVerificationTimeout is not a Go duration of at most 60s. The chart passes both to the agent, which refuses a larger value and exits at startup, so every HAProxy pod would crash-loop. Unaffected with haproxy.enabled=false, where neither key becomes an agent flag.
  • An unknown key under haproxy.agent fails the render naming it, alongside the existing guard on the haproxy.dataplane.* keys it replaced.
  • The app-root.map, mtls-error.map and hsts.map builders collapse into one shared macro. Rendered output is unchanged.

Removed

  • The SRV_N server-slot pool and its 192.0.2.1:1 disabled placeholders, the pod-names.map file, and the controller.config.templatingSettings.extraContext.serverSlots.{increment,minFree} knobs are gone — pod-named servers make them unnecessary. extraContext.serverSlots fails the render (operator input) with a message; the haproxy.org/scale-server-slots and haproxy-haptic.org/scale-server-slots Ingress annotations are tolerated but no longer do anything (warned via a Warning Event, since a tenant value must not abort a render).
  • BREAKING: haproxy.dataplane.* moved to haproxy.agent.*: logLevel, resources, extraEnv and service keep their meaning under the new prefix. haproxy.dataplane.validateConfig, haproxy.dataplane.debugSocketPath, haproxy.dataplane.aclFormat and haproxy.dataplaneBin configured the Data Plane API and are gone with it. The chart fails the render on every removed key, naming its replacement, rather than ignoring it.
  • BREAKING: the otel SPOA hub plugin is gone, along with spoaHub.plugins.otel. Tracing no longer needs it: HAProxy mints and propagates W3C trace context itself and the Vector sidecar derives OTLP spans from the access log, without a SPOE round-trip per request. Remove spoaHub.plugins.otel from your values — the chart fails the render with a message naming the replacement, since a leftover block would be written into a hub config for a plugin the image no longer contains.

Fixed

  • An install without the SPOA hub failed to load its configuration, and shared-rate-limit and request-schema annotations were silently ignored. The haptic-annotations library imported a spoa-hub-only macro at compile time, so any install without the hub (no Gateway API CRDs, no SPOA plugin) could not compile its config at all. With that fixed, the opt-in guards for haproxy-haptic.org/rate-limit-requests and haproxy-haptic.org/request-schema-* — which previously lived only in map producers the hub renders — now fail loud on a hub-less install instead of dropping the annotation (fail-open on a rate limit and on request validation).
  • The vendor annotation libraries left several config-injection sites unguarded after the map/named-defaults refactors. A capture-group rewrite-target (haproxy-ingress, nginx-ingress) wrote its raw value into a replace-path line — a newline injected a directive — while its literal twin was already guarded; nginx-ingress also left proxy-connect-timeout, permanent-redirect-code/temporal-redirect-code and upstream-hash-by unvalidated (directive / balance injection), and its session-cookie-hash and auth-tls-secret-not-found render comments interpolated the tenant value. haproxy-ingress did the same in its secure-verify-ca-secret/secure-crt-secret/auth-tls-secret comments. Finally, a basic-auth password hash from a referenced Secret (haproxytech, haproxy-ingress) was checked only against the operator-tunable hash regex, whose default ^.*$ accepts a space — enough to append groups or other keywords to the userlist line. Every such value now routes through the shared guards: routing/presentation values are denied at admission and warned-and-skipped under reconcile, the Secret-name comments are name-free, and the hash and upstream-hash-by fetch fail closed.
  • Map values carrying a space or a ; were silently corrupted on the runtime path. The HAProxy runtime CLI splits a map value at the first space and treats a ; as a command separator, so every hsts.map value with includeSubDomains was truncated when it reached a running worker, and a redirect location, app-root path or mTLS error page with a space fared the same. Those values are URL-encoded now and decoded in the configuration with url_dec(1).
  • The same corruption affected the ingress-written maps. reqhdr-host.map (haproxytech set-host, nginx upstream-vhost), reqhdr-connection.map, reqhdr-xfwd-prefix.map and path-rewrite.map (nginx and haproxy-ingress rewrite-target) stored their values unencoded, so an upstream Host, a Connection value, a forwarded prefix or a rewrite path with a space or a ; broke when applied over the runtime CLI. Every writer URL-encodes and every reader decodes with url_dec(1).
  • The Ingress render was superlinear above ~1500 routes. The three path maps each recomputed the whole route-ownership map — once per shard, per map — over every Ingress in the cluster. That work is computed once per render now (shared.ComputeIfAbsent), which the path maps read from.
  • A Gateway route matching on a RegularExpression path got no filters, no timeout and no resource fields in the access log. path-regex.map pointed straight at a backend, so txn.gw_rule_id was never set for those routes. It emits the same GW_ROUTE_ID qualifier as the exact and prefix maps now.
  • A Gateway URLRewrite or RequestRedirect with ReplacePrefixMatch against the / prefix no longer concatenates the replacement onto the whole path (/x became /barx instead of /bar/x). The remainder is taken from the matched prefix's byte length rather than a regex.
  • A Gateway header modifier value containing % is inserted literally instead of being re-read as a log-format string, which could fetch request-scoped state into a tenant's own header.
  • [security] A Gateway API CORS filter could smuggle HAProxy directives or leak request state into a response header. An HTTPRoute/GRPCRoute type: CORS filter's allowMethods, allowHeaders, exposeHeaders and allowOrigins reached the emitted access-control-* headers and the origin -m reg ACL unguarded, so a % was re-read as a log-format fetch (e.g. exposeHeaders: ["%ci"] leaked the client source IP into a tenant-controlled header) and a " or control character could break out of the directive. List values are now HAProxy-log-format escaped and control characters rejected; an origin with a " or control character is rejected — the offending route's CORS block warns-and-skips on reconcile and is denied at admission, while legitimate methods, headers and origins render unchanged.
  • haproxy.dataplane.validateConfig: true works: the Dataplane API hands its validate_cmd the transaction file in DATAPLANEAPI_TRANSACTION_FILE and substitutes nothing, so the previous haproxy -c -f %s failed every push with Cannot open configuration file/directory %s. The chart now installs a wrapper script that reads the variable.
  • [security] A cross-namespace backendRef without a covering ReferenceGrant could still receive traffic. In a weighted HTTPRoute/GRPCRoute rule the weighted map listed every ref, so an unpermitted ref that sorted first took the rule's whole traffic share; TCPRoute and TLSRoute passthrough backends never checked the grant at all. Every backend selector now shares one validity verdict (util-backend-ref-valid): unpermitted and unresolvable refs get no backend section, their weighted share answers 500 (gw-invalid-backend), and TCPRoute/TLSRoute drop the ref.
  • The weight share of an invalid backendRef in a multi-ref rule now returns 500, as the Gateway API spec requires, instead of leaking to the valid siblings or hitting a placeholder backend (503) depending on list order.
  • The OTLP span transform in the Vector library no longer triggers a VRL E620 compilation warning at every Vector start or reload.
  • Gateway API request latency and throughput no longer degrade with the number of HTTPRoutes. Every route emitted ~4 http-request lines into the HTTP frontend — each running a regex for a PathPrefix — and HAProxy evaluated all of them, linearly, on every request. An HTTPRoute whose rules all match on path alone is now resolved by the host/path maps plus two new O(1) maps (route-winner.map, route-backend.map); the per-route chain is emitted only for an HTTPRoute where some rule matches on more than the path, which is the only case whose winner depends on the request. Measured at ~500 routes: 2,022 → 112 frontend http-request lines, 2,165 → 4,562 qps and 0.455 → 0.221 ms p50 at one connection, 23,480 → 52,540 qps and 46.1 → 9.6 ms p99 at 64 connections.
  • Two Gateway API routes sharing one hostname and path could leave a request with no backend. The set of candidate route IDs was built twice — sorted in the route analysis, in emission order in the path map — and the two were then compared as strings. A group of two or more routes whose IDs sort differently than they are emitted never matched, so the request fell through to the default backend and answered 404. Both sides now read one string, built once. Reachable with two Gateways serving the same hostname and path, which is what GatewayHTTPListenerIsolation exercises.
  • nginx-ingress annotation values could smuggle HAProxy directives or map entries. Values reaching a config line, a whitespace-delimited map key/value, or an unquoted ACL operand were interpolated without a guard, so a control character (or a quote/space where the context forbids it) could inject a second directive or routing-map entry that haproxy -c still accepts. server-alias, canary-by-header/-value/-pattern/-cookie/-weight, custom-request/-response-headers, and session-cookie-name/-path now warn-and-skip the offending value (deny at admission); auth-url/-signin/-method, auth-realm, and limit-rps/-rpm/-connections fail closed. Header values are HAProxy-log-format escaped and the realm is quote-escaped, and canary regex/exact match values are single-quoted, so a legitimate %, " or backslash (e.g. a \d canary pattern) is preserved rather than executed or rejected.
  • Resource-scoped chart validation recognizes every store alias attached to the object under admission, so a duplicate GVR alias cannot bypass a reject-mode check.
  • Warning Events were lost whenever the ingress library was disabled. The ClusterRole's events grant was gated on controller.templateLibraries.ingress.enabled, but recordEvent is a generic engine function that the governance, haptic-annotations, ingress-annotations-compat and vendor annotation libraries all call independently — so the rendered configuration kept its recordEvent call sites while the grant disappeared, and every Event they emitted was refused. The grant no longer follows any single library.
  • Any pod in the cluster could reach the Dataplane API port and the cache's backend-fetch port with haproxy.networkPolicy.allowExternal at its default true. NetworkPolicy rules are additive, so the permissive port-matching-everything rule unioned with the narrow rules meant to fence those ports off, and those rules never restricted anything. Permissive mode now emits the port ranges around both, making each narrow rule the only path to its port.
  • Client-certificate verification could disappear silently. When the auth-tls-secret CA could not be resolved, the render dropped the ca-file … verify required clause but kept serving the host — and because every HTTPS host terminates on one shared bind, the route simply answered with ordinary server-side TLS. An Ingress naming a missing Secret is now rejected at admission, and a route whose Secret is deleted afterwards is denied with 503.
  • An upstream TLS connection could go unverified while the operator believed it was checked. Naming backend-ca-secret is the request to verify the upstream certificate, but an absent Secret — without backend-verify also set explicitly — fell through to ssl verify none. It now fails the same way an explicit backend-verify: on already did. An unresolvable Secret denies only that route with 503, rather than aborting the render for the whole fleet.
  • Basic auth could serve a route with no authentication at all. Four paths reached a backend with no http-request auth rule and no userlist: auth-type: basic with an absent or empty auth-secret on the native library, and an unresolvable auth-secret on the haproxy-ingress, haproxytech and nginx-ingress libraries. All four now reject at admission and serve 503 under reconcile. The nginx path checks the Secret before folding into satisfy: any.
  • A client could forge the client-certificate identity headers. auth-tls-cert-header sets X-SSL-Client-CN, -DN and -Cert only when a certificate is presented, and nothing removed a caller-supplied copy first — so a request over plain HTTP carried the caller's own values to the application. The headers are now stripped before being set, as is api-key-consumer-header, which had the same shape.
  • External-auth response headers (nginx.ingress.kubernetes.io/auth-response-headers, haproxy-ingress.github.io/auth-headers-succeed, the oauth2-proxy desugaring) are stripped from the inbound request before the auth plugin's value is set. Previously a set-header skipped for a header the auth reply omitted let a client-supplied copy (e.g. X-Auth-Groups: admin) survive to the backend.
  • API-key authentication and consumer-group authorization fail closed with 503 while their referenced Secrets are absent from the watch cache, instead of rejecting an Ingress during Secret propagation races.
  • Creating a Gateway can no longer move an existing Gateway's per-pod listener port. Same-slot collisions were resolved in key order, so an arriving Gateway could evict the incumbent and take over its port in the same render — silently scoping clients still addressing that port to a different Gateway's routes, across tenants. Slots are handed out oldest-first, so a Gateway's port depends only on its own key and those of older Gateways.
  • Gateway API traffic reports the correct owning resource. The owner-identity cascade read a variable scope nothing writes, so the Gateway branch never fired and requests fell through to txn.resource_id = "gtw/<namespace>" — the key every per-resource feature map (WAF policy, external auth, rate limit, cache, schema validation) is looked up with.
  • A single mirrored route no longer breaks every gRPC route on the same frontend. A mirror filter emitted a frontend-wide option http-buffer-request, which deadlocks a gRPC stream until it dies with a 408. All sites now emit a Content-Length-gated http-request wait-for-body instead and consult the per-backend override map, so a route's request-buffering: off is not defeated by an unrelated mirrored route. A chunked request body is no longer mirrored.
  • A body-inspecting WAF policy on a route with a gRPC backend is rejected at admission instead of failing at runtime, and detect mode waits only for a body whose length is declared. Coraza buffers a complete body and ships no protobuf processor, so a gRPC stream could never satisfy it: streaming calls timed out with 408 while unary calls kept working. An existing route records a WafBodyPolicyOnGRPCRoute Warning Event rather than being taken down.
  • When two Ingresses claim the same host, path and path type, the route resolves to the older Ingress (by creationTimestamp, tiebroken by namespace/name) instead of the arbitrary render-order winner, matching ingress-nginx's oldest-wins. An nginx canary Ingress no longer competes for its main's base route, and the losing Ingress gets a RouteConflict Warning Event.
  • EndpointSlices that do not expose a Service's requested named target port are no longer rendered as HAProxy servers using the Service port, which could send application traffic to unrelated pods matched by a broad Service selector.
  • Installing without cert-manager works out of the box: when cert-manager provisioning is enabled (the default) but its API is absent, the chart generates a self-signed default-ssl-cert Secret instead of silently skipping the Certificate — previously every render failed with TLS Secret not found and the HAProxy pods never became ready. An existing Secret is left untouched (#78).
  • helm upgrade from 0.1.0 no longer fails with Secret "…-webhook-cert" … cannot be imported into the current release: invalid ownership metadata. The chart's webhook serving certificate Secret is named <release>-webhook-tls, so it no longer contends with the cert-manager-provisioned Secret 0.1.0 left behind — which can be deleted at leisure. This removes the manual pre-step previously documented for it; controller.webhook.secretName still overrides the name.
  • Enabling several vendor annotation libraries at once no longer exceeds Kubernetes' per-object size limit, which previously failed to install with a bare etcdserver: request is too large and was worked around by a documented "enable at most one" restriction.
  • Forcing spoaHub.enabled=true with every plugin disabled is rejected at render time instead of installing a broken combination where the sidecar's own config was orphan-deleted out from under it.
  • The bundled SPOA hub no longer crashes with SIGSEGV during dlclose when a plugin is removed during a config reload; retired plugin instances are still drained and destroyed.
  • HAProxy's restrictive NetworkPolicy (haproxy.networkPolicy.allowExternal: false) allowed only port 8404, so Prometheus could reach neither Vector endpoint. Both Vector ports are now allowed alongside 8404.
  • The default HAProxy bootstrap configuration applies the same configurable extraContext.hardStopAfter drain bound as the rendered configuration, so the initial worker cannot remain stuck after the first reload.
  • Generated configurations no longer emit haproxy -c warnings from proxy-mode mismatches or unsupported TLS settings: TCP frontends use tcplog, forwardfor is scoped to HTTP frontends, the quiet status endpoint retains a log target with normal logs suppressed, and the AWS-LC-incompatible tune.ssl.default-dh-param default was removed.
  • The three shipped log-derived cache metrics (cache_status_total, cache_age_seconds_total, cache_uncacheable_total) were silently dropped from values.yaml while the library and docs still promised them, so cache dashboards went empty on upgrade. Restored with the missing origin_refused_sharing reason, and a build check fails if a shipped entry goes missing again.
  • Cache bypass paths strip every private Varnish protocol header before the application sees it and preserve downstream Vary and Cache-Control: private protections while Varnish is unavailable or the method is not cacheable.
  • CORS no longer destroys the cache's Vary. The rule used set-header, replacing the whole field, so a CDN in front of HAPTIC saw only Vary: Origin and could serve one caller's keyed response to another. It appends now.
  • The nginx.ingress.kubernetes.io/enable-cors and haproxy-ingress.github.io/cors-enable annotations answer the CORS preflight in HAProxy with a synthetic 204 carrying the Access-Control-* headers, matching ingress-nginx, instead of forwarding it to the backend.
  • The nginx.ingress.kubernetes.io/cors-allow-origin and haproxy-ingress.github.io/cors-allow-origin annotations accept a comma-separated allow-list with single-level *. subdomain wildcards, match it against the request Origin and echo the matched origin back (adding Vary: Origin for non-wildcard lists). Previously the value was emitted verbatim, so a multi-origin list produced an invalid header. A malformed origin now fails the render.
  • The haproxy.org/cors-* annotations mirror the upstream HAProxy Kubernetes Ingress Controller: a non-wildcard cors-allow-origin is a regex matched against the request Origin and the response echoes the matched origin, headers are added via http-after-response, and the cors-allow-methods/cors-allow-headers/cors-max-age defaults track upstream.
  • haproxy-ingress.github.io/auth-secret also parses an auth-key htpasswd Secret (haproxy-ingress's and ingress-nginx's native format), so a Secret migrated from those controllers authenticates. The previous Secret shape still works.
  • haproxy.org/auth-realm normalizes spaces to dashes instead of failing the render, and its default realm is Protected-Content. The sanitize_auth_realm toggle is removed.
  • nginx.ingress.kubernetes.io/proxy-ssl-secret client certificates deploy under the correct path: the rendered crt value is the bare filename instead of an ssl/-prefixed path that doubled to ssl/ssl/<name> and made HAProxy reject the config.
  • The Gateway API template library no longer requires the Ingress library; its shared hostname-to-map-key helper moved into the always-loaded base library, so controller.templateLibraries.gateway can be enabled with ingress disabled.
  • nginx.ingress.kubernetes.io/limit-rate-after is documented as different rather than supported. It maps to the bandwidth filter's min-size — the smallest chunk HAProxy forwards, not nginx's "start throttling after N bytes".
  • A tenant could hijack another tenant's host and path by putting a newline or space in an Ingress or HTTPRoute path. The path was written verbatim into the shared routing map, whose keys are whitespace-delimited, so a control character or space smuggled a second map line pointing a victim's host+path at the attacker's backend — content haproxy -c accepts. Ingress and Gateway path values are now guarded before they reach the routing map: denied at admission, and warned-and-skipped with an InvalidPath Warning Event under reconcile so one hostile object can't brick the fleet.
  • A tenant could hijack routing or inject frontend directives by putting a double quote or newline in an HTTPRoute/GRPCRoute match value or URL-rewrite/redirect path. These values land inside double-quoted HAProxy ACL patterns and http-request directives in the Gateway frontend; a " broke out and appended attacker-controlled ACL terms (always-match routing hijack) and a newline split the line into an arbitrary directive — all valid config that haproxy -c, the determinism check and the load gate accept. Gateway API does not charset-validate RegularExpression, header, query-param or rewrite values, so they are now guarded before emission: denied at admission, and warned-and-skipped with an InvalidMatch/InvalidURLRewrite/InvalidRedirect Warning Event under reconcile.
  • A tenant could inject frontend/backend directives through header, CORS, response-body, TLS-option and OAuth values that were emitted with an inadequate guard. Gateway RequestHeaderModifier/ResponseHeaderModifier and backendRef header values, haproxy-haptic.org/request-set-header / response-set-header values, the shared CORS cors-allow-methods/-headers/-max-age/-expose-headers values, and fixed-response/mock-response content-types and bodies are now emitted inside double quotes with "/$/\ escaped and control characters rejected — closing a single-quote break-out ('evil' # commented out the host-match gate, applying a header to all traffic) and an unescaped $ that HAProxy expanded as an environment variable, leaking controller env into a response body. Header and CORS values additionally double any % (a log-format string reads %[...] at request time, so a tenant value could otherwise read request-scoped internal state into its own header). Gateway per-listener spec.tls.options (TLS min/max version, cipher suites) and the OAuth oauth-uri-prefix are guarded before they reach the shared HTTPS bind line and the auth routing maps. All are denied at admission and warned-and-skipped under reconcile. Secret-absent warning comments (api-key-secret, auth-secret, hmac-secret) no longer interpolate the tenant-controlled Secret name into the rendered config.
  • The haproxy.org/* (haproxytech) annotation library could inject frontend/backend directives through many values emitted with an inadequate guard. cors-allow-methods/-headers/-max-age and request-set-header/response-set-header values are now emitted inside double quotes with "/$/\ escaped and % doubled (a header value is a log-format string), closing the old single-quote break-out; the cors-allow-origin regex is single-quoted (strong quoting, so its backslashes survive) and rejects a ' or control char; path-rewrite, timeouts, load-balance, check-http/check-interval, cookie-persistence(-no-dynamic) and server-proto reject control characters (and, for bare-token values, whitespace) before emission. Routing values are denied at admission and warned-and-skipped under reconcile; the security-critical src-ip-header, rate-limit-* and auth-realm fail closed. The Secret-absent server-ca/server-crt warning comments no longer interpolate the tenant-controlled Secret name into the rendered config.
  • The haproxy-ingress.github.io annotation library passed tenant values into HAProxy config without an adequate guard. A newline or space in server-alias/server-alias-regex or a regex path-type path smuggled a second host/path map entry (cross-tenant route hijack); a newline, space or quote in timeout-*, health-check-uri/backend-check-interval, session-cookie-name/-keywords/-domain, agent-check-*, the headers name and the auth-realm split its directive; a newline in auth-url/auth-signin/auth-method/oauth-uri-prefix/oauth-headers injected an auth routing-map entry — all valid config that haproxy -c accepts. Every such value now routes through the shared guard/escape utilities: map keys/values and directive tokens reject whitespace and control characters, the headers value is emitted inside double quotes with "/$/\ escaped and % doubled, and auth-realm is escaped inside its quoted literal. Routing values (aliases, paths, timeouts, cookies, health checks) are denied at admission and warned-and-skipped under reconcile; security values (secure-sni/secure-verify-hostname, ssl-ciphers-backend/ssl-cipher-suites-backend, auth-realm) fail closed.

[0.2.0-alpha.1] - 2026-07-05

Added

  • Support for every Gateway API release, adapted at runtime: watchedResources entries accept an ordered apiVersions candidate list and an optional flag, templateSnippets/validationTests accept requires/requiresFields (features whose resources or schema fields aren't served are stripped at config load), and a CRD watch reinitializes the controller when a watched CRD is installed, upgraded, or removed — no Helm operation or pod restart needed. Resolved versions are exposed via resources.<name>.APIVersion() and /debug/vars/effectiveConfigResolution; controller validate resolves the same way against --schema-dir. A required resource with no served version fails startup with a clear error instead of hanging.
  • The config's embedded validationTests are now enforced at every gate: the admission webhook denies a failing HAProxyTemplateConfig at kubectl apply, and the controller rejects a failing config at startup and on every change — the last-good config keeps serving, and failing test names surface on status.validationErrors. Previously only the controller validate CLI ran them.
  • Kubernetes Events on the HAProxyTemplateConfig: a Warning/ValidationFailed Event when a config change fails validation and a Normal/Validated Event on recovery, so failures surface in kubectl describe and kubectl get events.
  • HAProxy 3.4 support: the controller is built, validated, and released for HAProxy 3.4.
  • Typed watched resources: chart templates access watched resources through compile-time typed top-level globals (e.g. gateways, httproutes), type-checked at controller startup against schemas from the live apiserver or --schema-dir. See ADR-0010.
  • spec.k8sResources declares full Kubernetes resources reconciled via Server-Side Apply, owned by the HAProxyTemplateConfig CR (garbage-collected on uninstall); a partial-ownership mode (haproxy-haptic.org/ownership: partial) lets chart-owned and haptic-owned entries coexist on one resource.
  • spec.validators declares pluggable validator sidecars consulted by the admission webhook (per-entry socket, file-glob routing, timeout); /healthz reports each socket's reachability.
  • The admission webhook hot-reloads its TLS certificate from the mounted Secret, so a cert-manager renewal is served within about a minute without a controller restart. The --webhook-cert-secret-name flag is replaced by --webhook-cert-dir (env WEBHOOK_CERT_DIR).
  • Gateway API RequestMirror filters, including multiple and percentage/fraction mirrors, backed by the bundled mirror SPOA plugin.
  • Ingress spec.defaultBackend support, both rules-less (catch-all) and combined with spec.rules.
  • HAProxy responses now carry a Server: haptic header.
  • New metrics: haptic_haproxy_reloads_total (the canonical reload/SLO signal), haptic_dataplane_api_operations_total, runtime fast-path counters (haptic_runtime_fast_path_*), and per-plugin SPOA metrics via the hub's /metrics endpoint.
  • spec.dataplane.configPublishInterval, reloadVerificationTimeout, and syncTimeout are now tunable; spec.watchedResources.<name>.debounceInterval adds a per-resource batching override.
  • New spoa-hub container image bundling the SPOE hub plus seven plugin libraries (mirror, coraza, external-auth, fingerprinting, maxmind, otel, sso-auth), Cosign-signed with a CycloneDX SBOM.
  • External authentication (auth-url-style annotations) wired end to end via the spoa-hub external-auth plugin.

Changed

  • Leader-election timing defaults raised from 15s/10s/2s to 30s/20s/5s (leaseDuration/renewDeadline/retryPeriod): more headroom against apiserver or CPU-starvation stalls, at the cost of slower crash-failover (up to ~35s; voluntary handoffs still release the lease immediately). Tune via spec.controller.leaderElection (#57).
  • Config validation now runs asynchronously with latest-wins coalescing: rapid successive HAProxyTemplateConfig edits validate only the newest change, and an in-flight validation no longer blocks config processing or leaves a new leader without a config (#55).
  • store: on-demand watched resources keep far less memory resident: the informer cache strips object bodies down to index and identity fields; templates still read the full object, fetched live on access. See ADR-0012.
  • All log output now shares one structured logfmt format (client-go and other third-party lines included), and routine per-request/per-reconcile success lines moved from INFO to DEBUG — failures, denials, config changes, and leadership transitions stay at INFO. Set LOG_LEVEL=DEBUG for the previous verbosity.
  • Runtime-eligible server changes (pod-IP rotation, port, admin state) now reach the live HAProxy workers in milliseconds via a runtime fast path instead of waiting for the next scheduled deploy; structural changes still take the rate-limited reload path. See ADR-0011. The reconciler-level debounce was removed — batching is per-watcher via spec.watchedResources.<name>.debounceInterval (default raised from 100ms to 2s; "0" fires on every change).
  • Content-only changes now apply to the live HAProxy workers via the runtime API without a reload: map files, TLS certificates (HAProxy 3.2+), SSL CA files (HAProxy 3.2+, e.g. frequent trust-bundle rotation), and frontend maxconn. Adding or removing files and all other structural changes still reload, and a failed runtime apply falls back to a reload, so the result always converges.
  • controller validate now requires --schema-dir (or HAPTIC_SCHEMA_DIR) for typed-access templates; the embedded Gateway schema fallback was removed. Production (live apiserver) is unaffected.

Fixed

  • Fixed controller crash-loop and Gateway status-apply failures on clusters running older Gateway API releases (v1.1–v1.5) whose schemas lack fields the bundled chart exercises (#59).
  • A replica that loses the leader-election lease now reinitializes and re-enters the election loop instead of running as a permanent follower — on single-replica deployments, a missed lease renewal could previously stall deployments until a pod restart (#57).
  • Runtime map updates are now verified by read-back and fall back to a reload when the live map did not converge — a map write acknowledged but lost by HAProxy could previously latch stale routing until an unrelated reload (#48).
  • Eliminated intermittent 503s during rolling restarts of single-replica backends: a newly-Ready pod's endpoint change reaches the live workers within option redispatch's rescue window even when it coincides with an in-flight structural deploy, runtime applies retry across a concurrent reload, and deleting an unreferenced certificate no longer schedules a stray second reload that briefly blacked out the runtime socket (#67).
  • Reloads are bounded to one per minDeploymentInterval under concurrent churn, and auxiliary-file updates batch into the main config sync's single reload instead of triggering separate DataPlane API auto-reloads.
  • Resource status no longer goes stale: statuses are written even when a change produces no config diff (a Gateway with no attached routes no longer sticks at Programmed=Unknown), and sustained event bursts no longer drop status or deployment events.
  • HAProxyCfg.status.deployedToPods now reflects reality: a failed deploy keeps the pod's last successfully-deployed checksum plus a lastError instead of reading as converged, failed deployments are retried on the next reconcile instead of waiting for the drift timer, and a startup race no longer leaves a deployed pod unreported.
  • Cross-pod config drift is now detected and repaired: the deployer tracks each pod's actual post-sync state, and a render race no longer leaves a newer render undeployed.
  • Credentials-Secret rotation now takes effect immediately, and content updates to watched Secrets/ConfigMaps are no longer dropped.
  • gRPC: requests hitting the default backend get a proper trailers-only grpc-status: 12 response instead of a connection-breaking 404, GRPCRoute method matching works on TLS + ALPN h2 and plaintext h2c, and gRPC over h2c multiplexes with HTTP/1.1 on the same bind.
  • Gateway TLS listeners on non-default ports now bind correctly.
  • The renderer fails fast when the config references a map file it didn't register (previously the file was deleted as unreferenced, breaking every subsequent reload), and each render sees one consistent snapshot of the watched-resource stores.
  • store: on-demand resources no longer trigger a full list or one API fetch per resource on per-key reads, drift cycles, and the /debug/vars endpoints.
  • Ingress pathType: Exact now preserves trailing slashes (/foo/ no longer matches /foo).
  • controller validate renders with the same context as production (capabilities, extraContext promotion), so a template can no longer pass validation yet render differently in production.
  • HAProxy 3.3 configs are now validated against the v3.3 DataPlane API schema (was v3.2), and DataPlane API versions newer than the newest bundled client clamp down to it instead of falling back to the oldest v3.0 client.
  • Watched-resource admission (e.g. Ingress) now gets the same ~9s internal validation deadline as HAProxyTemplateConfig admission (was 5s), so large-config dry-runs aren't prematurely admitted without validation.
  • Status patches, admission overlays, and owned-resource applies resolve a resource's plural via the cluster RESTMapper, so a CRD with an irregular plural no longer targets a nonexistent GroupVersionResource.
  • Hardened component lifecycle: re-acquired leadership no longer stacks orphaned event subscriptions (previously endless "critical drops" log spam), event-loop panics recover instead of silently killing the loop, and the controller no longer schedules a spurious iteration restart right after startup.
  • HTTP content stores no longer misclassify an empty 200 OK as 304 Not Modified (stale content), SSL-certificate Secret publishing retries on write conflicts, and backends whose only change is on the default-server line now update instead of staying stale.
  • HAProxy file paths in rendered configs use slash-only semantics regardless of host OS, fixing the controller binary on Windows (no-op on Linux).
  • Bundled SPOA hub updated to v0.7.3: config reloads quiesce in-flight SPOE dispatches before retiring the old plugin set, fixing lost mirror/OTLP work around reloads (#47); unhandled SPOE messages surface via a WARN log and spoa_messages_unhandled_total.

Removed

  • The dormant webhook-cert-rotation pipeline and its haptic_webhook_cert_expiry_timestamp_seconds / haptic_webhook_cert_rotations_total metrics (it never rotated certs) — replaced by the webhook-cert hot-reload above.
  • namespaceSelector on watchedResources entries (was never wired up). Scope via labelSelector or separate controller instances.
  • Management of HAProxy program sections — HAProxy removed the section in 3.3 and client-native dropped the model. Rendered configs may still contain program sections on older HAProxy versions, but the controller no longer parses, diffs, or normalizes them.

Helm chart

Added

  • New nginx-ingress template library for nginx.ingress.kubernetes.io/* annotation compatibility (disabled by default): routing and redirects (rewrite-target, app-root, server-alias, default-backend, from-to-www-redirect, permanent-redirect/temporal-redirect with code overrides, ssl-redirect/force-ssl-redirect — 308 by default, tunable via the nginxHttpRedirectCode extraContext var), session affinity (session-cookie-*), rate limiting and throttling (limit-rps/limit-rpm/limit-whitelist, limit-rate/limit-rate-after), backend TLS (proxy-ssl-*), request/response header rewrites (upstream-vhost, x-forwarded-prefix, connection-proxy-header, proxy-cookie-domain/-path, proxy-redirect-from/-to), retries (proxy-next-upstream/-tries), body-size limits (proxy-body-size), auth (auth-*, auth-secret-type, satisfy), and request mirroring (mirror-target, via the spoa-hub mirror plugin).
  • haproxy-ingress library additions: oauth: oauth2_proxy (desugars onto the external-auth machinery), rewrite-target, rate limiting (limit-rps/limit-rpm/limit-whitelist — previously advertised but emitting no config), proxy-body-size, ssl-ciphers-backend/ssl-cipher-suites-backend, agent health checks (agent-check-*), default-backend-redirect/-code, server-alias/server-alias-regex, and raw directive injection (config-frontend/config-global/config-defaults).
  • haproxytech library additions: rate-limit-whitelist and ssl-redirect-port.
  • Gateway API support extended: TLSRoute (Terminate and Passthrough), TCPRoute (L4 forwarding), RequestMirror filters (via the spoa-hub mirror plugin), static Gateway addresses (per-Gateway LoadBalancer Service, multi-IP), spec.infrastructure propagation, ListenerSet routing, GEP-91 frontend client-certificate validation, HTTPRoute/GRPCRoute cookie session persistence (GEP-1619), HTTPRoute retry, per-listener TLS options (GEP-2907), and BackendTLSPolicy validation.subjectAltNames.
  • spoaHub values block and SPOA hub sidecar (hub v0.7.3 plus seven plugin libraries: mirror, coraza, external-auth, fingerprinting, maxmind, otel, sso-auth), auto-rendered when any plugin is enabled; a spoa-hub template library wires the HAProxy-side SPOE config, and the hub's runtime config reloads without a pod restart.
  • nginx-ingress and haproxy-ingress libraries wire the external-auth annotation family (auth-url, auth-signin, auth-method, auth-headers-*, client-mTLS auth-tls-*) to the external-auth plugin, plus the Coraza WAF annotations (/waf, modsecurity-snippet) with per-resource opt-in and a default-on dispatch mode.
  • controller.validators block and a validator sidecar consulted by the admission webhook, auto-enabled and auto-wired with the spoa-hub sidecar, so admission-time WAF/TOML validation works out of the box.
  • The HAProxyTemplateConfig admission webhook now also runs the config's embedded validationTests, denying a failing config at kubectl apply; its timeoutSeconds is 10, and failurePolicy: Ignore still admits-with-warning when the webhook is slow or unreachable.
  • The ingress library now emits a Warning Event (reason BackendUnresolved) on each Ingress whose backend Service or named port can't be resolved — visible in kubectl describe ingress — so a permanent Service-name typo is distinguishable from a propagation race; the Event is removed automatically once the Service appears (#66).
  • Four more default PrometheusRule alerts (individually toggleable): configRejected, haproxyPodsRejected, noHAProxyPods, and criticalEventsDropped.
  • controller.templateLibraries.gateway.experimentalChannel gates the validationTests that assert Experimental-channel HTTPRoute fields (sessionPersistence, retry) — since Gateway API v1.6 the Standard and Experimental channels ship an identical CRD set, so the channel can't be auto-detected.
  • Template libraries can declare default templatingSettings.extraContext values, merged at the lowest precedence so operator overrides still win.
  • Always-on local peers localinstance section so opted-in stick-tables survive reloads: rate-limit counters (haproxy.org/rate-limit-*, nginx-ingress limit-rps/limit-connections, haproxy-ingress limit-rps/limit-rpm) now persist across config reloads instead of resetting.
  • New values: haproxy.initialConfig (HAProxy bootstrap ConfigMap), controller.config.dataplane.{configPublishInterval,reloadVerificationTimeout,syncTimeout}, controller.config.extraContext.hardStopAfter (default 10s, emits hard-stop-after so old workers don't accumulate across reloads), and haproxy.dataplane.aclFormat (dataplane access-log format override).
  • The chart-static haptic-haproxy LoadBalancer Service is now rendered via the controller's spec.k8sResources (Server-Side Apply with an OwnerReference), folding non-default Gateway listener ports into the same Service.
  • HAProxy NetworkPolicy opens to all TCP ports when haproxy.networkPolicy.allowExternal: true, so dynamic Gateway listener ports work; restrictive mode is unchanged.
  • RBAC: the ClusterRole gains customresourcedefinitions read access (schema resolution), cluster-wide Event write verbs (ingress library), and cluster-wide plus namespace-scoped services write verbs (gateway library, per-Gateway Services); ClusterRole and webhook rules now cover every declared API-version candidate of each watched resource.

Changed

  • BREAKING: path matching order is now selected by controller.config.routing.regexMatchOrder (default/last), replacing the removed path-regex-last template library. Operators with templateLibraries.pathRegexLast.enabled: true must switch to routing.regexMatchOrder: last. See ADR-0005.
  • BREAKING: ingressClass.name and gatewayClass.name default from haproxy to haptic. Operators replacing an incumbent controller set them back to haproxy (or update their manifests to ingressClassName: haptic / gatewayClassName: haptic).
  • BREAKING: pod-spec scheduling, runtime, and metadata fields moved under namespaced podSpec: blocks on both Deployments. Container-, Deployment-, and chart-wide fields are unchanged. Rename the following keys in custom values files:
Previous New
imagePullSecrets controller.podSpec.imagePullSecrets
podAnnotations controller.podSpec.podAnnotations
podLabels controller.podSpec.podLabels
priorityClassName controller.podSpec.priorityClassName
topologySpreadConstraints controller.podSpec.topologySpreadConstraints
podSecurityContext controller.podSpec.podSecurityContext
nodeSelector controller.podSpec.nodeSelector
tolerations controller.podSpec.tolerations
affinity controller.podSpec.affinity
terminationGracePeriodSeconds controller.podSpec.terminationGracePeriodSeconds
dnsPolicy controller.podSpec.dnsPolicy
dnsConfig controller.podSpec.dnsConfig
hostAliases controller.podSpec.hostAliases
runtimeClassName controller.podSpec.runtimeClassName
haproxy.priorityClassName haproxy.podSpec.priorityClassName
haproxy.topologySpreadConstraints haproxy.podSpec.topologySpreadConstraints
haproxy.nodeSelector haproxy.podSpec.nodeSelector
haproxy.tolerations haproxy.podSpec.tolerations
haproxy.affinity haproxy.podSpec.affinity
haproxy.dnsPolicy haproxy.podSpec.dnsPolicy
haproxy.dnsConfig haproxy.podSpec.dnsConfig
haproxy.hostAliases haproxy.podSpec.hostAliases
haproxy.runtimeClassName haproxy.podSpec.runtimeClassName
haproxy.podAnnotations haproxy.podSpec.podAnnotations
haproxy.shareProcessNamespace haproxy.podSpec.shareProcessNamespace
haproxy.terminationGracePeriodSeconds haproxy.podSpec.terminationGracePeriodSeconds
haproxy.podSecurityContext haproxy.podSpec.podSecurityContext
  • The gateway library now supports every Gateway API release, resolved at runtime: each Gateway kind is an optional watched resource with an ordered apiVersions candidate list (core kinds v1/v1beta1, GRPCRoute v1/v1alpha2, TLSRoute v1/v1alpha3/v1alpha2, TCPRoute v1/v1alpha2, ReferenceGrant v1/v1beta1, BackendTLSPolicy v1/v1alpha3). Installing or upgrading Gateway API CRDs activates support at runtime — no helm upgrade needed; features whose fields don't exist in an older release's schemas stay inactive there. The Helm-render-time .Capabilities gate is removed.
  • The GatewayClass object is now created at runtime by the controller (Server-Side Apply, owned by the HAProxyTemplateConfig) instead of by a Helm template, so it exists exactly when the gatewayclasses CRD is served. Upgrade note: helm upgrade removes the previously Helm-owned GatewayClass and the controller recreates it within one reconcile; existing Gateways are unaffected.
  • Per-route annotation policies are now applied by shared frontend rules reading per-backend/per-host map files instead of inline per-backend/per-ingress rules: proxy-body-size, upstream header overrides (set-host, upstream-vhost, x-forwarded-prefix, connection-proxy-header), literal rewrite-target, ssl-redirect/force-ssl-redirect, host redirects (redirect-to, request-redirect, permanent-redirect/temporal-redirect), HSTS, app-root, auth-tls-error-page, from-to-www-redirect, default-backend-redirect, and ssl-redirect-port. Adding or changing these is now a map-only, reload-free update; behavior, status codes, and config-injection guards are unchanged (app-root, auth-tls-error-page, and hsts-max-age gain previously missing injection guards).
  • The deployed DataPlane API now addresses storage (maps/certs/general files) by relative paths resolved against the HAProxy base dir — the same identifiers HAProxy uses — so map content changes apply via the runtime API without a reload. The dataplane.mapsDir/sslCertsDir/generalStorageDir values are unchanged.
  • The auto-generated DataPlane API password (credentials.dataplane.password left empty) is now a random 32-char value instead of a deterministic hash of release name and namespace. Existing installs are unaffected (the password is preserved from the existing Secret on helm upgrade). GitOps note: under ArgoCD/Flux (no cluster lookup at render time), an empty password regenerates on every sync — set credentials.dataplane.password explicitly for those setups.
  • The validating admission webhook now provisions its own self-signed TLS certificate by default (webhook.certManager.enabled defaults to false; validity via webhook.selfSigned.certValidityDays, default 10y, reused across upgrades), so admission validation works out of the box without cert-manager. cert-manager stays opt-in and is recommended for production (automatic rotation); the self-signed cert is not auto-rotated. Upgrade note: a release that used the previous cert-manager default holds a webhook Secret without Helm ownership metadata; before upgrading, either keep cert-manager (--set webhook.certManager.enabled=true) or delete the old Secret (kubectl delete secret <release>-webhook-cert -n <namespace>) so the chart can recreate it.
  • Default haproxyVersion is now 3.4 (was 3.2); the controller and HAProxy pod images default to the HAProxy 3.4 series. Override haproxyVersion to stay on an older series.
  • Template libraries (gateway, nginx-ingress, haproxy-ingress, haproxytech) are now packaged as conditional subcharts: a release stores only the source of the libraries it enables, so an install with all libraries enabled no longer exceeds the Kubernetes 1 MiB release-Secret limit. No values changes (the same controller.templateLibraries.<x>.enabled flags), and the rendered HAProxyTemplateConfig is byte-identical.
  • The controller's startup probe is now enabled by default (30 × 10s budget): startup runs the config's embedded validationTests, so time-to-first-healthy can exceed the liveness budget on slow or contended nodes — previously the liveness probe could kill the controller mid-initialization.
  • haproxy.ports.http/https defaults shift from 8080/8443 to 80/443 so dst_port equals the Gateway listener port; explicit overrides are kept.
  • Dataplane minDeploymentInterval default raised to 5s (was 2s), throttling reload-inducing structural deploys; endpoint changes still apply instantly via the controller's runtime fast path. EndpointSlice watches keep debounceInterval: "0" for instant rolling-restart reaction.
  • HAProxy defaults timeout connect lowered from 5000 to 100 (100 ms). Backends are pod IPs over the CNI; 100 ms fails fast on a SYN to a just-terminated pod so option redispatch retries. Operators on slow networks restore 5000 via extraContext.timeout_connect.
  • gateway library: the cluster-wide configmaps watch now defaults to store: on-demand (it is only read by name for BackendTLSPolicy CA bundles), keeping references instead of every ConfigMap body resident.
  • extraDeploy now accepts both list and dict formats.
  • haproxy.org/pod-maxconn quantizes the pod count to the next power of 2 to avoid reload cascades on scaling.
  • Compound resource names (derived from haptic.fullname) now truncate to the 63-char label limit; releases with very long names will see affected resources renamed on upgrade (run helm diff upgrade first).
  • Removed the "TLS Certificate Expiry" Grafana dashboard panel — the controller no longer emits haptic_webhook_cert_expiry_timestamp_seconds (the webhook cert is hot-reloaded; see the controller section above).

Security

  • Annotation values interpolated into the HAProxy config — CIDR lists (allow/deny lists, rate-limit and satisfy whitelists) and single-value annotations (upstream header overrides, proxy-ssl-*, proxy-cookie-domain/-path, the auth realm) — are now validated against strict charsets, closing a config-injection vector where a crafted annotation containing a newline could inject arbitrary HAProxy directives into the rendered config.

Fixed

  • Fixed controller crash-loop and Gateway status-patch failures on clusters running older Gateway API releases (v1.1–v1.5) or Standard-channel installs without the TCPRoute CRD (#59).
  • Named Service-port references (port.name) in Ingress, HTTPRoute, GRPCRoute, and SSL-passthrough backends now resolve to the correct numeric port (previously silent 503s or empty backends). When the referenced Service isn't in the controller's store yet, the backend renders degraded (placeholder slots, 503 for that route) and converges once the Service appears, instead of failing the whole render and denying the Ingress at admission (#50); a Service that exists but lacks the named port still fails loudly.
  • defaults now sets option redispatch and base.yaml filters out not-ready/terminating endpoints, eliminating the single-replica rolling-restart 503 windows.
  • Reserved-slot server addresses are now config-driven across reloads (removed the HAProxy server-state-file machinery); placeholders use the unroutable 192.0.2.1:1 sentinel. See ADR-0011.
  • haproxy.dataplane.validateConfig: false now actually skips the dataplane's haproxy -c (the flag was misplaced), cutting raw-config push time ~130ms → ~18ms.
  • haproxytech: haproxy.org/check, check-interval, and scale-server-slots are now honored instead of silently ignored, and IP access control reads the canonical haproxy.org/allow-list/deny-list annotations (deprecated whitelist/blacklist honored as fallback) — the real annotations previously emitted no ACL.
  • haproxy-ingress: maxconn-server, maxqueue-server, initial-weight, backend-check-interval, and health-check-port/-fall-count/-rise-count now render their default-server keywords (previously validated but never emitted), the deprecated whitelist-source-range alias is honored, and the path-type annotation now actually routes.
  • Gateway TLS: an unspecified tls.mode defaults to Terminate per spec (the listener was previously skipped silently), and a BackendTLSPolicy with no resolvable CA returns 503 instead of downgrading to plaintext.
  • The bundled validationTests now pass in any release namespace (the shared SSL fixture no longer hardcodes haproxy-haptic/default).
  • The chart fails fast at install with actionable guidance when webhook.certManager.enabled=true but the cert-manager CRDs are absent, instead of leaving the controller pod stuck in ContainerCreating.
  • Basic-auth snippets no longer fail when the referenced auth Secret is briefly absent from the render snapshot.
  • PrometheusRule default alerts HAProxyControllerHighQueueDepth/HAProxyControllerNoLeader now reference metrics the controller actually emits (the old expressions never fired).
  • networkPolicy.ingress.webhook.from and networkPolicy.egress.kubernetesApi defaults switched to ipBlock 0.0.0.0/0 + ::/0, so a host-network apiserver can reach the webhook on clusters enforcing NetworkPolicy (previously silent admission failures).
  • The validating-webhook configuration now sources watchedResources from the merged libraries, so library-declared resources are actually validated.

[0.1.0] - 2026-03-09

Added

  • Template-driven HAProxy configuration: Generate HAProxy configs using Scriggo templates (Go-based, Jinja2-like syntax) with full access to Kubernetes resources, built-in utility functions, and modular template snippets
  • Embedded validation tests: Declarative test fixtures and assertions for testing HAProxy configurations within template libraries; run via haptic-controller validate --test <name>
  • Dry-run validation webhook: Admission webhook that intercepts CREATE/UPDATE on opted-in watched resources (Ingress, HTTPRoute, GRPCRoute by default), renders templates with the proposed object overlaid on the live store, and rejects the write if rendering or HAProxy validation fails
  • Multi-architecture container images: linux/amd64, linux/arm64, linux/arm/v7
  • HAProxy version support: 3.0, 3.1, 3.2, 3.3 — version-specific images tagged accordingly
  • Supply chain security: Container images, binaries, and Helm charts signed with Cosign (keyless OIDC); SBOM attestations in SPDX format
  • Prometheus metrics: Reconciliation timing, template rendering duration, validation results, and Kubernetes API latencies
  • Leader election for high availability: Multiple controller replicas with automatic leader election; hot-standby replicas continue watching and validating; configurable failover timing
  • Stall detection: Components detect when blocked and report unhealthy via /healthz, enabling automatic pod restart via Kubernetes liveness probes
  • Configurable deployment timeout: deploymentTimeout in dataplane config (default: 30s) to recover from stuck deployments
  • Server slot preservation: Preserve HAProxy server slots during rolling deployments to enable zero-reload runtime API updates via currentConfig template context
  • HAProxy Ingress annotation compatibility: 56 haproxy-ingress.github.io/* annotations via the haproxy-ingress template library
  • Dataplane API concurrency limiting: maxParallel config option to limit concurrent API operations, preventing timeouts for large configurations
  • CRD content compression: HAProxyCfg content compressed with zstd when exceeding configPublishing.compressionThreshold (default 1 MiB), reducing etcd storage
  • HAProxyGeneralFile CRD: Publish general files (error pages, etc.) as Kubernetes custom resources with compression support
  • HAProxyCRTListFile CRD: Publish crt-list files as Kubernetes custom resources with compression support
  • semver_gte template filter: Version comparison for gating features on HAProxy version (e.g., semver_gte(haproxyVersion, "3.3"))
  • Template-driven status patches: Templates can register status patches for any Kubernetes resource via statusPatch() function, with outcome-keyed variants (rendered, deployed, renderFailed, deployFailed) applied automatically based on pipeline phase
  • Backend diff field diagnostics: Reconciliation log now includes which BackendBase fields caused backend updates, aiding diagnosis of false diffs from parser round-trip asymmetries
  • Status patch helper functions: condition(), transitionTime(), and toJSON() template functions for building Kubernetes status conditions with stable transition timestamps

Changed

  • Reconciliation triggering: Leading-edge triggering with a 5s refractory period; no latency for isolated changes, bursts during that window are batched into a single reconciliation
  • Parallel Dataplane API operations: Operations execute in parallel within each priority group, reducing sync time for large configurations
  • Balance directive: balance roundrobin moved to defaults section to prevent silent behavior change when upgrading to HAProxy 3.3 (which changed the default balance algorithm from roundrobin to random)
  • Go runtime 1.26.1: Green Tea GC replaces manual GOGC tuning

Helm chart

Added

  • Initial Helm chart deploying the controller and HAProxy pods (2 replicas by default)
  • Separate controller Service (ClusterIP for operational endpoints) and HAProxy Service (configurable LoadBalancer/ClusterIP)
  • Default NetworkPolicy for HAProxy instances
  • Leader election support with configurable replica count
  • Default SSL certificate configuration via controller.defaultSSLCertificate
  • Modular template library system with composable libraries merged at Helm render time (enable/disable via controller.templateLibraries.<name>.enabled):
    • base.yaml: Core HAProxy template structure with extension points
    • ingress.yaml: Kubernetes Ingress support (path types: Exact, Prefix, ImplementationSpecific; TLS termination; default backend)
    • gateway.yaml: Gateway API support — HTTPRoute and GRPCRoute are watched and routed; traffic splitting, request/response header modification, URL rewrites, and Gateway/Route status patches are emitted. TLS/TCP/UDP listeners are reflected in each Gateway's supportedKinds status but TLSRoute/TCPRoute/UDPRoute resources are not watched or routed
    • haproxytech.yaml: haproxy.org/* annotation compatibility (backend config snippets, SSL passthrough, CORS, basic auth)
    • ssl.yaml: TLS/SSL features
    • haproxy-ingress.yaml: 56 haproxy-ingress.github.io/* annotation compatibility (enabled by default)
  • Gateway API status reporting: Gateway conditions (Accepted, Programmed), listener status, HTTPRoute/GRPCRoute parent status with Accepted and ResolvedRefs conditions
  • Ingress status reporting: LoadBalancer addresses propagated to Ingress .status.loadBalancer
  • HAProxy built-in Prometheus exporter enabled by default on the status frontend (/metrics on port 8404)
  • Grafana dashboard annotations for leader transitions and controller pod starts
  • Auto-generated Dataplane API credentials stored in a Secret (deterministic 32-char SHA256 of release-name + namespace; preserved across upgrades from the existing Secret)
  • haproxy.sysctls for setting kernel parameters on HAProxy pods via pod-level securityContext
  • haproxy.podAnnotations for custom pod annotations on HAProxy pods (supports Helm template expressions)
  • haproxy.shareProcessNamespace to enable process namespace sharing between containers (required for signal-based sidecar reload, e.g., SPIFFE/SPIRE mTLS agents)
  • haproxy.shmStats.enabled to persist stats counters across HAProxy reloads via shared memory (requires HAProxy 3.3+); automatically provisions /dev/shm emptyDir volume with auto-calculated size
  • haproxy.nbthread to control HAProxy thread count (auto-calculated from CPU requests by default)
  • haproxy.dataplane.validateConfig to control server-side config validation
  • haproxy.dataplane.debugSocketPath to enable Unix socket for runtime profiling of the Dataplane API sidecar
  • controller.config.dataplane.maxParallel to limit concurrent Dataplane API operations
  • controller.statusPatches.enabled to disable status patch writes during migration from another ingress controller
  • extraDeploy for deploying arbitrary Kubernetes resources alongside the chart (supports Helm templating)
  • extraEnv, haproxy.extraEnv, haproxy.dataplane.extraEnv for custom environment variables on all containers
  • global-settings-*, defaults-settings-*, and frontend-extra-* extension points for customizing HAProxy global/defaults sections and early frontend directives via template snippets
  • status-patches-* and status-extra-* extension points for custom status and Prometheus endpoint configuration
  • template post-processor type for declarative output transformations in postProcessing
  • guid directives on all frontends, backends, and servers for stable object identification

Changed

  • Dataplane API credentials consolidated into credentials.dataplane section; auto-generated if not provided
  • Basic auth userlists are named auth_<secretNs>_<secretName> and deduplicated per Secret; each Ingress references its userlist via http_auth(). Differs from the official HAProxy Ingress Controller's per-Ingress {namespace}-{ingressName} naming so multiple Ingresses sharing the same Secret produce a single userlist (significant speedup for bcrypt hashes)
  • Production-ready default resource requests and limits: controller (100m CPU / 512Mi memory), HAProxy (250m CPU / 1Gi memory), dataplane sidecar (50m CPU / 256Mi memory)
  • sidecars, extraVolumes, extraVolumeMounts and their haproxy.* counterparts support Helm template expressions

Removed

  • image.appendHaproxyVersion value (HAProxy version suffix is now always included in controller image tag)
  • haproxy.dataplane.credentials section (use credentials.dataplane instead)
Found a problem on this page? Report it or edit the page with the pencil icon above the title.