Skip to content

NewHabitz architecture and integration specification

  • Document: RFC 2026-08-NH-002
  • Status: Draft (v4). This document replaces v3.
  • Author: Mikael Karon, GoodHabitz Architecture Team
  • Date: 2026-08-27
  • Audience: system architects, lead engineers, security reviewers
  • Language: ASD-STE100 Simplified Technical English

0. How to read this document

Each section marks its statements as shipped or proposed. Shipped means checked against goodhabitz/newhabitz/*. Proposed means no code does this yet. A section without a mark is shipped.

A line that starts with Rule is a decision to follow. The sentences after it give the reason. Skip the reason if you accept the rule.

Section 14 holds each suggestion: a change to a repository that exists today. A suggestion is neither a rule nor a proposal, and it sits under the repository that must change.

A statement marked undecided is work in progress. The platform needs an answer, the answer is not given yet, and the text holds one option so that a reader can argue with something. Take it as a question, and not as a decision.

Six specifications give the design of the learning platform. This document holds the rules of the platform, and each of the six holds one part of that design. Where the two meet, the specification below is the authority on the learning path, and this document is the authority on the platform.

SpecificationWhat it binds here
RFC 001, Dynamic course protocol and state topologyThe domain model, and the machine that comes from it (sections 3.9 and 4.7)
RFC 002, LMS shell architecture and universal routingThe lms application and its frontend (section 3.10)
RFC 003, The activity playerplay, and each environment that runs it (section 4.7)
RFC 004, Distributed design system through a Shadcn registryThe views of play, and their delivery
RFC 005, Event sourcing and snapshot backendThe progress application (section 3.11)
RFC 006, Headless CMS integration and course composerThe cms application (section 3.9)

0.2 The three levels

LevelWhat it isDoes it move?
ApplicationOne Platformatic application in a runtime. It owns one part of the behaviour, and a database if it needs oneIt moves between moduliths
DomainA group of applications with one business meaning. It is the boundary of ownershipIt changes only when the business changes
ModulithA deployment group. One repository, one Watt runtime, one release, for example account-modulithIt is the thing that applications move between

application is the term of Platformatic. The directory is applications/. The gateway key is applications.

Domains and moduliths are two different groups, and they do not have to agree. An application has one domain and one modulith at a time. A domain can span moduliths. A modulith can hold more than one domain.

Therefore the modulith of an application is a deployment choice, and a change of that choice does not change a call site. Section 7.1 gives the rule that makes this true.

0.3 Vocabulary

Section 0.2 gives the three levels. These are the other terms that this document fixes.

TermWhat it means here
Domain modelThe shape of what a learner runs. RFC 001 holds the draft, and this document fixes none of it (section 3.9)
Domain objectOne thing of that model. This document uses the term where the model will give a name
DefinitionThe JSON document that carries a domain object. cms writes it, and it is the artifact that each reader takes
MachineThe XState machine that nh-play builds from a definition (section 4.7)
SnapshotThe state of one version of one machine, including each state name and the internal queue of events
Progress structureWhat a learner keeps: completion, a score and a position, with a version of its own. It holds no state of a machine, so it crosses a version (section 3.11)
playThe common player: the actor, the conversions and the route derivation that nh-play gives. A frontend adds the components
CatalogThe vocabulary of play: a name for each view, and a schema for its properties
ViewA name and a set of properties that a state of the machine emits. A frontend renders it
TargetOne authority string of the identity model, for example t:<tenantId> (section 5.4)
QueueThe work of one application, in its own namespace of the KeyValue store (section 4.5)

0.4 Where to start

If you areRead
Adding an application0.2, 1.1, 1.3, 3.1, then 5 and 6 for its identity and its rows
Adding a call between two applications7.1, 7.2, then 5.5 for the credential
Adding asynchronous work4.5, then 3.5 for the surface that shows it
Publishing an event to a third party3.6
Adding an agent, or any AI work1.5 and 3.14, then 5 and 6 for the authority that a tool call carries
Working on the learning path0.1, 0.3, then 3.9, 3.10, 3.11 and 4.7
Reviewing security5, 6 and 9
Setting up a machine2
Deploying11 and 12

1. Structure

graph TB
    subgraph modulith ["one modulith — one repo, one Watt runtime, one release"]
        direction TB
        GW["gateway, the entrypoint — @platformatic/gateway"]
        FE["frontend — one application or more"]
        A["application — @platformatic/db"]
        B["application — @platformatic/service"]
        GW -->|"HTTP over the mesh"| FE
        GW -->|"HTTP over the mesh"| A
        GW -->|"HTTP over the mesh"| B
        B -->|"HTTP, generated client"| A
    end
    client([client]) -->|HTTPS| GW
    A --> PG[(PostgreSQL)]
    A -->|"queue, cache, locks"| KV[("KeyValue, on Valkey")]
    B --> KV

Each modulith has one gateway. All traffic to a modulith goes through it.

1.1 The choice of modulith is open

Rule: choose the modulith for each application from deployment needs and scalability needs. Applications can share one modulith, or each can have its own. Neither is a default. The choice can change later. Nothing else in this document depends on it, except where a section states a start dependency (section 1.5).

Use these criteria:

CriterionIt argues for a separate modulith when…
Release rateOne application must release alone. Applications in one modulith release together
Resource profileThe load or the resource shape is very different, not only larger
Fault isolationA fault must stay contained. One runtime is one process, with no limit per application
ComplianceA policy needs separate data or separate workloads
Domain cohesion(argues for sharing) Applications in one domain change together, so they cost less to release together
Cost(argues for sharing) Each modulith adds a release train, a gateway, and infrastructure

These are not criteria: the number of applications, the number of engineers, or the quantity of code. A split for size adds cost and gives no isolation and no independent release, because no criterion above asked for them. “The modulith feels large” is not a reason.

The domain of an application is not open. Business meaning sets it. Only the modulith changes.

1.2 The shape inside a modulith

Platformatic gives two shapes. The choice sets where the load goes.

ShapeArrangementUse it for
PyramidOne entrypoint sends work to several applications below itOne public surface over several applications. Each modulith here has this shape
FunnelSeveral identical CPU-bound instances send work to one application that does the I/OWork that fills a CPU core: media, documents, transformation, inference

You can use both in one runtime.

Rule: the entrypoint does routing, authentication, composition, and coordination only. Each request passes through the gateway, and the gateway also terminates identity (section 5). Domain work there loads the one component that you cannot scale around.

Rule: a funnel needs idempotent workers and explicit backpressure. The platform can retry work on another instance. The application that receives the work is the narrow point.

A modulith already scales each application separately. Therefore “this application needs more capacity” is not a reason to split a modulith.

The gateway is the exception, and the order of the two problems matters. A split gives each new modulith a gateway with the same work. Move the work out of the entrypoint first. Then look again at the group.

1.3 Repository scaffold

The pyramid of section 1.2 gives this layout. One entrypoint at the top, and the applications below it.

PathContents
watt.jsonautoload over applications/, and entrypoint: "gateway"
applications/gateway@platformatic/gateway. It composes the applications and terminates authentication
applications/<application>A @platformatic/db application, a @platformatic/service application, or a @platformatic/node application where the code brings its own server
applications/<frontend>A web frontend, on @platformatic/vite today. A modulith can hold more than one. The gateway proxies each at its own prefix, and one of them can take /
packages/clientsTyped SDKs. Hey API generates them from the composed OpenAPI document of the gateway
packages/sharedTypes and helpers for more than one application
packages/cliProposed. It does not exist yet

A @platformatic/db application reads the PostgreSQL schema. It then gives a REST/OpenAPI interface and a GraphQL interface from that schema.

Rule: code that brings its own server is an application, and not a service beside the runtime. @platformatic/node runs a plain Node server as an application of the runtime, and @platformatic/vite does the same for a frontend. Therefore such code sits behind the gateway of its modulith, takes the identity headers of section 5, and moves between moduliths under the rule of section 1.1, the same as each other application.

Rule: reads use the generated surface. Writes are hand-written. A write has rules and effects that the row alone does not hold: storage and CDN state, provisioning steps, ACL stamps, and version rules. A generated INSERT keeps none of them. Therefore each entity switches the generated writes off with db.openapi.ignoreRoutes and db.graphql.resolvers.

Rule: the PostgreSQL schema is a published interface. Migrate it in two steps. The generated interface follows the columns, so a rename or a type change breaks each generated client with no change in any hand-written file. Add the new column, move each reader, then remove the old column in a later release.

1.4 Naming

nh- marks a shared library of the platform. No modulith name has that prefix. The design system uses arc- instead, because it is one product with its own name (section 4.8).

SlotFormExamples
Modulithone wordaccount, dam
Deployment<modulith>-modulithaccount-modulith
Databaseone database has one owner application. The name does not have to match that applicationaccount for auth, file for file
Package in a repository@goodhabitz/<repo>-<part>account-auth, dam-clients
Shared library@goodhabitz/nh-<family>[-<part>]nh-auth-acl, nh-fetch-core
Design system@goodhabitz/arc[-<part>]arc-tokens, arc-react

Do not use a reserved word as a modulith name. The generated clients make TypeScript symbols from it, and switch does not compile.

1.5 Platform infrastructure

The platform operates these centrally. An application uses them and starts none of its own.

What the platform operatesContents, or what it doesThe rule for using it
PostgreSQLThe system of record. One database per owner applicationData that you cannot lose
KeyValue, on ValkeyQueues, locks, cache entries, short-lived results. One namespace per user of the storeWork in progress, and data that you can build again
The object store, on S3Files, and each asset that a CDN delivers. One prefix per application in a shared bucket (section 11.1)A byte stream, and anything too large for a row
agentEach agent and each other piece of AI work, for each application that asks (section 3.14)Ask it. Hold no access to a model, and run no agent of your own

The first three are stores and the fourth is not. Each of the four is one thing that the platform runs, and an application that started a second one would hold state, a credential or a cost that no other application can see.

A store that an application reads at start limits the availability of that application. Section 3.4 gives the effect.

What the platform reads at start

Read at startEffect when it fails
The params snapshotThe application does not start (section 3.4)
The PostgreSQL schema of a @platformatic/db applicationThat application does not start, and its interface does not exist
The GraphQL schema of each composed applicationThe gateway does not start, so the modulith has no edge (section 8)
@platformatic/db-authorization, before nh-auth-aclThe application stops at start, not at the first request (section 6.4)
The registration of an event name in hookNothing. The producer starts, records the failure, and tries again (section 3.6)

Rule: a read at start is a hard dependency. Add one only with a reason. Each row above turns a fault in one component into a failed start of another. A value that an application can read later belongs on the request path.

1.6 The target map

One picture of the whole platform: each modulith, each application, and each platform service that they use at run time.

The map asserts no grouping. Section 1.1 leaves the modulith of each proposed application open, so the two moduliths that exist today hold their own applications, and each proposed application sits in a group of its own that names no modulith. A line appears only where a section of this document states it.

graph TB
    client([browser]) -->|"HTTPS, one origin"| EDGE["AWS ALB — a path for each modulith"]

    subgraph acct ["account-modulith"]
      direction TB
      AGW["gateway"]
      AFE["frontend"]
      AUTH["auth"]
    end

    subgraph dam ["dam-modulith"]
      direction TB
      DGW["gateway"]
      DFE["frontend"]
      FILE["file"]
    end

    subgraph open ["proposed — the modulith of each is open (section 1.1)"]
      direction LR
      PARAMS["params"]
      JOB["job"]
      HOOK["hook"]
      LIC["license"]
      CMS["cms"]
      LMS["lms"]
      PROG["progress"]
      ATT["attest"]
      NOTIF["notification"]
    end

    EDGE --> AGW
    EDGE --> DGW
    AGW --> AFE
    AGW --> AUTH
    DGW --> DFE
    DGW --> FILE

    subgraph plat ["platform services, in the cluster"]
      direction LR
      ZIT["Zitadel — the OIDC provider"]
      PG[("PostgreSQL — one database per owner")]
      KV[("KeyValue, on Valkey")]
      S3[("S3")]
      RP["Redpanda — no application uses it yet"]
      AGENT["agent — each agent, for each caller. Mastra runs inside it"]
      OTEL["Grafana Alloy — each application exports to it"]
    end

    AUTH -->|"verify a token"| ZIT
    AUTH --> PG
    FILE --> PG
    FILE --> S3
    PARAMS --> PG
    HOOK --> PG
    NOTIF --> PG
    PROG --> PG
    LIC --> PG
    CMS --> PG
    LMS --> PG
    JOB --> KV
    CMS -->|"publish a definition"| FILE
    LMS -->|"read a definition"| FILE
    LMS -->|"events and the structure"| PROG
    PROG -->|"a restore point"| LMS
    ATT -->|"read a stream, write a verdict"| PROG
    ATT -->|"read a definition"| FILE


    SUB(["a subscriber of a tenant"])
    PROV(["the billing provider"])
    DELIV(["the email and SMS providers"])
    MODEL(["the providers of a model"])
    HOOK -->|"a signed webhook"| SUB
    LIC -->|"read the state"| PROV
    NOTIF -->|"a rendered message"| DELIV
    AGENT -->|"a prompt, and a completion"| MODEL

Four things to read from it.

One origin reaches each modulith (section 7.1). The ALB routes on a path, so a browser sees one host whatever the grouping is.

A gateway is the only way in. Each application sits behind the gateway of its modulith, and the header spoofing row of section 9 makes an application unreachable from outside the cluster.

PostgreSQL is shared infrastructure with one owner for each database (section 1.5). The KeyValue store is shared with one namespace for each user of it, which is what lets job read across them (section 3.5).

Four lines leave the platform. hook posts a signed webhook to a subscriber of a tenant, license reads the state of the billing provider, notification hands a rendered message to an email or SMS provider, and agent sends a prompt to a provider of a model. Each other line stays inside the cluster.

2. The local development flow

2.1 One repository composes the stack

goodstack is the workspace repository. It runs no service of its own and publishes no package. It composes.

Rule: the stack repository holds no application code. A change to an application happens in the repository of that application. The stack repository holds the compose files, the devcontainer definition, the manifests, and a submodule for each repository.

2.2 Each repository arrives as a submodule

Rule: a submodule with a compose.yaml runs as a service. A submodule without one is there for source access.

SubmoduleRole
account, dam, arcEach carries its own compose.yaml, so the stack runs it as a hot-reload service against the bind-mounted source
packages/*Source access for the shared packages. No service
zitadelSource access for the Zitadel repository. No service
specsSource access for each specification of the platform, this document included. No service

A change in zitadel/ does not change what the stack runs. The Zitadel service comes from a pinned remote include, and the submodule is there to read and to change the code. This is the one place where the source in front of you and the service beside you are two different things.

Rule: never pin a submodule at the head of a merge-request branch. GitLab rebases a merge request when it merges, so that commit does not survive. Pin a tag, or the commit that landed on the default branch.

2.3 A pinned tag brings in each backing service

Rule: each backing service comes from its own repository, at a tag, through the remote include: of Compose. A version bump is therefore a tag change in one file.

ServicePurpose
caddyThe reverse proxy, and local TLS for *.local.goodhabitz.com
postgresThe databases of the applications
zitadel, zitadel-loginThe OIDC provider
mock-oidc, mock-samlExternal providers, brokered through Zitadel, for a test of single sign-on
rustfsS3-compatible object storage
cdn-edge-authThe CDN edge authentication for an asset
mailpitAn SMTP sink, which catches each outbound email
grafanaTraces and logs. Off by default, and one entry brings up its collector and its backends

Each service has its own host locally, and production has one origin. Section 7.1 gives one origin for a browser, and the local stack gives each application its own host behind Caddy. A cookie covers them because its domain is the parent. Therefore a cross-origin fault is visible locally and invisible in production, and an assumption of one origin is not testable locally.

2.4 The devcontainer is the tool container

The workspace container carries the toolchain, the CLIs and the tests. It does not run the applications: each application runs as its own service, against the bind-mounted source, as uid 1000, so a file that it writes is not owned by root. Hot reload comes from wattpm dev.

Rule: bring the stack up from the host, and not from inside the devcontainer. A compose up inside the container mounts an empty directory that root owns, and the fault arrives later as a permission error on a path inside a repository.

Rule: the repository root compose.yaml comes first in the devcontainer file list. Compose takes the project directory from the first file, and each relative path resolves against it. The order therefore decides what the workspace bind mount points at.

runServices lists what starts.

To debug with breakpoints, run the application yourself. A script stops the container of that application and starts a shim that points Caddy at the workspace, so the host and the session keep working. Watt enables the Node inspector only when the editor debugger starts it, therefore the run needs Auto Attach or a JavaScript debug terminal. The attach reaches the worker threads as well, so a breakpoint and hot reload work together.

2.5 Provisioning applies the Terraform of the deployed environments

nh stack apply runs in dependency order. It creates the PostgreSQL databases from databases.yaml and the buckets from buckets.yaml.

Rule: the local identity structure comes from the same Terraform as a deployed environment. The Zitadel plugin applies terraform/local with OpenTofu, and that configuration creates the organizations, the project, the OIDC application, the roles and the grants. Therefore the identity model of section 5 is the model that a developer works against, and not an approximation of it. It also creates the local login accounts and the service users of the applications, which never exist in a deployed environment.

Rule: Zitadel is not manifest-driven. Change the Terraform. A change to an organization, to a project, or to a redirect URI belongs in that configuration and not in the Zitadel console, because the next apply reverts a change made in the console.

The apply then writes its outputs into the .env of each application. The OIDC client id and secret reach both applications, and the service-user tokens reach account, which owns the management of an organization. Each application reads its own .env and must restart to take a new value.

An apply is idempotent, so a repeat re-applies unchanged Terraform and writes the same values. A full reset is one compose down -v from the host, which removes the Zitadel database and the OpenTofu state together.

This is the local half of section 11.1, with the same intent and a different mechanism. A resource is declared in a file that the team owns, and a tool creates it. The cluster has an operator and a Helm value, and the local stack has a manifest and a CLI. Therefore a new resource must appear in two places, and the two can drift.

2.6 What a developer needs once

StepWhy
A GitLab token with read_api and read_registryThe npm registry of section 4, and the container images of each service
One docker login on the hostThe devcontainer uses the Docker daemon of the host, so one login covers each pull from inside the container
A hosts entry for each service nameEach service has its own host under *.local.goodhabitz.com, and the browser must resolve it
The root certificate of Caddy, trusted on the hostThe devcontainer trusts it already. The browser of the host does not
git submodule update --initNothing else fetches the submodules

GETTING_STARTED.md in the stack repository holds the commands and the current detail of each step. This table is the set of things that a document about the architecture must state, because each one is a decision and not an instruction.

Rule: bump nh-cli to move each plugin. One package brings the plugins for each service, so one version change updates each of them. A service image moves by its tag in compose.yaml, and that needs a rebuild of the devcontainer.

Rule: @goodhabitz/* stays excluded from the pnpm cooldown. pnpm 11 refuses a package that is younger than a day, by default. A first-party package comes from our own registry and carries no age to test, so a release of the same day would be refused and the stack would run the previous version while the manifest claims the new one.

3. Application inventory

3.1 Shipped

ApplicationDomainModulith todayDatabaseResponsibility
authaccountaccountaccountIdentity, sessions, OIDC brokerage with Zitadel, user provisioning
filedamfileAsset metadata, S3 storage, CDN delivery

account is the only domain that is set today. The modulith today column shows where an application deploys now, not a property of it.

The auth row repeats account in three columns. That is a coincidence. Nothing makes a domain, a modulith, and a database share a name.

The table lists only applications that own part of the behaviour. Each modulith also has a gateway and at least one frontend (section 1.3).

3.2 auth — routes

The auth application owns the OIDC flow. Behind the gateway its routes have the prefix /api/v1/auth.

RoutePurpose
POST /loginStart the OIDC authorization-code flow with Zitadel
POST /logoutEnd the session and clear the cookies
POST /refreshRotate the session with the refresh token
GET /sessionGive the state of the session
GET /whoamiGive the identity and the roles of the caller

The gateway sets these routes to access: "skip". A caller must reach them without a token, because they are how a caller gets one.

Rule: only this flow reads or writes a token. Each other part of the platform uses the identity headers from the gateway (section 5).

3.3 Proposed applications

None of these exists. There is no repository, no database, and no code. This is a set of applications, not a deployment plan.

ProposedResponsibilityNote
cmsAuthoring: a definition, drafts, review, comments. It publishes to the asset storeSection 3.9
paramsConfiguration key/values, with toggles and experiments as richer selections on the same keySection 3.4
jobThe admin surface over the queue of each application. Each application runs its own jobs with nh-jobSection 3.5
lmsLearner portal: enrolment, delivery, classes, sessions. Its frontend is the shell of the learnerSection 3.10
progressLearner progress: the stream of events, the snapshot, and the completion, the score and the position that come from themSection 3.11. A write appends, so CRUD is the wrong shape
hookWebhook delivery to external services for NewHabitz eventsSection 3.6
licenseEntitlements, seat counts, term dates. Billing eventsSection 3.7. Kill Bill is the main candidate
attestIt says whether the progress of one learner is correct, from the stream and the definitionSection 3.13
agentOne place that runs each agent and each other piece of AI work, for each application that asks. Section 1.5 operates it centrallySection 3.14
notificationMessages to a person. It renders a template with the data of a caller and sends the result on a channel, for example email or SMSSection 3.8

The modulith for each of these is open (section 1.1).

3.4 params (proposed)

A key gives one typed value for one caller. Keys show hierarchy, so you can address a subtree. The delimiter is not chosen.

Toggles and experiments are the same key with a richer selection. They are not separate subsystems.

UseVariationsSelection
Configuration valueonenone. The value is the value
Configuration value, scopedone per rulea rule matches the targets of the caller
Toggletwoa rule matches
Experimenttwo or morerandom, then stable for that client

Availability

Rule: configuration is a hard dependency at start. An application that cannot read its configuration does not start. There is no local copy of the last good values.

Therefore params is a single point of failure for the start of the platform. The design does not remove this. Operate around it:

  • The availability target of params is the union of the targets of its dependents.
  • No single instance and no database failover can stop a platform start.
  • Dependents start after params, and a dependent that restarts during an outage stays down.
  • The start path of params depends on PostgreSQL only. More dependencies move the failure point.

The modulith of params is constrained, unlike the others. Each dependent reads params at start, so the edge of params is on the start path of the platform. Two limits follow. params must not share a modulith with an application that reads params at start. A read on the start path uses a service secret and not a user token, so a platform start does not depend on Zitadel.

Delivery

ReaderHow it gets values
A backend, at startIt reads a snapshot. Later changes arrive from a poll with an ETag, applied to a live value
A browser, at first paintThe server resolves the values and writes them into the HTML. See below
A browser, after thatIt fetches its resolved values again

Rule: a browser receives values, never rules. No targeting rule and no experiment assignment runs in a browser. Therefore each browser payload belongs to one caller, and only a private cache can hold it.

Authoring is a separate surface: CRUD over keys, variations, and rules.

Start values in the HTML

A frontend that fetches configuration before the first render adds a round trip. A frontend that renders first shows the wrong state. Therefore the values travel with the document: one shared library resolves a declared set of keys on the server and writes them into the head, and its browser half reads them at start and gives the values synchronously.

Rule: declare the set of keys. Never send all of them. The client and any cache of the page can read anything in the HTML.

Rule: a shared cache must not hold a page with scoped values. Keep scoped keys out of the set, or match the cache scope to the values. Section 7.3 gives the same rule for API responses.

The server half writes from the snapshot that the application already holds. Therefore a render does no configuration I/O and cannot fail on it.

The write path

Rule: each key declares a type, and the store validates each write. An untyped store makes each consumer parse the value again, and a bad value then fails at read time in production.

Rule: record each change and give a one-step return to the previous value. A configuration change has the effect of a deployment without the process of one.

Rule: secrets do not go in params. Any caller that a rule matches can read a value. Consumers hold values in memory and on disk. The audit log keeps old values. Secrets stay in the deployment path. Put this rule in the README of the application as well.

Experiments

The server assigns a bucket once and carries it in a cookie. Resolution is then a filter on a bucket column, so the platform needs no evaluation engine. A bucket identifies a client and not a person, so one person can sit in two arms and a client that refuses cookies is never counted correctly.

Rule: an exposure record goes to telemetry, and not to a database. “Bucket B saw variation V of key K” is a high-volume append-only write that the resolution path never reads. The application emits it with nh-otel (section 10), and analysis happens in the telemetry backend. This adds no table, no growth in the database of params, and no write on the read path.

Rules select on the nh-auth targets of the caller, for example t:<tenant>. There is no target kind for entitlements today (section 5.4).

3.5 job (proposed)

One application is the admin surface over the queue of each other application. It runs no work and creates none. nh-job (section 4.5) is the package that each application uses to hold a queue and run its handlers.

Rule: job reads and manages each queue in the KeyValue store. Each application writes its queue into its own namespace (section 1.5), and job works across those namespaces. It calls no application, so it needs no secret for one.

This does not break the boundary rule in section 7.1. That rule covers a database, which one application owns. The KeyValue store is shared platform infrastructure, and a namespace in it is not a private database.

Two effects follow, and both help. job shows and manages the queue of an application that is down, because the work is in the store and not in that application. And one page costs one read and not one call for each application, so the fan-out problem does not arise.

Rule: job restarts a job. The owner runs it. A restart returns the record to the pending state, and the worker of the owner application then picks the work up. Therefore job needs no handler and no code of any application, and the authority for the work stays where the work lives.

Three rules come with that.

A restart goes through nh-job, for the same reason as a read.

A restart of a complete job clears the record of that id. The id is the idempotency key, so a repeat gives completed and the cached result until something removes it. A restart is therefore a decision to do the work again, and not a retry of a failed attempt only.

Each restart records the operator. The run acts as the stored subject (section 4.5), so a restart has two identities: the subject of the work, and the person who asked for the run. Both belong in the record. Security row deferred impersonation gives the reason.

job does not create work. A new job comes from the application that owns it, so the surface of job covers a queue that exists and nothing more.

Schedules

Watt holds a scheduler array at the root of watt.json. Each entry has a name, a cron expression, and a callbackUrl, and can also set method, headers, body, enabled, and maxRetries. The runtime makes one HTTP request when the cron fires. It does nothing more.

Rule: a callbackUrl points at the application that owns the work. The route then acts at once when one attempt is enough, or enqueues locally when the work needs a retry, deduplication, a concurrency limit, or recovery of a stopped run.

The scheduler fires once or more, after a retry or a change of coordinator. Therefore a route on a schedule must tolerate a repeat. A route that enqueues gets this from the id: derive the id from the scheduled time, and not from the clock at handler time. The same trigger then gives the same id, and a repeat gives duplicate. A route that acts at once must do the same check itself.

A missed schedule stays missed. The scheduler keeps no state across a restart, so there is no catch-up. Work that must be known to have run needs a check.

A schedule that calls through a gateway also needs a credential.

Section 3.6 sets this scheduler beside the cron of hook, and states which work belongs to each.

Availability

No application depends on job, at start or on a request. job is an operator surface, so an outage of it hides the work and stops none of it.

The store behind a queue is a single point of failure for the deferred work of that application. The reaper recovers a job only when the store survives.

3.6 hook (proposed)

One application delivers webhooks to external services for events in NewHabitz. An external service subscribes to a queue. hook then posts each event to the URL of that subscriber.

@platformatic/pg-hooks gives this. It is a @platformatic/db application, not a library, so it fits the scaffold in section 1.3 without a wrapper. It stores queues, messages, and cron entries in PostgreSQL. It elects one leader with a PostgreSQL advisory lock. The leader delivers each message and runs each cron entry. Each other process can create a queue and store a message.

SurfacePurpose
POST /queues/Create a queue with a callbackUrl, a method, headers, and maxRetries
POST /messages/Enqueue an event for a queueId, with a body
POST /cron/Repeat a message for a queueId on a schedule

It also gives delayed delivery, automatic retries, and a dead letter queue.

Storage, and the difference from a job queue

hook uses PostgreSQL. A job queue uses KeyValue (section 4.5). Both choices follow the rule in section 1.5. A webhook is a promise to a third party, so the platform cannot lose it. Queued internal work can be built again.

Two schedulers, two purposes

The platform has two schedule mechanisms. They are not alternatives.

MechanismPurposeWhere it runs
The Watt scheduler array (section 3.5)Start internal work on a schedule. It calls the application that owns the workIn each application that owns scheduled work
The POST /cron/ surface of hookRepeat an outbound delivery to a subscriber on a scheduleIn hook only

Therefore do not replace one with the other. A time trigger for internal work belongs in the Watt scheduler. A repeated message to an external URL belongs in the cron of hook.

The two also differ in durability. The Watt scheduler keeps no state across a restart. The cron rows of hook are in PostgreSQL with an elected leader, so they survive a restart. Section 3.5 states what a caller must do about a missed internal schedule.

The delivery path is one leader. One process delivers each message for a database. That bounds throughput, and it makes the health of the leader an operational concern.

Signed requests

Rule: sign each request with a published standard. Do not write a scheme. A receiver must be able to check that an event came from NewHabitz. A receiver must also be able to use a verification library instead of its own code. The package gives no signature, so hook adds one.

The standard is Standard Webhooks. It fixes the headers webhook-id, webhook-timestamp and webhook-signature, and it fixes the signed string: the id, the timestamp and the body, joined by a full stop. Therefore a receiver uses a library and needs no knowledge of our choices. Its adopters are the services that our subscribers already receive webhooks from, so many subscribers hold a working verifier now.

Standard Webhooks allows two algorithms, and the version prefix of the signature names which one signed it. Use ed25519, which is v1a. The platform signs with a private key and publishes the public key. A receiver needs no secret from us, and a rotation needs no exchange with a subscriber. The platform already publishes keys for Zitadel, so the pattern is familiar here. HMAC-SHA256, which is v1, stays available for a subscriber that cannot do ed25519.

Three requirements come with the choice:

  • Give each message a unique id, and reject a repeat of one. The signature covers the id, so a receiver that records each id it has accepted has the whole defence against a repeated delivery. The id is the same value that section 3.6 makes stable for an at-least-once delivery.
  • Reject an old timestamp. The signature covers the timestamp, and a window bounds how long a captured request stays useful.
  • Allow two active keys. A rotation needs an overlap.

The signature binds the message, and not the destination. A subscriber that serves more than one endpoint therefore reads the event name to know what arrived, and it does not take the endpoint as proof of that.

The producer contract

A queue of the package holds one callbackUrl, so one queue is one subscriber. Therefore the fan-out needs an owner.

Rule: a producer posts one event. hook does the fan-out. A producer must not read subscriptions and must not know a subscriber.

hook therefore adds a subscription table above the queues of the package. A row joins a tenant, an event name, and one queue. A post of an event reads the rows that match the tenant and the event name, and then writes one message for each queue. Two effects follow. A subscriber manages a subscription, which is the surface that a subscriber expects. A producer sees one call, and the number of subscribers does not change it.

Rule: an event name is <domain>.<entity>.<action>. Three parts, in lower case, with a singular entity and an action in the past tense. An example is license.entitlement.granted.

Each part earns its place: the domain gives the owner, the entity names the thing that changed, and the action says what happened to it. The past tense is the part that matters most, because an event reports a fact and a name in the present tense reads as an instruction. A name cannot change later, so each of these small rules is permanent from the first release.

Rule: each payload carries a version, and a subtype or a reason goes in the payload. An event is a published interface. A payload that follows the row shape of the producer makes each schema change a breaking change for each subscriber. The payload can change under its version. The name cannot change at all.

Rule: a published name never changes. A subscription holds the name, and a rename gives a subscriber no event and no error. Therefore a new meaning gets a new triple, and the old name runs until each subscriber moves. The version in the payload covers a change of shape, and it does not cover a change of name.

Two things follow for the catalogue. A producer that stops an event must remove the name deliberately, because a registration at start never removes one. The catalogue must show how many subscriptions hold a name, because a removal is safe only at zero.

Rule: the producer owns each event name in its own domain, and declares it at start. The registration is idempotent, so a restart and a rollback are safe. Therefore the catalogue needs no central registry, and a new name needs no release of hook. A release pipeline is the other place for this step, and it becomes the right place only if hook starts to reject an undeclared name.

Rule: a failed registration does not stop the start of the producer. The producer records the failure and tries again. This keeps hook off the start path of each producer (section 1.5). An unregistered name stops a new subscription, and it does not stop an event.

A subscription to a name that no producer declared is an error at that moment. This is where a wrong name must fail, because a subscriber that waits for an event that never comes gets no message and no error.

Rule: an event carries ids and a tenant, not a full row. A subscriber then reads the current state through the API. This keeps an event small, and it removes the leak of a field that the subscriber must not read.

Tenant scope

Rule: a subscriber receives events from its own tenant only. A subscription belongs to one tenant. hook delivers an event to that subscription only when the tenant of the event matches the tenant of the subscription.

This needs no new mechanism. The subscription, queue, message, and cron rows take the ACL columns from section 6. The tenant target t:<tenantId> then filters them. Therefore one rule covers a read of a subscription and a delivery to it.

The callbackUrl needs an allowlist

A tenant gives that URL. See the a webhook destination row of section 9.

Rule: treat delivery as at-least-once. The documentation does not state the guarantee, and retries exist. Therefore tell each receiver to make its handler idempotent, and give each event a stable id.

Rule: give the dead letter queue an owner and an alert. A queue that nobody reads hides each failed delivery.

Authentication

@platformatic/pg-hooks gives its own authentication for its role rules. The platform uses nh-auth only (section 5.5). Therefore hook needs a re-wire before adoption. nh-auth-user assembles the identity, and the ACL from section 6 guards the subscription, queue, message, and cron entities.

This is not extra work for hook alone. It is the same wiring as each other application. The tenant scope above is that ACL.

3.7 license (proposed)

One application owns entitlements, seat counts, and term dates. A billing platform holds the subscriptions and the invoices behind it, and Kill Bill is the main candidate. That platform is not a Platformatic application, so license is the face of it inside the platform.

What license exposes:

SurfacePurpose
A read of the entitlement state of a tenantThe question that a delivery or a seat check asks
A platform event on a change of that stateEach subscriber then reads the new state. Section 3.6 gives the name and the payload
A route that receives a change from the providerThe one inbound path. Nothing else reaches it

Rule: license is the only reader of the provider. No other application calls the provider, and no other application holds a credential for it. Therefore a change of provider is a change in one application, and section 7.1 stays true for each caller.

Rule: the entitlement state that the platform reads lives in license. A read on the request path must not call the provider. license holds the current state in its own database, and the inbound route updates it. Therefore the availability of a delivery does not depend on the availability of the provider, and a slow provider does not slow a page.

Rule: license owns the map from a tenant of the platform to a tenant of the provider. The provider has its own multi-tenancy, and the two id spaces are separate. A wrong map crosses tenants in a billing record, so this map is a security surface and not a configuration value.

This is the pattern that section 5.4 describes, with a second provider. auth maps a Zitadel org and a Zitadel subject to an internal id already, so license follows a shape that the platform has rather than a new one.

Rule: a change from the provider enters through license, and leaves as one platform event. license updates its own state and then posts that event to hook. A direct path from the provider to a subscriber avoids the event catalogue, the tenant filter, and the signature of the platform.

Rule: the inbound route is idempotent. The provider retries, so the same change can arrive more than once. A repeat must not give a second platform event.

Rule: the provider authenticates as a Zitadel machine identity. nh-auth-jwt verifies that token the same as any other, and nh-auth-user gives it a real identity with declared targets. Therefore this path needs no exception to section 5.5.

The database of the provider is not the database of license. Section 7.1 forbids a query across a database, and this is the same rule with an external system on the other side.

The credential that license uses to call the provider is an outbound credential for an external system. Section 5.5 governs authentication into the platform, so it is no exception to it.

3.8 notification (proposed)

One application sends a message to a person. A caller posts the data, a template id, a channel, and a recipient. notification renders that template with that data and delivers the result on that channel. Email and SMS are the first two channels.

SurfacePurpose
A send of one messageThe data, a template id, a channel, and a recipient. The application renders and delivers
A read of the state of one sendAccepted, delivered, or failed, for one message id
Authoring of a templateCRUD over each template and each version of it

Rule: a caller sends data and a template id. A caller never sends a rendered body. The words of a message then have one owner, so a change of wording is a new version of a template and not a release of each caller. A caller that sends a body also takes each constraint of the channel on itself: the length of an SMS, the markup of an email, and the escape rules of both.

Rule: a template declares its channel. An email and an SMS of the same event are two templates, because the length, the markup, and the fields differ. A caller therefore names a template and a channel that agree, and a mismatch is an error at the request and not a bad message at the recipient.

Rule: a template declares the fields that it needs, and a render validates the data against them. An absent field must fail at the request. A renderer that prints an empty value sends a message with a hole in it, and the platform learns of it from the recipient.

Rule: the data of a render is escaped for the channel. A value from a caller reaches a person, so an HTML email needs HTML escaping and a link needs URL escaping. This is the injection class of section 9, on a path that leaves the platform.

Delivery

Rule: a send is accepted and then delivered, and the delivery runs in notification. A provider is slow and fails, so a caller must not wait for it. The application takes nh-job (section 4.5), holds its own queue, and runs its own handlers, the same as each other application with asynchronous work.

Rule: the caller gives the id, and it derives that id from the work. Section 3.5 gives the reason: the id is the idempotency key, so a client retry that carries the same id gives one message and not two. A person receives a duplicate when this rule is broken, which is the failure that is visible outside the platform.

Rule: notification is the only holder of a credential of a delivery provider. No other application calls a provider. Therefore a change of provider is a change in one application, and a new channel is a change in one application as well. This is the rule of section 3.7 with a second kind of provider.

Rule: a recipient belongs to the tenant of the caller. The message, the template, and the record of a send take the ACL columns of section 6, and the tenant target filters them. A caller that could name any address turns one API into a way to send mail as NewHabitz to anyone.

How this differs from hook

Both applications deliver something outside the platform, and they are not alternatives. hook posts a signed event to a machine that subscribed to it (section 3.6). notification renders words for a person who did not subscribe to anything. The subscriber of hook chooses its own URL and verifies a signature; the recipient of notification reads an email or an SMS.

Therefore a producer that wants both posts one event to hook and one message to notification. A path from an event of hook to a message of notification is possible later, and it is not needed for either application to work.

3.9 cms (proposed)

One application authors what a learner runs. RFC 006 gives its design, and RFC 001 gives the model of what it writes.

Rule: the domain model is specified once, in RFC 001. This document says “domain object”. The model is not agreed, so a name or a structure asserted here is a decision that the business has not made, and four readers would then hold it. This document states what the platform needs from a definition and fixes nothing else. Each name arrives with the model.

Rule: the definition is the artifact. A domain object leaves cms as one JSON document, and each reader takes that document. A row of a content store is not the artifact. Therefore a change of content store is a migration of one application, and no reader changes.

Rule: the definition is a published interface, and it carries a version. lms delivers it, nh-play builds a machine from it (section 4.7), progress records against it (section 3.11), attest rebuilds a result from it (section 3.13), and each report reads it. A change to the shape reaches each of those five, so section 1.3 applies: add, move each reader, then remove.

Rule: a save validates the definition against the published schema. An invalid definition reaches a reader that cannot repair it.

Rule: a published version does not change. An edit is a new version. A learner runs against one version, and section 3.11 checks a result against the machine that the version builds. A version that changes under a learner therefore turns an honest run into a refused one. This is the rule of a frozen event name in section 3.6, with a document in place of a name.

The host and the editor

RFC 006 splits this application in two parts. A content store gives authentication, roles, persistence, workflow states, revision history and delivery. A composer gives the editor of that definition, and it mounts inside the administration surface of that store as one custom field.

Rule: the composer owns the definition. The host owns each concern around it. A structured document needs a visual editor, because such a document in a text area is not a surface that an author can use. Each other concern of an authoring tool is general, and RFC 006 names the candidates that give it.

What the host must giveWhy
Workflow states past a draft and a publicationA review by the legal team and a pending translation are states of a definition
Localization of a whole documentA locale carries a whole definition (below)
A custom field that mounts a React component, with the locale, the state and the permissions of the callerThe composer is that component
A field that cannot change after a creationThe id of a domain object
An audit trail of each change, with its author and its timeA definition is a record of the business

Rule: a locale can change the structure of a definition, and not only its words. The definition of one language can differ in structure from the definition of another. Therefore each locale carries a whole definition, and a translation is not a table of strings. A delivery gives the default locale when a locale is absent.

Rule: the composer runs alone as well. It reads the catalog for the views that it can offer, and it reaches its host through a thin adapter. The editor is then testable without that host, and a change of host is a change of the adapter.

3.10 lms (proposed)

One application brings a definition to a learner: enrolment, delivery, classes and sessions. Its frontend is the shell of the learner, and RFC 002 gives the design of that shell.

Rule: the machine holds the authority over navigation, and the URL follows it. The shell runs play (section 4.7), and each route comes from the state of its actor. A learner who types a route that the state does not allow gets no such view: the actor refuses the intent, and the browser returns to the route of the current state.

A router that owns the URL turns each rule of access into a guard beside it, and a rule in two places is a rule that disagrees with itself. The gap is a way into a view that a learner did not earn.

Rule: an entry check runs before a delivery opens. The check reads the enrolment of the learner, and it reads an active seat from license (sections 3.7 and 7.5). A refusal returns the learner to the dashboard through the same state machine, and not through a redirect beside it.

Rule: the shell owns the URL of the page. play owns the route inside a delivery. The shell holds the identity, the session, the URL of the browser and the read of a definition. It hands that definition and the progress of the learner to play, and play returns each event and each new state (section 4.7). Two of them in one page that both write the URL give a back button that leaves the delivery.

Rule: lms reads a definition. It never writes one. cms is the only writer (section 3.9). Therefore what a learner runs is the published document, and no delivery path can change it.

Rule: a session opens at the restore point that progress gives. lms reads that point when a learner returns, so the learner continues where they stopped and does not begin again. The point is the progress structure and never the stream, and never a snapshot of a machine (section 3.11).

3.11 progress (proposed)

One application holds the progress of each learner. RFC 005 gives its design. It holds three things.

What it holdsThe authority for
The enrolment: a learner, a definition, and the version of that definitionWhich definition each read and each write applies to
The stream of events: each event of the machine, in order, and never changedAnalysis of behaviour, and the training of a model
The progress structure: completion, a score and a position, with a version of its own. It is the transformed snapshotWhat a learner keeps, and what each reader sees

Rule: progress holds no state of a machine. play transforms a snapshot into the progress structure before it sends one, and transforms the structure back when it restores (section 4.7). Therefore the store holds nothing that names a state of one version of one machine, and no reader of it becomes a reader of that machine.

Rule: a write appends. The stream is the record of how a learner reached a state, so a write adds an event and changes none. The progress structure is a projection of that stream, and it is the only row that a write replaces.

One path in, two paths out

SurfaceWhat crosses it
A synchronizationA batch of events, the progress structure that follows them, and a sequence number
A resumeThe version of the definition, and the progress structure to restore from
A read of progressCompletion, a score and a position

Rule: a synchronization carries a sequence number. A client sends a batch, and a network repeats a batch. The number makes a repeat idempotent, and it makes a gap visible. This is the rule of the job id in section 4.5, with a batch of events in place of a job.

Rule: progress takes the batch and the structure that the client sends. The sequence number above makes the write idempotent, and that is the whole of the default path. play holds the machine, so the result that it reports is the result that the platform keeps.

Rule: progress records. It does not judge. A check on the write path costs each learner the time of a replay, and it needs the definition, which progress does not read. Section 3.13 gives attest, which asks that question after the fact.

Rule: an enrolment names the version of the definition, and a resume compares it. A resume gives the progress structure and the version that it belongs to. play builds a snapshot from that structure and starts the machine (section 4.7). A version that differs from the enrolment is therefore not a failure of a resume: the structure holds no state of a machine, so it crosses a version, and the comparison tells play which machine to build.

Rule: each measure of behaviour comes from the stream. Time on a task is the interval between the entry of a view and the event that leaves it. A struggle is a run of failed attempts at one step. An abandonment is a session with no event after it. A counter in the write path answers one question, and a stream answers a question that nobody has asked yet.

The stream feeds analysis and a model. These are the readers that the shape of the stream must satisfy, and they are the reason that the platform keeps each event rather than a count. Section 3.12 sets the retention of the stream against that use.

Undecided: how the stream reaches the data platform. Two transports fit, and the platform runs the parts of each already. Change data capture reads the PostgreSQL of progress and emits each row, which section 11.1 gives for a pipeline that keeps its own copy. A message bus has progress publish each event, which section 4.6 gives, with Redpanda as the candidate. This is an improvement on top of the application and not a condition of it, so progress ships before the answer.

3.12 Retention and erasure (proposed, work in progress)

Section 3.11 holds a record of a person that a write never changes, so an erasure and a retention period each need an answer. Both apply to progress first, and to each other application that keeps a record of a person.

The requirement is settled and the mechanism is not. The three statements below are requirements: each one follows from a law or from a rule that this document already holds. Each undecided statement after them holds one option, and the business has given no answer.

Open choiceThe option that this section holds
How an erasure removes a person from a record that a write never changesIt deletes the rows
Whether the stream expires, and after how longIt expires on a schedule, and the period is not set
Which application drives an erasure across the othersNot chosen

The requirements

Rule: an erasure reaches each store that holds the person. A learner appears in auth, in progress, in notification and in the log of a delivery. Each of those is a separate database (section 7.1), so an erasure is one operation of the platform and not one query.

Rule: a projection is rebuilt from the records that remain. The progress structure comes from the stream (section 3.11), so a rebuild after an erasure must not restore what the erasure removed. No backup of a projection reaches production.

Rule: an erasure is idempotent. It arrives more than once, and it runs as deferred work with the id of the request as its idempotency key (section 4.5).

A count and a measure survive an erasure. An aggregate that names no person is not a record of that person, so a report that counts a completion keeps its value after the rows behind it are gone.

The open choices

Undecided: an erasure deletes the rows of that person. The stream of section 3.11 is partitioned by enrolment, so the record of one learner is a set of rows that no other record reads, and a delete of them changes no other record. An append-only stream is a rule for a writer of that stream, and it is not a promise to keep a row after the person that it names asks the platform to remove it. A store that a delete cannot reach needs a different answer, and that answer is not written here.

Undecided: the stream carries a retention period, and the progress structure does not. The stream answers a question about behaviour, and that value falls with age. The structure is what a learner keeps. The period itself is not set, and the length of it is a decision for the business and not for this document.

Undecided: one application drives an erasure across the others. Each store holds part of the person, so something must call each of them in an order and record that it did. Which application owns that is not chosen.

3.13 attest (proposed)

One application says whether the progress of one learner is correct. It reads the stream and the progress structure from progress, reads the definition from dam, and writes its result back to progress. It is the only application that reads on both paths of section 13.

Rule: attest rebuilds the result from the stream. It builds the machine from the definition of the version that the enrolment names, replays each event of the stream, transforms its own result with the conversions of nh-play (section 4.7), and compares that structure with the one that progress holds. An equal pair is an attestation. A difference is a refusal, and it names the first event where the two parted.

Rule: three things line up, or a check refuses an honest learner. The version of the definition, the version of nh-play, and the sequence of the stream. A machine built from another version applies another set of rules, so a comparison then measures a difference of input and not a difference of result. Section 3.9 fixes a published version, and section 4.7 pins the package.

Rule: attest writes a verdict and nothing else. The stream and the progress structure keep the one writer that section 3.11 gives them. An application that could correct a result on its own authority would be a second source of truth for what a learner did.

Rule: a check is opt-in, and a delivery declares it. A certified assessment is the case that needs one. Each other delivery keeps what play reports and pays nothing.

A check runs after the fact, so no learner waits for it. progress accepts a batch and answers the client, and attest reads later. Therefore a slow check, a long stream and a busy dam each cost an operator and never a learner.

3.14 agent (proposed)

The platform runs one agent, and it is infrastructure in the sense of section 1.5: each application asks it for the result of an agent or of a workflow, and no application runs an agent of its own. It holds the access to a model, the tools that an agent may call, the memory of a conversation, and the record of each run.

It is infrastructure that deploys as an application. Section 1.5 gives three stores that a cluster component serves, and this one is a Platformatic application in a modulith. params is the same shape already: each other application depends on it, and it is an application (section 3.4). Therefore the rules of section 1.1, of section 1.3 and of section 5 hold for it with no exception, and it is one thing to operate rather than one for each caller.

Rule: agent holds the credential of a model provider. No other application calls a provider, and no other application holds a key for one. Therefore a change of provider or of model is a change in one place, and a cost, a rate limit and a retry each have one owner. This is the rule of section 3.7 and section 3.8 with a third kind of provider.

Rule: an agent acts as the caller, and never with more authority. The identity headers of section 5 reach agent, and each tool that calls another application forwards them with the generated client (section 7.2). The row filter of section 6 therefore applies to what a tool reads, and an agent reads nothing that the person who asked could not read.

Rule: a tool is an HTTP call to the application that owns the data. Section 7.1 holds here with no exception. An agent that read a database directly would be a second reader of a store that one application owns, and the row filter would not see it.

Rule: a run that a person waits for and a run that takes minutes are two surfaces. A model call is slow and it fails. A caller that waits therefore gets one attempt and a timeout that it chose, and a caller of a long run gets a handle at once and reads the result later.

The workflow of the framework carries a long run. Mastra gives a workflow as a graph with control flow, so the steps of one run need no queue beside it. This application is therefore the one place where section 4.5 is not the answer for asynchronous work, and a framework that already holds the model, the tools and the memory of a run is the right holder of its steps.

Undecided: what a long run survives. A restart of the application, a step that fails, and a run that waits for a person each need an answer, and the guarantees of that workflow decide it. Where the workflow carries a run across a restart, this application needs nothing more. Where it does not, nh-job (section 4.5) is the queue that does, and it holds the run and not the steps.

Rule: each run leaves a record. Which agent, which model, which version of the instructions, which tools it called, what it received and what it returned. The record goes in the PostgreSQL of the application (section 1.5), because a question about what an agent did arrives after the fact, and a trace of section 10 does not last long enough to answer it.

Rule: the memory of an agent is data of a tenant. Each row of a memory, a thread and a record of a run takes the ACL columns of section 6, and the tenant target filters them. A memory that crosses a tenant gives one customer the words of another.

Rule: an instruction of an agent is configuration with a version, and not a string in a release. A change of wording changes what the platform does, so it needs the record and the one-step return of section 3.4, and each record of a run names the version that ran.

A domain authors an agent

Rule: a domain authors its own agent, and agent installs it. An agent is a package of the repository that owns that domain (section 1.4), and the application declares a dependency on it. Therefore the instruction, the tools and the scorers of an agent live beside the data and the people that they concern, and one application still holds the model access, the memory and the record for each of them.

Rule: a package declares. The application runs. A package gives the declaration of an agent, of each tool and of each scorer. It holds no credential of a provider, no store and no server. Therefore a team writes an agent with no key of its own, and the rule above keeps its one owner.

A tool of such a package usually calls the application of its own domain, and it does so through the generated client, as each other caller does (section 7.2). The rule above still applies: the tool acts as the caller of the run, and the row filter of section 6 sees it.

A new agent is two merge requests

One in the repository of the domain, which publishes the package. One in mastra, the repository that holds agent, which adds the dependency and pins the version.

This is the shape that section 2.2 gives for a submodule and section 11.2 gives for the chart: what composes names a version, and a change of what it composes is a deliberate step. The cost is that an agent reaches production through a release of agent, and the gain is that a reader of one manifest sees each agent that runs.

A change of wording needs neither. The rule above makes an instruction configuration with a version, so a team changes the words of an agent through section 3.4 and waits for no merge request. The two merge requests are for a new agent, a new tool, and a change of code.

What Mastra gives

Mastra is an open-source TypeScript framework, and it is the candidate for this application. It gives an agent with its instructions, its model and its tools in one declaration; a workflow as a graph, for work with more than one step; a tool with a schema; a memory with semantic recall; retrieval over documents; and a scorer for an evaluation. It reaches many providers of a model.

Rule: the framework brings its own server, so @platformatic/node runs it (section 1.3). Therefore this application sits behind the gateway of its modulith, it takes the identity headers of section 5 like each other application, its modulith is the open choice of section 1.1, and the platform gains no component to operate. This is the shape of section 3.6, where a package of Platformatic becomes the application, with a framework in place of that package.

Undecided: the store for a vector. Semantic recall and retrieval each need one, and each other part of this application needs only PostgreSQL. An extension of the PostgreSQL that section 1.5 already operates adds no component, and a store of its own is a fourth store to run. Neither is chosen, and an agent with no recall and no retrieval needs neither.

4. Package inventory

The platform publishes these to the GitLab npm registry under @goodhabitz. An install needs a read_api token.

Rule: name the package that owns the code. Do not use an umbrella package. A symbol with two names does not tell a reader where it lives. A family below is a group, and not a name to depend on. A dash means the package stands alone.

PackageFamilyPurpose
nh-auth-corenh-authThe overlaps check and the shared options
nh-auth-jwtnh-authIt verifies the Zitadel token and gives the claims
nh-auth-contextnh-authIt owns the header names and mints them at the edge
nh-auth-usernh-authIt assembles request.user from headers or a secret
nh-auth-forwardnh-authIt builds the outbound headers of a call
nh-auth-httpnh-authThe Fastify guard
nh-auth-graphqlnh-authThe mercurius guard
nh-auth-aclnh-authRow filters for Platformatic DB
nh-auth-fetchnh-authIt renews a session in a browser
nh-auth-urqlnh-authThe same, for a GraphQL client
nh-auth-sessionnh-authIt reads the session of the caller from auth. The browser counterpart of nh-auth-context
nh-auth-testnh-authFixtures and helpers
nh-fetch-corenh-fetchfetch middleware for a browser and for Node
nh-oclif-corenh-oclifThe base commands and the document streaming of a CLI
nh-oclif-authnh-oclifThe login and the session of a CLI
nh-otelOpenTelemetry. Each application registers it
nh-configBase TypeScript, oxlint, and oxfmt configuration
nh-job (proposed)The queue of an application, and what job reads (section 4.5)
nh-bus (proposed)One interface over each message bus (section 4.6)
nh-play (proposed)play, the common player: a machine from a domain object, and progress from a snapshot (section 4.7)
arc-tokensarcThe design tokens, from Figma. No runtime dependency, so any framework reads them
arc-stylesarcOne import that brings Tailwind, the tokens, and the map that binds them to a component
arc-reactarcThe Arc components for React, one entry point for each
arc-vuearcThe same components for Vue 3, from the same source
arc-iconsarcThe Arc icons, as SVG path data
arcarcThe shadcn registry and the Storybook host. It exports cn

Each of the nh-auth, nh-fetch and nh-oclif families also publishes an umbrella package and a documentation package. The rule above forbids a dependency on the umbrella. The table lists each package that holds code, and those are the names to depend on.

Section 5.1 gives the same nh-auth packages by layer, because the identity model needs the order in which they run.

4.1 nh-fetch

SurfacePackageBehaviour
createFetch(middlewares)nh-fetch-coreIt combines middlewares into one function with the shape of fetch
createDedupe()nh-fetch-coreIdentical requests at the same time share the promise of the first. Each caller gets its own readable clone
createLock()nh-fetch-coreMutual exclusion with the Web Locks API. Each caller still makes its own request and waits its turn
createCrossTab()nh-fetch-coreOne document makes the request and the others use its response
createWhen(rules)nh-fetch-coreIt applies a sub-chain to matched requests only, by origin, method, path, and accepted version

createWhen is a filter, not a router. It does not change a path or a host.

nh-fetch-core has no runtime dependency, and nothing in it needs Node or a framework.

Rule: put createDedupe above createLock and createCrossTab. Put anything that reads a body above them as well.

4.2 nh-oclif

SurfacePackageBehaviour
BaseCommand, FormattedCommandnh-oclif-coreBase commands with --output and --format
documentStream, bodyStream, fromFile, fromStdin, fromArgs, isStdinPipednh-oclif-coreJSON document streaming from a file, from stdin, or from a flag. It reads NDJSON and an array batch
documentArg, idArgnh-oclif-coreArguments that exclude stdin
toOutput, OclifErrornh-oclif-coreOutput plumbing and the error type
auth loginnh-oclif-authAn oclif plugin for the OIDC login of a CLI: a browser flow with an authorization code and PKCE over a loopback redirect, a device grant, or client credentials
readAuth, writeAuth, clearAuth, authPathnh-oclif-authThe session of one CLI on disk
startLoopbacknh-oclif-authThe loopback listener for the browser flow

nh-oclif-auth builds its authenticated fetch on nh-fetch-core and nh-auth-fetch. It renews the session on a 401 and writes the rotated refresh token under a lock that holds across processes. Section 9.3 gives the rules for that middleware.

4.3 nh-otel

SurfacePackageBehaviour
The default exportnh-otelThe registration of OpenTelemetry for one application
redactionSpanProcessorPluginnh-otelA Fastify plugin that removes a value from a span
The redaction corenh-otelThe rules that the processor and the plugin share

Each application registers this and exports OTLP to PLT_OTEL_URL. Section 10 gives the destination.

4.4 nh-config

SubpathPackageContents
./tsconfignh-configThe base TypeScript configuration
./oxlintnh-configThe base oxlint configuration
./oxfmtnh-configThe base oxfmt configuration

nh-config has no bare entry point. Each repository takes the subpath that it needs.

4.5 nh-job (proposed)

One package gives an application its own queue. It wraps @platformatic/job-queue, owns the layout of the keys and the shape of the records in the KeyValue store, and gives one API to enqueue, to read, and to manage. Section 3.5 is the application that reads what it writes.

Rule: a job runs in the application that created it, and an application enqueues into its own queue only. @platformatic/job-queue runs work in in-process handlers, so each application with asynchronous or long work takes this package, holds its own queue, and runs its own handlers. No application writes a job into the queue of another: a caller that needs deferred work elsewhere calls that application, and that application decides whether to defer. Section 7.1 stays the one rule for a call between applications.

This places the handler. It does not limit the reach of the work. A handler calls another application with the generated client of that application (section 7.2), the same as a route does. Therefore a job in one application can drive an operation in another, and that other application needs no queue of its own.

Three results follow. A handler reaches the code of its own application directly, with no HTTP hop. No application holds authority to run a handler inside another. A move of an application to another modulith takes its queue along and changes nothing else.

What each queue gives

@platformatic/job-queue gives the queue, retry with backoff, deduplication, a concurrency limit, and a reaper for a stopped worker. The Watt scheduler gives the time trigger (section 3.5). nh-job adds the parts that the platform needs and the package does not give: the layout in the store, the identity model below, and the read and manage API that job uses.

@platformatic/job-queue takes a storage implementation. An in-memory store loses work at a restart, and a filesystem store serves one node only. Therefore nh-job uses the platform KeyValue store (section 1.5), which holds state of this kind and adds no infrastructure, with one namespace for each application.

Rule: the durability of queued work is the durability of the configured store, and no more. Work that must leave a durable record writes that record to the PostgreSQL database of the application.

Rule: nh-job owns the layout of the keys and the shape of the records. Both are a contract between each writer and job. nh-job writes them and reads them, therefore no application and no UI parses a key by hand. A change of layout is then one release of nh-job, and not a search across each repository.

The job id is the idempotency key

The caller gives the id. An active id gives duplicate. A complete id gives completed and the cached result.

Rule: derive each id from the work. A random id turns a client retry into a second run.

The cached result expires after resultTTL. Therefore a late repeat is a new run.

Identity

The request that enqueued the work is complete before the handler runs. Therefore the handler cannot replay the forwarded headers.

Rule: to act as the user, store the subject at enqueue time and derive the authority again at run time. Do not store a credential. A credential in a queue outlives its request, and a retried job outlives its expiry.

Rule: a job that loses its authority fails. The derivation reads the roles that are current at run time. When those roles no longer allow the work, the job fails and goes to the dead letter queue. It does not run with reduced targets, because a smaller result set looks like a completed job and hides the permission change.

A handler that calls another application forwards that identity, the same as a route does (section 7.2). Therefore the derived identity crosses an application boundary, and the receiver cannot tell a job from a request. That is deliberate: one identity model covers both.

4.6 nh-bus (proposed)

One package gives one interface over each message bus. An application publishes with emit(message) and subscribes with on(topic, listener), and a message is an object with a topic. mqemitter is the library below it.

This extracts a pattern that ships already. auth holds a provisioning event bus on that library: it publishes each provisioning event on a topic, and any plugin subscribes to one event or to a wildcard without knowing that an identity provider exists. @platformatic/sql-events uses the same library with its Redis transport for a GraphQL subscription, so the dependency is in the tree. Therefore each application that wants a bus repeats the wiring today, and each one chooses its own topic shape.

Rule: the transport is configuration, and not code in an application. A change of bus is then a configuration change, and a second bus is a second configuration.

CaseTransport
A test, and one processThe in-memory emitter of the library. No broker, and no cluster for a local run
More than one process, todayThe Redis transport, against the Valkey of section 1.5. It needs no new infrastructure, and @platformatic/sql-events runs this pair already
A bus with no transport in the ecosystemWrite one. Kafka is that case, so a Redpanda transport is an extension of this package (section 11.1)

Rule: the interface promises what the weakest transport gives. A transport is where each guarantee of a bus lives, and the library states none of its own. Therefore a consumer must tolerate a repeat and must not assume an order.

Rule: a topic is the event name of section 3.6, and the separator is .. The library defaults to / and takes the separator as an option, so one naming rule covers a webhook and a topic: three parts, in lower case, with a singular entity and a past-tense action. The wildcards then give a subscription to a domain or to an entity for no extra work.

Rule: a bus carries an event. A call stays HTTP. Section 7.1 is the rule for a request that waits for an answer, and this package does not change it. A request over a bus is a remote call that no trace follows.

Rule: a consumer derives its authority at run time. A message arrives with no request behind it, so the rule of section 4.5 applies: store the subject with the message, and derive the authority again when the consumer runs.

4.7 nh-play (proposed)

One package gives play, the common player. It wraps the Play architecture of @xmachines/*, and it adds the two conversions that belong to this platform.

ConversionDirection
A domain object to a machineThe definition builds an XState machine
A snapshot and the progress structureA snapshot gives the structure, and the structure builds a snapshot

The two conversions carry the same weight. Each one is a boundary between a form that the business owns and a form that a library owns. The second conversion is what keeps a record that outlives the machine, because a store that holds only a snapshot holds nothing that does.

Rule: the conversion between a snapshot and the progress structure runs in both directions, and the direction back is lossy. The structure holds no part that belongs to one machine: a state name, the internal queue of events, and the part of a context that no id of the definition keys. Therefore a snapshot that comes from the structure is correct for a start, and it is not the snapshot that was written.

The definition and the progress structure stay the two records that the platform keeps. A machine and a snapshot stay derived.

Rule: each conversion is a function that runs in a browser and in Node. It needs no actor, no view and no document of a browser. Therefore a server runs the same function that play runs, which is what lets attest rebuild a result with no frontend (section 3.13).

Rule: two sides that compare a result run the same version of this package. The check in section 3.11 holds only where both sides build the machine and the structure the same way. A difference of version between them therefore reads as a difference of result, and an honest client is refused. Pin the version that the server runs to the version that play ships.

Play separates three concerns. An actor holds the logic as an XState machine and knows nothing of a framework. A runtime adapter binds that actor to a browser, to Node, or to a router. A view reads a snapshot and renders. A route derives from the state of the actor rather than from its own configuration, so the logic keeps authority over navigation.

Rule: nh-play gives play. A frontend gives the view. Section 1.3 allows more than one frontend, and a player that each frontend writes again is a player that each frontend gets wrong differently. Therefore the machine, the two conversions and the route derivation live in this package, and a frontend adds the components.

A machine comes from a domain object

Rule: the machine is derived from the definition, and not written for it. A definition is the document of section 3.9, which cms authors and lms delivers. nh-play reads that definition and builds the machine. A machine that an author writes for each definition makes a new domain model a release of play, and it holds the rules in two places.

Rule: the definition holds the structure and the rules. The package holds the conversion. The domain model is not agreed (section 3.9), so a structure or an order inside play is a decision that the business has not made. Therefore the definition declares how a learner advances and when each part of it is complete, and nh-play builds the states and the transitions from those declarations.

Two things follow. A new way to advance is a new value in the definition, and not a release of play. And RFC 001 is the document to change when the model is agreed, because it holds the shape.

Therefore the domain model is an input of this package, and a change to it reaches play. Section 1.3 makes a published schema a two-step migration, and this is the same obligation with a different reader.

Progress comes from a snapshot

A snapshot is the state of one version of one machine: each state name, the context, and the internal queue of events. The progress structure is what a learner keeps: completion, a score and a position (section 3.11).

Rule: progress owns the progress structure. nh-play converts a snapshot to it, and a structure back to a snapshot. The two directions are the whole interface between play and the store, and each one has its own moment.

DirectionWhen it runsWhy
A snapshot to the progress structureAfter each transition, with the batch that section 3.11 synchronizesIt gives the read surface, each report and each export a form that no machine owns
The progress structure to a snapshotOn each resume, because a resume gives the structure and not a snapshot (section 3.11)It starts the machine where the learner stopped

Rule: the conversion is keyed by the ids that the definition declares. The structure holds the state of each part of a definition by the id that the definition gives it, and it holds no state name and no path of a machine. A name and a path belong to one machine, so a structure that holds either cannot cross a version.

Rule: the progress structure carries a version of its own. The structure outlives each machine that wrote it, the same as the payload of an event in section 3.6. A reader of the structure therefore needs no knowledge of the definition, of the machine, or of the library below it.

A view comes from the catalog

The catalog is the vocabulary of play: a name for each view, and a schema for the properties of that view. A state of the machine names a view, and it builds those properties from its context.

Rule: the catalog holds each name and each schema. It holds no component. A view is then data: a name and a set of properties. A test drives a whole definition in Node and reads each view as data, with no browser and no component (section 12). A frontend supplies the component for a name, and it builds that component on Arc (section 4.8).

Rule: a render validates the properties against the schema. A view arrives from a machine and a component renders it, so the two agree only where something checks. A failure at that boundary is a clear error in one place, and not a missing value in a page.

An adapter fits the environment

An adapter binds the actor to the place where it runs. Each route of play comes from the state of the actor, and the adapter decides where that route goes.

Where play runsEach route goes to
Inside the shell of lmsMemory. The shell owns the URL of the page (section 3.10)
Alone: a development run, a test, or an exportThe URL of the browser

Rule: an embedded play writes no history of the browser. One that writes there takes the URL of its host, and a back button then leaves the delivery. Therefore the environment chooses the adapter, and the machine, the conversions and the views stay the same in each environment.

4.8 arc

One repository holds the design system: the tokens, the components, the icons, and the registry that publishes them. Storybook is at arc.goodhabitz.dev, with a build for each framework.

Rule: a frontend takes its components from Arc. Section 1.3 allows more than one frontend, and a component that each frontend writes again is a component that each one gets wrong differently. This is the rule of section 4.7 for a view, one level below it.

LayerWhat it gives
The tokensCSS variables, a Tailwind theme and preset, typed objects, and constants for React Native. They carry no runtime dependency, so a framework outside this list reads them
The componentsOne source in Mitosis generates the package of React and the package of Vue, so the two do not drift
The registryThe shadcn registry at arc.goodhabitz.dev. It is content-negotiated: the CLI receives the registry document, and a browser receives Storybook

Rule: a frontend depends on the package of its own framework. arc-react and arc-vue hold the components, and arc holds the registry and cn. This is the rule of section 4 against an umbrella package, with a second reason: a dependency on the root gives a frontend no component.

Rule: a Tailwind build points @source at the package of the framework. Tailwind 4 generates a class only where it reads one, and it does not read the dist of a dependency. Therefore an import of arc-styles gives the tokens and the variable map, and the @source line is what makes each class exist.

5. Identity and authorization

Rule: only the edge verifies the token. An application behind a gateway trusts the headers from that gateway and never verifies a JWT again.

sequenceDiagram
    autonumber
    participant C as client
    participant GW as modulith gateway
    participant SVC as db application
    participant PG as PostgreSQL
    C->>GW: request + access_token cookie
    GW->>GW: nh-auth-jwt — verify signature and audience with Zitadel
    GW->>GW: nh-auth-context — remove client copies, mint identity headers
    GW->>SVC: forward x-user-sub, x-user-email, x-user-roles, x-user-targets, x-tenant-id, x-session-id
    SVC->>SVC: nh-auth-user — assemble request.user
    SVC->>SVC: nh-auth-http / nh-auth-graphql — overlap the needed targets
    SVC->>PG: nh-auth-acl — add acl_effective && ARRAY[...]
    PG-->>SVC: permitted rows only
    SVC-->>GW: response
    GW-->>C: response

5.1 Package map

Section 4 lists these packages as an inventory. This table gives the order in which they run.

LayerPackageResponsibility
Corenh-auth-coreThe overlaps check and the shared AccessOptions
Tokennh-auth-jwtIt verifies the Zitadel JWT and gives accessClaims and idClaims
Headers, writernh-auth-contextIt owns the header names and mints them at the edge
Headers, readernh-auth-userIt assembles request.user from headers or a service secret. It owns deriveTargets
Relaynh-auth-forwardgetHeaders() relays the service secret, the incoming credential, and the identity headers
Guardsnh-auth-http, nh-auth-graphqlverifyTargets for Fastify. nhAuthDirectiveCheck for mercurius
Row ACLnh-auth-aclRow filters for Platformatic DB (section 6)
Client sessionnh-auth-fetch, nh-auth-urqlThey renew an expired session once (section 9.3)
Client sessionnh-auth-sessionIt reads the session of the caller from auth, and tells a revoked session from a signed-out one
Testingnh-auth-testFixtures and helpers

Rule: a gateway uses context, not forward. The two names give the opposite impression, because a gateway does forward each request. context writes new headers from a verified token. forward only relays a credential that already exists.

The error is quiet. A gateway with forward verifies the token and then forwards nothing from it. Each application sees no identity, so the ACL denies every row and responses are empty. An integration suite calls applications directly and does not detect this.

5.2 The header contract

nh-auth-context owns these names and removes each client copy before it mints its own.

HeaderContents
x-user-subThe Zitadel subject
x-user-emailThe verified email
x-user-rolesTenant-scoped roles, as <tenantId>:<role>
x-user-targetsThe derived target strings
x-tenant-idThe Zitadel org id
x-session-idThe session identifier

5.3 Token transport and access modes

The access token arrives in an Authorization: Bearer header or in the access cookie. The header wins when both are present. Bearer is the only accepted scheme. The id token comes from its cookie only.

nh-auth-jwt gives each path in routes[] one mode:

ModeMeaning
requiredReject the request without valid claims
verifyVerify a token when present, admit the request when absent
skipNo token handling

The session cookies

auth sets four cookies. Each attribute comes from configuration.

CookieContentsPath
sessionThe state of the OIDC flow: the state, the nonce, the code verifier, and the return target. @fastify/secure-session encrypts it/
access_tokenThe access token/
id_tokenThe id token/
refresh_tokenThe refresh token/api/v1/auth/refresh

The three token cookies set HttpOnly. Each of the four sets Secure, a SameSite value, and a Domain value.

SameSite is none. A frame of another site withholds a lax cookie, and that blocks the OIDC flow. The platform runs in a frame everywhere, so none is the only value that works.

Rule: the platform controls each subdomain of the cookie domain. Each subdomain of that domain receives the cookie. Therefore content from a tenant, from a third party, or from an external host goes on a different domain. The cookie domain row of section 9 gives the effect of a breach of this rule.

Rule: the OIDC flow works inside a frame. A top-level redirect leaves the frame, therefore the flow needs a top-level navigation or a separate window. The state cookie needs the same attributes as the token cookies, because the flow fails when the browser withholds it, and the failure looks like a lost state and not like a cookie problem.

Two of the suggestions in section 14.1 follow from SameSite=none.

5.4 Target grammar

TargetMeaning
u:<sub>The principal by name. Absent when only a secret gave the authority
t:<tenantId>Tenant membership
r:<grantingOrgId>:<role>One per real role, scoped to the org that gave it

Each target arrives in two id spaces. nh-auth-context mints the set above from the verified token of the provider. It mints a second set from the account_id and tenant_id values that account writes into the user metadata of Zitadel, which the token carries as a claim. Therefore x-user-targets names the caller twice: once with the ids of the provider, and once with ours.

The second set adds and never replaces. A caller that account never provisioned holds no metadata and gets no second set. A role takes our tenant id only when the org that gave it is the org of the caller, because the metadata holds the id of that org and of no other. The Root org gives super-admin and is not a tenant, so a grant from it keeps the id of the provider.

A row therefore names a principal in one space or the other. An overlap is a set operation, so a row that holds either id is reachable. Each table names the space it holds, and no table depends on the space another one holds.

request.user is a ForwardedUser with sub?, email, tenantId, roles, and targets. roles is always ['user'], so no value from the network can replace it.

A target kind for entitlements does not exist. A fourth kind would change nh-auth-user, the header contract, and each acl_defaults value already written.

Rule: a route that is not an entity route must call extractForwardedUser(request). Without that call request.user has no value.

The internal id and the external id are separate columns

auth holds this pattern already. A row has id, which is our own GUID, and external_id, which is the id that an external system minted. tenants.external_id is the Zitadel org id, accounts.external_id is the Zitadel subject, and sessions.external_id is the Zitadel session id.

Rule: a foreign key uses the internal id. An external id is a lookup column. An external system owns its id space, and it can change the shape of it. A key that points at an external id then carries that risk into each table that joins to it.

Rule: each external system gets its own mapping table. auth shows this with a second one: tenant_provisioning and account_provisioning hold the id of the system that drives provisioning. Therefore a third system needs a third table and no change to the first two.

Rule: a row names our id where it holds one. auth does this. An account row names u:<accounts.id> and t:<accounts.tenant_id>, a tenant row names t:<tenants.id>, and a session row names u:<sessions.account_id>. A column of ours needs no lookup and no mapping table, and it keeps the id space of a provider out of a filter. A row that holds no id of ours names the id of the provider, which external_id gives.

A column entry reads one column of the row and cannot follow a reference (section 6.2). Therefore the id must be on the row itself. Where a table names its tenant by a foreign key alone, that key is our tenant id, which is exactly what the second target space matches.

A row can detach, and a later row can take the same external id. Uniqueness therefore applies to the live rows and not to each row, so the history of one external id reads as equality on one string.

Section 3.7 gives the same pattern for a second provider.

5.5 Authentication between applications

A security review must read this section.

Rule: nh-auth-user handles the secret. A guard never does. Its secrets option declares a header, the accepted values, and the full ForwardedUser that a match mints, including its targets.

The forwarded-headers strategy runs first and the secret strategy second. Therefore a backend that sends a secret and forwarded identity acts as the user. A call with a secret only gets the identity of that application.

After the assembler there is one check: overlap the needed targets against request.user.targets. Therefore one guard covers people and applications, and no enforcement point reads a secret.

Rule: nh-auth is the only authentication mechanism. This holds for a caller from outside and for a call between applications. There is no second path, and no mechanism gives a caller a bypass of the row filter.

Therefore a package that gives its own authentication needs a re-wire before adoption. Move its secret to the secrets option of nh-auth-user, and guard each entity with the ACL from section 6. Section 3.6 gives one case.

Rule: a secret is for one callee, and the callee declares its targets. A caller of more than one application holds one secret for each of them, and each of those secrets gets the targets that the callee needs and no more. No secret holds a platform-wide target.

A single secret with a target for each callee is much larger when it leaks, because one value then acts against each application. Therefore accept the extra secrets.

Section 14.2 gives two suggested improvements to this mechanism.

6. Row-level ACL

nh-auth-acl filters rows with a PostgreSQL array overlap between the ACL of the row and the targets of the caller. It reads request.user and never reads headers. It needs @platformatic/db-authorization to load first.

6.1 Columns

ALTER TABLE assets
ADD COLUMN acl_defaults TEXT[] DEFAULT '{}',
ADD COLUMN acl_share TEXT[] DEFAULT '{}',
ADD COLUMN acl_effective TEXT[]
GENERATED ALWAYS AS (COALESCE(acl_defaults, '{}') || COALESCE(acl_share, '{}')) STORED;
CREATE INDEX assets_acl_effective_gin ON assets USING GIN (acl_effective);

PostgreSQL generates acl_effective, and queries match that column only. Never write to it. All three columns are hidden from the generated OpenAPI and GraphQL surfaces.

Rule: the two writable columns take no NOT NULL. Platformatic types a NOT NULL column as required on a save, so NOT NULL here makes each writer state the value that the plugin exists to compute, and a writer that states it owns it (section 6.3). Null and empty mean one thing to a read in any case, because an overlap against null is not true. Therefore such a row matches no caller, which is what an empty ACL means. Keep the default, so a row written outside the hook holds the empty array.

Rule: the generated column coalesces each side. Concatenation reads one null side as empty. Two null sides give NULL, and acl_effective is then not an array. An overlap against null is not true, so a read behaves on such a row. IS NULL and = '{}' answer differently, and a caller that reads the column receives null where it expects a list. COALESCE keeps the column an array in each row.

6.2 The overlap includes the action

The overlap uses <target>:<action> strings, not bare targets. Each entity declares the entries that a new row gets:

{
"name": "@goodhabitz/nh-auth-acl",
"options": {
"writeSurfaces": "reject",
"entities": {
"asset": {
"entries": [
{ "type": "literal", "kind": "r", "value": "{PLT_ROOT_ORG_ID}", "role": "super-admin", "verbs": ["find", "save"] },
{ "type": "column", "kind": "u", "from": "ownerId", "verbs": ["find", "save", "delete", "share"] },
{ "type": "column", "kind": "t", "from": "tenantId", "verbs": ["find"] }
]
}
}
}
}

An entry names one principal and the verbs that principal gets. type says where the value comes from: literal holds it in the configuration, and column reads it off the row. The kind, the role and the verbs come from the configuration alone. Therefore the same row gives the same ACL, whoever writes it, and a write that carries no request gives the same ACL as a write that carries one.

Several entries may name one principal, and their verbs then join. "*" stamps the target with no action, which the overlap reads as each action.

Rule: a column entry names a column that holds the id of a principal. A local key that points at another row of ours is such an id, because a target arrives in our id space as well (section 5.4). A column entry reads one column and cannot follow a reference, so the id must be on the row. A column that holds neither gives a well-formed entry that matches nobody, and no check can catch that: the plugin cannot know which of your columns holds an id.

At start the plugin registers one user-role rule per entity, registers a save hook that stamps acl_defaults from the row, and adds fastify.acl.

A find for subject S in tenant T, whose metadata gives our ids A and N, runs:

SELECTFROM assets
WHERE acl_effective && ARRAY[
'u:S','t:T','r:T:user','u:S:find','t:T:find','r:T:user:find',
'u:A','t:N','r:N:user','u:A:find','t:N:find','r:N:user:find'
]

The array holds the caller twice, once per id space, so a row that names either is reachable (section 5.4).

Reads and writes are separate actions. Therefore a tenant can read a row and cannot write it. One array without actions cannot show that difference.

6.3 Writes and sharing

An overlap cannot filter an INSERT. Therefore:

  • Update and delete add an overlap on the save or delete action. A caller changes only rows that give it that action.
  • Insert uses the save hook, which stamps acl_defaults from the row. The caller does not give the ACL.

The hook runs on each save of a guarded entity, and it needs no request. Therefore a write on a trusted path gets the same ACL as a write from a person: a provisioning job, a service account and a login callback each give what the row names. A write that states acl_defaults itself owns the column, and the hook then stamps nothing.

Rule: a save that carries a request may not set the ACL columns. The hook removes them and records that it did. This holds for each caller, including one that a secret authenticated: an exemption from the rules is not a licence to write these columns.

Treat that removal as defence in depth and not as a boundary. Only a save carries the hook, so an insert does not, and a caller that gives its own primary key takes a path that the rule does not guard. Therefore a guarded entity must keep its generated write surfaces closed. The plugin reports each one that stays open, by entity, at start. writeSurfaces: "reject" turns that report into a failed start.

Rule: an entity with no configuration is refused, not allowed. @platformatic/db-authorization wraps each entity, and one with no rule denies each request that carries a ctx. Absence is therefore safe, and it reports nothing. A table that only an operator may reach declares one literal entry for the platform role instead, which puts it under the report above.

acl_share is the part that can change. fastify.acl gives authorize, grant, shares, revoke, allowed, columnsFor and recomputeDefaults:

await fastify.acl.authorize(request, { entity: 'asset', action: 'share', id })
await fastify.acl.grant(request, {
entity: 'asset',
id,
target: { kind: 'user', id: granteeSub },
actions: ['find']
})

You cannot revoke a default. A share never changes acl_defaults.

The entries of an entity are a function of each row, so a whole table rebuilds from the current configuration in one statement:

const changed = await fastify.acl.recomputeDefaults({ entity: 'asset' })

It writes only the rows that differ and answers how many, so zero means that nothing had drifted and a second run changes nothing. It is how a table with rows from before the ACL gets them, and how a table follows a change of configuration. Expose it to an operator and never to a person: account gives POST /admin/acl/recompute/:entity behind the target of the platform role.

6.4 Failure modes

SymptomCause
fastify.platformatic has no value at startThe plugin loaded before @platformatic/db-authorization. A guard stops the application at start, not at the first request
Each query gives no rowsrequest.user has no value, or the rows have an empty acl_effective. An absent x-user-sub gives the anonymous role. An empty acl_effective shows rows from before the ACL, which recomputeDefaults fills (section 6.3)
One entity gives no rows, and the rest give rowsA column entry reads a column that holds no id of a principal, so each row names a principal that does not exist. Section 6.2 gives the rule
A row is reachable by an operator and by nobody elseThe row was written before the entries of its entity named a principal of the caller. Rebuild the entity
The application stops at start with a report of write surfacesA guarded entity still exposes a generated route or mutation, and writeSurfaces is reject. Close the surface, or take the warning
A literal entry stops the startIts value is blank. An environment variable that the configuration names must hold a value, because a blank segment is not a target

7. Calls between applications

7.1 One convention

Rule: each call from one application to another is HTTP with a URL.

The runtime enforces this. Each application runs in its own V8 isolate with its own module registry and heap. Therefore there is no shared object and no import across the boundary. The boundary has the shape of a network interface, so it continues to work across a real network.

Colocation changes the transport only:

Caller and calleeTransportAddress
Same runtimeThe in-process mesh. Worker threads carry HTTP over MessagePort. No TCP port, no TLS, no DNShttp://<application>.plt.local
Different runtimesHTTP or HTTPS to the gateway of the calleeThat gateway origin, and the prefix of the application

The mesh is a transport improvement, not a different model. It serializes each body, and it cannot cross a process.

Rule: an application reads the data of another application through the HTTP surface of that application only. No query crosses a database, and no credential gives one application access to the database of another. A shared database removes each benefit of the boundary and keeps the cost.

This rule covers a database. The KeyValue store is shared platform infrastructure (section 1.5), so a read of another namespace in it is not a breach. Such a read needs the library that owns the layout of those keys, and not knowledge of the keys. Section 3.5 gives the one case.

Therefore there is no transaction across two applications. Work that changes data in two applications is eventual, and it needs a stated pattern: the caller repeats an idempotent write, or the owner enqueues the second step in its own queue (section 3.5). Choose the pattern in the design, not at the first failure.

Rule: the browser sees one origin, whatever the grouping is. A path selects the modulith, and one edge in front of the gateways routes on that path. Therefore a browser makes no cross-origin call, the platform needs no CORS between two moduliths, and a regroup changes no origin in a frontend.

Section 5.3 gives the cookie attributes and what they mean for a session.

Therefore a move of an application relocates code and deployment, not the contract. A caller changes a baseUrl. Four things move with the application:

What movesEffect
Its databaseA @platformatic/db application owns a database, and ownership moves with it
Its place behind a gatewayIt arrives behind a different edge and gets the route policies of that gateway
Its schedulesThe scheduler array is at the root, so entries and their callbackUrl values must move
What the mesh gave at no costAcross a network the caller must add TLS, a credential, a retry policy, and a timeout policy. This part of a split is work, not configuration

7.2 Use the generated clients

Rule: do not write a raw fetch call to another application. Use the generated client of that modulith. Hey API generates it from the composed OpenAPI document of the gateway, so one package covers each application that the modulith exposes.

import { createXClient } from '@goodhabitz/<modulith>-clients/<application>'
import { getHeaders } from '@goodhabitz/nh-auth-forward'
const client = createXClient({
baseUrl: 'http://<application>.plt.local',
headers: getHeaders(request)
})

baseUrl holds the colocation and is the only value that changes after a regroup. The code of the caller is the same in both cases.

Rule: build the outbound headers with nh-auth-forward. Do not write them by hand. getHeaders relays the service secret, the incoming credential, and the identity headers, so the call acts as the user. The header names are the contract of nh-auth-context (section 5.2), and a hand-written set goes stale when that contract changes.

7.3 Use the HTTP cache

Rule: use HTTP cache semantics before you write a cache inside an application.

The Watt cache is an undici interceptor below fetch, so it caches the outbound requests of a caller. Each call is HTTP (section 7.1), so one layer covers mesh calls, calls between moduliths, and third-party APIs. A cache inside an application serves one caller, needs manual invalidation, and repeats per worker.

ConcernHow it works
Who decidesThe producer. Cache-Control on the response, and s-maxage, because this is a shared cache
Which originshttpCache.origins is an allowlist. Nothing is cached until you add the origin. cacheByDefault gives a fallback TTL — use it rarely
Where entries liveIn memory per runtime by default. Applications in one runtime share the store
A shared cacheValkey, through httpCache.store. Entries survive a restart and replicas share them. Each read costs a network hop
InvalidationgetInvalidateHttpCache() from @platformatic/globals, by exact key or by tag. Tags travel in cacheTagsHeader

Tag invalidation lets a write path expire the entries that it changed. That need is the usual reason for a cache inside an application.

Rule: a shared cache must not hold a response that depends on the caller. Row-level ACL filters by the targets of the caller, so every ACL response belongs to one caller. Declare private or no-store, or keep the origin out of httpCache.origins. An error here gives the rows of one tenant to another, with no error message.

7.4 Traffic today — shipped

The gateway of dam composes the GraphQL schema of account from PLT_ACCOUNT_ORIGIN. That is the only link between two moduliths. No application calls another over HTTP yet.

7.5 Proposed traffic (proposed)

CallerCalleeProtocolPurpose
cmsdamPOST RESTPublish a definition
lmsdamGET REST/GraphQLRead a definition
lmsprogressPOST RESTSynchronize a batch of events and the snapshot that follows them
lmsprogressGET RESTRead the snapshot and the version of a definition, to resume
attestprogressGET RESTRead the stream and the progress structure of one learner
attestprogressPOST RESTWrite the verdict of a check
attestdamGET RESTRead the definition of the version that the enrolment names
any applicationagentPOST RESTAsk for the result of an agent or of a workflow
agentany applicationGET RESTA tool call, as the caller (section 3.14)
lmslicenseGET RESTCheck an active seat
licenseauthPOST RESTUpdate entitlements after a billing event
any clientaccountPOST /api/v1/auth/refreshRenew a session after a 401

8. Gateway composition

Composition uses @platformatic/gateway. Each entry in applications selects REST composition, GraphQL composition, or a plain proxy.

{
"gateway": {
"applications": [
{ "id": "admin", "proxy": { "prefix": "/admin" } },
{ "id": "frontend", "proxy": { "prefix": "/" } },
{
"id": "file",
"openapi": { "url": "/documentation/json", "prefix": "/api/v1/file" },
"proxy": { "prefix": "/api/v1/file", "routes": ["/*"] },
"graphql": true
},
{
"id": "account",
"origin": "{PLT_ACCOUNT_ORIGIN}",
"graphql": { "graphqlEndpoint": "/graphql", "composeEndpoint": "/graphql" }
}
],
"graphql": { "graphiql": true }
}
}
  • Frontends. A proxy entry sends a prefix to a frontend. More than one frontend can sit behind one gateway, so each needs its own prefix, and the entry for / must come last. A first match wins, and / matches each path.
  • REST. The gateway reads each OpenAPI document, mounts it at the prefix, and serves one merged specification. Hey API generates the clients from that document.
  • GraphQL. The gateway reads each schema at start and merges them into one endpoint. An entry with origin composes a remote modulith over HTTP.
  • A nested field across two applications needs entity configuration. A merge gives one endpoint. It does not join the types.

8.1 The version is in the prefix

Each route of an application sits behind /api/v1/<application>, which ships today. That prefix is the version of the HTTP surface, and it is separate from the version of a payload in section 3.6 and from the version of a schema in section 1.3.

The three rules below are proposed. No second prefix exists, so nothing has run this path yet.

Rule: a change that a client cannot ignore is a new prefix, and the old prefix keeps working. The gateway composes both, so /api/v1/<application> and /api/v2/<application> are two entries of the same application. Hey API then generates a client for each, and createWhen of section 4.1 selects by the accepted version. Therefore a caller moves when it chooses to, and a release of the callee does not break it.

Rule: an additive change stays in the prefix that it is in. A new field, a new route and a new optional parameter break no client, and a new prefix for each of them gives two surfaces to keep and no reader a reason to move.

Rule: a prefix that no caller uses is removed. Two live prefixes are two paths to guard, to test and to keep in the ACL. The telemetry of section 10 shows the last caller of a prefix, so a removal is a fact and not a hope.

8.2 Introspection

A gateway that exposes GraphiQL also exposes introspection. A text match on __schema is not enough, because a caller can avoid it. Mercurius also skips a preValidation hook after a query-cache hit, so a guard there does not run.

Rule: detect introspection from the parsed AST in a Fastify hook.

9. Security considerations

Rule: name a row, and do not number it. Each row below has a name, and each reference in this document uses that name. A row that arrives in the middle then renames nothing.

9.1 Shipped

Each row here applies to the platform as it runs today.

PropertyWhat to check
Header spoofingnh-auth-context removes client identity headers before it mints its own. A path that avoids the gateway avoids that step. An application must not be reachable from outside the cluster. An integration suite that calls an application directly tests an unprotected surface, so a mis-wired gateway can pass CI
Secret scopeA service secret mints an identity with configured targets. A platform-wide caller crosses tenants, so control rotation carefully. Check that each secret declares the narrowest target set that its caller needs (section 5.5)
ImpersonationA secret with forwarded identity keeps the caller as the user. This is deliberate. A compromised backend can act as each user whose request it handles
Anonymous is deniedAn absent x-user-sub gives the anonymous role with no targets, so the overlap matches nothing. The result is no rows, not all rows, and there is no error message. A broken identity chain looks like absent data
Rows from before the ACLA row with an empty acl_effective is invisible to each caller. A rebuild is a correctness requirement for a table that gets ACL columns after it holds data (section 6.3)
Plain routesA route without extractForwardedUser has no user, and the route then decides the result (section 5.4)
Token audienceThe access token and the id token verify against separate audiences. One audience for both lets one token replace the other
Cache scopeA per-caller response marked cacheable in the shared cache reaches the next caller. Check private, no-store, or an absent origin (section 7.3)
IntrospectionSection 8.2
Refresh-token rotationZitadel rotates the refresh token at each use and rejects the previous one. Two refreshes from one cookie state log the user out. createDedupe below the refresh transport prevents this
The cookie domainEach session cookie sets Domain to goodhabitz.com or goodhabitz.dev, so each subdomain of that domain receives it (section 5.3). The platform controls each subdomain, which is what makes this safe. Therefore the check is on a change: a new subdomain that serves content from a tenant, or from a third party, or from an external host, gets a path to each session
CSRF with SameSite=noneThe configured value is none, so a cross-site top-level POST carries the session of the user, and no repository in the stack holds a CSRF plugin. Credentialed CORS at the gateway is not a defence. Section 5.3 gives the answer: an Origin check at the gateway, for a cookie request only

9.2 With a proposed application

Each row here arrives with the application that it names, and none applies today.

PropertyIt arrives withWhat to check
Configuration is not a secret storeparamsRules make values readable, consumers hold them on disk, and the audit log keeps old values (section 3.4)
A webhook destinationhookA tenant gives a callbackUrl, so a tenant can point the platform at an internal address, at plt.local, or at a cloud metadata endpoint. Check for an allowlist of destinations and for egress control. This is a server-side request forgery path (section 3.6)
A webhook tenant filterhookA subscriber receives events from its own tenant only (section 3.6). The filter is the ACL overlap on the tenant target. Check that each event carries a tenant, and that a queue row cannot match an event from another tenant
Deferred impersonationA queueA handler mints an identity from a stored subject, so it can act as any user, and it can forward that identity to another application (section 4.5). Check four limits: the derivation reads the roles that are current at run time, and a job that lost its authority fails; an enqueue caller cannot set a subject that is not its own; each run records the session that enqueued the work; and a restart from job records the operator beside that subject, because a restart makes one person cause a run as another
The inbound path of licenselicenseThe provider authenticates as a Zitadel machine identity (section 3.7). Check the targets of that machine user, because it can write entitlements
A result that a client reportsattestA check is opt-in (section 3.13), so the default path keeps what play reports. Check that each delivery whose result must hold against a learner who controls the environment has one, that the setting is a property of the delivery and not of the client that asks for it, and that attest writes a verdict and never a result
An instruction that arrives as dataagentA model reads what a tool returns and what a person typed, and it cannot tell either from its own instructions. A row of one tenant, a document of a customer and a field of a form each reach the model as text. Check that a tool call carries the identity of the caller and passes the row filter (section 3.14), that a tool which writes needs a target that the caller holds, and that no tool takes a URL or a host from what a model produced
Erasure of a learnerprogressSection 3.11 holds a stream that a write never changes. Check that an erasure removes the person from each of the three stores of that section, and that a rebuilt projection does not restore what an erasure removed. Section 3.12 holds the requirements, and its mechanism is undecided

9.3 Session renewal in the client

createReplayAfterRefresh in nh-auth-fetch reacts to a 401 only, and you cannot configure that. A 403 means an authenticated caller without permission, and a new token does not change that answer.

Rule: one replay only. A 401 on a session that the client just renewed is a real denial.

Rule: stamp your own credential below this middleware. A cookie hides this rule, because the platform attaches a cookie at send time. A bearer token does not: the code that sets the header must run again for the replay, and that happens only inside this middleware.

The other order looks correct, which is why the rule needs a statement. It reads as “add the credential, then handle a failure of it”. In that order the token is stamped once, before the 401, so the replay sends the token that the server rejected. The measured result is a 401 and one wasted refresh.

10. Errors and observability

10.1 One error shape

Rule: an application throws httpErrors from @fastify/sensible, and it registers that plugin with sharedSchemaId: "HttpError". One shared schema id gives one error shape in the composed OpenAPI document of section 8. Therefore a generated client has one error type for each application, and a caller writes one handler and not one for each callee.

Rule: a package exposes only the errors that it throws. A caller that catches an error of a package needs the name of it, and a list that holds more than the package throws sends a reader to look for a case that cannot arrive.

Rule: a status code carries the meaning, and a body carries the detail. Section 9.3 makes a client renew a session on a 401 and not on a 403, so a wrong code turns a permission failure into a login loop.

10.2 Telemetry

Each application registers nh-otel and exports OTLP to PLT_OTEL_URL, which points at the Grafana Alloy of section 11.1. A query in Grafana filters by environment.

Section 4.3 gives the surface of the package, and section 3.4 sends an exposure record along the same path.

11. Kubernetes, operators and Helm

11.1 An application declares its infrastructure

Rule: an application declares the infrastructure that it needs in its Helm values. An operator provisions it. A database, an IAM role, a topic, a certificate, and a DNS record each come from a value in a file that the application owns. There is no ticket, and no merge request against an infrastructure repository.

Terraform holds the foundation that the operators run on: the cluster, the network, and the RDS instance. Each thing above that foundation belongs to an operator.

Therefore the lifecycle of a resource is the lifecycle of its Helm release. A release that goes away takes its database with it. An operator also reconciles, so it corrects a drift without a plan step.

ConcernCustom resourceOperator
A database and its user, on the RDS instancePostgres, PostgresUserext-postgres-operator
A topic, a user, and an ACLTopic, User, RedpandaRoleThe Redpanda operator
A TLS certificate, for TLS inside the clusterCertificatecert-manager
An IAM role, and the Pod Identity association that gives it to a service accountRole, Policy, and the associationThe ACK IAM controller, and the ACK EKS controller
A secret from an external store, and a copy across namespacesExternalSecret, SecretStoreThe External Secrets Operator
Telemetry collectionAlloyThe Grafana Alloy operator
A DNS recordnone. An Ingress gives itExternal DNS
Ingress routing, and TLS terminationAn IngressAn AWS ALB, which the AWS Load Balancer Controller creates from that Ingress
A change-data-capture connectorKafkaConnect, KafkaConnectorStrimzi, for Debezium

A bucket is shared, and an application declares its access. The platform operates the object store, and each application writes under its own prefix in a shared bucket. Therefore an application creates no bucket, and what a deployment declares is the Pod Identity association of the row above, which is what lets it write there. One store then serves each application, and a new application adds an association.

Each operator runs in the operators namespace. Grafana Alloy runs in observability as a daemon set, and the Alloy operator manages it. Alloy is the collector of section 10: each application exports its OTLP to it, and its Faro receiver takes the telemetry of a browser. The platform runs no OpenTelemetry collector.

This names the edge of section 7.1. An AWS ALB terminates each request and its TLS, and External DNS derives the record from the Ingress. cert-manager issues the certificates inside the cluster, and not the certificate of the edge. External DNS supports more than one provider, so the platform is not tied to one DNS or CDN. It also never deletes a record.

Rule: the edge is a role, and not a product. An AWS ALB fills that role today. The requirement is that a chart declares its routing in a standard resource, so another controller can take the role. The DNS records do not change with it, because External DNS reads the same Ingress.

Rule: a chart declares an Ingress. The controller of the cluster serves that resource, so a change of controller stays in the cluster and reaches no chart.

Rule: change data capture serves analytics and a migration. Debezium reads the PostgreSQL of an application and emits each row, which suits a pipeline that owns its own copy of the data. An event between two applications comes from hook instead, where the name is frozen, the payload has a version, and the payload carries ids (section 3.6).

A topic needs no new infrastructure. The Redpanda cluster and its operator run already, and a topic is a value in a file. No application of the platform publishes to a topic today. Therefore the reason to start is a need that a queue cannot meet: one event with many internal consumers, a consumer that needs the history, an order for one key, or a volume above what one delivery process carries (section 3.6).

11.2 The chart is one line

Rule: a modulith chart holds no resource of its own. The whole template of dam is one include:

{{- include "common.platformatic" . }}

The common chart renders each resource from the values. It comes from an OCI registry, and Chart.yaml pins the version exactly.

Rule: never write a custom resource by hand. Fill in a value. The chart owns the shape of each resource, so a change to that shape is one release of the chart.

Each repository holds one base file and one file for each environment: values.yaml, values-staging.yaml, values-production.yaml, and values-review.yaml where a review environment exists.

ValueWhat it declares
imageThe digest-pinned image. CI substitutes it at deploy time
port, metricsPort, replicasThe runtime and the scale
healthChecksThe liveness path, the readiness path, and the startup probe
resourcesThe requests and the limits
service, ingress, podDisruptionBudgetThe network and the disruption budget
databasesA database, a user, and the name of the secret that holds the credential
awsSecretAn IAM role, a store, and an ExternalSecret that brings one secret into the namespace
serviceAccount.iamThe Pod Identity association of the service account
env, extraEnvVarsThe environment of the runtime

Rule: a probe must tolerate a slow boot. Each probe reaches the main thread of the runtime, and that thread can be unresponsive for tens of seconds while the module graph loads. Therefore the startup probe carries the boot, with a generous failure count, and the liveness and readiness timeouts stay above one second. A budget below the progressDeadlineSeconds of the deployment makes a slow boot fail through the probe, with a restart and a clear event, rather than through a stale condition.

Rule: each variable of the runtime is declared once. env and extraEnvVars are two lists, and one name belongs in one of them. Section 14.3 gives the change that makes the chart hold this rule.

The KeyValue store of section 1.5 has no operator in this list, so it is the one store that an application cannot declare. Section 14.3 gives the suggestion.

12. Continuous integration

12.1 The pipeline is composed, not written

Rule: each pipeline is built from to-be-continuous components. A repository adds a component and gives it inputs. It overrides a job where it must, and it does not copy a template.

ComponentVersionWhat it gives
node5.4.0Install, build, lint, audit, and the scoped registry configuration
semantic-release4.3.0The version, the tag, the changelog, and the publish
docker8.5.1The image build, an SBOM, and a Trivy scan
helm9.6.3The package, the lint, kube-score, and the deploy for each environment
terraform, terraform-aws9.3.2The configuration as code of Zitadel
helmfile4.2.0The Kubernetes deploy of Zitadel

Rule: pin each component exactly, and never a floating major. A template release must be a deliberate change, and not something that arrives in the middle of a sprint.

Rule: the audit level is high in each repository. The default of the node component is lower, and it gates on more than the platform can act on. Override the finding, and never suppress the gate.

12.2 When a pipeline runs

Rule: a pipeline runs on the default branch, on a merge request, and on a manual or triggered run. A push to a branch with no merge request gets none. Without that last rule the same commit runs twice, because a push pipeline and a merge-request pipeline both start when one push creates the branch and the merge request together.

Rule: a release branch must appear in the workflow rules. A branch that the rules do not admit gets no pipeline at all, so the release can never run. The branches-ref input of the release component and the branches list of the release configuration must also agree.

12.3 The version comes from semantic-release

semantic-release computes the version on a release branch. SEMREL_INFO_ON then exposes it to the Docker job and to the Helm jobs, so one commit gives one version to the package, the image, and the release.

12.4 The deploy

The Helm component deploys each environment from its own values file. Staging is a goodhabitz.dev host and production is a goodhabitz.com host, which section 5.3 needs, because a session cookie of production carries the production domain.

EnvironmentState today
StagingOn, for account and dam
ProductionOn for account. Off for dam, which needs DNS and a certificate first
ReviewOff for each application here. Where a review environment exists, a deploy is two manual plays, the image and then the release

The deploy runs upgrade --install --atomic --timeout 600s, so a failed release rolls back on its own.

kube-score runs with a named list of ignored checks. Each ignored check is a choice of the shared chart, except the image tag, because the deploy job substitutes the image and the lint job does not.

13. The learning domain

The platform carries more than one domain, and this is the one that delivers the product: an author writes, a learner runs, and the platform records what happened. Each rule of sections 1 to 12 holds here without a change. This section gives the parts, and where each one meets the next.

PartWhat it isWhere its rules are
cmsThe authoring surfaceSection 3.9
damThe store of a definition and of each asset that a delivery needsSections 3.1 and 11.1
lmsThe delivery surface, and the shell of the learnerSection 3.10
progressThe stream and the progress structure of one learnerSections 3.11 and 3.12
playThe common player, from nh-play, inside a frontend. It holds the machine and runs the two conversions, which a server can run as wellSection 4.7
attestIt rebuilds a result from the stream and says whether the record is correctSection 3.13

13.1 The structure

graph LR
    author([an author]) --> CMS["cms — authoring"]
    learner([a learner]) --> SHELL

    subgraph delivery ["what a learner opens"]
      direction TB
      SHELL["lms frontend — the shell"]
      PLAY["play — the common player"]
      SHELL --> PLAY
    end

    CMS -->|"publish"| DAM
    DAM[("dam — the definition, and each asset")]
    LMS["lms — delivery"]
    PROG["progress"]

    SHELL --> LMS
    LMS -->|"read a definition"| DAM
    LMS -->|"events, and the structure"| PROG
    PROG -->|"a restore point"| LMS
    PROG --> PGD[("PostgreSQL")]
    ATT["attest"]
    ATT -->|"read a stream, write a verdict"| PROG
    ATT -->|"read a definition"| DAM
    DPLAT(["the data platform — analysis, and a model"])
    PROG -.->|"the stream. The transport is undecided"| DPLAT

The definition travels one way, and the record of a learner travels both. cms writes and dam holds. lms reads and never writes a definition, so what a learner runs is the published document. progress sits on the second path, and lms both writes to it and reads a restore point from it.

play is the usual holder of a machine, and not the only one. It builds the machine from the definition, and the conversions run in Node as well, which is what lets attest rebuild a result with no frontend (sections 3.13 and 4.7).

attest crosses the two paths. Each other part reads on one of them. A check needs the record of a learner and the definition that the record was made against, so this application reads progress and dam together, and writes a verdict back.

13.2 The interaction

sequenceDiagram
    autonumber
    participant A as an author
    participant CMS as cms
    participant DAM as dam
    participant L as lms
    participant P as play
    participant PR as progress
    participant AT as attest
    A->>CMS: edit a definition
    CMS->>CMS: validate against the published schema
    CMS->>DAM: publish the definition, with its version
    Note over A,DAM: authoring ends
    L->>DAM: read the definition of this delivery
    L->>PR: read the restore point
    PR-->>L: the progress structure, and its version
    L->>P: the definition, and the structure
    P->>P: build the machine from the definition
    P->>P: build a snapshot from the structure
    P->>P: transform the snapshot into the progress structure
    P-->>L: each event, and the structure after it
    L->>PR: synchronize the batch, with a sequence number
    PR->>PR: append the stream, and store the structure
    Note over P,AT: a check, where declared
    AT->>PR: read the stream and the structure
    AT->>DAM: read the definition of that version
    AT->>AT: replay, transform, and compare
    AT->>PR: write the verdict

The publish ends the authoring, and each step after it reads what that publish put in dam. The last four steps run later, and only where a delivery declares a check (section 3.13). Each other step runs while a learner is present.

13.3 What each part owns

PartIt ownsIt readsIt writes
cmsThe definition, until it publishes oneIts own store, and the views that it can offerA definition, to dam
damEach published definition, and each assetWhat cms gives it
lmsThe enrolment, the class, the session, and the URL of the pageA definition from dam, and a restore point from progressEach event and each structure, to progress
progressThe stream and the progress structure of one learnerWhat lms sendsIts own three stores (section 3.11)
playThe machine, and the route inside a delivery. It runs the conversions that nh-play definesThe definition and the structure that lms gives itEach event, and the structure after it, back to lms
attestThe verdict of a checkThe stream and the structure from progress, and the definition from damThe verdict, to progress

Three boundaries carry the design, and each one has its rule where the part is specified. The definition is the artifact and not a row of a store (section 3.9). The snapshot of a machine never leaves progress (section 3.11). The machine is derived from the definition and never written for it (section 4.7).

14. Suggested improvements

A suggestion is a change to a repository that exists. It is not a rule and not a proposal, so it has its own section.

The convention. A suggestion sits under the repository that holds the code to change, and each one gives four things in this order: the problem, the change, the effect, and the bound where one applies. A section that explains why the problem exists keeps the facts and points here. Therefore a reader who owns a repository sees one list, and no suggestion appears twice.

14.1 account

Suggestions 1 and 2 come from SameSite=none, which the frame case makes necessary (section 5.3).

1. Set Partitioned on each session cookie.

  • Problem. No repository in the stack sets that attribute, so each session cookie is an unpartitioned third-party cookie in a frame. It works only while the browser of the reader allows one.
  • Change. Add the attribute in auth. Section 3.2 says that only the OIDC flow reads or writes a token, so a second place that sets a session cookie gives two owners and two sets of attributes. The cookie serializer in the stack supports the attribute, so this is a configuration value and not a new library.
  • Effect. The key of a partitioned cookie has two parts: the origin that set it, and the site of the top-level page. Therefore the platform in the site of a customer gets a session for that site alone, the same person on our own site has a separate session, and a second customer gives a third. A count of sessions is then not a count of people, and a support question about a lost session needs the top-level site in the answer.

2. Add a CSRF defence to each state-changing route.

  • Problem. No repository in the stack holds a CSRF plugin. SameSite=none is the configured value, so the platform does not get what lax gives, and a cross-site top-level POST carries the session of the user. The credentialed CORS at the gateway is not a defence: a simple POST needs no preflight, and a top-level form POST is not a CORS request at all.
  • Change. Check the Origin header at the gateway. Reject a state-changing cookie request unless that header is the origin of the platform, and reject the request when the header is absent.
  • Effect. One hook in one place, because each request passes one gateway and one origin (section 7.1). It needs no change in a frontend, and it works inside a frame, because a document of the platform in a frame sends the origin of the platform.
  • Bound. The defence applies to a cookie only, and never to a bearer token. A forged request needs a credential that a browser attaches by itself. A cookie is such a credential, and an Authorization: Bearer header is not. A third party calls the API with a bearer token and sends no Origin header, so a rule over each request would reject each of those calls. Therefore a client that is not a browser must use a bearer token.

3. Remove the unused cross-origin configuration.

  • Problem. The gateway of account configures credentialed CORS with an allowlist of origins, and auth takes a pattern of origins for a return target. One origin needs neither (section 7.1).
  • Change. Check each of them against a real request, and remove what nothing uses.
  • Effect. A cross-origin path that allows credentials is a surface to defend, and a surface that nothing needs is one to delete.

14.2 nh-auth

Suggestions 1 and 2 are about the credential of a call between two applications. Each type of secret is a standing grant of authority. A static string and a signed token differ in shape and not in kind, so neither is a smaller risk than the other by itself (section 5.5).

1. A permanent call uses a Zitadel service account. Section 14.3 gives the operator that would create one from a deploy.

  • Problem. A standing call between two applications holds a secret that never expires.
  • Change. The caller holds the credential of a service account and gets an access token. nh-auth-jwt verifies that token the same as a token of a person, and nh-auth-user derives the targets from real roles.
  • Effect. A token expires, so a leak has a short life. A grant lives in Zitadel with each other grant, so one place shows who can call what. A rotation needs no exchange with a callee. And the callee holds a list of principals, not a list of secrets. Section 3.7 uses this shape already.

2. Where a secret must stay, sign it.

  • Problem. A static value makes each callee hold the target set of each caller in its own configuration, so a new call needs a change in the callee and a release of it.
  • Change. The caller signs a token and names the targets that the call needs. The secrets option of nh-auth-user takes both types, and an absent type is a denied type. Therefore a callee that declares no signer behaves exactly as it does today, and the change goes in one application at a time.
  • Effect. A new call between two applications is a change in the caller alone. The callee holds a list of signers instead of a target set for each caller, so it releases when its own behaviour changes and not when a caller arrives.
  • Bound. The callee declares which signers it trusts and the ceiling of each one, so the static configuration becomes a list of signers and the widest target set that each may assert. A token above that ceiling fails. A token expires in minutes and names one callee, which keeps the property that the per-callee secret rule gives.

14.3 cluster-applications

1. Add an operator for the KeyValue store.

  • Problem. Each other store is self-service. An application declares a database or a topic in its own values, or the association that gives it access to the shared object store, and an operator provisions it (section 11.1). The KeyValue store has no operator, so it is the one store that an application cannot declare. Section 1.5 makes it platform infrastructure, section 4.5 puts each job queue in it, and section 7.3 uses it for the shared HTTP cache, so three parts of this document depend on a store that a team cannot ask for in the same way.
  • Change. Add an operator that takes a custom resource for a namespace in the store, and a template in the common chart that renders it. The values then look like the databases block.
  • Effect. One model covers each store. A namespace lives and dies with the Helm release that declared it, the credential arrives as a secret the same as a database credential, and section 1.5 stops being two rules with two mechanisms.
  • Bound. The operator declares a namespace, and not a cluster. The store stays one piece of platform infrastructure, because section 3.5 has job read across namespaces in it.

2. Add an operator for a Zitadel service account, and a value that declares one.

  • Problem. Section 14.2 makes a Zitadel service account the answer for a permanent call between two applications. Nothing creates that account from a deploy today, so a person creates it by hand, and a credential then reaches the application by a path that no file records. An account that a deploy does not own also outlives the application that needed it.
  • Change. Add an operator for Zitadel, and a template in the common chart that renders its custom resource. An application then declares its service account and the targets that it needs in its own values, the same as it declares a database.
  • Effect. The operator creates the account and writes the credential as a secret, which the External Secrets Operator already brings into a namespace. Therefore the grant lives beside the application that holds it, a review of one values file shows what that application can call, and a release that goes away takes its account with it.
  • Bound. The operator creates an account and a grant. It does not create a person and does not change a role of a tenant, because section 5 keeps a role of a person in the identity model and not in a chart.

3. Make the common chart reject a variable that two lists declare.

  • Problem. env and extraEnvVars are two lists of the same thing, and the chart joins them. One name in both therefore renders twice, and Kubernetes takes the last value. A reader of the values file sees the value that it expects, and the deployed pod holds the other one. This is a defect of the chart, and section 11.2 carries it as a rule that each author must remember.
  • Change. The template fails the render when one name appears in both lists, and the message names it. helm template and helm lint then give the error, so a merge request shows it.
  • Effect. The rule moves from each author to the chart, where one release fixes it for each repository. The pipeline of section 12 finds the fault before a deploy, and not at the end of one.
  • Bound. The change is one guard in the template. It joins no list differently and renames no value, so each values file that follows the rule renders exactly as it does now.

15. References

15.1 Specifications of the learning platform

Each specification below is a page in the Product space of Confluence. Section 0.1 states what each one binds.

SpecificationTitle
RFC 001Dynamic Course Protocol & State Topology
RFC 002LMS Shell Architecture & Universal Routing
RFC 003The Activity Player (Standalone Content Engine)
RFC 004Distributed Design System for Learning via Shadcn Registry
RFC 005Event Sourcing and Snapshot Backend for Progress Tracking
RFC 006Headless CMS Integration and Course Composer

Rule: a reference names a specification, and not a version of it. Each page above is a draft that changes, so this document points at the page and repeats none of its content. A statement of this document that depends on one of them says so at the point of use.

15.2 Standards

Each standard below is named in a rule of this document. A rule states the standard at the point of use, and this table gives the source of it.

StandardWhere a rule uses it
Standard WebhooksThe headers, the signed string and the algorithm of an outbound webhook (section 3.6)
RFC 8032, EdDSA and ed25519The signature algorithm of that webhook (section 3.6)
RFC 2104, HMACThe alternative for a subscriber that cannot verify ed25519 (section 3.6)
OpenID Connect Core 1.0The authorization-code flow and the claims of a token (sections 3.2 and 5)
RFC 7636, PKCEThe browser flow of a CLI (section 4.2)
RFC 8628, the OAuth 2.0 device grantThe second login mode of a CLI (section 4.2)
The OpenAPI SpecificationThe generated REST surface, and the document that a client is generated from (sections 1.3, 7.2 and 8)
The GraphQL specificationThe generated GraphQL surface, and the composed schema (sections 1.3 and 8)
RFC 6265bis, HTTP cookiesThe attributes of each session cookie, and SameSite (section 5.3)
CHIPS, a cookie with an independent partitioned stateThe Partitioned attribute that section 14.1 suggests
OTLP, the OpenTelemetry protocolThe export of each trace and each metric (section 10)
The Web Locks APIMutual exclusion in a browser (section 4.1)
ASD-STE100, Simplified Technical EnglishThe language of this document