Architecture and Guarantees

One page for what nfltr guarantees, which part guarantees it, and what it does not. Every guarantee is a rule the code enforces and a test protects; the evidence section says how strong each kind of test is. Claims the sources do not support are listed under Not guaranteed and Known gaps.


At a glance

This page describes main as of 2026-10-26; the evidence and known gaps were re-checked on 2026-11-06. The code is the source of truth. Each claim rests on nfltr's internal specification, which cites the code and the test for every rule. The code and the specification are not public; this page states their rules in plain words. Agent orchestration (nfltr-orch) is not part of nfltr and is not publicly available.

Architecture

+------------------------+                +------------------------+
| MACHINE  nfltr         |                | MACHINE  nfltr         |
|  cat serve, up, http   |                |  cat pipe, cat cp, ... |
+-----------|------------+                +-----------|------------+
            | outgoing connection                     | outgoing connection
            v                                         v
+----------------------------------------------------------+
| RELAY  nfltr.xyz or your own relay                       |
|  routes frames by agent id, checks machine certificates,|
|  lists connected agents (presence), mailboxes            |
+----------------------------------------------------------+
 data plane between machines: link, multiplexed sessions
 with per-session flow control (tunnels, TCP)

Which layer guarantees what

Two layers give guarantees. A guarantee of a lower layer is not a promise of a higher one: link says nothing about the relay's routing.

link: the data-plane library

link multiplexes sessions over one connection that the caller dials and secures. It imports only the Go standard library. The wire format is version 2.

The relay

Production guarantees and other evidence

A guarantee here is a rule the code enforces and a check protects. How strong the check is differs. This table keeps the kinds apart.

Kind of evidenceWhat it showsWhat it does not show
Unit and contract tests, run on every change. Examples: the mailbox contract tests.The rule holds in the tested cases.Behaviour under a real network or a real fleet.
TLA+ model checking. The state machines of link and the relay are TLA+ specs, checked with TLC before their code and on every push. Every past bug in an area is a mutant the checker must catch, and steps recorded by the Go tests are checked against the specs. What TLA+ found in nfltr.Every reachable state of the checked configurations keeps the invariants, and the recorded runs of the code match the spec.Configurations larger than the ones checked, and code paths no recorded run takes.
Chaos lab on Kubernetes (kind). Experiments kill relays and Redis, partition and delay the network, and skew clocks. Shared steady-state probes check the result.Real relays under network faults.There is no general unit-level clock-skew test.
link fuzzing and soak. Decoders are fuzzed (4.85 hours in total, no crasher in the library). Soaks under faults ran 2.89 hours on macOS and Linux CI with leak and integrity checks.The wire decoders and sessions survived what was run.The project's own target for both is 24 hours. It has not been met.

Missing from this evidence:

Trust boundaries

Who trusts whom, and what each party can do if it misbehaves.

PartyTrusted toNot trusted toControl
RelayAuthenticate machines, route frames inside an account, hold mailboxes and presence.Read content, decide, or vouch for a peer's key.Organisation sessions (nfltr cat with --allow-group) and end-to-end HTTP tunnels are encrypted between the two machines.

Contracts per primitive

Each contract states the guarantee, the bound and the failure. Where the sources give no rule, the row says so.

Streams (link sessions)

QuestionContract
BackpressureEach side announces a per-session window (128 KiB to 1 GiB). A sender may have at most the peer's window unacknowledged. The receiver returns credit as bytes leave its buffer. A slow reader stalls only its own session.
OrderingBytes of one session arrive in order. There is no ordering between sessions.
CancellationReset ends a session and the peer reads an error. CloseWrite half-closes: the peer reads io.EOF and can still write. Abort reaches the far end through every hop as a typed code. Deadlines are each side's own; link carries none from hop to hop.
Terminationio.EOF is a deliberate end. A reset, a closed connection, a dead peer or a drain deadline reads an error that is not io.EOF. After a go-away, no new sessions open; open ones run to their end.
Dead peerDetected within Interval + Timeout. Idle connections cost a few bytes.
LimitsSessions per connection are bounded by the peer's announced maximum. An open past it ends the connection (ErrFull). A data payload is at most 1 MiB per frame.
HTTP through the relayA client that opts in has at most 4 response chunks and 4 upload chunks in flight per transaction. Any exit before the end sends a cancel frame. Clients that do not opt in get bodies whole.
A2A streamsWhen an agent's connection ends, every pending call ends with ErrConnectionClosed (gRPC Unavailable, retryable), never io.EOF.

Not specified in the sources: a fairness guarantee between sessions beyond "every session keeps getting its turn", and any end-to-end encryption in link itself. link does no encryption. Confidentiality needs TLS or an encrypted payload inside the session.

Mailbox

A mailbox is a named, ordered queue of opaque envelopes owned by one account. Only the owner's principals append, pull or acknowledge. The relay does not read payloads.

QuestionContract
DeliveryAt least once. A pull never acknowledges. An envelope has a relay-assigned increasing seq.
Lost acknowledgementThe envelope is returned again with a higher delivery_count. The reader dedups by seq. An ack removes every envelope up to and including through_seq.
Consumer goneEnvelopes stay until acknowledged or expired. A reader that is away loses nothing before expiry. A full mailbox refuses the append with MAILBOX_FULL.
ExpiryA time-to-live up to 7 days. Unset or 0 means 7 days. After expires_at the relay drops the envelope, and a pull returns only unexpired ones.
Duplicates from the senderThe same idempotency_key from the same sender within 24 hours returns the first seq with duplicate = true and stores nothing. At most 1000 keys are remembered per mailbox.
LimitsPayload 1 MiB. 500 envelopes and 8 MiB per mailbox. 64 MiB and 256 mailboxes per account. A pull returns at most 100 and waits at most 25 s.
WebhooksA POST is at most 64 KiB and 600 per minute. Refusals are counted (dropped_rate_limited, dropped_mailbox_full, rejected_too_large) and the next envelope carries dropped_before.
Your own relay (beta)Mailboxes and webhooks are kept in the relay's data directory. A restart keeps them, with their unacknowledged envelopes.

Presence

Presence lists the connected agents an account may reach, with their labels, and a change feed.

QuestionContract
What it isA snapshot of connected agents plus a cursor taken before the snapshot. Watching from that cursor misses nothing.
StalenessAn agent counts as connected until the relay ends its stream. Heartbeats run every 15 s. With heartbeat checks on, the relay ends a session that sent no frame for 45 s. Its reaper evicts agents idle for more than 180 s (it scans every 60 s). A frozen process can stay listed until then.
A listed agent may be deadA call to a frozen but listed agent is answered NotFound before acceptance, so nothing ran.
FeedWatch is a long poll that waits at most 25 s and returns at most 256 events, oldest first. The revision orders events within one relay process.
Cursor resetIf the cursor is older than the relay's history, or from another relay process, the response sets reset. Changes may be missing. The client must list again.
Several relay replicasWith Redis down, a replica's feed shows only its own agents.
LabelsLabels are advertised data. The relay filters on exact values and never interprets keys. Labels set by the operator at connect cannot be changed by UpdateLabels.

Typed errors

Clients branch on a machine-readable reason, never on message text.

ClassMeaningExamples
TransientThe sources mark the error retryable. The same call may succeed later.Relay errors AGENT_OFFLINE, RELAY_STARTING and DIRECTORY_UNAVAILABLE (marked retryable), RELAY_DRAINING (carries a retry hint). link: ErrGoAway (open on another connection; Pool and Node do), ErrConnClosed (open a new session).
PermanentThe call is refused or wrong, and the sources mark no retry. Change the request or the setup.UNAUTHENTICATED, LIMIT_EXCEEDED, INVALID_ARGUMENT. A replaced session (FailedPrecondition) is final.

Rules that protect the classes:

The full list is generated from code: the ErrorReason enum in the relay protocol.

Threat model

Scope: the hosted relay at nfltr.xyz, your own relay (beta), the nfltr command and the dashboard. The full model, with a table per boundary, is internal; this section summarizes it.

Who can read your traffic, and what the relay sees

What the relay always sees: the account, agent ids, labels, which machine connects to which, when, frame sizes and timing. For tunnels, it sees the traffic of ordinary HTTP tunnels and shares (TLS ends at the relay) and raw TCP bytes. It does not see end-to-end TLS passthrough content.

Organisation sessions (nfltr cat with --allow-group) are TLS between the two machines with their machine certificates: the relay sees the two machines, the service, sizes and timing, not the bytes. End-to-end encryption has the details.

Key management and revocation

KeyWhere it livesRevocation or rotation
Machine key and certificateThe key is made on the machine (machine.key, mode 0600, beside the config) and never leaves it; the relay signs a certificate for it that lasts 24 hours and renews itself.nfltr machines revoke ends the streams the machine opened with Unauthenticated and refuses its renewals; a machine off for longer than its certificate enrols again.
Dashboard sessionA signed cookie naming a server-side session, 24 hours by default.Logout or removing the account from the allow-list ends it. A WebSocket opened before logout stays open until it closes (finding F-27).
Webhook secretA bearer secret is hashed at the relay. A GitHub secret is derived from the relay's seal key.DELETE /api/v1/webhooks/<id> removes the hook. Rotating the seal key means re-creating every GitHub hook.
Your own relay's machine CA and TLS certificateIts data directory. Clients pin the TLS certificate or trust a CA.Stop the relay, delete its machine-ca/, machines and admin-enrol-code (a new machine CA) or its tls/ directory (a new TLS certificate), start it and repeat the client step on each machine, with a new code after a new CA. Certificate files you provide can be replaced in place: the relay serves the new pair within about 10 seconds, without a restart. A machine enrolled without --account is its own account.
Release signing keyHeld outside the repository; the public keys are embedded in install.sh and nfltr update.The installer verifies the signature before the checksum.

Device compromise

What is compromisedWhat an attacker getsWhat limits it
A machine, and its machine keyThe key acts as that machine, inside its account and groups. It can pick its own agent id under the machine. There is no per-service credential.Revoke the machine (nfltr machines revoke). This is audit finding F-20, residual.
The relayIt can deny service, drop or delay frames, observe metadata and route frames inside an account. It could serve a tampered install.sh.Release signature checks in install.sh and nfltr update. End-to-end TLS inside sessions.

What persists locally

WhereWhatHow long
Relay (hosted), as inventoried on 2026-10-03Accounts, key hashes, machine records, mailbox envelopes until acked or expired, weekly activity, page-visit counts, contact messages, service logs, daily database backups.Contact messages 180 days. Mailbox envelopes at most 7 days. Retention of the others is not confirmed in the sources.
Your own relayA data directory with its machine CA, machines file, TLS certificate and file-backed stores.Yours. A data directory readable by group or others is refused.

Not guaranteed

Known gaps

These are open findings and claims the sources do not support. They are listed so you do not assume them.

Sources

Each claim above is a rule of nfltr's internal specification. The specification cites the code that enforces each rule and the test that protects it, and a check fails the build when a cited file or symbol no longer exists. The code, the specification, the security audit and the threat model live in private repositories, so this page states their rules instead of linking them. What is published:

Next