End-to-End Encryption in NFLTR: Architecture and Security Analysis

1. Introduction

NFLTR is an open-source tunneling platform that connects clients to agents (services running behind NAT/firewalls) through a relay server. In its default mode, the relay terminates TLS and can observe plaintext traffic — a trust model acceptable for development but insufficient for production workloads handling sensitive data.

This document specifies four independent E2EE tracks that eliminate server-side plaintext visibility. Each track is activated by user configuration and addresses a distinct communication pattern:

• Track A (TLS Passthrough) — HTTP/gRPC tunnels where the agent terminates TLS directly, and the server performs SNI-based blind routing.
• Track B (WireGuard VPN) — Full Layer 3 network tunnels using the WireGuard protocol with Curve25519 key exchange.
• Track C (P2P Direct) — File and message transfers via TCP hole-punching with AES-256-CTR encryption, bypassing the server entirely when possible.
• Track D (A2A Messaging) — Structured inter-agent communication with X25519-ECDH key exchange and AES-256-GCM authenticated encryption.

All four tracks share a fundamental design principle: the relay server never possesses session keys and cannot decrypt payload data.

2. Terminology and Conventions

The key words "MUST", "MUST NOT", "SHOULD", "MAY" in this document are to be interpreted as described in RFC 2119 [1].

3. Threat Model

3.1. Adversary Capabilities

The E2EE architecture is designed to protect against a compromised or malicious relay server operator. The adversary model assumes:

• The adversary has full control of the relay server, including access to all server-side memory, disk, and network interfaces.
• The adversary can observe, record, replay, modify, delay, or drop any traffic passing through the relay.
• The adversary can inject messages into any relay stream.
• The adversary does NOT have access to the agent's or client's private keys, local memory, or filesystem.
• The adversary cannot break the underlying cryptographic primitives (AES-256, X25519, SHA-256, ChaCha20-Poly1305).

3.2. Security Goals

3.3. Out of Scope

• Metadata privacy — The relay server can observe connection timing, SNI hostnames, packet sizes, and connection patterns. Traffic analysis resistance is not a goal.
• Endpoint compromise — If the agent or client device is compromised, E2EE provides no additional protection.
• Denial of service — The relay can always drop traffic. Availability is not guaranteed by E2EE.

4. System Architecture

4.1. Relay Server Model

In standard (non-E2EE) mode, the NFLTR relay server terminates TLS, inspects HTTP headers, and proxies requests to agents over gRPC streams. When E2EE is enabled, the server transitions to a blind relay model:

The server's role in E2EE mode is reduced to:

1. Connection routing — extracting the SNI hostname from the TLS ClientHello (the only plaintext visible) to determine which agent should receive the connection.
2. Byte forwarding — bidirectional copying of opaque bytes between the client TCP connection and the agent's gRPC stream.
3. Metadata management — storing agent identity material (TLS certificate fingerprints, WireGuard public keys) for client verification.

4.2. E2EE Track Overview

5. Track A: TLS Passthrough

Track A provides E2EE for HTTP, gRPC, and generic TCP tunnels by having the agent terminate TLS directly, rather than the relay server. Activated with the --e2ee flag on the agent.

5.1. Certificate Generation

When an agent starts with --e2ee and no certificate files are provided, it generates a self-signed X.509 certificate:

The certificate fingerprint is computed as:

fingerprint = SHA-256(DER-encoded leaf certificate)
display     = colon-separated uppercase hex (e.g., AB:CD:EF:...)

This fingerprint is transmitted to the relay server and stored in the endpoint registry, enabling clients to perform trust-on-first-use (TOFU) or out-of-band fingerprint verification.

Users MAY provide their own certificates via --e2ee-cert and --e2ee-key flags for certificates issued by a private CA.

5.2. TLS Configuration

The agent's TLS configuration enforces modern cipher suites with mandatory forward secrecy:

MinVersion: TLS 1.2
CurvePreferences: [X25519, P-256]

TLS 1.2 Cipher Suites (explicit):
  TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
  TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256

TLS 1.3 Cipher Suites (Go runtime auto-selected):
  TLS_AES_128_GCM_SHA256
  TLS_AES_256_GCM_SHA384
  TLS_CHACHA20_POLY1305_SHA256

All configured cipher suites use ECDHE key exchange, ensuring forward secrecy for every connection. Non-ECDHE suites (RSA key transport) are explicitly excluded.

Certificate serving uses the GetCertificate callback, enabling hot-rotation without dropping active connections.

5.3. ACME Integration

Agents MAY obtain publicly-trusted certificates from Let's Encrypt using the ACME TLS-ALPN-01 challenge [7].

The ACME flow:

1. Agent specifies --acme-domain <fqdn>.
2. autocert.Manager issues a TLS-ALPN-01 challenge, requiring the agent to be reachable on port 443 via the E2EE routing path.
3. Certificates are cached to disk (DirCache) with 0700 permissions.
4. On renewal, the new certificate fingerprint is pushed to the relay server via the _fingerprint control frame on the gRPC stream.
5. HostPolicy restricts issuance to the explicitly-configured domain list.

ACME certificates eliminate the browser security warning associated with self-signed certificates while maintaining the same E2EE guarantees — the private key never leaves the agent.

5.4. SNI Extraction and Routing

The relay server extracts the SNI hostname from the TLS ClientHello without terminating the TLS handshake. The extraction algorithm:

1. Read the 5-byte TLS record header. Verify ContentType = 0x16 (Handshake).
2. Read the handshake body (length from record header).
3. Parse the ClientHello: skip ProtocolVersion (2 bytes), Random (32 bytes), SessionID (length-prefixed), CipherSuites (length-prefixed), CompressionMethods (length-prefixed).
4. Iterate TLS extensions. For extension type 0x0000 (server_name), parse the SNI hostname.
5. Return the SNI hostname and all peeked bytes for replay to the destination.

The SNI hostname is resolved to an agent ID by stripping the configured base domain suffix. For example, with base domain e2ee.nfltr.xyz:

SNI: myapp.e2ee.nfltr.xyz → Agent ID: myapp
SNI: fresh-crest.nfltr.xyz → ShareStore.Validate("fresh-crest") → Agent ID

When a ShareStore is configured, the resolver also checks whether the subdomain prefix is a valid share code (two-word pair), enabling memorable E2EE URLs without a dedicated .e2ee. subdomain.

5.5. Port Multiplexing

NFLTR multiplexes E2EE TLS passthrough traffic and standard gRPC traffic on a single TCP port (typically 443) using an SNI-based multiplexer:

The sniMuxListener implements net.Listener. E2EE connections are routed in a separate goroutine, and the multiplexer immediately returns to accepting the next connection. Non-E2EE connections are returned from Accept() with a prefixConn wrapper that replays the already-read ClientHello bytes, so the gRPC server's TLS stack processes them normally.

5.6. Blind TCP Relay

Once routing is determined, the relay creates a bidirectional byte stream between the client's TCP connection and the agent's gRPC ConnectTCP stream:

1. Client's raw TCP bytes (complete TLS ClientHello + all subsequent TLS records) are forwarded to the agent via gRPC.
2. The agent receives the complete TLS handshake, terminates TLS locally, and handles the decrypted request.
3. Response bytes (encrypted by the agent's TLS stack) flow back through the gRPC stream to the client.

The relay sets TargetAddr: "__e2ee__" and E2EE: true on the TCP endpoint to signal the agent that this is an E2EE passthrough connection. The server performs no TLS termination, no certificate validation, no payload inspection — it is a pure TCP byte forwarder.

In multi-pod deployments, cross-pod relay is supported via podDialer.DialTCP(), maintaining the blind relay property across pod boundaries.

5.7. Certificate Rotation

Certificates can be rotated without disconnecting the agent:

1. The agent generates or loads a new certificate.
2. The new certificate is atomically swapped via mutex-protected update.
3. The agent sends a _fingerprint control frame with the new SHA-256 fingerprint over the existing gRPC stream.
4. The relay server updates the endpoint registry with the new fingerprint.
5. Existing TLS connections continue using the old certificate; new connections use the new one.

For ACME certificates, this process is automatic — the autocert.Manager handles renewal and the fingerprint update is triggered on successful certificate acquisition.

6. Track B: WireGuard VPN

Track B provides full Layer 3 VPN tunnels using the WireGuard protocol [3]. The implementation runs entirely in userspace — no root privileges or kernel modules are required.

6.1. Key Generation

WireGuard keys are Curve25519 key pairs [4]:

1. Generate 32 bytes from crypto/rand.
2. Apply Curve25519 clamping:
     privateKey[0]  &= 248
     privateKey[31]  = (privateKey[31] & 127) | 64
3. Compute publicKey = X25519(privateKey, Basepoint)
4. Encode both keys as base64 for transport.

Clamping ensures the private key is a valid Curve25519 scalar, preventing small-subgroup attacks.

6.2. Key Exchange Protocol

WireGuard public keys are exchanged during agent connection setup via gRPC metadata headers:

IP addresses are allocated from the CGNAT range 100.64.0.0/10 [5]. The server hub is always .1; agents receive sequential addresses starting at .2. The IPAM subsystem follows the standard in-memory/Redis/SQL factory pattern for single-pod and multi-pod deployments.

6.3. Tunnel Construction

The WireGuard tunnel is constructed entirely in userspace using wireguard-go [6] and the gvisor network stack:

1. netstack.CreateNetTUN() creates a gvisor-backed virtual network interface (TUN device).
2. device.NewDevice() creates the wireguard-go device, bound to a custom RelayBind (see §6.4) instead of a real UDP socket.
3. IPC configuration is applied: private key, peer public key, allowed IPs, and a synthetic endpoint 127.0.0.1:51820 for handshake initiation.
4. The device is brought up (equivalent to wg-quick up).
5. MTU is set to 1420 (WireGuard standard).

6.4. Relay Binding

The RelayBind replaces wireguard-go's standard UDP socket binding with in-process byte channels that interface with the gRPC stream:

The RelayBind implements wireguard-go's conn.Bind interface:

• Send(bufs, endpoint) — wireguard-go calls this to emit encrypted packets. The packets are placed on a buffered outbound channel (256 packets).
• Deliver(data) — incoming encrypted WG packets from the gRPC stream are fed into the inbound channel.
• Outbound() — returns the channel of outgoing encrypted packets for transmission via gRPC.

WireGuard packets are transported over the existing gRPC Subscribe stream with TransactionId: "__wg__", eliminating the need for a separate UDP transport or additional ports.

6.5. Multi-Peer Hub Routing

The server-side WGHub runs its own wireguard-go device that routes encrypted packets between peers. Each connected agent is added as a WireGuard peer with a unique synthetic port (starting at 51821).

The hub uses a ForwardingTUN instead of a full gvisor network stack. The routing loop:

1. Receive — An encrypted WG packet arrives from Agent A via gRPC. DeliverFrom() feeds it to the hub's WG device.
2. Decrypt & Route — The hub device decrypts the outer WG layer, reads the destination IP from the inner IP header, and looks up the target peer in the AllowedIPs routing table.
3. Re-encrypt & Send — The hub device encrypts the packet for the target peer (Agent B) using Agent B's WG session keys and sends it via HubBind.Send().
4. Deliver — HubBind routes the encrypted packet to Agent B's sender callback, which writes it to Agent B's gRPC stream.

6.6. Exit-Node Forwarding

When an agent operates as a WireGuard exit node, it creates a ForwardingTUN with a gvisor TCP forwarder that intercepts all incoming TCP SYN packets and proxies them to the real LAN:

gvisor options:
  AllowExternalLoopbackTraffic: true
  HandleLocal: false
  Promiscuous mode: enabled
  Spoofing: enabled (for arbitrary destination IPs)

For each TCP connection:
  1. gvisor intercepts SYN from the WG tunnel.
  2. TCP forwarder calls net.DialTimeout("tcp", dstAddr, 10s).
  3. Bidirectional io.Copy between gvisor TCP conn and real LAN conn.

The SOCKS5 proxy (client-side) implements RFC 1928 [2] with CONNECT command support, dialing through the WireGuard tunnel's gvisor network stack.

7. Track C: Peer-to-Peer Direct Transfer

Track C enables direct machine-to-machine file and message transfers that bypass the relay server entirely when connectivity permits. Encryption is always-on.

7.1. Signaling Protocol

P2P signaling messages flow over the A2A (agent-to-agent) gRPC stream using message type p2p.v1. Signal message types:

The shared secret Token is transmitted only in the offer message — it is never retransmitted in the answer. The Token is used for mutual authentication and key derivation (§7.3, §7.4).

7.2. Connectivity Establishment

Connectivity is established by racing multiple connection strategies simultaneously:

1. STUN discovery — A minimal RFC 5389 STUN Binding Request is sent to stun.l.google.com:19302 to discover the external (server-reflexive) IP address. The response is parsed for XOR-MAPPED-ADDRESS (preferred) or MAPPED-ADDRESS.
2. Host candidate gathering — All non-loopback IPv4 addresses are enumerated.
3. Parallel connection racing — One goroutine accepts on the local listener; N goroutines dial each remote candidate simultaneously. The first successfully authenticated connection (either direction) wins.
4. Timeout — 30 seconds for the entire connectivity establishment process. If no direct connection succeeds, fallback to relay (§7.6).

7.3. Mutual Authentication

After a raw TCP connection is established, both peers perform a 4-step HMAC-based mutual authentication handshake:

Step 1: Each side generates 32 bytes from crypto/rand (challenge).
Step 2: Each side sends its challenge and reads the peer's challenge.
Step 3: Each side computes:
          response = HMAC-SHA256(token, sessionID + ":" +
                     myChallenge + theirChallenge)[:16]
        and sends the 16-byte response.
Step 4: Each side reads the peer's response and verifies using
        hmac.Equal() (constant-time comparison).

This handshake proves both peers possess the same Token (pre-shared via the signaling channel) without revealing it to a network observer. The challenge-response structure prevents replay attacks.

7.4. Key Derivation

After successful authentication, a symmetric session key and directional IVs are derived:

// Challenges sorted lexicographically for canonical ordering
first, second = sort(myChallenge, theirChallenge)

// 32-byte AES-256 key
key = HMAC-SHA256(token, "p2p-key:" || sessionID || ":" || first || second)

// 16-byte IVs (direction-specific to prevent CTR stream collision)
IV_dial    = HMAC-SHA256(key, "p2p-iv-dial2accept")[:16]
IV_accept  = HMAC-SHA256(key, "p2p-iv-accept2dial")[:16]

// Assignment by role:
if isDialer:
  writeIV = IV_dial,   readIV = IV_accept
else:
  writeIV = IV_accept, readIV = IV_dial

The canonical ordering of challenges ensures both peers derive identical key material regardless of who initiated the connection. Direction-specific IV labels guarantee the two CTR streams never share a (key, IV) pair, which would be catastrophic for CTR mode security.

7.5. Stream Encryption

All data after the handshake is encrypted using AES-256-CTR:

encryptedConn = {
  read:  cipher.StreamReader{S: CTR(AES-256, readIV),  R: conn}
  write: cipher.StreamWriter{S: CTR(AES-256, writeIV), W: conn}
}

The encryptedConn wraps the raw TCP connection, transparently encrypting all writes and decrypting all reads. The CTR mode counter advances independently for each direction.

File integrity is verified with SHA-256 checksums computed before transmission and verified on receipt. Path traversal is prevented by filepath.Base() sanitization on received filenames.

7.6. Relay Fallback

When direct P2P connectivity cannot be established (symmetric NAT, restrictive firewalls), both peers fall back to relay mode:

• Both sides exchange SignalRelay messages.
• A RelayConn wraps the A2A gRPC stream as a net.Conn, chunking writes to 256 KB to fit within gRPC frame limits.

8. Track D: Agent-to-Agent E2EE Messaging

Track D provides end-to-end encrypted structured messaging between agents. Unlike Tracks A–C (which are opt-in), A2A E2EE is enabled by default.

8.1. ECDH Key Exchange

Key exchange uses X25519 Elliptic Curve Diffie-Hellman [8]:

Initiator (Agent A):
  1. Generate ephemeral X25519 keypair: privA, pubA = X25519.GenerateKey()
  2. Send pubA via "a2a-e2ee-init" message on gRPC stream.
  3. Await "a2a-e2ee-ack" with pubB.
  4. Compute shared = privA.ECDH(pubB)

Responder (Agent B):
  1. Receive "a2a-e2ee-init" with pubA.
  2. Generate ephemeral X25519 keypair: privB, pubB = X25519.GenerateKey()
  3. Compute shared = privB.ECDH(pubA)
  4. Send pubB via "a2a-e2ee-ack" message.

Both ephemeral keys are generated fresh per session using crypto/rand, providing forward secrecy. Compromise of any long-term credential (API keys, certificates) does not expose past A2A message content.

8.2. Key Derivation

key = SHA-256("nfltr-a2a-e2ee-v1" || shared_secret)    // 32 bytes

The domain separator "nfltr-a2a-e2ee-v1" prevents cross-protocol key reuse. The resulting 32-byte key is used for AES-256-GCM encryption.

8.3. AEAD Encryption

Messages are encrypted using AES-256-GCM with authenticated associated data (AAD):

Encryption:
  nonce  = crypto/rand (12 bytes, GCM standard nonce size)
  aad    = "nfltr-a2a-e2ee-v1:" || transactionID
  output = nonce || GCM.Seal(nonce, nonce, plaintext, aad)

Decryption:
  nonce     = ciphertext[:12]
  payload   = ciphertext[12:]
  plaintext = GCM.Open(nil, nonce, payload, aad)

Cipher identifier: "x25519-aes-256-gcm"
Headers:
  x-a2a-e2ee: "true"
  x-a2a-e2ee-cipher: "x25519-aes-256-gcm"

The AAD includes the transaction ID, binding each encrypted message to its specific transaction context. This prevents the relay from swapping encrypted messages between different conversations. Random nonces ensure uniqueness without coordination between peers.

9. Converged E2EE Identity

When an agent uses both Track A (TLS Passthrough) and Track B (WireGuard), a combined fingerprint is computed to bind both E2EE tracks to a single agent identity:

combined = SHA-256(TLS_fingerprint || WG_public_key)
display  = colon-separated uppercase hex

This combined fingerprint is stored in the endpoint registry as E2EEIdentityMetadata:

{
  "tls_fingerprint":      "AB:CD:EF:...",
  "wg_public_key":        "base64...",
  "combined_fingerprint": "12:34:56:..."
}

A client verifying the combined fingerprint confirms that both the TLS passthrough path and the WireGuard tunnel terminate at the same physical agent — preventing a scenario where one track is man-in-the-middled while the other is not.

Identity updates are handled atomically via the E2EEIdentityUpdater interface, which merges incoming identity fields with existing stored values. If either the TLS fingerprint or WG public key changes, the combined fingerprint is automatically recomputed.

10. Relay Integrity Verification

NFLTR includes a cryptographic audit trail that allows E2EE peers to verify that the relay server faithfully forwarded all bytes without tampering.

10.1. Hash Chain Construction

Both the sender and receiver independently maintain a SHA-256 hash chain over every data frame relayed during a session:

node[0] = SHA-256(frame_0_data)
node[i] = SHA-256(node[i-1] || frame_i_data)     for i > 0

The chain head after N frames constitutes the session's integrity commitment.

10.2. Commitment Exchange

At any point during or after a session, either peer can request a commitment from the other:

commitment = (hex(chain_head), frame_count)

Verification succeeds if and only if both peers report the same chain head hash and frame count.

10.3. Tamper Detection

The audit trail is thread-safe (sync.Mutex protected) and can be reset between sessions via Reset().

11. Security Considerations

11.1. Metadata Leakage

The relay server can observe: SNI hostnames (Track A), WireGuard handshake initiation packets (Track B), signaling messages (Track C), and connection timing/volume for all tracks. NFLTR does not provide traffic analysis resistance. Users requiring metadata privacy should layer additional protections (e.g., Tor, VPN) beneath NFLTR.

11.2. Self-Signed Certificate Trust

Track A with self-signed certificates relies on trust-on-first-use (TOFU) via fingerprint verification. Users SHOULD verify fingerprints through an out-of-band channel on first connection. ACME certificates from Let's Encrypt provide CA-backed trust but require DNS control.

11.3. P2P Relay Fallback Limitation

Track C relay fallback mode does not apply the P2P AES-256-CTR encryption layer. Data in relay fallback is protected only by gRPC transport TLS. This is a known limitation documented in §7.6. Users requiring guaranteed E2EE should use Track A or Track D.

11.4. WireGuard Hub Plaintext Exposure

The WireGuard hub (§6.5) momentarily holds plaintext IP headers in memory during the decrypt→route→re-encrypt step. This is inherent to WireGuard's mesh routing model and is documented in §6.5. For zero server-side plaintext requirements, use Track A.

11.5. CTR Mode IV Management

Track C uses AES-256-CTR, which is catastrophically broken if the same (key, IV) pair is reused. The key derivation function (§7.4) prevents this by deriving direction-specific IVs with distinct labels ("p2p-iv-dial2accept" vs "p2p-iv-accept2dial") and incorporating per-session random challenges.

11.6. Nonce Uniqueness (Track D)

Track D uses random 12-byte GCM nonces. With AES-256-GCM, the probability of nonce collision reaches 50% after approximately 2^48 encryptions under the same key. Since keys are ephemeral per-session, this limit is not practically reachable.

11.7. Forward Secrecy Properties

12. Cryptographic Primitives Summary

12.1. Standard Compliance

13. References

1. Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
2. Leech, M., et al., "SOCKS Protocol Version 5", RFC 1928, March 1996.
3. Donenfeld, J. A., "WireGuard: Next Generation Kernel Network Tunnel", NDSS 2017.
4. Bernstein, D. J., "Curve25519: New Diffie-Hellman Speed Records", PKC 2006.
5. Weil, J., et al., "IANA-Reserved IPv4 Prefix for Shared Address Space", RFC 6598, April 2012.
6. Donenfeld, J. A., "wireguard-go: Go Implementation of WireGuard", https://git.zx2c4.com/wireguard-go.
7. Shoemaker, R. B., "Automated Certificate Management Environment (ACME) TLS Application-Layer Protocol Negotiation (ALPN) Challenge Extension", RFC 8737, February 2020.
8. Langley, A., Hamburg, M., and S. Turner, "Elliptic Curves for Security", RFC 7748, January 2016.
9. Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, August 2018.
10. Dworkin, M., "Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC", NIST SP 800-38D, November 2007.
11. Perrin, T., "The Noise Protocol Framework", Revision 34, July 2018.