nfltr-pipe

One end listens and prints an address; whoever holds it dials. What one end writes to its standard input the other reads on its standard output, both ways at once, so any command that reads or writes a stream can run across two machines.


What it does

nfltr-pipe is an open-source command (Apache-2.0), github.com/nfltr/pipe, built on the relay's public client SDK.

a$ nfltr-pipe listen
listening; give the other end this address (a secret: whoever holds it can connect):
nf1...
b$ nfltr-pipe dial nf1...

An address on dial's command line is visible to the machine's other users (ps); on a shared machine pass it with --address-file instead (below).

Who closes what and when, and what happens when a peer stops or vanishes, is a checked TLA+ model in the repository's spec/, and the tests' session ends are checked against it.

Install

Release binaries for Linux, macOS and Windows on amd64 and arm64 are on the releases. Each release carries SHA256SUMS, the SHA-256 of every binary, and SHA256SUMS.sshsig, its signature by nfltr-pipe's own release key in the repository's release/allowed_signers, Ed25519 SHA256:f6CMaXVMYmUA1RGI6EmJ/0SNrAWN8xS7UQgCiXdwkaM. Checking both needs ssh-keygen from OpenSSH 8.1 or later (ssh -V prints the version); its Good "nfltr-release" signature line names the key's fingerprint, which is the one above. The commands take the latest release; download/<version> in place of latest/download takes another.

Linux (on arm64, nfltr-pipe_linux_arm64):

$ f=nfltr-pipe_linux_amd64; u=https://github.com/nfltr/pipe/releases/latest/download
$ for x in $f SHA256SUMS SHA256SUMS.sshsig; do curl -fsSLO $u/$x; done
$ curl -fsSLO https://raw.githubusercontent.com/nfltr/pipe/main/release/allowed_signers
$ ssh-keygen -Y verify -f allowed_signers -I nfltr-release -n nfltr-release -s SHA256SUMS.sshsig < SHA256SUMS
$ grep " $f\$" SHA256SUMS | sha256sum -c && mkdir -p ~/.local/bin && install -m 755 $f ~/.local/bin/nfltr-pipe

macOS (on Intel, nfltr-pipe_darwin_amd64), which checks the sum with shasum:

$ f=nfltr-pipe_darwin_arm64; u=https://github.com/nfltr/pipe/releases/latest/download
$ for x in $f SHA256SUMS SHA256SUMS.sshsig; do curl -fsSLO $u/$x; done
$ curl -fsSLO https://raw.githubusercontent.com/nfltr/pipe/main/release/allowed_signers
$ ssh-keygen -Y verify -f allowed_signers -I nfltr-release -n nfltr-release -s SHA256SUMS.sshsig < SHA256SUMS
$ grep " $f\$" SHA256SUMS | shasum -a 256 -c && mkdir -p ~/.local/bin && install -m 755 $f ~/.local/bin/nfltr-pipe

~/.local/bin is where nfltr's installer puts nfltr; add it to PATH if it is not there yet.

Windows, in PowerShell (on arm64, nfltr-pipe_windows_arm64.exe):

PS> $ProgressPreference = "SilentlyContinue"; $f = "nfltr-pipe_windows_amd64.exe"; $u = "https://github.com/nfltr/pipe/releases/latest/download"
PS> foreach ($x in $f, "SHA256SUMS", "SHA256SUMS.sshsig") { Invoke-WebRequest "$u/$x" -OutFile $x }
PS> Invoke-WebRequest "https://raw.githubusercontent.com/nfltr/pipe/main/release/allowed_signers" -OutFile allowed_signers
PS> cmd /c "ssh-keygen -Y verify -f allowed_signers -I nfltr-release -n nfltr-release -s SHA256SUMS.sshsig < SHA256SUMS"; if ($LASTEXITCODE) { throw "SHA256SUMS is not signed by the release key" }
PS> $sum = (Select-String -Path SHA256SUMS -SimpleMatch " $f").Line.Split(" ")[0]
PS> if ((Get-FileHash $f -Algorithm SHA256).Hash -ne $sum) { throw "$f does not match SHA256SUMS" }
PS> New-Item -ItemType Directory -Force "$HOME\bin" | Out-Null; Move-Item -Force $f "$HOME\bin\nfltr-pipe.exe"

cmd /c hands SHA256SUMS to ssh-keygen byte for byte, which a PowerShell pipe does not; add $HOME\bin to PATH if it is not there yet.

With Go, built from source instead of the signed binary:

$ go install github.com/nfltr/pipe/cmd/nfltr-pipe@latest

The listener's relay is --relay when given, else $NFLTR_PIPE_RELAY, else the one nfltr config set-relay saved (with its pin), else nfltr.xyz. Only the saved relay's address and pin are read, never an account or key. The dialler takes its relay from the address: dial refuses --relay and --relay-http (exit 1) and ignores $NFLTR_PIPE_RELAY.

On a private relay, --cert-sha256 (or $NFLTR_RELAY_CERT_SHA256, or the pin nfltr config set-relay saved) pins the relay's certificate on either end. --direct off keeps the session relayed (a listener needs its machine certificate for that); the default, --direct on, takes a direct path when the networks allow one.

nfltr-pipe version prints the version, commit and Go version; -h on any command prints its usage and exits 0.

Examples

Each example runs as written in the repository's tests, both ends on one machine against a relay started in the test. addr is a file holding the listener's address, readable by its owner only: paste it with umask 077; cat > addr, so it never shows on a command line. dial refuses an address file that others can read. --address-file - reads the address from standard input's first line instead, and the rest of standard input goes to the session:

dst$ (cat addr; cat notes.txt) | nfltr-pipe dial --address-file -

Copy a directory

src$ tar -C ~/project -cf - . | nfltr-pipe listen
dst$ nfltr-pipe dial --address-file addr | tar -C ~/project -xf -

dial exits 0 only after the sender's last byte and its end arrived. A transfer that fails part way exits non-zero on both sides, and tar sees a truncated archive, not a complete one.

A database, dump straight into restore

src$ sqlite3 app.db .dump | nfltr-pipe listen
dst$ nfltr-pipe dial --address-file addr | sqlite3 app.db

No dump file on either side. The same shape works for any dump and restore pair that reads and writes a stream, such as pg_dump and psql.

Follow a log on another machine

server$ tail -F app.log | nfltr-pipe listen
laptop$ nfltr-pipe dial --address-file addr | grep --line-buffered ERROR

Stopping the reader (Ctrl-C) resets the session; the server's nfltr-pipe exits 6.

In a script

Retry when the relay or the listener was not reached in time (5), discard a copy that was cut (6):

src$ nfltr-pipe listen < backup.tar
dst$ for try in 1 2 3; do
       nfltr-pipe dial --address-file addr > backup.tar; code=$?
       [ $code -eq 5 ] || break; sleep 10
     done
dst$ case $code in 0) echo complete ;; 6) rm backup.tar; echo cut ;; *) echo "failed: $code" ;; esac

More, in the README: a disk image with progress, a container image without a registry, the clipboard, a TCP service for one connection, and ssh and rsync through a listener in front of sshd.

Exit codes

nfltr-pipe and nfltr-send share one table, so a number means the same in both. 8 and 11 are not used; nfltr-pipe has no exit for a stop (a stopped end is 6).

exit code meaning
0 both directions ended and every byte was taken
1 usage bad flags, address or configuration
2 auth the relay refused this machine (revoked, or off for more than 90 days): nfltr config login
3 not_found nothing listens at the address (or it already served its session)
4 refused the listener or the relay refused the session
5 unreachable the relay or the listener was not reached in time
6 reset the session was cut: the other end failed, was stopped or vanished, or this end's input or output failed, or it was stopped
7 e2ee_pin_mismatch the listener is not the one the address names
9 balance_exhausted the serving account's balance is exhausted
10 e2ee_client_refused the listener refused this end's proof
12 direct_required no direct path to a listener serving with no account, or that path was lost

Compatibility

From v0.1.3 the flags, variables, exit codes and address format above are nfltr-pipe's v1 contract: later releases keep them, and a listener and a dialler of different releases work together, both ways. The full contract is the repository's docs/v1-contract.md.

Who needs an account

Dialling needs none. Listening needs none either, while the bytes can go peer to peer:

A listener serves one session; a second dial of the same address is reset. A peer that is killed without a reset ends the other end's session at once when relayed, and within about 30 seconds on a direct path.