Home /Modules/advantage/README.md

README.md

advantage/transport/README.md · 185 line(s) · markdown

advantage / transport — secure acquisition layer

A secure, fast, and idempotent transport layer for searching, reviewing, obtaining, downloading, and storing remote data over HTTP(S), SFTP, and SSH, with Diffie-Hellman / ECDH key exchange and an encrypted-at-rest key store.

It is built on OpenSSL (the audited library the repository's own crypto/README.md recommends for protecting real data) and on the system OpenSSH client for SSH/SFTP. It deliberately does not hand-roll TLS or the SSH protocol.

Components

FileLangResponsibility
keystore.{h,c}CAES-256-GCM encrypted secret store; PBKDF2-HMAC-SHA-256 key derivation; atomic writes
kex.{h,c}CDH (FFDHE2048/3072) and ECDH (X25519, P-256) key agreement → HKDF-SHA-256 session key
digest.{h,cpp}C++SHA-256 content addressing / integrity
uri.{h,cpp}C++URI parsing for https/http/sftp/ssh/file
http_client.{h,cpp}C++TLS-verified HTTP(S) GET/HEAD with redirects, timeouts, size caps
sftp_client.{h,cpp}C++SSH/SFTP download via OpenSSH scp (argv-only, no shell)
acquire.{h,cpp}C++Idempotent, content-addressed acquisition facade (search/review/obtain/download/store)
bulk.{h,cpp}C++Rate-limited, parallel bulk fetcher over a list of URLs, with bounded retries/backoff
transport_cli.cppC++transport command-line front end
bulk_cli.cppC++bulkfetch command-line front end

Building produces a reusable static library libadvantagetransport.a (all of the above except the CLI mains) plus the transport and bulkfetch binaries.

Build

make            # transport, bulkfetch, libadvantagetransport.a + .so
make lib        # just the static library
make shared     # just the shared library (+ soname/dev symlinks)
make pc         # generate the pkg-config file
make test       # runs the safety + functional battery (test.sh)

Requires OpenSSL development headers (libssl/libcrypto); bulk fetching uses pthreads. For sftp:// and ssh:// fetches, the OpenSSH scp binary must be on PATH at runtime.

Install

make install PREFIX=/usr/local        # honors DESTDIR for staged installs
make uninstall PREFIX=/usr/local

Installs: transport + bulkfetch to $(BINDIR); libadvantagetransport.a and the versioned shared library (libadvantagetransport.so.1.0.0 + .so.1 soname + .so dev symlink) to $(LIBDIR); headers to $(PREFIX)/include/advantage/transport/; and advantagetransport.pc to $(LIBDIR)/pkgconfig/. Overridable: PREFIX, BINDIR, LIBDIR, INCLUDEDIR, PKGCONFIGDIR, DESTDIR.

Linking from another program

After install, via pkg-config:

c++ your_app.cpp $(pkg-config --cflags --libs advantagetransport) \
    $(pkg-config --libs openssl) -lpthread -o your_app
# headers resolve as: #include <acquire.h>, <bulk.h>, <keystore.h>, ...

Or against the in-tree static library without installing:

c++ your_app.cpp -I/path/to/transport \
    /path/to/transport/libadvantagetransport.a \
    $(pkg-config --libs openssl) -lpthread -o your_app

Public headers: acquire.h, bulk.h, keystore.h, kex.h, digest.h, uri.h, http_client.h, sftp_client.h.

Usage

# Review remote metadata without downloading the body (HTTP HEAD)
transport review https://example.com/file.bin

# Obtain + download + store (idempotent, content-addressed)
transport fetch  https://example.com/file.bin --store ./store
transport fetch  sftp://user@host/path/file   --store ./store
transport fetch  file:///etc/hostname          --store ./store

# Pin expected content; a mismatch fails and discards the bytes
transport fetch  https://example.com/f --sha256 <hex> --store ./store

# Encrypted key store (passphrase via $ADVANTAGE_KS_PASS, never argv)
export ADVANTAGE_KS_PASS='…'
transport keystore secrets.ks put api_token 's3cr3t'
transport keystore secrets.ks get api_token
transport keystore secrets.ks list
transport keystore secrets.ks rm  api_token

# Demonstrate a key exchange + derived session key
transport kex x25519        # also: p256, ffdhe2048, ffdhe3072

Bulk / parallel fetching

Fetch a list of URLs you control into one content-addressed store, in parallel, with a global rate limit and bounded retries:

# list file: one "URL [sha256]" per line; '#' comments and blanks ignored
bulkfetch sources.txt --store ./store --jobs 8 --rate 200 --retries 3

A template is provided in sources.example.txt — edit it to point only at sources you are authorized to fetch. The bulk fetcher applies the same secure defaults as transport (TLS verify-on, strict SSH host-key checking, size caps, idempotent content addressing).

Security model

Idempotency & integrity

Verified behavior

The build and the following were exercised locally:

Run the whole battery any time with make test (16 checks; no public-internet egress required — the HTTPS case uses a local self-signed server).

Notes / limits