lumbda/examples/cuda-fanout/plans/bend-http-deploy.md

Bend HTTP deploy — Caddy + DNS proposals (NOT applied)

Phase 1 of bend's HTTP path (port 8321) lands inside the lumbda repo in this commit: gpu-worker.lsp now listens on both :8320 wire and :8321 HTTP, sharing one handle-request dispatcher. This document holds the operational deltas — to be applied by fox in ~/git/foxhop-pillar, ~/git/foxhop-states, and ~/git/proxy.unturf.com after review.

Two access tiers, both supported, both compatible:

Audience	URL	Gate	What changes here
Anyone (public adoption)	http://localhost:8321/ (their own machine)	Browser localhost exception	Nothing in fox's infra — users run make gpu-worker on their own host.
Fox (personal remote access)	https://bend.unturf.com/ (proposed)	proxy.unturf.com @trusted remote_ip allowlist	proxy.unturf.com vhost + ai.foxhop.net vhost + DNS A record

The playground placeholder stays http://localhost:8321/ to teach the right pattern to visitors. Fox personally pastes the remote URL into the field when traveling.

1. Public DNS — bend.unturf.com A record

Whatever zone-authority manages unturf.com (per CLAUDE.md the PowerDNS master lives on proxy.uncloseai.com:22222):

bend.unturf.com.  3600  IN  A  142.93.73.64

The IP is the existing proxy.unturf.com edge (the same address the @trusted matcher in ai-foxhop-net.sls already accepts).

Apply via:

ssh -i ~/.ssh/digitalocean -p 22222 root@proxy.uncloseai.com \
    "pdnsutil add-record unturf.com bend A 3600 142.93.73.64 && \
     pdns_control notify unturf.com"

2. proxy.unturf.com — new public vhost

File: ~/git/proxy.unturf.com/ingress/Caddyfile

Insert a new vhost block (place near the other *.unturf.com ones; exact location not load-bearing since Caddy matches by host):

# bend.unturf.com — personal remote access to fox's bend mesh.
# NOT public: gated by a trusted-IP allowlist of fox's known IPs.
# Routes Host-rewritten through ai.foxhop.net so the existing
# @trusted edge-IP pattern at the LAN Caddy continues to gate.
bend.unturf.com {
    @trusted_fox {
        # TODO(fox): replace with fox's known IPs. Examples:
        # remote_ip <home-public-ip>
        # remote_ip 100.64.0.0/10            # Tailscale CGNAT range
        # remote_ip <work-public-ip>
        remote_ip <FOX-IP-PLACEHOLDER>
    }

    handle @trusted_fox {
        reverse_proxy https://ai.foxhop.net {
            header_up Host bend.foxhop.net
            header_up X-Real-IP {http.request.remote.host}
            header_up X-Forwarded-For {http.request.remote.host}
            header_up X-Forwarded-Proto {http.request.scheme}
            transport http {
                tls
                tls_server_name ai.foxhop.net
            }
        }
    }

    handle {
        respond "Access denied — personal endpoint" 403
    }
}

3. ai.foxhop.net Caddy — new LAN vhost

File: ~/git/foxhop-pillar/caddy/ai-foxhop-net.sls

Insert between the existing home.foxhop.net block (lines 74-88) and the next vhost. Mirrors the home.foxhop.net pattern exactly — edge-IP gated, reverse_proxies to a specific worker on the LAN.

    bend.foxhop.net {

        # === Trusted sources - ONLY edge proxy ===
        @trusted {
            remote_ip 142.93.73.64       # edge / proxy.unturf.com
        }

        handle @trusted {
            # Forward HTTP/8321 of the bend worker mesh.
            # Single backend today (3090-ai); fan out across the mesh
            # once the single-host path is verified.
            reverse_proxy 3090-ai.foxhop.net:8321
        }

        handle {
            respond "Access denied - bend backend (edge proxy required)" 403
        }
    }

Apply via the usual pillar workflow:

cd ~/git/foxhop-pillar
git add caddy/ai-foxhop-net.sls
git commit -m "caddy: bend.foxhop.net vhost — proxy to 3090-ai:8321"
git push
# then trigger highstate on ai.foxhop.net via salt-master (home.foxhop.net)

4. Worker hosts — start bend on every GPU server

Per the discussion: each GPU server runs gpu-worker.lsp on both 8320 + 8321 so the Caddy vhost above can reverse_proxy to whichever host is healthy.

Today the proposal pins backend to 3090-ai.foxhop.net:8321 (single host MVP). To verify the dual-port mode is live on 3090-ai:

ssh 3090-ai.foxhop.net "ss -ltn | grep -E ':832[01]'"
# expect both 8320 and 8321 in LISTEN

A systemd unit (or whatever process manager 3090-ai uses for the current wire-only worker) needs the unit's ExecStart to keep the same args — no flag changes required since both ports come up by default. Confirm gpu-worker.lsp was redeployed after this commit lands in lumbda.

5. Future — fan-out across the 3-node mesh

Once 3090-ai HTTP path is verified, the ai.foxhop.net vhost can upgrade to round-robin / failover across the existing mesh:

    bend.foxhop.net {
        @trusted { remote_ip 142.93.73.64 }
        handle @trusted {
            reverse_proxy 3090-ai.foxhop.net:8321 \
                          ai.foxhop.net:8321 \
                          cammy.foxhop.net:8321 {
                lb_policy least_conn
                health_uri /             # or a dedicated /healthz once added
                health_interval 10s
            }
        }
        handle { respond "Access denied" 403 }
    }

The fleet logic in examples/cuda-fanout/bend.lsp (round-robin + failover for native callers) and Caddy's lb_policy are independent — both can fan across the same mesh; neither needs to know about the other.

What we explicitly chose NOT to do

• No public-internet bend endpoint without auth. bend.unturf.com is fox-only via trusted-IP. Public-internet bend would need authentication first (Phase 2 dependency).
• No bend.foxhop.net split-horizon DNS for public access. Keeping bend.foxhop.net LAN-only and bend.unturf.com as the personal- remote name avoids the split-horizon confusion that bites every six months.
• No factory recipe handling in this commit. Phase 2 lands server-side compilation of .lsp recipes after auth ships.