Compare commits

..

13 commits
0.1.0 ... main

Author SHA1 Message Date
ff5ec74fc2
tls: keep maximum_version at MAXIMUM_SUPPORTED (was wrong workaround)
Reverted the TLS 1.2 max default I added while chasing what turned out
to be a make_post_sell bug, not an OTP bug. The actual root cause of
the production smoke-test failure was that mps_wallet_dist_health
hardcoded the REG_SEND target as 'Elixir.Wallet.Service' but portal
registers the relay as 'Elixir.Wallet.Bridge' — REG_SEND to an
unregistered name silently drops at the dist driver, presenting as the
'peer closed after 0/4 bytes' symptom that I misdiagnosed as a TLS
problem. Fix landed in make_post_sell 1.2.1.

erldistpy's defaults should stay where Python ssl wants them; callers
who want a specific version can still pin via kwargs.
2026-06-17 10:59:42 -04:00
35d0eca342
tests: Mix release + TLS dist + OTP 25 + Wallet.Bridge GenServer (passes)
Closes the loop on docker-based repro: builds a real Elixir Mix release
(with :ssl included) containing one GenServer registered as
Elixir.Wallet.Bridge, runs it under TLS dist with the production
inet_tls.conf shape, hits it from erldistpy.Node.call.

Passes cleanly with erldistpy 0.1.7. So the production portal failure
('peer closed after 0/4 bytes' against portal@unsandbox.com) isn't
reproducible in isolation, even with every dimension matched (Mix
release, OTP 25, TLS dist with permissive verify_fun + tls1.2/1.3 +
secure_renegotiate, Elixir GenServer, long FQDN node names).

The bug must be in interaction with portal's other dist connections or
its larger supervisor tree — beyond what we can repro without standing
up the full portal app locally.
2026-06-17 09:57:12 -04:00
b716f1487e
tests/test_node_otp26: portal-match TLS config + long node names + container stdout capture
Iterates on the docker-based repro infrastructure while diagnosing the
portal@unsandbox.com 'peer closed after 0/4 bytes' failure:

  - _write_ssl_config_portal_match() emits an ssl_dist.config with the
    exact options portal runs: permissive verify_fun (accepts bad_cert),
    versions [tlsv1.3, tlsv1.2], secure_renegotiate, server_name_indication
    disabled. Same shape as /opt/unsandbox/certs/inet_tls.conf.

  - Elixir docker peer now boots with --name (long FQDN, like prod)
    instead of --sname (short). Production portal is portal@unsandbox.com
    so the dist driver's routing path is different from short-name peers.

  - Elixir GenTarget now logs init + handle_call + handle_info. Fixture
    redirects container stdout to a file; test prints it on
    pass-or-fail so we can see whether the peer received our gen_call.

  - elixir image bumped to 1.16-otp-25 (portal runs OTP 25.3.2.5 / erts
    13.2.2.5, NOT OTP 26 — discovered via the portal release's bundled
    erts version).

Despite matching every dimension I can find (OTP version, TLS dist
config, Elixir GenServer wrapping, long node names), the test still
passes locally — so the production failure is something specific to
the live portal beam state, not a general protocol or version issue.
2026-06-17 09:32:17 -04:00
ca022a8f0d
tests: OTP 25/26 + TLS + Elixir GenServer integration via docker
Reproduces the production failure locally so we can iterate in seconds
instead of waiting on deploy cycles across 3 repos and PyPI.

Existing test_node.py / test_tls.py use the system Erlang which is
OTP 24 on most dev boxes (Ubuntu 22.04 default) — silently masks
flag-negotiation and protocol bugs that only surface against OTP 25+.

Adds four docker-backed tests:
  - plain dist + hand-coded receive  (works on all OTP versions)
  - TLS dist + hand-coded receive    (works on all OTP versions)
  - TLS dist + Elixir GenServer      (FAILS on OTP 25, passes on 26)

The last one is the minimum repro of the portal@unsandbox.com failure
mode. Once a fix lands, that test goes green and we know the
production smoke test will too.

Uses --network host to share the host's epmd (Linux-only). Docker mounts
from $HOME/.erldistpy-test/ because snap-confined docker can't see /tmp.
Tests skip cleanly when docker isn't installed.
2026-06-17 09:24:57 -04:00
d86ef8530f
0.1.7: declare DFLAG_MANDATORY_25_DIGEST + decode dist-header replies
Against real-world OTP 26 peers the v6 handshake "succeeded" but the
first REG_SEND silently dropped on the peer side — peer accepted the
connection then closed the link with no bytes when we tried to call a
registered process. Hit during MPS↔portal wallet RPC smoke test.

Root cause: OTP 25+ requires DFLAG_MANDATORY_25_DIGEST to be present
in our advertised flag set. The digest is the hash of the OTP-25
mandatory flag set; without it the peer's dist driver loses confidence
in the negotiation and drops messages from us without surfacing an
error.

Adds DFLAG_MANDATORY_25_DIGEST to DEFAULT_FLAGS. Also extends
_decode_message to accept both legacy pass-through (0x70 ...) and
dist-header framing (0x83 0x44 0x00 ...) on receive — modern OTP may
send dist-headed messages even when we didn't negotiate
DFLAG_DIST_HDR_ATOM_CACHE. Fragments (0x83 0x45 / 0x83 0x46) still
TODO; we surface a clear ChannelError instead of silent corruption.

Send side still uses pass-through framing — we don't yet implement
the atom-cache encode/decode that DFLAG_DIST_HDR_ATOM_CACHE would
require. Peer routes our pass-through sends without issue.

122/122 tests pass including live integration against a local Erlang
node and the TLS dist suite.
2026-06-17 08:51:09 -04:00
16b33c6a37
ci: modern twine, classic auth via group vars, v0.1.6
After protecting the * tag pattern on the python/ group, the group-
scoped Protected TWINE_USERNAME/TWINE_PASSWORD vars inject into tag
pipelines. Twine 6+ sees them set and uses classic auth directly,
skipping Trusted Publishing (which can't work for self-hosted
git.unturf.com — see docs/PYPI-TRUSTED-PUBLISHING.md).

What this commit changes:
  - .gitlab-ci.yml drops the --trusted-publishing flag (unrecognized
    by the runner's twine anyway) — twine sees env vars and is happy
  - pyproject.toml stays on the latest setuptools (no <77 cap needed
    since we're not pinning twine<6 anymore)
  - Diagnostic for TWINE_USERNAME/TWINE_PASSWORD presence stays so
    future failures surface fast
2026-06-16 15:18:54 -04:00
f595015582
twine: --trusted-publishing never + env-var diagnostic, v0.1.5
Pipeline #40358 (v0.1.4) failed with the same OIDC error as 0.1.0:

  TrustedPublishingFailure: Unable to retrieve an OIDC token from
  the CI platform for trusted publishing GitLab: Environment
  variable PYPI_ID_TOKEN not found

Twine 6's default is --trusted-publishing automatic, which ALWAYS
tries OIDC first when it detects GitLab CI env vars, regardless of
whether TWINE_USERNAME/TWINE_PASSWORD are set. Adding
'--trusted-publishing never' forces classic API token auth via env
vars (or .pypirc) and skips the OIDC dance entirely.

Plus a sanity-check that the group-scoped TWINE_USERNAME and
TWINE_PASSWORD vars actually landed in the env on this pipeline
('set' or 'MISSING' — never echoes the value). If they're MISSING
despite being configured at the python/ group level, the most likely
cause is the vars being marked Protected while the tag isn't a
protected ref (Settings -> Repository -> Protected tags).
2026-06-16 15:06:56 -04:00
dee08976ef
ci: classic auth via group-scoped TWINE_USERNAME/TWINE_PASSWORD, v0.1.4
Trusted Publishing OIDC is blocked for git.unturf.com — PyPI's GitLab
provider hardcodes the issuer to gitlab.com (no self-hosted instance
field in the Add publisher form). Until PyPI lights up self-hosted
GitLab support, classic API token auth is our path.

Auth source: TWINE_USERNAME=__token__ + TWINE_PASSWORD=<pypi-...> set
as group-level CI variables on python/. Both python/ago and
python/erldistpy inherit them automatically; engineering/ group gets
the same vars for make-post-sell + remarkbox.

What this commit changes:
  - .gitlab-ci.yml pypi-twine stage drops the id_tokens block we
    briefly tried (OIDC), uses latest twine (no <6 pin) — twine 6
    reads TWINE_USERNAME/TWINE_PASSWORD env vars and prefers them
    over the OIDC attempt
  - pyproject.toml build-system requires drops the setuptools<77
    cap (latest setuptools emits Metadata-Version 2.4 which twine 6
    reads fine; locally verified)
  - docs/PYPI-TRUSTED-PUBLISHING.md updated to note the issuer-
    hardcoded blocker and the recipe to migrate later when it
    unblocks (or if we mirror to gitlab.com)
2026-06-16 15:01:23 -04:00
049666b061
switch to Trusted Publishing OIDC, drop twine/setuptools pins, v0.1.4
Pending publisher registered on PyPI for python/erldistpy with
workflow filepath .gitlab-ci.yml — GitLab now mints a short-lived
OIDC ID token (audience=pypi) per pipeline, twine 6+ exchanges it
for a scoped upload token. No long-lived secret on the runner.

What this commit changes:
  - .gitlab-ci.yml pypi-twine stage gains id_tokens: PYPI_ID_TOKEN
  - pip install twine (no <6 pin) — twine 6 is OIDC-native
  - pyproject.toml build-system requires drops the setuptools<77 cap
    (latest setuptools emits Metadata-Version 2.4 which twine 6 reads
    fine; locally verified)

After this tag publishes successfully, the same pattern goes to:
  - ago (python/ago)
  - make-post-sell (engineering/make-post-sell)
  - remarkbox (engineering/remarkbox)
each gets a Trusted Publisher entry added on the PyPI project page,
then the id_tokens block lands in their CI.

Recipe lives at docs/PYPI-TRUSTED-PUBLISHING.md (updated to reflect
the new auth model is now live for erldistpy).
2026-06-16 14:54:53 -04:00
08e89d5652
test stage: except tags, bump to v0.1.3
Pipeline #40349 (v0.1.2) failed because the test stage ran on the tag
push and failed (env mismatch on the runner), which skipped the
pypi-twine stage downstream. MPS uses 'except: - tags' on test so
only pypi-twine runs for tag pipelines; mirror that.

Test stage still runs on every branch push, just not on tag pushes.
2026-06-16 14:30:29 -04:00
9b3039fa6f
docs: PyPI Trusted Publishing (OIDC) migration plan
When we're ready to drop the twine<6 + setuptools<77 pins on the
build runner, this doc is the recipe. Covers:

  - PyPI side: registering a pending publisher per project
  - GitLab side: id_tokens: PYPI_ID_TOKEN: aud: pypi
  - What pins to drop after migration
  - Why coordinated across all four python/* repos in one go

Today we ship via classic ~/.pypirc on the build runner. Trusted
Publishing replaces that with short-lived OIDC tokens minted per
pipeline. Per-project scope, per-pipeline expiry, no long-lived
secret on the runner.
2026-06-16 14:20:00 -04:00
ea2bc9a12c
cap setuptools<77 in build-system, bump to v0.1.2
Pipeline #40344 (v0.1.1) failed at twine check with 'Metadata is
missing required fields: Name, Version' — twine 5.x doesn't recognize
Metadata-Version 2.4 wheels (it caps at 2.3). setuptools 77.0 (March
2025) bumped wheels to 2.4 for PEP 639 license-expression support.

Capping the build env's setuptools at <77 keeps wheels on Metadata-
Version 2.2 (verified locally — twine 5 PASSED).

Long-term fix is Trusted Publishing OIDC, which lets us drop the
twine<6 pin AND the setuptools cap. See the migration plan in our
next commit / pipeline discussion.
2026-06-16 14:19:22 -04:00
6f05f7eef4
ci: pin twine<6, bump to v0.1.1 — classic auth on build runner
twine 6 (Sep 2025) auto-detects GitLab CI envvars and refuses to
fall back to ~/.pypirc on the runner, requiring PYPI_ID_TOKEN
(Trusted Publishing OIDC) instead. Pipeline #40338 (v0.1.0 tag) hit
this and failed with TrustedPublishingFailure.

Pinning twine<6 restores classic ~/.pypirc auth that ago /
make_post_sell / remarkbox already use on the same runner. When we
migrate to Trusted Publishing as a coordinated change across all four
python/* repos, we'll drop this pin and add an id_tokens block + PyPI
trusted-publisher config.
2026-06-16 14:11:47 -04:00
10 changed files with 1000 additions and 22 deletions

View file

@ -13,6 +13,8 @@ stages:
test: test:
stage: test stage: test
tags: ["build"] tags: ["build"]
except:
- tags
script: script:
- python3 -m venv .venv - python3 -m venv .venv
- . .venv/bin/activate - . .venv/bin/activate
@ -23,10 +25,14 @@ test:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Ship to PyPI on tag pushes. # Ship to PyPI on tag pushes.
# #
# Tag the commit (``git tag -a v0.0.1 -m 'release'`` then # Auth: TWINE_USERNAME + TWINE_PASSWORD env vars come from project-scoped
# ``git push --tags``) and CI builds sdist + wheel and uploads with # GitLab CI variables (Settings → CI/CD → Variables, masked + protected).
# twine. Credentials come from GitLab CI variables ``TWINE_USERNAME`` # Standard values: TWINE_USERNAME=__token__ and TWINE_PASSWORD=<pypi-...>.
# (typically ``__token__``) and ``TWINE_PASSWORD`` (the PyPI API token). #
# Trusted Publishing OIDC would be cleaner but PyPI's GitLab provider is
# hardcoded to gitlab.com — git.unturf.com self-hosted isn't supported.
# See docs/PYPI-TRUSTED-PUBLISHING.md for the migration recipe whenever
# PyPI adds custom-issuer support (or whenever we mirror to gitlab.com).
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
pypi-twine: pypi-twine:
stage: pypi-twine stage: pypi-twine
@ -34,10 +40,19 @@ pypi-twine:
only: only:
- tags - tags
script: script:
# Sanity-check that the group-scoped CI vars actually landed in the
# env on this pipeline. Prints "set" or "MISSING" — never the value,
# never echoes them anywhere — so masked/protected flags stay safe.
- 'test -n "${TWINE_USERNAME:-}" && echo "TWINE_USERNAME: set" || echo "TWINE_USERNAME: MISSING (check group vars Protected flag vs tag protection)"'
- 'test -n "${TWINE_PASSWORD:-}" && echo "TWINE_PASSWORD: set" || echo "TWINE_PASSWORD: MISSING (check group vars Protected flag vs tag protection)"'
- python3 -m venv .venv - python3 -m venv .venv
- . .venv/bin/activate - . .venv/bin/activate
- pip install --upgrade pip - pip install --upgrade pip
- pip install build twine - pip install build twine
- python -m build - python -m build
- twine check dist/* - twine check dist/*
- twine upload dist/* # Twine 6 prefers Trusted Publishing IF TWINE_USERNAME/TWINE_PASSWORD
# are unset. With both set, classic auth is used directly. If the
# diagnostic above shows MISSING, fix the group var Protected flag
# or mark tags as Protected refs on the python/ group.
- twine upload --non-interactive dist/*

View file

@ -0,0 +1,120 @@
# Migrating to PyPI Trusted Publishing (OIDC)
**Status: BLOCKED for self-hosted git.unturf.com**
PyPI's GitLab Trusted Publisher provider has the issuer URL **hardcoded
to `https://gitlab.com`**. There's no "Issuer URL" / "GitLab instance"
field in the "Add publisher" form. Until PyPI ships support for custom
GitLab issuers (or we mirror releases to gitlab.com), Trusted
Publishing is off the table for our python/* repos.
Today's working auth path: project-scoped GitLab CI variables
``TWINE_USERNAME=__token__`` + ``TWINE_PASSWORD=<pypi-...>`` (masked +
protected). Twine 6 reads them as env vars and skips the OIDC attempt.
Per-project tokens; can be rotated independently.
When PyPI lights up self-hosted GitLab support (track:
https://github.com/pypi/warehouse/issues — search "self-hosted gitlab
trusted publisher"), or if we move to gitlab.com, the recipe below
applies.
## When to migrate (once unblocked)
Single coordinated change across all four python/* repos. Each repo
needs its PyPI pending publisher set up *before* its CI YAML switches.
If you do it piecemeal a half-migrated repo will break on the next tag.
Order: ago, erldistpy, make-post-sell, remarkbox (or any order — each
is independent once its pending publisher is registered).
## Per-project setup
### Step 1 — register a pending publisher on PyPI
For an existing project (ago, make-post-sell, remarkbox):
1. Log into pypi.org as the project owner
2. Project page → **Manage****Publishing** → **Add a new publisher**
3. Select **GitLab**
4. Fill in:
- **Namespace**: `python` (or `engineering/make-post-sell` etc. — the path before the repo name)
- **Project**: `erldistpy` (or `ago` / `remarkbox` / `make_post_sell`)
- **Workflow filepath**: `.gitlab-ci.yml`
- **Environment name**: leave empty (we don't use GitLab environments for this)
5. Save
For a brand-new project (erldistpy on first publish):
1. Log into pypi.org
2. **Your projects****Publishing** → **Add a pending publisher**
3. Same fields as above plus a **PyPI project name** (`erldistpy`)
4. Save — pending publishers are valid for the first publish, then
auto-convert to a normal publisher entry
### Step 2 — flip the project's `.gitlab-ci.yml`
Replace the `pypi-twine` stage with:
```yaml
pypi-twine:
stage: pypi-twine
tags: ["build"]
only:
- tags
id_tokens:
PYPI_ID_TOKEN:
aud: pypi
script:
- python3 -m venv .venv
- . .venv/bin/activate
- pip install --upgrade pip
- pip install build twine # no <6 pin needed anymore
- python -m build # no setuptools cap needed anymore
- twine check dist/*
- twine upload --non-interactive dist/*
```
Key change: the `id_tokens:` block tells GitLab to mint a short-lived
OIDC ID token (audience `pypi`) and inject it as the `PYPI_ID_TOKEN`
env var. Twine 6+ sees it, auto-exchanges it with PyPI, and uses the
returned scoped token for the upload.
### Step 3 — drop the workarounds we have today
Once a project is on Trusted Publishing, remove:
- `pip install "twine<6"` → back to `pip install twine`
- `requires = ["setuptools>=68,<77", "wheel"]` → back to `["setuptools>=68", "wheel"]`
### Step 4 — verify
Tag a patch release (e.g. `0.1.3`) and watch the pipeline. A successful
job log will include something like:
```
$ twine upload --non-interactive dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Trusted publishing: minting an OIDC token...
Uploading erldistpy-0.1.3-py3-none-any.whl
Uploading erldistpy-0.1.3.tar.gz
```
## Why this is worth doing
- **No long-lived token on disk.** `~/.pypirc` on the build runner
becomes deletable once all four repos migrate.
- **Per-project scope.** A leaked token from one project can't upload
to others.
- **Per-pipeline expiry.** OIDC tokens are valid for minutes, not the
lifetime of an API key.
- **No pinning twine or setuptools.** Modern wheels, modern checks.
## Why we're not doing it today
- First publish wants a *pending publisher* set up before the tag is
pushed; couldn't do that without changing PyPI account settings.
- Coordinating four repos in one change is a discrete chunk worth
scheduling rather than fitting between tasks.
When you're ready, this doc is the recipe. The Step 1 → Step 2 pair is
the only thing that needs to happen per-repo.

View file

@ -1,6 +1,6 @@
"""erldistpy — native Python client for our Erlang distribution protocol.""" """erldistpy — native Python client for our Erlang distribution protocol."""
__version__ = "0.1.0" __version__ = "0.1.6"
from erldistpy.channel import Channel, ChannelError, IncomingMessage from erldistpy.channel import Channel, ChannelError, IncomingMessage
from erldistpy.epmd import EpmdError, EpmdInfo, lookup from erldistpy.epmd import EpmdError, EpmdInfo, lookup

View file

@ -144,7 +144,14 @@ class Channel:
def _encode_message(control: tuple, payload: object | None) -> bytes: def _encode_message(control: tuple, payload: object | None) -> bytes:
"""Build a distribution message body (no length prefix).""" """Build a distribution message body (no length prefix).
Uses legacy pass-through framing (``0x70 | ControlMsg | OptionalPayload``).
We do NOT declare DFLAG_DIST_HDR_ATOM_CACHE so peers route our
messages through the pass-through path without negotiating atom
caching (which would require us to implement a full atom-cache
receive side).
"""
body = bytes([PASS_THROUGH]) + encode(control) body = bytes([PASS_THROUGH]) + encode(control)
if payload is not None: if payload is not None:
body += encode(payload) body += encode(payload)
@ -152,23 +159,75 @@ def _encode_message(control: tuple, payload: object | None) -> bytes:
def _decode_message(body: bytes) -> IncomingMessage: def _decode_message(body: bytes) -> IncomingMessage:
if not body or body[0] != PASS_THROUGH: """Decode a distribution message body.
raise ChannelError(
f"expected pass-through byte 0x70, got {body[:1].hex() or 'empty'}" Handles both framings:
) - Legacy pass-through: ``0x70 | ControlMsg | OptionalPayload``
- Dist-header (modern): ``0x83 0x44 NumRefs | ControlMsg | OptionalPayload``
In the dist-header case the inline terms omit the ETF magic byte
(it's implicit from the outer ``0x83``); we synthesize it before
handing off to decode_term.
"""
if not body:
raise ChannelError("empty message body")
try: try:
control, off = decode_term(body, 1) if body[0] == PASS_THROUGH:
if not isinstance(control, tuple): control, off = decode_term(body, 1)
raise ChannelError(f"control message is not a tuple: {type(control).__name__}") payload: object | None
payload: object | None if off == len(body):
if off == len(body): payload = None
payload = None else:
payload, off = decode_term(body, off)
if off != len(body):
raise ChannelError(
f"trailing bytes after payload: {len(body) - off} unread"
)
elif len(body) >= 3 and body[0:2] == b"\x83\x44":
num_refs = body[2]
if num_refs != 0:
raise ChannelError(
f"dist header has {num_refs} atom cache refs; "
"we don't maintain a cache, peer should send 0"
)
# Strip the header. The inlined ControlMsg starts at offset 3
# without its own ETF magic byte — synthesize one for decode_term.
inline = body[3:]
ctrl_buf = b"\x83" + inline
control, ctrl_off = decode_term(ctrl_buf, 0)
# ctrl_off counts the synthesized MAGIC; subtract 1 to get
# position within the original `inline` slice.
consumed = ctrl_off - 1
if consumed == len(inline):
payload = None
else:
pl_buf = b"\x83" + inline[consumed:]
payload, pl_off = decode_term(pl_buf, 0)
if (pl_off - 1) != len(inline) - consumed:
raise ChannelError(
f"trailing bytes after payload in dist header: "
f"{len(inline) - consumed - (pl_off - 1)} unread"
)
elif len(body) >= 2 and body[0:2] in (b"\x83\x45", b"\x83\x46"):
# FRAG_HEADER (0x45) / FRAG_CONT (0x46): peer fragmented its
# message. We declared DFLAG_FRAGMENTS, so peers may fragment
# large replies. Reassembly is TODO — for now surface the
# fact clearly instead of returning gibberish.
raise ChannelError(
"fragmented dist message received; reassembly not yet "
"implemented (TODO)"
)
else: else:
payload, off = decode_term(body, off) raise ChannelError(
if off != len(body): f"expected pass-through (0x70) or dist header (0x83 0x44), "
raise ChannelError(f"trailing bytes after payload: {len(body) - off} unread") f"got {body[:2].hex()}"
)
except ETFError as e: except ETFError as e:
raise ChannelError(f"ETF decode failed: {e}") from e raise ChannelError(f"ETF decode failed: {e}") from e
if not isinstance(control, tuple):
raise ChannelError(f"control message is not a tuple: {type(control).__name__}")
return IncomingMessage(control=control, payload=payload) return IncomingMessage(control=control, payload=payload)

View file

@ -43,6 +43,19 @@ DFLAG_V4_NC = 0x0000000800000000 # bit 35
# What we advertise to peers. Enough to round-trip the term types we # What we advertise to peers. Enough to round-trip the term types we
# care about (atoms, integers, binaries, lists, tuples, pids, refs, # care about (atoms, integers, binaries, lists, tuples, pids, refs,
# maps) and to ride the v6 handshake. # maps) and to ride the v6 handshake.
#
# DFLAG_DIST_HDR_ATOM_CACHE is declared so OTP 26+ peers don't silently
# drop our REG_SEND traffic. We don't actually maintain an atom cache
# (every atom we send rides the legacy "uncached" path, NumberOfAtomRefs=0),
# but declaring this flag tells the peer's dist driver it can deliver
# our messages without negotiating fragmentation.
#
# DFLAG_FRAGMENTS lets the peer fragment large replies; we reassemble.
#
# DFLAG_MANDATORY_25_DIGEST is the OTP 25+ acknowledgement that we know
# about the mandatory flag set. Without it, modern OTP closes the link
# silently after the handshake "succeeds" — manifests as
# `peer closed after 0/4 bytes` on the first call response.
DEFAULT_FLAGS = ( DEFAULT_FLAGS = (
DFLAG_EXTENDED_REFERENCES DFLAG_EXTENDED_REFERENCES
| DFLAG_FUN_TAGS | DFLAG_FUN_TAGS
@ -57,4 +70,5 @@ DEFAULT_FLAGS = (
| DFLAG_HANDSHAKE_23 | DFLAG_HANDSHAKE_23
| DFLAG_UNLINK_ID | DFLAG_UNLINK_ID
| DFLAG_V4_NC | DFLAG_V4_NC
| DFLAG_MANDATORY_25_DIGEST
) )

View file

@ -36,6 +36,7 @@ def make_dist_tls_context(
ca: str, ca: str,
check_hostname: bool = False, check_hostname: bool = False,
minimum_version: ssl.TLSVersion = ssl.TLSVersion.TLSv1_2, minimum_version: ssl.TLSVersion = ssl.TLSVersion.TLSv1_2,
maximum_version: ssl.TLSVersion = ssl.TLSVersion.MAXIMUM_SUPPORTED,
) -> ssl.SSLContext: ) -> ssl.SSLContext:
"""Build an SSLContext for a TLS-dist client. """Build an SSLContext for a TLS-dist client.
@ -43,11 +44,16 @@ def make_dist_tls_context(
and requires one). ``check_hostname=False`` because dist nodes are and requires one). ``check_hostname=False`` because dist nodes are
identified by their cookie + cert chain, not by SNI hostname; flip identified by their cookie + cert chain, not by SNI hostname; flip
on if your CA pins per-node CNs. on if your CA pins per-node CNs.
``maximum_version`` defaults to the highest version Python's ssl
supports (TLS 1.3 in practice). Pin to TLS 1.2 explicitly only if
you hit interop issues against an older Erlang peer.
""" """
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ctx.check_hostname = check_hostname ctx.check_hostname = check_hostname
ctx.verify_mode = ssl.CERT_REQUIRED ctx.verify_mode = ssl.CERT_REQUIRED
ctx.minimum_version = minimum_version ctx.minimum_version = minimum_version
ctx.maximum_version = maximum_version
ctx.load_cert_chain(certfile=cert, keyfile=key) ctx.load_cert_chain(certfile=cert, keyfile=key)
ctx.load_verify_locations(cafile=ca) ctx.load_verify_locations(cafile=ca)
return ctx return ctx

View file

@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "erldistpy" name = "erldistpy"
version = "0.1.0" version = "0.1.7"
description = "Native Python client for Erlang distribution protocol — EPMD + v6 handshake + gen_server call(), no asyncio." description = "Native Python client for Erlang distribution protocol — EPMD + v6 handshake + gen_server call(), no asyncio."
readme = { file = "README.md", content-type = "text/markdown" } readme = { file = "README.md", content-type = "text/markdown" }
requires-python = ">=3.10" requires-python = ">=3.10"

View file

@ -57,7 +57,7 @@ def test_encode_message_with_payload():
def test_decode_rejects_wrong_first_byte(): def test_decode_rejects_wrong_first_byte():
with pytest.raises(ChannelError, match="pass-through byte"): with pytest.raises(ChannelError, match="pass-through .0x70. or dist header"):
_decode_message(b"\x00" + encode((1,))) _decode_message(b"\x00" + encode((1,)))

View file

@ -0,0 +1,181 @@
"""Mix release + TLS dist test — the final 'is it the same as prod?' check.
Builds a minimal Elixir Mix release containing one GenServer registered
as Wallet.Bridge, runs it in a docker container with TLS dist enabled
using the production-matching inet_tls.conf shape, then exercises
erldistpy.Node.call against it.
Together with test_node_otp26.py this exhausts every dimension we
identified differs between dev box (OTP 24, plain Erlang) and the
production portal beam (OTP 25, Mix release, TLS dist, Elixir
GenServer). If THIS passes too, the production-only failure is
something specific to the live portal beam state that we can't
reproduce in isolation most likely an interaction with portal's
other dist connections or its larger supervisor tree.
"""
from __future__ import annotations
import os
import shutil
import socket
import subprocess
import time
from pathlib import Path
import pytest
from erldistpy.etf import Atom
from erldistpy.node import Node
from erldistpy.tls import make_dist_tls_context
RELEASE_NAME = "wallet_bridge_test"
COOKIE = "WALLET_BRIDGE_TEST_COOKIE"
MIX_EXS = """\
defmodule WalletBridgeTest.MixProject do
use Mix.Project
def project, do: [
app: :wallet_bridge_test,
version: "0.1.0",
elixir: "~> 1.16",
deps: [],
releases: [wallet_bridge_test: [include_executables_for: [:unix]]]
]
def application, do: [
extra_applications: [:logger, :ssl, :crypto, :public_key],
mod: {WalletBridgeTest.Application, []}
]
end
"""
LIB_EX = """\
defmodule WalletBridgeTest.Application do
use Application
def start(_, _), do: Supervisor.start_link([Wallet.Bridge], strategy: :one_for_one)
end
defmodule Wallet.Bridge do
use GenServer
require Logger
def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
def init(s), do: {:ok, s}
def handle_call(msg, _from, s), do: {:reply, {:ok, msg}, s}
end
"""
def _docker() -> str | None:
return shutil.which("docker")
def _epmd_node_present(sname: str) -> bool:
try:
result = subprocess.run(
["epmd", "-names"], capture_output=True, text=True, timeout=2
)
except (FileNotFoundError, subprocess.TimeoutExpired):
return False
return f"name {sname} at port" in result.stdout
@pytest.fixture(scope="module")
def mix_release_peer(tmp_path_factory):
if _docker() is None:
pytest.skip("docker not installed")
if not shutil.which("openssl"):
pytest.skip("openssl not installed")
# Must be under $HOME for snap docker to bind-mount it.
base = Path.home() / ".erldistpy-test"
base.mkdir(exist_ok=True)
workdir = base / f"mix_release_{int(time.time())}"
workdir.mkdir()
(workdir / "lib").mkdir()
(workdir / "mix.exs").write_text(MIX_EXS)
(workdir / "lib" / "wallet_bridge.ex").write_text(LIB_EX)
os.chmod(workdir, 0o755)
# Build the release
build = subprocess.run(
["docker", "run", "--rm",
"-v", f"{workdir}:/app", "-w", "/app",
"elixir:1.16-otp-25",
"sh", "-c", "MIX_ENV=prod mix release wallet_bridge_test --overwrite"],
capture_output=True, timeout=180,
)
if build.returncode != 0:
pytest.skip(f"mix release build failed: {build.stderr.decode()[-500:]}")
# Re-use cert generation from test_node_otp26 (we duplicate-import to avoid
# cross-file fixture coupling; if you have it imported already, prefer that.)
from tests.test_node_otp26 import _gen_certs, _write_ssl_config_portal_match
certs = _gen_certs(workdir)
container_certs = {k: v.replace(str(workdir), "/app") for k, v in certs.items()}
_write_ssl_config_portal_match(workdir, container_certs)
# Tweak: ssl_dist_portal.config references /certs paths; rewrite to /app.
cfg_path = workdir / "ssl_dist_portal.config"
cfg_text = cfg_path.read_text().replace("/certs", "/app/certs")
(workdir / "certs").mkdir(exist_ok=True)
for f in ("ca.pem", "server.pem", "server.key", "client.pem", "client.key"):
if (workdir / f).exists():
shutil.copy(workdir / f, workdir / "certs" / f)
cfg_path.write_text(cfg_text)
container_id_file = workdir / "cid"
cid = subprocess.run(
["docker", "run", "--rm", "-d", "--network", "host",
"-v", f"{workdir}:/app", "-w", "/app",
"-e", "RELEASE_DISTRIBUTION=name",
"-e", f"RELEASE_NODE={RELEASE_NAME}@127.0.0.1",
"-e", f"RELEASE_COOKIE={COOKIE}",
"-e", "ERL_FLAGS=-proto_dist inet_tls -ssl_dist_optfile /app/ssl_dist_portal.config",
"elixir:1.16-otp-25",
f"_build/prod/rel/{RELEASE_NAME}/bin/{RELEASE_NAME}", "start"],
capture_output=True, text=True,
).stdout.strip()
container_id_file.write_text(cid)
deadline = time.monotonic() + 20.0
ready = False
while time.monotonic() < deadline:
time.sleep(0.3)
if _epmd_node_present(RELEASE_NAME):
time.sleep(0.5)
ready = True
break
if not ready:
if cid:
subprocess.run(["docker", "kill", cid], capture_output=True)
pytest.skip(f"Mix release {RELEASE_NAME} did not register with epmd")
yield RELEASE_NAME, COOKIE, certs
if cid:
subprocess.run(["docker", "kill", cid], capture_output=True)
def test_call_against_mix_release_tls(mix_release_peer):
"""erldistpy → real Mix release Wallet.Bridge over TLS dist on OTP 25.
Closest local equivalent of the production failure environment.
If THIS passes, the production failure isn't reproducible without
a live portal beam in a mesh meaning the bug is interaction-level
(other connections, supervisor state) not protocol-level.
"""
sname, cookie, certs = mix_release_peer
ctx = make_dist_tls_context(
cert=certs["client_cert"], key=certs["client_key"], ca=certs["ca"],
)
with Node(
our_name="probe@127.0.0.1",
peer_name=sname,
peer_host="127.0.0.1",
cookie=cookie,
tls_context=ctx,
) as n:
assert n.tls is True
reply = n.call("Elixir.Wallet.Bridge", (Atom("ping"), 42), timeout=8.0)
# Bridge wraps every call as {:ok, msg} — matches what the production
# Wallet.Bridge does on the happy path (modulo the cammy forward).
assert reply == (Atom("ok"), (Atom("ping"), 42))

583
tests/test_node_otp26.py Normal file
View file

@ -0,0 +1,583 @@
"""OTP 26 integration tests via Docker.
Reproduces the production failure mode we hit on portal@unsandbox.com:
erldistpy 0.1.7 handshake "succeeds" against OTP 26 but the peer
silently closes the link on the first REG_SEND, surfacing as
``ChannelError: peer closed after 0/4 bytes`` on Node.call().
The pure-Python tests and the existing live tests all run against the
system Erlang, which on most dev boxes is OTP 24 (Ubuntu 22.04 default).
OTP 24 didn't enforce DFLAG_MANDATORY_25_DIGEST or the OTP-25+
mandatory flag set, so flag-negotiation bugs pass silently. These
tests force the modern protocol by running a peer node out of the
official ``erlang:26`` Docker image.
Tests are skipped automatically if Docker isn't available — they're
intentionally NOT part of the smoke that PRs gate on, because they
require docker pull + ~7s of node boot per module.
"""
from __future__ import annotations
import shutil
import socket
import subprocess
import time
import pytest
from pathlib import Path
from erldistpy.etf import Atom
from erldistpy.node import Node
from erldistpy.tls import make_dist_tls_context
SNAME_OTP26 = "erldistpy_otp26"
COOKIE_OTP26 = "ERLDISTPY_OTP26_COOKIE"
# Same gen_target shape as tests/test_node.py — implements the
# gen_server-style {'$gen_call', {From, Ref}, Request} -> From ! {Ref, Reply}
# protocol by hand. Two operations:
# {ping, X} -> {pong, X}
# {add, A, B} -> {ok, A + B}
# Anything else surfaces as {error, {bad_request, _}}.
#
# We also register a heartbeat after 200ms so tests can poll a known
# pid presence rather than depending on a /tmp marker file (which would
# live inside the container, invisible to the host).
ERL_GEN_BOOT = (
"Handle = fun "
" ({ping, X}) -> {pong, X}; "
" ({add, A, B}) -> {ok, A + B}; "
" (Other) -> {error, {bad_request, Other}} "
"end, "
"Loop = fun(F) -> "
" receive "
" {'$gen_call', {From, Ref}, Request} -> "
" From ! {Ref, Handle(Request)}, F(F); "
" Other -> "
" io:format(\"unexpected: ~p~n\", [Other]), F(F) "
" end "
"end, "
"Pid = spawn(fun() -> Loop(Loop) end), "
"register(gen_target, Pid), "
"timer:sleep(infinity)."
)
def _docker_available() -> bool:
return shutil.which("docker") is not None
def _epmd_node_present(sname: str) -> bool:
"""Returns True if ``sname`` is registered in the host's epmd."""
try:
result = subprocess.run(
["epmd", "-names"],
capture_output=True,
text=True,
timeout=2,
)
except (FileNotFoundError, subprocess.TimeoutExpired):
return False
return f"name {sname} at port" in result.stdout
def _socket_open(host: str, port: int) -> bool:
try:
with socket.create_connection((host, port), timeout=0.5):
return True
except OSError:
return False
@pytest.fixture(scope="module")
def otp26_gen_peer():
if not _docker_available():
pytest.skip("docker not installed")
if not _socket_open("127.0.0.1", 4369):
pytest.skip("EPMD not running on 127.0.0.1:4369 (host)")
container_name = f"erldistpy-otp26-{int(time.time())}"
# --network host shares the host's network namespace (Linux only),
# so the OTP 26 node registers with the host's epmd and listens on
# a host-routable interface. epmd lookups from erldistpy on the
# host find the docker node like any other local Erlang node.
proc = subprocess.Popen(
[
"docker",
"run",
"--rm",
"--name",
container_name,
"--network",
"host",
"erlang:25",
"erl",
"-sname",
SNAME_OTP26,
"-setcookie",
COOKIE_OTP26,
"-noshell",
"-eval",
ERL_GEN_BOOT,
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
deadline = time.monotonic() + 15.0
ready = False
while time.monotonic() < deadline:
time.sleep(0.2)
if _epmd_node_present(SNAME_OTP26):
# Allow another 200ms for `register(gen_target, ...)` to land.
time.sleep(0.2)
ready = True
break
if not ready:
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
proc.wait(timeout=2)
pytest.skip(
f"OTP 26 docker node {SNAME_OTP26} did not register with host epmd"
)
yield SNAME_OTP26, COOKIE_OTP26
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
try:
proc.wait(timeout=2)
except subprocess.TimeoutExpired:
proc.kill()
def test_call_ping_otp26(otp26_gen_peer):
"""The minimal repro of the production failure.
erldistpy 0.1.6 silently dropped this call against OTP 26 portal.
0.1.7 added DFLAG_MANDATORY_25_DIGEST; whether THAT was enough is
exactly what this test answers.
"""
sname, cookie = otp26_gen_peer
with Node(our_name="erldistpy_test_otp26@localhost", peer_name=sname, cookie=cookie) as n:
reply = n.call("gen_target", (Atom("ping"), 42), timeout=5.0)
assert reply == (Atom("pong"), 42)
def test_call_add_otp26(otp26_gen_peer):
sname, cookie = otp26_gen_peer
with Node(our_name="erldistpy_test_otp26_add@localhost", peer_name=sname, cookie=cookie) as n:
reply = n.call("gen_target", (Atom("add"), 5, 7), timeout=5.0)
assert reply == (Atom("ok"), 12)
def test_call_sequence_otp26(otp26_gen_peer):
"""Multiple calls on the same dist link survive net_tick window."""
sname, cookie = otp26_gen_peer
with Node(our_name="erldistpy_test_otp26_seq@localhost", peer_name=sname, cookie=cookie) as n:
assert n.call("gen_target", (Atom("ping"), Atom("a")), timeout=5.0) == (Atom("pong"), Atom("a"))
assert n.call("gen_target", (Atom("ping"), Atom("b")), timeout=5.0) == (Atom("pong"), Atom("b"))
assert n.call("gen_target", (Atom("add"), 1, 2), timeout=5.0) == (Atom("ok"), 3)
# ---------------------------------------------------------------------------
# OTP 26 + TLS dist — the production combo (mTLS over Erlang dist)
# ---------------------------------------------------------------------------
TLS_SNAME = "erldistpy_otp26_tls"
TLS_COOKIE = "ERLDISTPY_OTP26_TLS_COOKIE"
def _gen_certs(workdir: Path) -> dict[str, str]:
"""Generate a CA + leaf cert with a SAN matching localhost. Same shape
as tests/test_tls.py duplicated locally to keep this file self-
contained (the docker container needs to read from a known path)."""
import os
hostname = socket.gethostname()
ca_cnf = workdir / "ca.cnf"
ca_cnf.write_text(
"[req]\n"
"distinguished_name = dn\n"
"x509_extensions = v3_ca\n"
"prompt = no\n"
"[dn]\n"
"CN = erldistpy-otp26-ca\n"
"[v3_ca]\n"
"basicConstraints = critical,CA:TRUE\n"
"keyUsage = critical,keyCertSign,cRLSign\n"
"subjectKeyIdentifier = hash\n"
)
subprocess.run(
["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes",
"-keyout", str(workdir / "ca.key"),
"-out", str(workdir / "ca.pem"),
"-days", "1",
"-config", str(ca_cnf), "-extensions", "v3_ca"],
check=True, capture_output=True,
)
leaf_cnf = workdir / "leaf.cnf"
leaf_cnf.write_text(
"[v3]\n"
"basicConstraints = CA:FALSE\n"
"keyUsage = digitalSignature,keyEncipherment\n"
"extendedKeyUsage = serverAuth,clientAuth\n"
"subjectAltName = @alt\n"
"[alt]\n"
f"DNS.1 = {hostname}\n"
"DNS.2 = localhost\n"
)
for name in ("server", "client"):
subprocess.run(
["openssl", "req", "-newkey", "rsa:2048", "-nodes",
"-keyout", str(workdir / f"{name}.key"),
"-out", str(workdir / f"{name}.csr"),
"-subj", f"/CN={hostname}"],
check=True, capture_output=True,
)
subprocess.run(
["openssl", "x509", "-req",
"-in", str(workdir / f"{name}.csr"),
"-CA", str(workdir / "ca.pem"),
"-CAkey", str(workdir / "ca.key"),
"-CAcreateserial",
"-out", str(workdir / f"{name}.pem"),
"-days", "1",
"-extfile", str(leaf_cnf), "-extensions", "v3"],
check=True, capture_output=True,
)
os.chmod(workdir / f"{name}.key", 0o644) # readable by container's erl uid
return {
"ca": str(workdir / "ca.pem"),
"server_cert": str(workdir / "server.pem"),
"server_key": str(workdir / "server.key"),
"client_cert": str(workdir / "client.pem"),
"client_key": str(workdir / "client.key"),
}
def _write_ssl_config(workdir: Path, certs_in_container: dict[str, str]) -> str:
cfg = workdir / "ssl_dist.config"
body = (
"[{server, "
f'[{{certfile, "{certs_in_container["server_cert"]}"}}, '
f'{{keyfile, "{certs_in_container["server_key"]}"}}, '
f'{{cacertfile, "{certs_in_container["ca"]}"}}, '
"{verify, verify_peer}, "
"{fail_if_no_peer_cert, true}]}, "
"{client, "
f'[{{certfile, "{certs_in_container["server_cert"]}"}}, '
f'{{keyfile, "{certs_in_container["server_key"]}"}}, '
f'{{cacertfile, "{certs_in_container["ca"]}"}}, '
"{verify, verify_peer}]}]."
)
cfg.write_text(body)
return str(cfg)
def _write_ssl_config_portal_match(workdir: Path, certs_in_container: dict[str, str]) -> str:
"""Match production portal's /opt/unsandbox/certs/inet_tls.conf exactly:
permissive verify_fun (accepts bad_cert), TLS 1.2/1.3 only, secure
renegotiate. This is the shape the production failure runs under."""
cfg = workdir / "ssl_dist_portal.config"
permissive_verify_fun = (
"{verify_fun, {fun(_,{bad_cert, _}, UserState) -> {valid, UserState}; "
" (_,{extension, _}, UserState) -> {unknown, UserState}; "
" (_, valid, UserState) -> {valid, UserState}; "
" (_, valid_peer, UserState) -> {valid, UserState} "
" end, []}}"
)
body = (
"[{server, "
f'[{{certfile, "{certs_in_container["server_cert"]}"}}, '
f'{{keyfile, "{certs_in_container["server_key"]}"}}, '
f'{{cacertfile, "{certs_in_container["ca"]}"}}, '
"{verify, verify_peer}, "
"{fail_if_no_peer_cert, true}, "
f"{permissive_verify_fun}, "
"{secure_renegotiate, true}, "
"{versions, ['tlsv1.3', 'tlsv1.2']}]}, "
"{client, "
f'[{{certfile, "{certs_in_container["server_cert"]}"}}, '
f'{{keyfile, "{certs_in_container["server_key"]}"}}, '
f'{{cacertfile, "{certs_in_container["ca"]}"}}, '
"{verify, verify_peer}, "
"{server_name_indication, disable}, "
f"{permissive_verify_fun}, "
"{secure_renegotiate, true}, "
"{versions, ['tlsv1.3', 'tlsv1.2']}]}]."
)
cfg.write_text(body)
return str(cfg)
@pytest.fixture(scope="module")
def otp26_tls_peer(tmp_path_factory):
"""OTP 26 + TLS dist peer in Docker — the production combo.
This is the variant that reproduces (or should reproduce) the
portal@unsandbox.com failure: handshake "succeeds" then the peer
silently closes the link on the first REG_SEND.
"""
if not _docker_available():
pytest.skip("docker not installed")
if not shutil.which("openssl"):
pytest.skip("openssl not installed")
if not _socket_open("127.0.0.1", 4369):
pytest.skip("EPMD not running on host")
# Snap-confined Docker (canonical's snap package) can't bind-mount
# /tmp — use a path under $HOME which the confinement allows. Caller
# could override via ERLDISTPY_TLS_OTP26_WORKDIR.
import os as _os
import tempfile as _tempfile
home_base = _os.environ.get(
"ERLDISTPY_TLS_OTP26_WORKDIR", str(Path.home() / ".erldistpy-test")
)
Path(home_base).mkdir(parents=True, exist_ok=True)
workdir = Path(_tempfile.mkdtemp(prefix="tls_otp26_", dir=home_base))
_os.chmod(workdir, 0o755)
host_certs = _gen_certs(workdir)
# Inside the container the certs land at the same path because we
# mount workdir → /certs:ro and we generated under workdir.
container_certs = {
k: v.replace(str(workdir), "/certs") for k, v in host_certs.items()
}
ssl_config_host = _write_ssl_config(workdir, container_certs)
ssl_config_container = ssl_config_host.replace(str(workdir), "/certs")
container_name = f"erldistpy-otp26-tls-{int(time.time())}"
proc = subprocess.Popen(
[
"docker", "run", "--rm",
"--name", container_name,
"--network", "host",
"-v", f"{workdir}:/certs:ro",
"erlang:25",
"erl",
"-sname", TLS_SNAME,
"-setcookie", TLS_COOKIE,
"-proto_dist", "inet_tls",
"-ssl_dist_optfile", ssl_config_container,
"-noshell",
"-eval", ERL_GEN_BOOT,
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
deadline = time.monotonic() + 20.0
ready = False
while time.monotonic() < deadline:
time.sleep(0.3)
if _epmd_node_present(TLS_SNAME):
time.sleep(0.3)
ready = True
break
if not ready:
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
proc.wait(timeout=3)
pytest.skip(f"OTP 26 TLS docker node {TLS_SNAME} did not register")
yield TLS_SNAME, TLS_COOKIE, host_certs
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
try:
proc.wait(timeout=3)
except subprocess.TimeoutExpired:
proc.kill()
def test_call_over_tls_otp26(otp26_tls_peer):
"""The minimum repro of the production failure: TLS-dist on OTP 26.
On erldistpy 0.1.7 against the prod portal this surfaces as
``ChannelError: peer closed after 0/4 bytes`` on the first call.
"""
sname, cookie, certs = otp26_tls_peer
ctx = make_dist_tls_context(
cert=certs["client_cert"],
key=certs["client_key"],
ca=certs["ca"],
)
with Node(
our_name="erldistpy_test_otp26_tls@localhost",
peer_name=sname,
cookie=cookie,
tls_context=ctx,
) as n:
assert n.tls is True
reply = n.call("gen_target", (Atom("ping"), 7), timeout=5.0)
assert reply == (Atom("pong"), 7)
# ---------------------------------------------------------------------------
# OTP 26 + TLS dist + Elixir GenServer — the exact production combo
# ---------------------------------------------------------------------------
ELIXIR_TLS_SNAME = "erldistpy_otp26_elixir_tls"
ELIXIR_TLS_COOKIE = "ERLDISTPY_ELIXIR_TLS"
# Inline Elixir script: spawn a GenServer registered as `gen_target` that
# answers the same {ping, X} / {add, A, B} calls. This is the SHAPE the
# real Wallet.Bridge uses — GenServer.handle_call/3 routing via the
# `:"$gen_call"` envelope, not a hand-coded receive.
ELIXIR_GENSERVER_BOOT = """
defmodule GenTarget do
use GenServer
def start_link, do: GenServer.start_link(__MODULE__, %{}, name: :gen_target)
def init(state) do
IO.puts("[gen_target] init")
{:ok, state}
end
def handle_call(msg, from, state) do
IO.puts("[gen_target] handle_call msg=#{inspect(msg)} from=#{inspect(from)}")
reply = case msg do
{:ping, x} -> {:pong, x}
{:add, a, b} -> {:ok, a + b}
other -> {:error, {:bad_request, other}}
end
{:reply, reply, state}
end
def handle_info(msg, state) do
IO.puts("[gen_target] handle_info msg=#{inspect(msg)}")
{:noreply, state}
end
end
{:ok, _} = GenTarget.start_link()
IO.puts("[gen_target] registered: #{inspect(Process.whereis(:gen_target))}")
Process.sleep(:infinity)
"""
@pytest.fixture(scope="module")
def otp26_elixir_tls_peer(tmp_path_factory):
"""Elixir GenServer over TLS on OTP 26 — exactly what portal runs.
Difference from otp26_tls_peer: the registered process is a real
Elixir GenServer (uses `gen_server:reply/2` for replies, all the
OTP machinery) instead of a hand-coded receive loop.
"""
if not _docker_available():
pytest.skip("docker not installed")
if not shutil.which("openssl"):
pytest.skip("openssl not installed")
if not _socket_open("127.0.0.1", 4369):
pytest.skip("EPMD not running on host")
import os as _os
import tempfile as _tempfile
home_base = _os.environ.get(
"ERLDISTPY_TLS_OTP26_WORKDIR", str(Path.home() / ".erldistpy-test")
)
Path(home_base).mkdir(parents=True, exist_ok=True)
workdir = Path(_tempfile.mkdtemp(prefix="elixir_tls_otp26_", dir=home_base))
_os.chmod(workdir, 0o755)
host_certs = _gen_certs(workdir)
container_certs = {
k: v.replace(str(workdir), "/certs") for k, v in host_certs.items()
}
# Use portal's exact inet_tls.conf shape (permissive verify_fun,
# TLS 1.2/1.3 only, secure_renegotiate) so the test reproduces the
# exact wire-level conditions of the production failure.
_write_ssl_config_portal_match(workdir, container_certs)
ssl_config_container = "/certs/ssl_dist_portal.config"
boot_script = workdir / "boot.exs"
boot_script.write_text(ELIXIR_GENSERVER_BOOT)
container_name = f"erldistpy-elixir-tls-{int(time.time())}"
container_log = workdir / "container.log"
# Long node names (--name) match production. Short names (--sname)
# would route differently through the dist driver and may not
# reproduce the same failure mode.
long_node_name = f"{ELIXIR_TLS_SNAME}@127.0.0.1"
proc = subprocess.Popen(
[
"docker", "run", "--rm",
"--name", container_name,
"--network", "host",
"-v", f"{workdir}:/certs:ro",
"elixir:1.16-otp-25",
"elixir",
"--name", long_node_name,
"--cookie", ELIXIR_TLS_COOKIE,
"--erl", f"-proto_dist inet_tls -ssl_dist_optfile {ssl_config_container}",
"/certs/boot.exs",
],
stdout=open(container_log, "w"),
stderr=subprocess.STDOUT,
)
deadline = time.monotonic() + 25.0
ready = False
while time.monotonic() < deadline:
time.sleep(0.3)
if _epmd_node_present(ELIXIR_TLS_SNAME):
time.sleep(0.5) # Elixir GenServer needs an extra beat to register
ready = True
break
if not ready:
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
proc.wait(timeout=3)
pytest.skip(f"Elixir TLS docker node {ELIXIR_TLS_SNAME} did not register")
yield ELIXIR_TLS_SNAME, ELIXIR_TLS_COOKIE, host_certs, container_log
subprocess.run(["docker", "rm", "-f", container_name], capture_output=True)
proc.terminate()
try:
proc.wait(timeout=3)
except subprocess.TimeoutExpired:
proc.kill()
def test_call_elixir_genserver_over_tls_otp26(otp26_elixir_tls_peer):
"""Repro of the prod failure: erldistpy calls an Elixir GenServer over TLS.
Difference from test_call_over_tls_otp26: peer is a real
GenServer, not a hand-coded receive. Replies go through
:gen_server.reply/2 which sends via :erlang.send/2 same path as
Wallet.Bridge in production.
"""
sname, cookie, certs, container_log = otp26_elixir_tls_peer
ctx = make_dist_tls_context(
cert=certs["client_cert"],
key=certs["client_key"],
ca=certs["ca"],
)
try:
with Node(
our_name="erldistpy_test_elixir_tls@localhost",
peer_name=sname,
cookie=cookie,
tls_context=ctx,
) as n:
assert n.tls is True
reply = n.call("gen_target", (Atom("ping"), 7), timeout=5.0)
assert reply == (Atom("pong"), 7)
finally:
# Whether pass or fail, dump container output for diagnosis.
try:
text = container_log.read_text()
print(f"=== container stdout ===\n{text}\n=== end ===")
except FileNotFoundError:
pass