erldistpy/docs/PYPI-TRUSTED-PUBLISHING.md

Migrating to PyPI Trusted Publishing (OIDC)

Status: BLOCKED for self-hosted git.unturf.com

PyPI's GitLab Trusted Publisher provider has the issuer URL hardcoded to https://gitlab.com. There's no "Issuer URL" / "GitLab instance" field in the "Add publisher" form. Until PyPI ships support for custom GitLab issuers (or we mirror releases to gitlab.com), Trusted Publishing is off the table for our python/* repos.

Today's working auth path: project-scoped GitLab CI variables TWINE_USERNAME=__token__ + TWINE_PASSWORD=<pypi-...> (masked + protected). Twine 6 reads them as env vars and skips the OIDC attempt. Per-project tokens; can be rotated independently.

When PyPI lights up self-hosted GitLab support (track: https://github.com/pypi/warehouse/issues — search "self-hosted gitlab trusted publisher"), or if we move to gitlab.com, the recipe below applies.

When to migrate (once unblocked)

Single coordinated change across all four python/* repos. Each repo needs its PyPI pending publisher set up before its CI YAML switches. If you do it piecemeal a half-migrated repo will break on the next tag.

Order: ago, erldistpy, make-post-sell, remarkbox (or any order — each is independent once its pending publisher is registered).

Per-project setup

Step 1 — register a pending publisher on PyPI

For an existing project (ago, make-post-sell, remarkbox):

1. Log into pypi.org as the project owner
2. Project page → Manage → Publishing → Add a new publisher
3. Select GitLab
4. Fill in:
   • Namespace: python (or engineering/make-post-sell etc. — the path before the repo name)
   • Project: erldistpy (or ago / remarkbox / make_post_sell)
   • Workflow filepath: .gitlab-ci.yml
   • Environment name: leave empty (we don't use GitLab environments for this)
5. Save

For a brand-new project (erldistpy on first publish):

1. Log into pypi.org
2. Your projects → Publishing → Add a pending publisher
3. Same fields as above plus a PyPI project name (erldistpy)
4. Save — pending publishers are valid for the first publish, then auto-convert to a normal publisher entry

Step 2 — flip the project's .gitlab-ci.yml

Replace the pypi-twine stage with:

pypi-twine:
  stage: pypi-twine
  tags: ["build"]
  only:
    - tags
  id_tokens:
    PYPI_ID_TOKEN:
      aud: pypi
  script:
    - python3 -m venv .venv
    - . .venv/bin/activate
    - pip install --upgrade pip
    - pip install build twine        # no <6 pin needed anymore
    - python -m build                # no setuptools cap needed anymore
    - twine check dist/*
    - twine upload --non-interactive dist/*

Key change: the id_tokens: block tells GitLab to mint a short-lived OIDC ID token (audience pypi) and inject it as the PYPI_ID_TOKEN env var. Twine 6+ sees it, auto-exchanges it with PyPI, and uses the returned scoped token for the upload.

Step 3 — drop the workarounds we have today

Once a project is on Trusted Publishing, remove:

• pip install "twine<6" → back to pip install twine
• requires = ["setuptools>=68,<77", "wheel"] → back to ["setuptools>=68", "wheel"]

Step 4 — verify

Tag a patch release (e.g. 0.1.3) and watch the pipeline. A successful job log will include something like:

$ twine upload --non-interactive dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Trusted publishing: minting an OIDC token...
Uploading erldistpy-0.1.3-py3-none-any.whl
Uploading erldistpy-0.1.3.tar.gz

Why this is worth doing

• No long-lived token on disk. ~/.pypirc on the build runner becomes deletable once all four repos migrate.
• Per-project scope. A leaked token from one project can't upload to others.
• Per-pipeline expiry. OIDC tokens are valid for minutes, not the lifetime of an API key.
• No pinning twine or setuptools. Modern wheels, modern checks.

Why we're not doing it today

• First publish wants a pending publisher set up before the tag is pushed; couldn't do that without changing PyPI account settings.
• Coordinating four repos in one change is a discrete chunk worth scheduling rather than fitting between tasks.

When you're ready, this doc is the recipe. The Step 1 → Step 2 pair is the only thing that needs to happen per-repo.