docs: PyPI Trusted Publishing (OIDC) migration plan

When we're ready to drop the twine<6 + setuptools<77 pins on the
build runner, this doc is the recipe. Covers:

  - PyPI side: registering a pending publisher per project
  - GitLab side: id_tokens: PYPI_ID_TOKEN: aud: pypi
  - What pins to drop after migration
  - Why coordinated across all four python/* repos in one go

Today we ship via classic ~/.pypirc on the build runner. Trusted
Publishing replaces that with short-lived OIDC tokens minted per
pipeline. Per-project scope, per-pipeline expiry, no long-lived
secret on the runner.
This commit is contained in:
russell@unturf.com 2026-06-16 14:20:00 -04:00
parent ea2bc9a12c
commit 9b3039fa6f
No known key found for this signature in database

View file

@ -0,0 +1,111 @@
# Migrating to PyPI Trusted Publishing (OIDC)
Today's auth path: classic API token in `/home/gitlab-runner/.pypirc`
on `build.unturf.com`, used by twine 5.x. Works, but every project that
ships from this runner shares one token, rotation is manual, and we're
pinning `twine<6` + `setuptools<77` to keep the legacy fallback alive.
Trusted Publishing replaces that with short-lived OIDC tokens that
GitLab mints per-job and PyPI exchanges for an upload-only API token.
No long-lived secret on the runner. Per-project, audited per-pipeline.
## When to migrate
Single coordinated change across all four python/* repos. Each repo
needs its PyPI pending publisher set up *before* its CI YAML switches.
If you do it piecemeal a half-migrated repo will break on the next tag.
Order: ago, erldistpy, make-post-sell, remarkbox (or any order — each
is independent once its pending publisher is registered).
## Per-project setup
### Step 1 — register a pending publisher on PyPI
For an existing project (ago, make-post-sell, remarkbox):
1. Log into pypi.org as the project owner
2. Project page → **Manage****Publishing** → **Add a new publisher**
3. Select **GitLab**
4. Fill in:
- **Namespace**: `python` (or `engineering/make-post-sell` etc. — the path before the repo name)
- **Project**: `erldistpy` (or `ago` / `remarkbox` / `make_post_sell`)
- **Workflow filepath**: `.gitlab-ci.yml`
- **Environment name**: leave empty (we don't use GitLab environments for this)
5. Save
For a brand-new project (erldistpy on first publish):
1. Log into pypi.org
2. **Your projects****Publishing** → **Add a pending publisher**
3. Same fields as above plus a **PyPI project name** (`erldistpy`)
4. Save — pending publishers are valid for the first publish, then
auto-convert to a normal publisher entry
### Step 2 — flip the project's `.gitlab-ci.yml`
Replace the `pypi-twine` stage with:
```yaml
pypi-twine:
stage: pypi-twine
tags: ["build"]
only:
- tags
id_tokens:
PYPI_ID_TOKEN:
aud: pypi
script:
- python3 -m venv .venv
- . .venv/bin/activate
- pip install --upgrade pip
- pip install build twine # no <6 pin needed anymore
- python -m build # no setuptools cap needed anymore
- twine check dist/*
- twine upload --non-interactive dist/*
```
Key change: the `id_tokens:` block tells GitLab to mint a short-lived
OIDC ID token (audience `pypi`) and inject it as the `PYPI_ID_TOKEN`
env var. Twine 6+ sees it, auto-exchanges it with PyPI, and uses the
returned scoped token for the upload.
### Step 3 — drop the workarounds we have today
Once a project is on Trusted Publishing, remove:
- `pip install "twine<6"` → back to `pip install twine`
- `requires = ["setuptools>=68,<77", "wheel"]` → back to `["setuptools>=68", "wheel"]`
### Step 4 — verify
Tag a patch release (e.g. `0.1.3`) and watch the pipeline. A successful
job log will include something like:
```
$ twine upload --non-interactive dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Trusted publishing: minting an OIDC token...
Uploading erldistpy-0.1.3-py3-none-any.whl
Uploading erldistpy-0.1.3.tar.gz
```
## Why this is worth doing
- **No long-lived token on disk.** `~/.pypirc` on the build runner
becomes deletable once all four repos migrate.
- **Per-project scope.** A leaked token from one project can't upload
to others.
- **Per-pipeline expiry.** OIDC tokens are valid for minutes, not the
lifetime of an API key.
- **No pinning twine or setuptools.** Modern wheels, modern checks.
## Why we're not doing it today
- First publish wants a *pending publisher* set up before the tag is
pushed; couldn't do that without changing PyPI account settings.
- Coordinating four repos in one change is a discrete chunk worth
scheduling rather than fitting between tasks.
When you're ready, this doc is the recipe. The Step 1 → Step 2 pair is
the only thing that needs to happen per-repo.