docs: PyPI Trusted Publishing (OIDC) migration plan
When we're ready to drop the twine<6 + setuptools<77 pins on the build runner, this doc is the recipe. Covers: - PyPI side: registering a pending publisher per project - GitLab side: id_tokens: PYPI_ID_TOKEN: aud: pypi - What pins to drop after migration - Why coordinated across all four python/* repos in one go Today we ship via classic ~/.pypirc on the build runner. Trusted Publishing replaces that with short-lived OIDC tokens minted per pipeline. Per-project scope, per-pipeline expiry, no long-lived secret on the runner.
This commit is contained in:
parent
ea2bc9a12c
commit
9b3039fa6f
1 changed files with 111 additions and 0 deletions
111
docs/PYPI-TRUSTED-PUBLISHING.md
Normal file
111
docs/PYPI-TRUSTED-PUBLISHING.md
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
# Migrating to PyPI Trusted Publishing (OIDC)
|
||||||
|
|
||||||
|
Today's auth path: classic API token in `/home/gitlab-runner/.pypirc`
|
||||||
|
on `build.unturf.com`, used by twine 5.x. Works, but every project that
|
||||||
|
ships from this runner shares one token, rotation is manual, and we're
|
||||||
|
pinning `twine<6` + `setuptools<77` to keep the legacy fallback alive.
|
||||||
|
|
||||||
|
Trusted Publishing replaces that with short-lived OIDC tokens that
|
||||||
|
GitLab mints per-job and PyPI exchanges for an upload-only API token.
|
||||||
|
No long-lived secret on the runner. Per-project, audited per-pipeline.
|
||||||
|
|
||||||
|
## When to migrate
|
||||||
|
|
||||||
|
Single coordinated change across all four python/* repos. Each repo
|
||||||
|
needs its PyPI pending publisher set up *before* its CI YAML switches.
|
||||||
|
If you do it piecemeal a half-migrated repo will break on the next tag.
|
||||||
|
|
||||||
|
Order: ago, erldistpy, make-post-sell, remarkbox (or any order — each
|
||||||
|
is independent once its pending publisher is registered).
|
||||||
|
|
||||||
|
## Per-project setup
|
||||||
|
|
||||||
|
### Step 1 — register a pending publisher on PyPI
|
||||||
|
|
||||||
|
For an existing project (ago, make-post-sell, remarkbox):
|
||||||
|
|
||||||
|
1. Log into pypi.org as the project owner
|
||||||
|
2. Project page → **Manage** → **Publishing** → **Add a new publisher**
|
||||||
|
3. Select **GitLab**
|
||||||
|
4. Fill in:
|
||||||
|
- **Namespace**: `python` (or `engineering/make-post-sell` etc. — the path before the repo name)
|
||||||
|
- **Project**: `erldistpy` (or `ago` / `remarkbox` / `make_post_sell`)
|
||||||
|
- **Workflow filepath**: `.gitlab-ci.yml`
|
||||||
|
- **Environment name**: leave empty (we don't use GitLab environments for this)
|
||||||
|
5. Save
|
||||||
|
|
||||||
|
For a brand-new project (erldistpy on first publish):
|
||||||
|
|
||||||
|
1. Log into pypi.org
|
||||||
|
2. **Your projects** → **Publishing** → **Add a pending publisher**
|
||||||
|
3. Same fields as above plus a **PyPI project name** (`erldistpy`)
|
||||||
|
4. Save — pending publishers are valid for the first publish, then
|
||||||
|
auto-convert to a normal publisher entry
|
||||||
|
|
||||||
|
### Step 2 — flip the project's `.gitlab-ci.yml`
|
||||||
|
|
||||||
|
Replace the `pypi-twine` stage with:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
pypi-twine:
|
||||||
|
stage: pypi-twine
|
||||||
|
tags: ["build"]
|
||||||
|
only:
|
||||||
|
- tags
|
||||||
|
id_tokens:
|
||||||
|
PYPI_ID_TOKEN:
|
||||||
|
aud: pypi
|
||||||
|
script:
|
||||||
|
- python3 -m venv .venv
|
||||||
|
- . .venv/bin/activate
|
||||||
|
- pip install --upgrade pip
|
||||||
|
- pip install build twine # no <6 pin needed anymore
|
||||||
|
- python -m build # no setuptools cap needed anymore
|
||||||
|
- twine check dist/*
|
||||||
|
- twine upload --non-interactive dist/*
|
||||||
|
```
|
||||||
|
|
||||||
|
Key change: the `id_tokens:` block tells GitLab to mint a short-lived
|
||||||
|
OIDC ID token (audience `pypi`) and inject it as the `PYPI_ID_TOKEN`
|
||||||
|
env var. Twine 6+ sees it, auto-exchanges it with PyPI, and uses the
|
||||||
|
returned scoped token for the upload.
|
||||||
|
|
||||||
|
### Step 3 — drop the workarounds we have today
|
||||||
|
|
||||||
|
Once a project is on Trusted Publishing, remove:
|
||||||
|
|
||||||
|
- `pip install "twine<6"` → back to `pip install twine`
|
||||||
|
- `requires = ["setuptools>=68,<77", "wheel"]` → back to `["setuptools>=68", "wheel"]`
|
||||||
|
|
||||||
|
### Step 4 — verify
|
||||||
|
|
||||||
|
Tag a patch release (e.g. `0.1.3`) and watch the pipeline. A successful
|
||||||
|
job log will include something like:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ twine upload --non-interactive dist/*
|
||||||
|
Uploading distributions to https://upload.pypi.org/legacy/
|
||||||
|
Trusted publishing: minting an OIDC token...
|
||||||
|
Uploading erldistpy-0.1.3-py3-none-any.whl
|
||||||
|
Uploading erldistpy-0.1.3.tar.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
## Why this is worth doing
|
||||||
|
|
||||||
|
- **No long-lived token on disk.** `~/.pypirc` on the build runner
|
||||||
|
becomes deletable once all four repos migrate.
|
||||||
|
- **Per-project scope.** A leaked token from one project can't upload
|
||||||
|
to others.
|
||||||
|
- **Per-pipeline expiry.** OIDC tokens are valid for minutes, not the
|
||||||
|
lifetime of an API key.
|
||||||
|
- **No pinning twine or setuptools.** Modern wheels, modern checks.
|
||||||
|
|
||||||
|
## Why we're not doing it today
|
||||||
|
|
||||||
|
- First publish wants a *pending publisher* set up before the tag is
|
||||||
|
pushed; couldn't do that without changing PyPI account settings.
|
||||||
|
- Coordinating four repos in one change is a discrete chunk worth
|
||||||
|
scheduling rather than fitting between tasks.
|
||||||
|
|
||||||
|
When you're ready, this doc is the recipe. The Step 1 → Step 2 pair is
|
||||||
|
the only thing that needs to happen per-repo.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue