russell@unturf.com
dee08976ef
Trusted Publishing OIDC is blocked for git.unturf.com — PyPI's GitLab
provider hardcodes the issuer to gitlab.com (no self-hosted instance
field in the Add publisher form). Until PyPI lights up self-hosted
GitLab support, classic API token auth is our path.
Auth source: TWINE_USERNAME=__token__ + TWINE_PASSWORD=<pypi-...> set
as group-level CI variables on python/. Both python/ago and
python/erldistpy inherit them automatically; engineering/ group gets
the same vars for make-post-sell + remarkbox.
What this commit changes:
- .gitlab-ci.yml pypi-twine stage drops the id_tokens block we
briefly tried (OIDC), uses latest twine (no <6 pin) — twine 6
reads TWINE_USERNAME/TWINE_PASSWORD env vars and prefers them
over the OIDC attempt
- pyproject.toml build-system requires drops the setuptools<77
cap (latest setuptools emits Metadata-Version 2.4 which twine 6
reads fine; locally verified)
- docs/PYPI-TRUSTED-PUBLISHING.md updated to note the issuer-
hardcoded blocker and the recipe to migrate later when it
unblocks (or if we mirror to gitlab.com)
120 lines
4.3 KiB
Markdown
120 lines
4.3 KiB
Markdown
# Migrating to PyPI Trusted Publishing (OIDC)
|
|
|
|
**Status: BLOCKED for self-hosted git.unturf.com**
|
|
|
|
PyPI's GitLab Trusted Publisher provider has the issuer URL **hardcoded
|
|
to `https://gitlab.com`**. There's no "Issuer URL" / "GitLab instance"
|
|
field in the "Add publisher" form. Until PyPI ships support for custom
|
|
GitLab issuers (or we mirror releases to gitlab.com), Trusted
|
|
Publishing is off the table for our python/* repos.
|
|
|
|
Today's working auth path: project-scoped GitLab CI variables
|
|
``TWINE_USERNAME=__token__`` + ``TWINE_PASSWORD=<pypi-...>`` (masked +
|
|
protected). Twine 6 reads them as env vars and skips the OIDC attempt.
|
|
Per-project tokens; can be rotated independently.
|
|
|
|
When PyPI lights up self-hosted GitLab support (track:
|
|
https://github.com/pypi/warehouse/issues — search "self-hosted gitlab
|
|
trusted publisher"), or if we move to gitlab.com, the recipe below
|
|
applies.
|
|
|
|
## When to migrate (once unblocked)
|
|
|
|
Single coordinated change across all four python/* repos. Each repo
|
|
needs its PyPI pending publisher set up *before* its CI YAML switches.
|
|
If you do it piecemeal a half-migrated repo will break on the next tag.
|
|
|
|
Order: ago, erldistpy, make-post-sell, remarkbox (or any order — each
|
|
is independent once its pending publisher is registered).
|
|
|
|
## Per-project setup
|
|
|
|
### Step 1 — register a pending publisher on PyPI
|
|
|
|
For an existing project (ago, make-post-sell, remarkbox):
|
|
|
|
1. Log into pypi.org as the project owner
|
|
2. Project page → **Manage** → **Publishing** → **Add a new publisher**
|
|
3. Select **GitLab**
|
|
4. Fill in:
|
|
- **Namespace**: `python` (or `engineering/make-post-sell` etc. — the path before the repo name)
|
|
- **Project**: `erldistpy` (or `ago` / `remarkbox` / `make_post_sell`)
|
|
- **Workflow filepath**: `.gitlab-ci.yml`
|
|
- **Environment name**: leave empty (we don't use GitLab environments for this)
|
|
5. Save
|
|
|
|
For a brand-new project (erldistpy on first publish):
|
|
|
|
1. Log into pypi.org
|
|
2. **Your projects** → **Publishing** → **Add a pending publisher**
|
|
3. Same fields as above plus a **PyPI project name** (`erldistpy`)
|
|
4. Save — pending publishers are valid for the first publish, then
|
|
auto-convert to a normal publisher entry
|
|
|
|
### Step 2 — flip the project's `.gitlab-ci.yml`
|
|
|
|
Replace the `pypi-twine` stage with:
|
|
|
|
```yaml
|
|
pypi-twine:
|
|
stage: pypi-twine
|
|
tags: ["build"]
|
|
only:
|
|
- tags
|
|
id_tokens:
|
|
PYPI_ID_TOKEN:
|
|
aud: pypi
|
|
script:
|
|
- python3 -m venv .venv
|
|
- . .venv/bin/activate
|
|
- pip install --upgrade pip
|
|
- pip install build twine # no <6 pin needed anymore
|
|
- python -m build # no setuptools cap needed anymore
|
|
- twine check dist/*
|
|
- twine upload --non-interactive dist/*
|
|
```
|
|
|
|
Key change: the `id_tokens:` block tells GitLab to mint a short-lived
|
|
OIDC ID token (audience `pypi`) and inject it as the `PYPI_ID_TOKEN`
|
|
env var. Twine 6+ sees it, auto-exchanges it with PyPI, and uses the
|
|
returned scoped token for the upload.
|
|
|
|
### Step 3 — drop the workarounds we have today
|
|
|
|
Once a project is on Trusted Publishing, remove:
|
|
|
|
- `pip install "twine<6"` → back to `pip install twine`
|
|
- `requires = ["setuptools>=68,<77", "wheel"]` → back to `["setuptools>=68", "wheel"]`
|
|
|
|
### Step 4 — verify
|
|
|
|
Tag a patch release (e.g. `0.1.3`) and watch the pipeline. A successful
|
|
job log will include something like:
|
|
|
|
```
|
|
$ twine upload --non-interactive dist/*
|
|
Uploading distributions to https://upload.pypi.org/legacy/
|
|
Trusted publishing: minting an OIDC token...
|
|
Uploading erldistpy-0.1.3-py3-none-any.whl
|
|
Uploading erldistpy-0.1.3.tar.gz
|
|
```
|
|
|
|
## Why this is worth doing
|
|
|
|
- **No long-lived token on disk.** `~/.pypirc` on the build runner
|
|
becomes deletable once all four repos migrate.
|
|
- **Per-project scope.** A leaked token from one project can't upload
|
|
to others.
|
|
- **Per-pipeline expiry.** OIDC tokens are valid for minutes, not the
|
|
lifetime of an API key.
|
|
- **No pinning twine or setuptools.** Modern wheels, modern checks.
|
|
|
|
## Why we're not doing it today
|
|
|
|
- First publish wants a *pending publisher* set up before the tag is
|
|
pushed; couldn't do that without changing PyPI account settings.
|
|
- Coordinating four repos in one change is a discrete chunk worth
|
|
scheduling rather than fitting between tasks.
|
|
|
|
When you're ready, this doc is the recipe. The Step 1 → Step 2 pair is
|
|
the only thing that needs to happen per-repo.
|