# MPS-16: Bring Your Own Bucket (BYOB) — Primary S3 Per Shop

## Summary

Allow shops to use their own S3-compatible bucket as the **primary** storage
for all media (products, thumbnails, previews, karaoke tracks). Files are
uploaded directly to the shop's bucket — MPS never stores them on the
platform bucket.

Free trial users must bring their own bucket during onboarding (zero storage
cost for MPS during trial). Paid plan users can use BYOB or the MPS bucket.

## Current Architecture

- All uploads go to MPS DigitalOcean Spaces bucket (global, configured in INI)
- `request.secure_uploads_client` is a single global boto3 client
- Presigned URLs always use `request.app["bucket.secure_uploads.get_endpoint"]`
- Mirror S3 (`shop.mirror_s3_*`) is an async secondary copy

## Target Architecture

- Each shop can optionally specify a **primary** S3 bucket
- If configured, all uploads, presigned URLs, and thumbnail CDN URLs use the shop's bucket
- MPS bucket is never touched for BYOB shops
- Mirror S3 continues to work as a secondary copy (shop can mirror from their primary to another bucket)

## Model Changes

### Shop model (`models/shop.py`)

Reuse existing `mirror_s3_*` columns but add a new flag to indicate primary vs mirror:

```python
primary_s3_enabled = Column(Boolean, default=False)
# When True: mirror_s3_* columns are used as the PRIMARY bucket
# When False: mirror_s3_* columns are used as mirror (current behavior)
```

Or add separate columns for clarity:

```python
primary_s3_endpoint = Column(Unicode(256), nullable=True)
primary_s3_region = Column(Unicode(64), nullable=True)
primary_s3_bucket = Column(Unicode(128), nullable=True)
primary_s3_access_key = Column(Unicode(128), nullable=True)
primary_s3_secret_key = Column(Unicode(128), nullable=True)
primary_s3_cdn_endpoint = Column(Unicode(256), nullable=True)  # public CDN URL for thumbnails
primary_s3_enabled = Column(Boolean, default=False)
```

The CDN endpoint is critical — thumbnails and previews use public CDN URLs,
not presigned URLs. The shop owner must configure their bucket's CDN endpoint
(e.g., `https://mybucket.nyc3.cdn.digitaloceanspaces.com`).

### Properties

```python
@property
def has_primary_s3(self):
    return bool(
        self.primary_s3_enabled
        and self.primary_s3_endpoint
        and self.primary_s3_bucket
        and self.primary_s3_access_key
        and self.primary_s3_secret_key
        and self.primary_s3_cdn_endpoint
    )

@property
def media_cdn_endpoint(self):
    """Return the CDN endpoint for this shop's media."""
    if self.has_primary_s3:
        return self.primary_s3_cdn_endpoint
    return None  # caller falls back to request.app default
```

## Request Method Changes

### `request_methods.py`

Add a shop-aware S3 client factory:

```python
def add_shop_uploads_client(request):
    """Return S3 client for the current shop (BYOB or MPS default)."""
    shop = request.shop
    if shop and shop.has_primary_s3:
        import boto3
        session = boto3.session.Session()
        return session.client(
            "s3",
            region_name=shop.primary_s3_region,
            endpoint_url=shop.primary_s3_endpoint,
            aws_access_key_id=shop.primary_s3_access_key,
            aws_secret_access_key=shop.primary_s3_secret_key,
        )
    return request.secure_uploads_client  # default MPS bucket
```

Add `request.shop_uploads_client` as a reified request method.

### Bucket name resolution

```python
def get_shop_bucket_name(request):
    shop = request.shop
    if shop and shop.has_primary_s3:
        return shop.primary_s3_bucket
    return request.app["bucket.secure_uploads"]
```

Add `request.shop_bucket_name` as a reified request method.

## View Changes

### `views/product.py`

All S3 operations must use `request.shop_uploads_client` and
`request.shop_bucket_name` instead of `request.secure_uploads_client` and
`request.app["bucket.secure_uploads"]`:

- **Presigned GET** (downloads, line 45-83): use shop client + shop bucket
- **Presigned POST** (uploads, line 429-456): use shop client + shop bucket
- **Copy object** (line 344): use shop client + shop bucket
- **Delete object**: use shop client + shop bucket

### Template changes

All thumbnail/media URLs must resolve through the shop's CDN endpoint:

```jinja2
{# Before #}
{{ request.app["bucket.secure_uploads.get_endpoint"] }}/{{ product.s3_path }}/thumbnail1

{# After #}
{{ product.shop.media_cdn_endpoint or request.app["bucket.secure_uploads.get_endpoint"] }}/{{ product.s3_path }}/thumbnail1
```

Affected templates:
- `product.j2` (lines 13, 43, 75, 97, 112, 131-132)
- `cart.j2` (line 124)
- `shop.j2` (thumbnail rendering)
- `content.j2`
- `snippets/related_content.j2`

### `views/watch.py`

Watch mode JSON endpoint (line 96-99) must use shop CDN endpoint:

```python
cdn = shop.media_cdn_endpoint or request.app["bucket.secure_uploads.get_endpoint"]
thumbnail_url = f"{cdn}/{product.s3_path}/thumbnail1?ts={product.updated_timestamp}"
```

### `lib/karaoke.py`

Karaoke downloads/uploads must use shop client + shop bucket.

### `lib/s3_mirror.py`

When primary_s3 is enabled, mirror source becomes the shop's bucket (not MPS).
Mirror destination is still the mirror_s3_* config.

## Settings UI

### New form section: `bucket-settings`

In `shop_settings.j2`, add a "Storage" or "Media Bucket" section:

- Endpoint URL (text input)
- Region (text input)
- Bucket name (text input)
- Access key (text input)
- Secret key (password input)
- CDN endpoint (text input, with help text: "Public URL for thumbnails")
- Enable checkbox
- Test connection button (reuse `test_mirror_connection` pattern)

### Validation

- Endpoint must start with `https://`
- All 6 fields required if any provided
- Connection test: list bucket, attempt a test PUT/GET/DELETE cycle
- CDN endpoint must be reachable (optional HEAD request)

## Onboarding for Trial Users

During shop creation (`/s/new`), after the shop is created and the user is
redirected to `/s/{shop_id}/settings`:

- If trial user (no paid plan), show a prominent "Set Up Storage" step
- Guide them through configuring their S3 bucket
- Trial shops cannot upload files until BYOB is configured
- Provide documentation links for DigitalOcean Spaces, AWS S3, Backblaze B2, MinIO

## Migration

- Add `primary_s3_*` columns (6 columns) to `mps_shop`
- All nullable, `primary_s3_enabled` server_default="0"
- Idempotent guards

## Tests

- Unit: has_primary_s3 property, media_cdn_endpoint property
- Integration: shop with BYOB config, verify client resolution
- Functional: enable BYOB via settings, verify connection test, verify upload uses shop bucket
