upload.unturf.com/README.rst

===========================================
PyraFiles - A Sophisticated Pyramid Project
===========================================

**PyraFiles** is a **public domain** application built on the `Pyramid <https://trypyramid.com>`_ framework. It demonstrates a flexible, multi-database design where each namespace (group of users and agents) maintains its own SQLite database for media uploads (images, audio, video). The main database (``main.db``) stores system-level data, including user accounts, namespaces, agents, and roles.

PyraFiles supports both human users and agents:

- **Users** authenticate via OTP (One-Time Password) email verification codes and interact through the web interface.
- **Agents** authenticate via JWT tokens and can interact programmatically with the application.

Key Features
============

- **Passwordless Login** using email verification codes (OTP).
- **Namespaces** to organize media files and control access.
- **Role-Based Access Control** with roles: *owner*, *editor*, and *reader*.
- **JWT Authentication** for agents, enabling programmatic access.
- **Per-Namespace SQLite Databases** for media uploads.
- **Public/Private** namespace and media visibility controls.
- **Admin Tools** for namespace and agent management.
- **OpenAPI Documentation** available for API interactions.
- **Configurable** via environment variables (including secret keys).

Git Repository
==============

The project is maintained at:

- `PyraFiles Git Repo <https://git.unturf.com/engineering/unturf/upload.unturf.com>`_

Since this is public domain, you can adapt and redistribute it freely.

Configuration via Environment
=============================

PyraFiles fetches settings from environment variables with sensible defaults:

- ``PYRAFILES_SECRET``  
  The secret key for session signing.  
  If missing, PyraFiles automatically generates a **random 64-character** string at runtime, which will log out all users.

- ``PYRAFILES_JWT_SECRET``  
  The secret key used for signing JWT tokens for agents.  
  If missing, PyraFiles automatically generates a **random 64-character** string at runtime, which will invalidate existing agent tokens.

- ``PYRAFILES_DB_URL``  
  Connection string for the main database. Default: ``sqlite:///data/main.db``.

- ``PYRAFILES_HOST`` and ``PYRAFILES_PORT``  
  The host and port to serve on. Defaults: ``0.0.0.0`` (host), ``6544`` (port).

- ``PYRAFILES_SMTP_HOST`` and ``PYRAFILES_SMTP_PORT``  
  SMTP server details. Defaults: ``localhost:25``.

- Any other environment variables you wish to incorporate can be accessed in the code.

Local Setup
===========

1. **Clone the Project**

   .. code-block:: bash

      git clone https://git.unturf.com/engineering/unturf/upload.unturf.com.git
      cd upload.unturf.com

2. **Create a Virtual Environment**

   .. code-block:: bash

      python3 -m venv venv
      source venv/bin/activate  # Linux/Mac
      # or
      venv\Scripts\activate.bat # Windows

3. **Install Dependencies**

   .. code-block:: bash

      pip install -r requirements.txt

4. **Initialize the Database**

   .. code-block:: bash

      python initialize_db.py

   This creates or updates the main database file (``data/main.db``) and ensures all tables are set up.

5. **Run the Application**

   .. code-block:: bash

      # Optionally set PYRAFILES_SECRET and PYRAFILES_JWT_SECRET if you want custom secrets.
      export PYRAFILES_SECRET="YOUR_OWN_LONG_RANDOM_STRING"
      export PYRAFILES_JWT_SECRET="YOUR_OWN_LONG_RANDOM_STRING"
      python app.py

   If ``PYRAFILES_SECRET`` and ``PYRAFILES_JWT_SECRET`` are **not** set, the app automatically generates 64-character secrets at runtime, which will log out all users and invalidate existing JWT tokens.

6. **Access the Application**

   Point your browser to `http://localhost:6544` or `http://<HOST>:<PORT>` according to your environment variables.

OTP Authentication
==================

PyraFiles uses a passwordless login system for users:

1. **Login with Email**

   - Users enter their email address on the login page.
   - A 6-digit verification code is sent to the provided email address.

2. **Verify with Code**

   - Users enter the 6-digit code on the verification page.
   - Upon successful verification, the user is logged in.

3. **Session Management**

   - User sessions are managed via signed cookies.
   - Sessions persist across browser restarts unless the server secret changes.

JWT Agent Authentication
========================

Agents authenticate with PyraFiles using Cookies or JWT tokens, allowing programmatic interaction:

1. **Generate Agent JWT**

   - **Owners** of a namespace can generate JWT tokens for agents.
   - Agents are assigned a role: *owner*, *editor*, or *reader*.
   - Each agent has a unique token, which includes their role and namespace ID.

2. **Authenticate with JWT**

   - Agents include the JWT in the `Authorization` header as a Bearer token:

     .. code-block:: http

        Authorization: Bearer <JWT_TOKEN>

   - The token is verified on each request, and the agent's role is used for access control.

3. **Access Control**

   - **Owner** agents can manage the namespace, including inviting users and creating other agents.
   - **Editor** agents can upload, edit, and delete media within the namespace.
   - **Reader** agents can view media if they have appropriate permissions.

4. **Token Revocation**

   - Owners can revoke agent tokens, which invalidates the JWT.

Example: Agent Workflow with JWT
================================

Below is a sample Bash script showing how an **agent** might upload a media file using a JWT token.

.. code-block:: bash

    #!/usr/bin/env bash
    #
    # Usage:
    #   ./agent_upload_jwt.sh /path/to/mediafile.jpg
    #
    # Description:
    #   Upload a media file to the PyraFiles server as an agent using a JWT token.

    BASE_URL="${BASE_URL:-http://localhost:6544}"
    JWT_TOKEN="${JWT_TOKEN:-your_agent_jwt_token}"
    NAMESPACE_SHORT_ID="${NAMESPACE_SHORT_ID:-your_namespace_short_id}"
    MEDIA_FILE="$1"

    if [[ -z "$MEDIA_FILE" ]]; then
      echo "Usage: $0 /path/to/mediafile.jpg"
      exit 1
    fi

    if [[ -z "$JWT_TOKEN" ]]; then
      echo "Error: JWT_TOKEN environment variable is not set."
      exit 1
    fi

    if [[ -z "$NAMESPACE_SHORT_ID" ]]; then
      echo "Error: NAMESPACE_SHORT_ID environment variable is not set."
      exit 1
    fi

    echo "Uploading media: $MEDIA_FILE"
    curl -s \
         -X POST \
         -H "Authorization: Bearer $JWT_TOKEN" \
         -F "media_file=@$MEDIA_FILE" \
         -F "is_public=on" \
         "$BASE_URL/namespace/$NAMESPACE_SHORT_ID/media/upload"

    echo ""
    echo "Upload complete."

OpenAPI Documentation
=====================

PyraFiles provides comprehensive OpenAPI documentation for all endpoints, making it easier to integrate agents and other services.

- **Swagger UI Documentation**

  Access the interactive API documentation at:

  - http://localhost:6544/docs

- **OpenAPI Specification**

  Download the OpenAPI YAML file:

  - http://localhost:6544/openapi.yaml

Docker Deployment
=================

Below is an example Dockerfile that:

- Uses **uWSGI** to run PyraFiles.
- Uses **Caddy** as a reverse proxy (and optionally HTTPS if configured).
- Defines a **volume** for databases (so they are stored on the host).

.. code-block:: dockerfile

   # Dockerfile for PyraFiles

   ###########################################################
   # Stage 1: Build the Python environment
   ###########################################################
   FROM python:3.9-slim AS builder

   WORKDIR /app
   COPY . /app

   # Install Python dependencies
   RUN pip install --no-cache-dir -r requirements.txt

   ###########################################################
   # Stage 2: Final image with Caddy + uWSGI + Python
   ###########################################################
   FROM caddy:2-alpine

   RUN apk add --no-cache python3 py3-pip uwsgi-python3

   WORKDIR /app

   # Copy the app from the builder
   COPY --from=builder /app /app

   # Copy Caddyfile for the reverse proxy
   COPY Caddyfile /etc/caddy/Caddyfile

   # Copy uWSGI configuration
   COPY uwsgi.ini /app/uwsgi.ini

   # Set environment variables
   ENV PYRAFILES_SECRET=""
   ENV PYRAFILES_JWT_SECRET=""
   ENV PYRAFILES_DB_URL="sqlite:///data/main.db"
   ENV PYRAFILES_HOST="0.0.0.0"
   ENV PYRAFILES_PORT="6544"

   # Expose HTTP and HTTPS
   EXPOSE 80 443

   # Define volume for data
   VOLUME ["/data"]

   # Start uWSGI and Caddy
   CMD ["/bin/sh", "-c", "\
        uwsgi --ini /app/uwsgi.ini & \
        caddy run --config /etc/caddy/Caddyfile \
      "]

.. note::

   - The ``/data`` volume is used to persist database files.
   - Ensure you mount a volume at ``/data`` when running the container.
   - Set ``PYRAFILES_SECRET`` and ``PYRAFILES_JWT_SECRET`` to persistent values in a production environment.

Tips
====

- For **production**, set up a real SMTP server or use a third-party service (e.g., Mailgun) and configure it via environment variables (``PYRAFILES_SMTP_HOST``, etc.).
- **Persist Secrets**: In production, set ``PYRAFILES_SECRET`` and ``PYRAFILES_JWT_SECRET`` to persistent values to avoid invalidating sessions and JWT tokens.
- **Mount Data Volume**: Mount the ``/data`` directory to persist database files and avoid data loss.
- **OpenAPI Integration**: Use the provided OpenAPI specification to generate client code or integrate with other services.

License and Public Domain
=========================

This project is in the public domain. You are free to use, adapt, and redistribute it without attribution or additional licensing.

If you find PyraFiles helpful, feel free to contribute back or share your enhancements!

Support and Contact
===================

- **Issues**: Please open tickets at the `PyraFiles Git Repo <https://git.unturf.com/engineering/unturf/upload.unturf.com>`_.
- **Contributions**: If you register an account, let us know, and we will grant you developer access to contribute.
- **General Inquiries**: Reach out to the maintainers directly.

We hope PyraFiles helps you get up and running quickly with a flexible media-sharing and multi-database infrastructure!

Enjoy and happy building!