From a8b0ee313d4b759efaa2379b454ea25bc1fe4bec Mon Sep 17 00:00:00 2001 From: Russell Ballestrini Date: Sat, 11 Jan 2025 20:18:34 -0500 Subject: [PATCH] working with revocations modified: README.rst modified: app.py modified: templates/manage_namespace.html.j2 modified: test_agent.sh --- README.rst | 206 +++++++----------- app.py | 335 +++++++++++++++++------------ templates/manage_namespace.html.j2 | 17 ++ test_agent.sh | 165 +++----------- 4 files changed, 319 insertions(+), 404 deletions(-) diff --git a/README.rst b/README.rst index cfec1e7..526a032 100644 --- a/README.rst +++ b/README.rst @@ -1,18 +1,19 @@ -=========================================== -pyrafiles - A Sophisticated Pyramid Project -=========================================== +================================ +PyraLogs - A Centralized Logging Hub +================================ -``pyrafiles`` is a **public domain** application built on the `Pyramid `_ framework. It demonstrates a flexible, multi-database design where each verified user maintains their own SQLite database for media uploads (images, audio, video). The main database (``main.db``) stores system- and user-level data, while each user’s personal DB file handles their specific uploads. +**PyraLogs** is a **public domain** application built on the `Pyramid `_ framework. It offers a flexible, agent-friendly solution for centralized logging, allowing environments and agents to log events to a central hub. PyraLogs supports a multi-database design where each namespace maintains its own SQLite database for logs. The main database (``main.db``) stores system- and user-level data, while each namespace’s personal DB file handles their specific logs. Key Features ============ - **Passwordless Login** using email verification codes. -- **Guest Sessions** for unverified visitors. -- **Per-user SQLite** databases for media uploads (limit ~30MB each). -- **Public/Private** media visibility controls. -- **Admin Tools** (import user records, manage site settings). -- **Agent-friendly** architecture: easily scriptable endpoints for login, verification, media uploads, etc. +- **Namespace-Based Logging** with per-namespace SQLite databases. +- **JWT Authentication for Agents**, enabling secure log submissions. +- **Token Versioning** for per-agent token revocation without affecting others. +- **Public/Private** namespace visibility controls. +- **Admin Tools** for managing namespaces, agents, and user access. +- **Agent-Friendly** architecture: easily scriptable endpoints for logging and management. - **Configurable** via environment variables (including secret keys). Git Repository @@ -20,32 +21,33 @@ Git Repository The project is maintained at: -- `pyrafiles Git Repo `_ +- `PyraLogs Git Repo `_ Since this is public domain, you can adapt and redistribute it freely. - Configuration via Environment ============================= -`pyrafiles` fetches settings from environment variables with sensible defaults: +PyraLogs fetches settings from environment variables with sensible defaults: -- ``PYRAFILES_SECRET`` +- ``PYRALOGS_SECRET`` The secret key for session signing. - If missing, ``pyrafiles`` automatically generates a **random 64-character** string at runtime & will log out all users out. + If missing, PyraLogs automatically generates a **random 64-character** string at runtime, which may log out all users. -- ``PYRAFILES_DB_URL`` +- ``PYRALOGS_JWT_SECRET`` + The secret key for signing JSON Web Tokens (JWTs). Defaults to the value of ``PYRALOGS_SECRET`` if not set. + +- ``PYRALOGS_DB_URL`` Connection string for the main database. Default: ``sqlite:///main.db``. -- ``PYRAFILES_HOST`` and ``PYRAFILES_PORT`` +- ``PYRALOGS_HOST`` and ``PYRALOGS_PORT`` The host and port to serve on. Defaults: ``0.0.0.0`` (host), ``6544`` (port). -- ``PYRAFILES_SMTP_HOST`` and ``PYRAFILES_SMTP_PORT`` +- ``PYRALOGS_SMTP_HOST`` and ``PYRALOGS_SMTP_PORT`` SMTP server details. Defaults: ``localhost:25``. - Any other environment variables you wish to incorporate can be accessed in the code. - Local Setup =========== @@ -53,8 +55,8 @@ Local Setup .. code-block:: bash - git clone https://git.unturf.com/engineering/unturf/upload.unturf.com.git - cd upload.unturf.com + git clone https://git.unturf.com/engineering/unturf/logs.unturf.com.git + cd logs.unturf.com 2. **Create a Virtual Environment** @@ -71,131 +73,77 @@ Local Setup pip install -r requirements.txt -4. **Run** +4. **Initialize the Database** - Create the database if this is the first time running the applicaiton. + Create the main database before running the application. .. code-block:: bash python initialize_db.py - Otherwise: +5. **Run** .. code-block:: bash - # Optionally set PYRAFILES_SECRET if you want a custom secret. - export PYRAFILES_SECRET="YOUR_OWN_LONG_RANDOM_STRING" - python main.py + # Optionally set PYRALOGS_SECRET if you want a custom secret. + export PYRALOGS_SECRET="YOUR_OWN_LONG_RANDOM_STRING" + python app.py - If ``PYRAFILES_SECRET`` is **not** set, the app automatically generates a 64-char secret at runtime & log out all users. + If ``PYRALOGS_SECRET`` is **not** set, the app automatically generates a 64-character secret at runtime, which may log out all users. -5. **Access** +6. **Access** Point your browser to `http://localhost:6544` or `http://:` according to your environment variables. - Example: Agent Workflow Script ============================== -First of all, everything is documented as OpenAPI for agentic flows. - -* http://localhost:6544/docs -* http://localhost:6544/openapi.yaml - Below is a sample Bash script showing how an **agent** might: -1. Start the session (to get a cookie). -2. Log in with an email. -3. Prompt the user for a verification code. -4. Complete the verification. -5. Upload a file. +1. Obtain a JWT for logging. +2. Send logs to the PyraLogs server using the JWT. .. code-block:: bash #!/usr/bin/env bash # # Usage: - # MAX_COOKIE_AGE=1800 ./agent_upload.sh /path/to/somefile.jpg + # ./agent_send_log.sh "Your log message" # # Description: - # Upload a file to the pyrafiles server as an agent, making it public. - # - If cookies.txt is newer than MAX_COOKIE_AGE seconds (default 31536000, ~1 year), - # we skip re-login. - # - Otherwise, we prompt for OTP again. + # Send a log message to the PyraLogs server using a JWT. # # Notes: - # 1. If not set, MAX_COOKIE_AGE defaults to 31536000 (one year). - # 2. We assume the server runs on http://localhost:6544. Adjust BASE_URL if needed. - + # - Replace with your actual JWT token. + # - Replace with your namespace's short ID. + # - Adjust BASE_URL if the server is running elsewhere. + BASE_URL="${BASE_URL:-http://localhost:6544}" - EMAIL="${EMAIL:-agent@example.com}" - MEDIA_FILE="$1" - # Default to 1 year in seconds if not provided: - MAX_COOKIE_AGE="${MAX_COOKIE_AGE:-31536000}" - - if [[ -z "$MEDIA_FILE" ]]; then - echo "Usage: MAX_COOKIE_AGE= $0 /path/to/mediafile" + JWT_TOKEN="" + LOG_MESSAGE="$1" + + if [[ -z "$LOG_MESSAGE" ]]; then + echo "Usage: $0 \"Your log message\"" exit 1 fi - - # Check if cookies.txt is 'fresh enough': - COOKIE_FRESH=0 - if [[ -f cookies.txt ]]; then - # 'stat -c %Y' returns the file's mod time on Linux; 'stat -f %m' on macOS/BSD - MOD_TIME=$(stat -c %Y cookies.txt 2>/dev/null || stat -f %m cookies.txt 2>/dev/null) - NOW=$(date +%s) - AGE=$((NOW - MOD_TIME)) - - if (( AGE < MAX_COOKIE_AGE )); then - COOKIE_FRESH=1 - fi - fi - - if [[ "$COOKIE_FRESH" -eq 1 ]]; then - echo "Reusing existing cookies (file age < $MAX_COOKIE_AGE seconds)." - echo "Skipping OTP prompt." - else - echo "Starting new session (cookie missing or stale)..." - curl -s -c cookies.txt -b cookies.txt "$BASE_URL/" >/dev/null - - echo "Logging in with email: $EMAIL" - curl -s -c cookies.txt -b cookies.txt \ - -X POST \ - -F "email=$EMAIL" \ - "$BASE_URL/auth/login" - - echo "" - echo "Check your console or email for the 6-digit verification code." - read -p "Enter the 6-digit code: " VERIFICATION_CODE - - echo "Verifying code..." - curl -s -c cookies.txt -b cookies.txt \ - -X POST \ - -F "code=$VERIFICATION_CODE" \ - "$BASE_URL/auth/verify" - - echo "Cookies refreshed." - fi - - echo "" - echo "Uploading media: $MEDIA_FILE (public)" - curl -s -c cookies.txt -b cookies.txt \ - -X POST \ - -F "media_file=@$MEDIA_FILE" \ - -F "is_public=on" \ - "$BASE_URL/media/upload" - - echo "" - echo "Upload complete. File is public." + echo "Sending log message..." + RESPONSE=$(curl -s -X POST "$BASE_URL/webhook" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -d "{ + \"message\": \"$LOG_MESSAGE\", + \"level\": \"INFO\" + }") + echo "Server Response: $RESPONSE" Dockerfile with Caddy + uWSGI ============================= Below is an example Dockerfile that: -- Uses **uWSGI** to run ``pyrafiles``. +- Uses **uWSGI** to run PyraLogs. - Uses **Caddy** as a reverse proxy (and optionally HTTPS if configured). - Defines a **volume** for databases (so they are stored on the host). @@ -232,7 +180,7 @@ Below is an example Dockerfile that: # Copy uWSGI config if desired # For example (assume you created uwsgi.ini in your repo): # [uwsgi] - # module = main:app + # module = app:app # master = true # processes = 4 # socket = 127.0.0.1:8080 @@ -241,22 +189,22 @@ Below is an example Dockerfile that: COPY uwsgi.ini /app/uwsgi.ini # Environment variables (optional overrides). - # If PYRAFILES_SECRET is empty, the app itself generates a random 64-char secret & logs all users out. - ENV PYRAFILES_SECRET="" - ENV PYRAFILES_DB_URL="sqlite:///data/main.db" - ENV PYRAFILES_HOST="0.0.0.0" - ENV PYRAFILES_PORT="6544" + # If PYRALOGS_SECRET is empty, the app itself generates a random 64-char secret & may log out all users. + ENV PYRALOGS_SECRET="" + ENV PYRALOGS_DB_URL="sqlite:///data/main.db" + ENV PYRALOGS_HOST="0.0.0.0" + ENV PYRALOGS_PORT="6544" # Expose HTTP and HTTPS EXPOSE 80 EXPOSE 443 - # Define volume so host can persist user DBs outside the container - # We'll store DB files in /data + # Define volume so host can persist databases outside the container + # We'll store database files in /data VOLUME ["/data"] # Command starts uWSGI and then starts Caddy - # The app itself checks for PYRAFILES_SECRET and auto-generates one if missing. + # The app itself checks for PYRALOGS_SECRET and auto-generates one if missing. CMD ["/bin/sh", "-c", "\ uwsgi --ini /app/uwsgi.ini & \ caddy run --config /etc/caddy/Caddyfile \ @@ -264,8 +212,8 @@ Below is an example Dockerfile that: .. note:: - - We use ``/data`` as the volume. By default, the environment variable ``PYRAFILES_DB_URL`` is set to ``sqlite:///data/main.db``, so the main DB (and any user DB files) go inside ``/data``. - - For user DB files, your app can also interpret an environment variable (like ``PYRAFILES_DB_BASEPATH=/data``) if you want to make that path configurable in the code. + - We use ``/data`` as the volume. By default, the environment variable ``PYRALOGS_DB_URL`` is set to ``sqlite:///data/main.db``, so the main DB (and any namespace DB files) go inside ``/data``. + - For namespace DB files, your app can also interpret an environment variable (like ``PYRALOGS_DB_BASEPATH=/data``) if you want to make that path configurable in the code. - Make sure to **mount** a volume at ``/data`` when you run the container, e.g.: .. code-block:: bash @@ -273,36 +221,34 @@ Below is an example Dockerfile that: docker run -d \ -p 80:80 -p 443:443 \ -v /my/local/dbfolder:/data \ - --name pyrafiles \ - pyrafiles-image:latest - - - With that, any user database is stored in ``/my/local/dbfolder`` on the host. + --name pyralogs \ + pyralogs-image:latest + - With that, any namespace database is stored in ``/my/local/dbfolder`` on the host. Tips ==== -- For **production**, you likely want to set up a real SMTP server or third-party service (e.g. Mailgun, ImprovMV) and configure it via environment variables (``PYRAFILES_SMTP_HOST``, etc.). +- For **production**, you likely want to set up a real SMTP server or third-party service (e.g., Mailgun, SendGrid) and configure it via environment variables (``PYRALOGS_SMTP_HOST``, etc.). - Ensure you mount a volume for persistent SQLite files if you want to avoid data loss when containers are removed or replaced. - Because the project is **public domain**, you can adapt it without restriction, removing features, adding custom logic, etc. - +- **Security Reminder**: Always keep your ``PYRALOGS_SECRET`` and ``PYRALOGS_JWT_SECRET`` secure. Do not expose them in your code repositories or logs. License and Public Domain ========================= -This project is in the public domain. You are free to use, adapt, and redistribute it without attribution or additional licensing. - -If you find `pyrafiles` helpful, feel free to contribute back or share your enhancements! +This project is in the public domain. You are free to use, adapt, and redistribute it without attribution or additional licensing. +If you find PyraLogs helpful, feel free to contribute back or share your enhancements! Support and Contact =================== -- Issues: Please open tickets at the `git.unturf.com project page `_. -- For general inquiries, you can reach out to the maintainers directly. +- **Issues**: Please open tickets at the `PyraLogs project page `_. +- For general inquiries, you can reach out to the maintainers directly. -We hope `pyrafiles` helps you get up and running quickly with a flexible media-sharing and multi-DB infrastructure! +We hope PyraLogs helps you set up a centralized logging solution quickly and efficiently! -If you register an account, let me know and I will bless you as an developer to contribute. +If you register an account, let me know, and I will grant you developer access to contribute. -Enjoy and happy building! +Enjoy and happy logging! diff --git a/app.py b/app.py index 252e985..9b3ddb4 100644 --- a/app.py +++ b/app.py @@ -1,5 +1,6 @@ ############################################################################### -# app.py - PyraLogs Application with JWT-based Authentication and Refresh Mechanism +# app.py - PyraLogs Application with Agent-Specific JWT Authentication and Revocation +# Users do not have token_version as only agents use JWTs ############################################################################### import os import base64 @@ -149,36 +150,8 @@ def get_current_user(request): # User ID in session does not exist in the database; remove it del request.session["user_id"] - # No valid user in session; create a guest user - # Generate a unique guest username - while True: - suffix = "".join(random.choices(string.ascii_uppercase + string.digits, k=8)) - guest_username = f"Guest-{suffix}" - existing_user = s.query(User).filter_by(username=guest_username).first() - if not existing_user: - break # Unique username found - - # Generate UUID and short ID - user_uuid = uuid.uuid4() - new_user_id = str(user_uuid) - short_id = uuid_to_short_id(user_uuid) - - # Create a new guest user - guest_user = User( - id=new_user_id, - short_id=short_id, - email=None, # Guests don't have an email - username=guest_username, - is_verified=False, - ) - s.add(guest_user) - s.flush() # Use flush instead of commit in pyramid_tm - - # Store the user ID in the session - request.session["user_id"] = guest_user.id - - guest_user.dbsession = s # Attach dbsession to user - return guest_user + # No valid user in session; return None + return None def uuid_to_short_id(u): @@ -226,12 +199,12 @@ Base = declarative_base() # Association table for Namespace <-> User (with roles) namespace_user_association = Table( - 'namespace_user_association', + "namespace_user_association", Base.metadata, - Column('namespace_id', String, ForeignKey('namespaces.id')), - Column('user_id', String, ForeignKey('users.id')), - Column('role', String, nullable=False), # 'owner', 'editor', 'reader' - Index('ix_namespace_user', 'namespace_id', 'user_id', unique=True) + Column("namespace_id", String, ForeignKey("namespaces.id")), + Column("user_id", String, ForeignKey("users.id")), + Column("role", String, nullable=False), # 'owner', 'editor', 'reader' + Index("ix_namespace_user", "namespace_id", "user_id", unique=True), ) @@ -239,8 +212,8 @@ class User(Base): __tablename__ = "users" id = Column(String, primary_key=True) # UUID short_id = Column(String, unique=True, nullable=False) - email = Column(String, unique=True, nullable=True) # Guests don't have an email. - username = Column(String, unique=True, nullable=False) # user-chosen handle + email = Column(String, unique=True, nullable=True) + username = Column(String, unique=True, nullable=False) code_hash = Column(String, nullable=True) # bcrypt hash of code code_expires = Column(DateTime, nullable=True) # time limit for code is_verified = Column(Boolean, default=False) @@ -275,10 +248,27 @@ class Namespace(Base): back_populates="namespaces", ) + # Agents associated with the namespace + agents = relationship("Agent", back_populates="namespace") + def __repr__(self): return f"" +class Agent(Base): + __tablename__ = "agents" + id = Column(String, primary_key=True) # UUID + name = Column(String, nullable=False) + namespace_id = Column(String, ForeignKey("namespaces.id")) + token_version = Column(Integer, default=0) + created_at = Column(DateTime, default=datetime.datetime.utcnow) + + namespace = relationship("Namespace", back_populates="agents") + + def __repr__(self): + return f"" + + ################################################################################ # Jinja2 Environment and Custom Filters ################################################################################ @@ -430,11 +420,33 @@ class LogEntry(NamespaceBase): @view_config(route_name="home", renderer="home.html.j2") def home_view(request): s = request.dbsession - # Show public namespaces + # Get public namespaces public_namespaces = s.query(Namespace).filter(Namespace.is_public == True).all() + + user_namespaces = [] + if request.user and request.user.is_verified: + # Get namespaces the user has access to along with their roles + user_namespaces = [] + for ns in request.user.namespaces: + result = s.execute( + namespace_user_association.select().where( + namespace_user_association.c.namespace_id == ns.id, + namespace_user_association.c.user_id == request.user.id, + ) + ).first() + if result: + user_namespaces.append( + { + "name": ns.name, + "short_id": ns.short_id, + "role": result.role, + } + ) + return { "request": request, "public_namespaces": public_namespaces, + "user_namespaces": user_namespaces, } @@ -553,14 +565,15 @@ def logout_view(request): # JWT Helper Functions ################################################################################ -def generate_jwt_token(user, expires_delta=datetime.timedelta(days=365)): - """Generate a JWT for the given user.""" +def generate_jwt_token(agent): + """Generate a JWT for the given agent without an expiration time.""" payload = { - "user_id": user.id, - "username": user.username, - # Set 'iat' to issue time and 'nbf' to tomorrow to enforce daily refresh + "agent_id": agent.id, + "agent_name": agent.name, + "namespace_id": agent.namespace_id, + "token_version": agent.token_version, "iat": datetime.datetime.utcnow(), - "exp": datetime.datetime.utcnow() + expires_delta, + # "exp": datetime.datetime.utcnow() + expires_delta, # Removed to make token not expire } token = jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM) return token @@ -569,30 +582,17 @@ def generate_jwt_token(user, expires_delta=datetime.timedelta(days=365)): def verify_jwt_token(token): """Verify the JWT and return the payload if valid.""" try: - payload = jwt.decode(token, JWT_SECRET, algorithms=[JWT_ALGORITHM]) + payload = jwt.decode( + token, + JWT_SECRET, + algorithms=[JWT_ALGORITHM], + options={"verify_exp": False}, # Disable expiration verification + ) return payload except PyJWTError: return None -def issue_new_jwt_if_needed(old_token, user): - """Issue a new JWT if the token was last used on a previous day.""" - payload = verify_jwt_token(old_token) - if not payload: - return None # Invalid token - last_used = datetime.datetime.utcfromtimestamp(payload.get("last_used", 0)) - now = datetime.datetime.utcnow() - - if last_used.date() < now.date(): - # Update 'last_used' to now and issue a new token - new_payload = payload.copy() - new_payload["last_used"] = now.timestamp() - token = jwt.encode(new_payload, JWT_SECRET, algorithm=JWT_ALGORITHM) - return token - else: - return None # No need to issue a new token - - ################################################################################ # Profile and Namespace Management ################################################################################ @@ -615,7 +615,7 @@ def profile_get_view(request): namespace_user_association.c.user_id == user.id, ) ).first() - if result and result.role == 'owner': + if result and result.role == "owner": owner_namespaces.append(ns) return { @@ -651,7 +651,11 @@ def profile_post_view(request): return HTTPFound(location=request.route_url("profile")) -@view_config(route_name="create_namespace", request_method="GET", renderer="create_namespace.html.j2") +@view_config( + route_name="create_namespace", + request_method="GET", + renderer="create_namespace.html.j2", +) def create_namespace_get_view(request): if not request.user or not request.user.is_verified: return Response("You must be logged in to create a namespace.", status=403) @@ -694,7 +698,7 @@ def create_namespace_post_view(request): namespace_user_association.insert().values( namespace_id=namespace.id, user_id=user.id, - role='owner', + role="owner", ) ) s.flush() @@ -702,12 +706,18 @@ def create_namespace_post_view(request): # Create namespace database namespace_db_url = get_namespace_db_url(namespace.id) engine = create_engine( - namespace_db_url, connect_args={"check_same_thread": False}, poolclass=StaticPool + namespace_db_url, + connect_args={"check_same_thread": False}, + poolclass=StaticPool, ) NamespaceBase.metadata.create_all(engine) engine.dispose() - return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id)) + return HTTPFound( + location=request.route_url( + "manage_namespace", namespace_short_id=namespace.short_id + ) + ) @view_config(route_name="manage_namespace", renderer="manage_namespace.html.j2") @@ -727,15 +737,21 @@ def manage_namespace_view(request): users = [] for ur in user_roles: u = s.query(User).filter(User.id == ur.user_id).first() - users.append({ - 'user': u, - 'role': ur.role, - }) + users.append( + { + "user": u, + "role": ur.role, + } + ) + + # Get agents associated with the namespace + agents = s.query(Agent).filter(Agent.namespace_id == namespace.id).all() return { "request": request, "namespace": namespace, "users": users, + "agents": agents, } @@ -750,7 +766,11 @@ def update_namespace_view(request): namespace.is_public = is_public s.flush() - return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id)) + return HTTPFound( + location=request.route_url( + "manage_namespace", namespace_short_id=namespace.short_id + ) + ) @view_config(route_name="invite_user", request_method="POST") @@ -761,7 +781,7 @@ def invite_user_view(request): email = request.POST.get("email", "").strip().lower() role = request.POST.get("role", "").strip().lower() - if role not in ['owner', 'editor', 'reader']: + if role not in ["owner", "editor", "reader"]: return Response("Invalid role.", status=400) # Find or create the user @@ -803,47 +823,44 @@ def invite_user_view(request): email_body = f"You have been invited as a {role} to namespace '{namespace.name}'. Please log in to access it." send_email(user.email, "Namespace Invitation", email_body) - return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id)) + return HTTPFound( + location=request.route_url( + "manage_namespace", namespace_short_id=namespace.short_id + ) + ) -@view_config(route_name="generate_agent_jwt", request_method="POST", renderer="display_agent_jwt.html.j2") +@view_config( + route_name="generate_agent_jwt", + request_method="POST", + renderer="display_agent_jwt.html.j2", +) @owner_required def generate_agent_jwt_view(request): namespace = request.namespace - user = request.user + s = request.dbsession agent_name = request.POST.get("agent_name", "").strip() if not agent_name: return Response("Agent name is required.", status=400) - # Create a user entry for the agent without an email + # Generate UUID for the agent agent_uuid = uuid.uuid4() agent_id = str(agent_uuid) - agent_short_id = uuid_to_short_id(agent_uuid) - agent_user = User( + # Create an agent entry + agent = Agent( id=agent_id, - short_id=agent_short_id, - email=None, - username=agent_name, - is_verified=True, # Agents are verified + name=agent_name, + namespace_id=namespace.id, + token_version=0, ) - s = request.dbsession - s.add(agent_user) + + s.add(agent) s.flush() - # Assign the agent to the namespace with the appropriate role - s.execute( - namespace_user_association.insert().values( - namespace_id=namespace.id, - user_id=agent_user.id, - role='editor', # Agents can be given 'editor' role - ) - ) - s.flush() - - # Generate JWT for the agent - jwt_token = generate_jwt_token(agent_user) + # Generate JWT for the agent including the namespace_id + jwt_token = generate_jwt_token(agent) return { "request": request, @@ -852,6 +869,38 @@ def generate_agent_jwt_view(request): "jwt_token": jwt_token, } + +@view_config(route_name="revoke_agent", request_method="POST") +@owner_required +def revoke_agent_view(request): + namespace = request.namespace + s = request.dbsession + + agent_id = request.POST.get("agent_id") + if not agent_id: + return Response("Agent ID is required.", status=400) + + # Get the agent + agent = s.query(Agent).filter( + Agent.id == agent_id, + Agent.namespace_id == namespace.id, + ).first() + + if not agent: + return Response("Agent not found.", status=404) + + # Increment the agent's token_version to revoke its token + agent.token_version += 1 + s.flush() + request.session.flash(f"Access revoked for agent '{agent.name}'.") + + return HTTPFound( + location=request.route_url( + "manage_namespace", namespace_short_id=namespace.short_id + ) + ) + + ################################################################################ # Logging Views ################################################################################ @@ -865,47 +914,48 @@ def log_webhook_view(request): auth_header = request.headers.get("Authorization") if not auth_header or not auth_header.startswith("Bearer "): return Response("Missing or invalid Authorization header.", status=401) - token = auth_header[len("Bearer "):].strip() + token = auth_header[len("Bearer ") :].strip() payload = verify_jwt_token(token) if not payload: return Response("Invalid or expired token.", status=401) - # Get user and check permissions - user = s.query(User).filter(User.id == payload["user_id"]).first() - if not user: - return Response("User not found.", status=404) - request.user = user - user.dbsession = s # Attach dbsession to user + # Get agent from the JWT + agent_id = payload.get("agent_id") + if not agent_id: + return Response("Agent ID not specified in token.", status=400) + agent = s.query(Agent).filter(Agent.id == agent_id).first() + if not agent: + return Response("Agent not found.", status=404) - # Issue a new JWT if needed and include it in the response - new_token = issue_new_jwt_if_needed(token, user) - if new_token: - request.response.headers['X-New-JWT'] = new_token + # Check token_version + if payload.get("token_version") != agent.token_version: + return Response("Token has been revoked.", status=401) - # Get namespace from header or parameter - namespace_short_id = request.headers.get('X-Namespace') or request.params.get('namespace') - if not namespace_short_id: - return Response("Namespace not specified.", status=400) - namespace = s.query(Namespace).filter(Namespace.short_id == namespace_short_id).first() + # Get namespace from the agent + namespace = s.query(Namespace).filter(Namespace.id == agent.namespace_id).first() if not namespace: return Response("Namespace not found.", status=404) + request.namespace = namespace - # Check if user has sufficient permissions - if not check_namespace_permission(user, namespace, 'editor'): - return Response("You do not have permission to write to this namespace.", status=403) - - request.namespace = namespace # Attach namespace to request + # No need to issue a new token since tokens do not expire due to time # Access namespace_dbsession namespace_dbsession = request.namespace_dbsession - payload = request.json_body - message = payload.get('message', '') - level = payload.get('level', 'INFO') - meta_data = payload.get('metadata', None) - if meta_data: + # Parse payload + try: + json_payload = request.json_body + except json.decoder.JSONDecodeError: + return Response("Invalid JSON payload.", status=400) + message = json_payload.get("message", "") + if not message: + return Response("Log message is required.", status=400) + level = json_payload.get("level", "INFO") # Default to 'INFO' if not provided + meta_data = json_payload.get("metadata", None) + if meta_data is not None: meta_data = json.dumps(meta_data) + # Create log entry log_entry = LogEntry( message=message, level=level, @@ -917,13 +967,19 @@ def log_webhook_view(request): return Response("Log entry created.", status=201) -@view_config(route_name="view_namespace_logs", renderer="view_namespace_logs.html.j2") +@view_config( + route_name="view_namespace_logs", renderer="view_namespace_logs.html.j2" +) @reader_required def view_namespace_logs_view(request): namespace = request.namespace namespace_dbsession = request.namespace_dbsession - logs = namespace_dbsession.query(LogEntry).order_by(LogEntry.timestamp.desc()).all() + logs = ( + namespace_dbsession.query(LogEntry) + .order_by(LogEntry.timestamp.desc()) + .all() + ) return { "request": request, @@ -969,11 +1025,13 @@ def main(global_config=None, **settings): # Set up SQLAlchemy engine = create_engine( - DB_URL, connect_args={"check_same_thread": False}, poolclass=StaticPool + DB_URL, + connect_args={"check_same_thread": False}, + poolclass=StaticPool, ) session_factory_ = sessionmaker(bind=engine) Base.metadata.bind = engine - #Base.metadata.create_all(engine) + Base.metadata.create_all(engine) DBSession = scoped_session(session_factory_) register(DBSession) # Register with zope.sqlalchemy @@ -991,7 +1049,9 @@ def main(global_config=None, **settings): config.add_request_method(get_current_namespace, "namespace", reify=True) # Add namespace_dbsession to requests if namespace is set - config.add_request_method(add_namespace_dbsession, "namespace_dbsession", reify=True) + config.add_request_method( + add_namespace_dbsession, "namespace_dbsession", reify=True + ) # Routes config.add_route("home", "/") @@ -1004,15 +1064,24 @@ def main(global_config=None, **settings): # Namespace Management config.add_route("create_namespace", "/namespace/create") - config.add_route("manage_namespace", "/namespace/{namespace_short_id}/manage") + config.add_route( + "manage_namespace", "/namespace/{namespace_short_id}/manage" + ) config.add_route("update_namespace", "/namespace/{namespace_short_id}/update") config.add_route("invite_user", "/namespace/{namespace_short_id}/invite") + config.add_route( + "revoke_agent", "/namespace/{namespace_short_id}/revoke_agent" + ) # Route to generate JWTs for agents - config.add_route("generate_agent_jwt", "/namespace/{namespace_short_id}/generate_agent_jwt") + config.add_route( + "generate_agent_jwt", "/namespace/{namespace_short_id}/generate_agent_jwt" + ) # Logging config.add_route("log_webhook", "/webhook") - config.add_route("view_namespace_logs", "/namespace/{namespace_short_id}/logs") + config.add_route( + "view_namespace_logs", "/namespace/{namespace_short_id}/logs" + ) config.scan() return config.make_wsgi_app() diff --git a/templates/manage_namespace.html.j2 b/templates/manage_namespace.html.j2 index 41cb994..f8fbbaf 100644 --- a/templates/manage_namespace.html.j2 +++ b/templates/manage_namespace.html.j2 @@ -39,6 +39,23 @@

No users in this namespace.

{% endif %} +

Agents

+{% if agents %} +
    + {% for agent in agents %} +
  • + {{ agent.name }} +
    + + +
    +
    • + {% endfor %} +
+{% else %} +

No agents found.

+{% endif %} +

Generate JWT for Agent

diff --git a/test_agent.sh b/test_agent.sh index ed20a21..5447d46 100644 --- a/test_agent.sh +++ b/test_agent.sh @@ -1,150 +1,33 @@ #!/usr/bin/env bash -# -# A Bash script to demonstrate and test multiple authenticated routes -# in the pyrafiles application (or a similar Pyramid-based app). -# -# This script: -# 1. Checks if cookies.txt is older than a defined threshold (default 1 year). -# - If too old, it deletes cookies.txt so we force re-login. -# 2. If cookies.txt is still valid or absent, proceeds to the next steps: -# - If cookies.txt does not exist or was removed, it does the email + OTP flow. -# - If cookies.txt exists and is fresh, it reuses that session. -# 3. Fetches and prints the Profile page (GET /auth/profile) as a quick test. -# 4. Uploads a sample media file (POST /media/upload), marked public. -# 5. Parses the server's redirect "Location" header to extract user_short_id and media_short_id. -# 6. Views media details (GET /media///details). -# 7. Edits the newly uploaded media (POST /media///edit). -# 8. Deletes the media (POST /media///delete). -# 9. Lists all public media (GET /media/list). -# 10. Logs out (GET /auth/logout). -# -# The script exits on the first command error (set -e). -# Adjust BASE_URL, EMAIL, and file paths as needed. -# + # Usage: -# MAX_COOKIE_AGE= ./test_all_routes.sh /path/to/somefile.jpg -# By default, MAX_COOKIE_AGE=31536000 (1 year). -# If cookies.txt is younger than that, we skip re-login. Otherwise, we do the OTP flow. -# -# By default, uses http://localhost:6544 and agent@example.com +# ./test_agent.sh "Your log message" -set -e # Exit on first error +# Description: +# Sends a log message to the PyraLogs server using the provided JWT token. +# Adjust the BASE_URL and JWT_TOKEN variables as needed. -BASE_URL="${BASE_URL:-http://localhost:6544}" -EMAIL="${EMAIL:-agent@example.com}" -MEDIA_FILE="$1" -MAX_COOKIE_AGE="${MAX_COOKIE_AGE:-31536000}" # default ~1 year in seconds +# Variables +BASE_URL="http://127.0.0.1:6544" +JWT_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhZ2VudF9pZCI6IjMzMmM3ZjY2LWMzNTktNDg4Yi05MTIxLWIwN2Y1N2EyMTA3MCIsImFnZW50X25hbWUiOiJ0ZXN0LTkwMDAiLCJuYW1lc3BhY2VfaWQiOiI3YmM3Mjk5Yy1hYTgxLTRhYjQtYmNmMy00OTU5NzNhZjk3NzEiLCJ0b2tlbl92ZXJzaW9uIjowLCJpYXQiOjE3MzY2NDQyNjB9.fTZuLTFARjgTXIUtg2zGkfcZp3sAJYAasSa6MFxK-Gk" -if [[ -z "$MEDIA_FILE" ]]; then - echo "Usage: MAX_COOKIE_AGE= $0 /path/to/mediafile" +# Log message from the first argument +LOG_MESSAGE="$1" + +# Check if a log message was provided +if [[ -z "$LOG_MESSAGE" ]]; then + echo "Usage: $0 \"Your log message\"" exit 1 fi -echo "=== Checking cookies.txt freshness ===" +# Send the log message using curl +RESPONSE=$(curl -s -X POST "$BASE_URL/webhook" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -d "{ + \"message\": \"$LOG_MESSAGE\", + \"level\": \"INFO\" + }") -COOKIE_FRESH=0 -if [[ -f cookies.txt ]]; then - # 'stat -c %Y' returns the file modification time on Linux - # 'stat -f %m' does the same on BSD/macOS - MOD_TIME=$(stat -c %Y cookies.txt 2>/dev/null || stat -f %m cookies.txt 2>/dev/null) - NOW=$(date +%s) - AGE=$((NOW - MOD_TIME)) - - echo "cookies.txt is $AGE seconds old, threshold is $MAX_COOKIE_AGE." - if (( AGE < MAX_COOKIE_AGE )); then - COOKIE_FRESH=1 - echo "Reusing existing cookies.txt (fresh enough)." - else - echo "cookies.txt is too old. Removing it to force re-login." - rm -f cookies.txt - fi -else - echo "No cookies.txt found. A new session will be created." -fi - -if [[ "$COOKIE_FRESH" -eq 0 ]]; then - echo "" - echo "=== Starting new session and logging in with OTP flow ===" - curl -s -i -c cookies.txt -b cookies.txt "$BASE_URL/" >/dev/null - - echo "Logging in with email: $EMAIL (POST /auth/login)" - curl -s -i -c cookies.txt -b cookies.txt \ - -X POST \ - -F "email=$EMAIL" \ - "$BASE_URL/auth/login" - - echo "" - echo "Check your server logs or email for the 6-digit verification code." - read -p "Enter the 6-digit code: " VERIFICATION_CODE - - echo "Verifying code (POST /auth/verify)" - curl -s -i -c cookies.txt -b cookies.txt \ - -X POST \ - -F "code=$VERIFICATION_CODE" \ - "$BASE_URL/auth/verify" - - echo "Login/OTP flow complete. cookies.txt saved." -fi - -echo "" -echo "=== Quick check: Fetching Profile page (GET /auth/profile) ===" -curl -i -c cookies.txt -b cookies.txt "$BASE_URL/auth/profile" - -echo "" -echo "=== Uploading media (POST /media/upload) as public ===" -UPLOAD_RESPONSE=$(curl -i -c cookies.txt -b cookies.txt \ - -X POST \ - -F "title=TestUploadByAgent" \ - -F "media_file=@${MEDIA_FILE}" \ - -F "is_public=on" \ - "$BASE_URL/media/upload") - -echo "$UPLOAD_RESPONSE" -echo "" -echo "Parsing 'Location' header from upload response..." - -LOCATION_HEADER=$(echo "$UPLOAD_RESPONSE" | grep -i "^Location:" | head -n1 | sed 's/\r//g') -LOCATION_URL="${LOCATION_HEADER#Location: }" -LOCATION_URL=$(echo "$LOCATION_URL" | tr -d '[:space:]') -LOCATION_PATH="${LOCATION_URL#*//*/}" - -USER_SHORT_ID=$(echo "$LOCATION_PATH" | cut -d'/' -f2) -MEDIA_SHORT_ID=$(echo "$LOCATION_PATH" | cut -d'/' -f3) - -echo "user_short_id=$USER_SHORT_ID" -echo "media_short_id=$MEDIA_SHORT_ID" - -if [[ -z "$USER_SHORT_ID" || -z "$MEDIA_SHORT_ID" ]]; then - echo "Failed to parse user or media short ID. Exiting." - exit 1 -fi - -echo "" -echo "=== Viewing media details (GET /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/details) ===" -curl -i -c cookies.txt -b cookies.txt \ - "$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/details" - -echo "" -echo "=== Editing media title to 'RenamedByAgent' (POST /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/edit) ===" -curl -i -c cookies.txt -b cookies.txt \ - -X POST \ - -F "title=RenamedByAgent" \ - "$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/edit" - -echo "" -echo "=== Deleting the media (POST /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/delete) ===" -curl -i -c cookies.txt -b cookies.txt \ - -X POST \ - "$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/delete" - -echo "" -echo "=== Listing all public media (GET /media/list) ===" -curl -i -c cookies.txt -b cookies.txt "$BASE_URL/media/list" - -echo "" -echo "=== Logging out (GET /auth/logout) ===" -curl -i -c cookies.txt -b cookies.txt "$BASE_URL/auth/logout" - -echo "" -echo "Test complete. We exercised multiple routes, including login/verify, profile, " -echo "upload/delete/edit media, and logout. If no errors appeared, all requests succeeded!" +# Output the server's response +echo "Server Response: $RESPONSE"