engineering/logs.unturf.com
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

working with revocations

Browse source 
modified:   README.rst
	modified:   app.py
	modified:   templates/manage_namespace.html.j2
	modified:   test_agent.sh
This commit is contained in:
Russell Ballestrini 2025-01-11 20:18:34 -05:00
parent bed3607168
commit a8b0ee313d
4 changed files with 319 additions and 404 deletions
Download patch file Download diff file Expand all files Collapse all files

206
README.rst
View file

@ -1,18 +1,19 @@
===========================================
pyrafiles - A Sophisticated Pyramid Project
===========================================
================================
PyraLogs - A Centralized Logging Hub
================================
``pyrafiles`` is a **public domain** application built on the `Pyramid <https://trypyramid.com>`_ framework. It demonstrates a flexible, multi-database design where each verified user maintains their own SQLite database for media uploads (images, audio, video). The main database (``main.db``) stores system- and user-level data, while each user’s personal DB file handles their specific uploads.
**PyraLogs** is a **public domain** application built on the `Pyramid <https://trypyramid.com>`_ framework. It offers a flexible, agent-friendly solution for centralized logging, allowing environments and agents to log events to a central hub. PyraLogs supports a multi-database design where each namespace maintains its own SQLite database for logs. The main database (``main.db``) stores system- and user-level data, while each namespace’s personal DB file handles their specific logs.
Key Features
============
- **Passwordless Login** using email verification codes.
- **Guest Sessions** for unverified visitors.
- **Per-user SQLite** databases for media uploads (limit ~30MB each).
- **Public/Private** media visibility controls.
- **Admin Tools** (import user records, manage site settings).
- **Agent-friendly** architecture: easily scriptable endpoints for login, verification, media uploads, etc.
- **Namespace-Based Logging** with per-namespace SQLite databases.
- **JWT Authentication for Agents**, enabling secure log submissions.
- **Token Versioning** for per-agent token revocation without affecting others.
- **Public/Private** namespace visibility controls.
- **Admin Tools** for managing namespaces, agents, and user access.
- **Agent-Friendly** architecture: easily scriptable endpoints for logging and management.
- **Configurable** via environment variables (including secret keys).
Git Repository
@ -20,32 +21,33 @@ Git Repository
The project is maintained at:
- `pyrafiles Git Repo <https://git.unturf.com/engineering/unturf/upload.unturf.com>`_
- `PyraLogs Git Repo <https://git.unturf.com/engineering/unturf/logs.unturf.com>`_
Since this is public domain, you can adapt and redistribute it freely.
Configuration via Environment
=============================
`pyrafiles` fetches settings from environment variables with sensible defaults:
PyraLogs fetches settings from environment variables with sensible defaults:
- ``PYRAFILES_SECRET``
- ``PYRALOGS_SECRET``
The secret key for session signing.
If missing, ``pyrafiles`` automatically generates a **random 64-character** string at runtime & will log out all users out.
If missing, PyraLogs automatically generates a **random 64-character** string at runtime, which may log out all users.
- ``PYRAFILES_DB_URL``
- ``PYRALOGS_JWT_SECRET``
The secret key for signing JSON Web Tokens (JWTs). Defaults to the value of ``PYRALOGS_SECRET`` if not set.
- ``PYRALOGS_DB_URL``
Connection string for the main database. Default: ``sqlite:///main.db``.
- ``PYRAFILES_HOST`` and ``PYRAFILES_PORT``
- ``PYRALOGS_HOST`` and ``PYRALOGS_PORT``
The host and port to serve on. Defaults: ``0.0.0.0`` (host), ``6544`` (port).
- ``PYRAFILES_SMTP_HOST`` and ``PYRAFILES_SMTP_PORT``
- ``PYRALOGS_SMTP_HOST`` and ``PYRALOGS_SMTP_PORT``
SMTP server details. Defaults: ``localhost:25``.
- Any other environment variables you wish to incorporate can be accessed in the code.
Local Setup
===========
@ -53,8 +55,8 @@ Local Setup
.. code-block:: bash
git clone https://git.unturf.com/engineering/unturf/upload.unturf.com.git
cd upload.unturf.com
git clone https://git.unturf.com/engineering/unturf/logs.unturf.com.git
cd logs.unturf.com
2. **Create a Virtual Environment**
@ -71,131 +73,77 @@ Local Setup
pip install -r requirements.txt
4. **Run**
4. **Initialize the Database**
Create the database if this is the first time running the applicaiton.
Create the main database before running the application.
.. code-block:: bash
python initialize_db.py
Otherwise:
5. **Run**
.. code-block:: bash
# Optionally set PYRAFILES_SECRET if you want a custom secret.
export PYRAFILES_SECRET="YOUR_OWN_LONG_RANDOM_STRING"
python main.py
# Optionally set PYRALOGS_SECRET if you want a custom secret.
export PYRALOGS_SECRET="YOUR_OWN_LONG_RANDOM_STRING"
python app.py
If ``PYRAFILES_SECRET`` is **not** set, the app automatically generates a 64-char secret at runtime & log out all users.
If ``PYRALOGS_SECRET`` is **not** set, the app automatically generates a 64-character secret at runtime, which may log out all users.
5. **Access**
6. **Access**
Point your browser to `http://localhost:6544` or `http://<HOST>:<PORT>` according to your environment variables.
Example: Agent Workflow Script
==============================
First of all, everything is documented as OpenAPI for agentic flows.
* http://localhost:6544/docs
* http://localhost:6544/openapi.yaml
Below is a sample Bash script showing how an **agent** might:
1. Start the session (to get a cookie).
2. Log in with an email.
3. Prompt the user for a verification code.
4. Complete the verification.
5. Upload a file.
1. Obtain a JWT for logging.
2. Send logs to the PyraLogs server using the JWT.
.. code-block:: bash
#!/usr/bin/env bash
#
# Usage:
# MAX_COOKIE_AGE=1800 ./agent_upload.sh /path/to/somefile.jpg
# ./agent_send_log.sh "Your log message"
#
# Description:
# Upload a file to the pyrafiles server as an agent, making it public.
# - If cookies.txt is newer than MAX_COOKIE_AGE seconds (default 31536000, ~1 year),
# we skip re-login.
# - Otherwise, we prompt for OTP again.
# Send a log message to the PyraLogs server using a JWT.
#
# Notes:
# 1. If not set, MAX_COOKIE_AGE defaults to 31536000 (one year).
# 2. We assume the server runs on http://localhost:6544. Adjust BASE_URL if needed.
# - Replace <YOUR_NAMESPACE_JWT> with your actual JWT token.
# - Replace <NAMESPACE_SHORT_ID> with your namespace's short ID.
# - Adjust BASE_URL if the server is running elsewhere.
BASE_URL="${BASE_URL:-http://localhost:6544}"
EMAIL="${EMAIL:-agent@example.com}"
MEDIA_FILE="$1"
# Default to 1 year in seconds if not provided:
MAX_COOKIE_AGE="${MAX_COOKIE_AGE:-31536000}"
if [[ -z "$MEDIA_FILE" ]]; then
echo "Usage: MAX_COOKIE_AGE=<seconds> $0 /path/to/mediafile"
JWT_TOKEN="<YOUR_NAMESPACE_JWT>"
LOG_MESSAGE="$1"
if [[ -z "$LOG_MESSAGE" ]]; then
echo "Usage: $0 \"Your log message\""
exit 1
fi
# Check if cookies.txt is 'fresh enough':
COOKIE_FRESH=0
if [[ -f cookies.txt ]]; then
# 'stat -c %Y' returns the file's mod time on Linux; 'stat -f %m' on macOS/BSD
MOD_TIME=$(stat -c %Y cookies.txt 2>/dev/null || stat -f %m cookies.txt 2>/dev/null)
NOW=$(date +%s)
AGE=$((NOW - MOD_TIME))
if (( AGE < MAX_COOKIE_AGE )); then
COOKIE_FRESH=1
fi
fi
if [[ "$COOKIE_FRESH" -eq 1 ]]; then
echo "Reusing existing cookies (file age < $MAX_COOKIE_AGE seconds)."
echo "Skipping OTP prompt."
else
echo "Starting new session (cookie missing or stale)..."
curl -s -c cookies.txt -b cookies.txt "$BASE_URL/" >/dev/null
echo "Logging in with email: $EMAIL"
curl -s -c cookies.txt -b cookies.txt \
-X POST \
-F "email=$EMAIL" \
"$BASE_URL/auth/login"
echo ""
echo "Check your console or email for the 6-digit verification code."
read -p "Enter the 6-digit code: " VERIFICATION_CODE
echo "Verifying code..."
curl -s -c cookies.txt -b cookies.txt \
-X POST \
-F "code=$VERIFICATION_CODE" \
"$BASE_URL/auth/verify"
echo "Cookies refreshed."
fi
echo ""
echo "Uploading media: $MEDIA_FILE (public)"
curl -s -c cookies.txt -b cookies.txt \
-X POST \
-F "media_file=@$MEDIA_FILE" \
-F "is_public=on" \
"$BASE_URL/media/upload"
echo ""
echo "Upload complete. File is public."
echo "Sending log message..."
RESPONSE=$(curl -s -X POST "$BASE_URL/webhook" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $JWT_TOKEN" \
-d "{
\"message\": \"$LOG_MESSAGE\",
\"level\": \"INFO\"
}")
echo "Server Response: $RESPONSE"
Dockerfile with Caddy + uWSGI
=============================
Below is an example Dockerfile that:
- Uses **uWSGI** to run ``pyrafiles``.
- Uses **uWSGI** to run PyraLogs.
- Uses **Caddy** as a reverse proxy (and optionally HTTPS if configured).
- Defines a **volume** for databases (so they are stored on the host).
@ -232,7 +180,7 @@ Below is an example Dockerfile that:
# Copy uWSGI config if desired
# For example (assume you created uwsgi.ini in your repo):
# [uwsgi]
# module = main:app
# module = app:app
# master = true
# processes = 4
# socket = 127.0.0.1:8080
@ -241,22 +189,22 @@ Below is an example Dockerfile that:
COPY uwsgi.ini /app/uwsgi.ini
# Environment variables (optional overrides).
# If PYRAFILES_SECRET is empty, the app itself generates a random 64-char secret & logs all users out.
ENV PYRAFILES_SECRET=""
ENV PYRAFILES_DB_URL="sqlite:///data/main.db"
ENV PYRAFILES_HOST="0.0.0.0"
ENV PYRAFILES_PORT="6544"
# If PYRALOGS_SECRET is empty, the app itself generates a random 64-char secret & may log out all users.
ENV PYRALOGS_SECRET=""
ENV PYRALOGS_DB_URL="sqlite:///data/main.db"
ENV PYRALOGS_HOST="0.0.0.0"
ENV PYRALOGS_PORT="6544"
# Expose HTTP and HTTPS
EXPOSE 80
EXPOSE 443
# Define volume so host can persist user DBs outside the container
# We'll store DB files in /data
# Define volume so host can persist databases outside the container
# We'll store database files in /data
VOLUME ["/data"]
# Command starts uWSGI and then starts Caddy
# The app itself checks for PYRAFILES_SECRET and auto-generates one if missing.
# The app itself checks for PYRALOGS_SECRET and auto-generates one if missing.
CMD ["/bin/sh", "-c", "\
uwsgi --ini /app/uwsgi.ini & \
caddy run --config /etc/caddy/Caddyfile \
@ -264,8 +212,8 @@ Below is an example Dockerfile that:
.. note::
- We use ``/data`` as the volume. By default, the environment variable ``PYRAFILES_DB_URL`` is set to ``sqlite:///data/main.db``, so the main DB (and any user DB files) go inside ``/data``.
- For user DB files, your app can also interpret an environment variable (like ``PYRAFILES_DB_BASEPATH=/data``) if you want to make that path configurable in the code.
- We use ``/data`` as the volume. By default, the environment variable ``PYRALOGS_DB_URL`` is set to ``sqlite:///data/main.db``, so the main DB (and any namespace DB files) go inside ``/data``.
- For namespace DB files, your app can also interpret an environment variable (like ``PYRALOGS_DB_BASEPATH=/data``) if you want to make that path configurable in the code.
- Make sure to **mount** a volume at ``/data`` when you run the container, e.g.:
.. code-block:: bash
@ -273,36 +221,34 @@ Below is an example Dockerfile that:
docker run -d \
-p 80:80 -p 443:443 \
-v /my/local/dbfolder:/data \
--name pyrafiles \
pyrafiles-image:latest
- With that, any user database is stored in ``/my/local/dbfolder`` on the host.
--name pyralogs \
pyralogs-image:latest
- With that, any namespace database is stored in ``/my/local/dbfolder`` on the host.
Tips
====
- For **production**, you likely want to set up a real SMTP server or third-party service (e.g. Mailgun, ImprovMV) and configure it via environment variables (``PYRAFILES_SMTP_HOST``, etc.).
- For **production**, you likely want to set up a real SMTP server or third-party service (e.g., Mailgun, SendGrid) and configure it via environment variables (``PYRALOGS_SMTP_HOST``, etc.).
- Ensure you mount a volume for persistent SQLite files if you want to avoid data loss when containers are removed or replaced.
- Because the project is **public domain**, you can adapt it without restriction, removing features, adding custom logic, etc.
- **Security Reminder**: Always keep your ``PYRALOGS_SECRET`` and ``PYRALOGS_JWT_SECRET`` secure. Do not expose them in your code repositories or logs.
License and Public Domain
=========================
This project is in the public domain. You are free to use, adapt, and redistribute it without attribution or additional licensing.
If you find `pyrafiles` helpful, feel free to contribute back or share your enhancements!
This project is in the public domain. You are free to use, adapt, and redistribute it without attribution or additional licensing.
If you find PyraLogs helpful, feel free to contribute back or share your enhancements!
Support and Contact
===================
- Issues: Please open tickets at the `git.unturf.com project page <https://git.unturf.com/engineering/unturf/upload.unturf.com>`_.
- For general inquiries, you can reach out to the maintainers directly.
- **Issues**: Please open tickets at the `PyraLogs project page <https://git.unturf.com/engineering/unturf/logs.unturf.com>`_.
- For general inquiries, you can reach out to the maintainers directly.
We hope `pyrafiles` helps you get up and running quickly with a flexible media-sharing and multi-DB infrastructure!
We hope PyraLogs helps you set up a centralized logging solution quickly and efficiently!
If you register an account, let me know and I will bless you as an developer to contribute.
If you register an account, let me know, and I will grant you developer access to contribute.
Enjoy and happy building!
Enjoy and happy logging!

335
app.py
View file

@ -1,5 +1,6 @@
###############################################################################
# app.py - PyraLogs Application with JWT-based Authentication and Refresh Mechanism
# app.py - PyraLogs Application with Agent-Specific JWT Authentication and Revocation
# Users do not have token_version as only agents use JWTs
###############################################################################
import os
import base64
@ -149,36 +150,8 @@ def get_current_user(request):
# User ID in session does not exist in the database; remove it
del request.session["user_id"]
# No valid user in session; create a guest user
# Generate a unique guest username
while True:
suffix = "".join(random.choices(string.ascii_uppercase + string.digits, k=8))
guest_username = f"Guest-{suffix}"
existing_user = s.query(User).filter_by(username=guest_username).first()
if not existing_user:
break # Unique username found
# Generate UUID and short ID
user_uuid = uuid.uuid4()
new_user_id = str(user_uuid)
short_id = uuid_to_short_id(user_uuid)
# Create a new guest user
guest_user = User(
id=new_user_id,
short_id=short_id,
email=None, # Guests don't have an email
username=guest_username,
is_verified=False,
)
s.add(guest_user)
s.flush() # Use flush instead of commit in pyramid_tm
# Store the user ID in the session
request.session["user_id"] = guest_user.id
guest_user.dbsession = s # Attach dbsession to user
return guest_user
# No valid user in session; return None
return None
def uuid_to_short_id(u):
@ -226,12 +199,12 @@ Base = declarative_base()
# Association table for Namespace <-> User (with roles)
namespace_user_association = Table(
'namespace_user_association',
"namespace_user_association",
Base.metadata,
Column('namespace_id', String, ForeignKey('namespaces.id')),
Column('user_id', String, ForeignKey('users.id')),
Column('role', String, nullable=False), # 'owner', 'editor', 'reader'
Index('ix_namespace_user', 'namespace_id', 'user_id', unique=True)
Column("namespace_id", String, ForeignKey("namespaces.id")),
Column("user_id", String, ForeignKey("users.id")),
Column("role", String, nullable=False), # 'owner', 'editor', 'reader'
Index("ix_namespace_user", "namespace_id", "user_id", unique=True),
)
@ -239,8 +212,8 @@ class User(Base):
__tablename__ = "users"
id = Column(String, primary_key=True) # UUID
short_id = Column(String, unique=True, nullable=False)
email = Column(String, unique=True, nullable=True) # Guests don't have an email.
username = Column(String, unique=True, nullable=False) # user-chosen handle
email = Column(String, unique=True, nullable=True)
username = Column(String, unique=True, nullable=False)
code_hash = Column(String, nullable=True) # bcrypt hash of code
code_expires = Column(DateTime, nullable=True) # time limit for code
is_verified = Column(Boolean, default=False)
@ -275,10 +248,27 @@ class Namespace(Base):
back_populates="namespaces",
)
# Agents associated with the namespace
agents = relationship("Agent", back_populates="namespace")
def __repr__(self):
return f"<Namespace(name='{self.name}', is_public={self.is_public})>"
class Agent(Base):
__tablename__ = "agents"
id = Column(String, primary_key=True) # UUID
name = Column(String, nullable=False)
namespace_id = Column(String, ForeignKey("namespaces.id"))
token_version = Column(Integer, default=0)
created_at = Column(DateTime, default=datetime.datetime.utcnow)
namespace = relationship("Namespace", back_populates="agents")
def __repr__(self):
return f"<Agent(name='{self.name}', namespace_id='{self.namespace_id}')>"
################################################################################
# Jinja2 Environment and Custom Filters
################################################################################
@ -430,11 +420,33 @@ class LogEntry(NamespaceBase):
@view_config(route_name="home", renderer="home.html.j2")
def home_view(request):
s = request.dbsession
# Show public namespaces
# Get public namespaces
public_namespaces = s.query(Namespace).filter(Namespace.is_public == True).all()
user_namespaces = []
if request.user and request.user.is_verified:
# Get namespaces the user has access to along with their roles
user_namespaces = []
for ns in request.user.namespaces:
result = s.execute(
namespace_user_association.select().where(
namespace_user_association.c.namespace_id == ns.id,
namespace_user_association.c.user_id == request.user.id,
)
).first()
if result:
user_namespaces.append(
{
"name": ns.name,
"short_id": ns.short_id,
"role": result.role,
}
)
return {
"request": request,
"public_namespaces": public_namespaces,
"user_namespaces": user_namespaces,
}
@ -553,14 +565,15 @@ def logout_view(request):
# JWT Helper Functions
################################################################################
def generate_jwt_token(user, expires_delta=datetime.timedelta(days=365)):
"""Generate a JWT for the given user."""
def generate_jwt_token(agent):
"""Generate a JWT for the given agent without an expiration time."""
payload = {
"user_id": user.id,
"username": user.username,
# Set 'iat' to issue time and 'nbf' to tomorrow to enforce daily refresh
"agent_id": agent.id,
"agent_name": agent.name,
"namespace_id": agent.namespace_id,
"token_version": agent.token_version,
"iat": datetime.datetime.utcnow(),
"exp": datetime.datetime.utcnow() + expires_delta,
# "exp": datetime.datetime.utcnow() + expires_delta, # Removed to make token not expire
}
token = jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
return token
@ -569,30 +582,17 @@ def generate_jwt_token(user, expires_delta=datetime.timedelta(days=365)):
def verify_jwt_token(token):
"""Verify the JWT and return the payload if valid."""
try:
payload = jwt.decode(token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
payload = jwt.decode(
token,
JWT_SECRET,
algorithms=[JWT_ALGORITHM],
options={"verify_exp": False}, # Disable expiration verification
)
return payload
except PyJWTError:
return None
def issue_new_jwt_if_needed(old_token, user):
"""Issue a new JWT if the token was last used on a previous day."""
payload = verify_jwt_token(old_token)
if not payload:
return None # Invalid token
last_used = datetime.datetime.utcfromtimestamp(payload.get("last_used", 0))
now = datetime.datetime.utcnow()
if last_used.date() < now.date():
# Update 'last_used' to now and issue a new token
new_payload = payload.copy()
new_payload["last_used"] = now.timestamp()
token = jwt.encode(new_payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
return token
else:
return None # No need to issue a new token
################################################################################
# Profile and Namespace Management
################################################################################
@ -615,7 +615,7 @@ def profile_get_view(request):
namespace_user_association.c.user_id == user.id,
)
).first()
if result and result.role == 'owner':
if result and result.role == "owner":
owner_namespaces.append(ns)
return {
@ -651,7 +651,11 @@ def profile_post_view(request):
return HTTPFound(location=request.route_url("profile"))
@view_config(route_name="create_namespace", request_method="GET", renderer="create_namespace.html.j2")
@view_config(
route_name="create_namespace",
request_method="GET",
renderer="create_namespace.html.j2",
)
def create_namespace_get_view(request):
if not request.user or not request.user.is_verified:
return Response("You must be logged in to create a namespace.", status=403)
@ -694,7 +698,7 @@ def create_namespace_post_view(request):
namespace_user_association.insert().values(
namespace_id=namespace.id,
user_id=user.id,
role='owner',
role="owner",
)
)
s.flush()
@ -702,12 +706,18 @@ def create_namespace_post_view(request):
# Create namespace database
namespace_db_url = get_namespace_db_url(namespace.id)
engine = create_engine(
namespace_db_url, connect_args={"check_same_thread": False}, poolclass=StaticPool
namespace_db_url,
connect_args={"check_same_thread": False},
poolclass=StaticPool,
)
NamespaceBase.metadata.create_all(engine)
engine.dispose()
return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id))
return HTTPFound(
location=request.route_url(
"manage_namespace", namespace_short_id=namespace.short_id
)
)
@view_config(route_name="manage_namespace", renderer="manage_namespace.html.j2")
@ -727,15 +737,21 @@ def manage_namespace_view(request):
users = []
for ur in user_roles:
u = s.query(User).filter(User.id == ur.user_id).first()
users.append({
'user': u,
'role': ur.role,
})
users.append(
{
"user": u,
"role": ur.role,
}
)
# Get agents associated with the namespace
agents = s.query(Agent).filter(Agent.namespace_id == namespace.id).all()
return {
"request": request,
"namespace": namespace,
"users": users,
"agents": agents,
}
@ -750,7 +766,11 @@ def update_namespace_view(request):
namespace.is_public = is_public
s.flush()
return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id))
return HTTPFound(
location=request.route_url(
"manage_namespace", namespace_short_id=namespace.short_id
)
)
@view_config(route_name="invite_user", request_method="POST")
@ -761,7 +781,7 @@ def invite_user_view(request):
email = request.POST.get("email", "").strip().lower()
role = request.POST.get("role", "").strip().lower()
if role not in ['owner', 'editor', 'reader']:
if role not in ["owner", "editor", "reader"]:
return Response("Invalid role.", status=400)
# Find or create the user
@ -803,47 +823,44 @@ def invite_user_view(request):
email_body = f"You have been invited as a {role} to namespace '{namespace.name}'. Please log in to access it."
send_email(user.email, "Namespace Invitation", email_body)
return HTTPFound(location=request.route_url("manage_namespace", namespace_short_id=namespace.short_id))
return HTTPFound(
location=request.route_url(
"manage_namespace", namespace_short_id=namespace.short_id
)
)
@view_config(route_name="generate_agent_jwt", request_method="POST", renderer="display_agent_jwt.html.j2")
@view_config(
route_name="generate_agent_jwt",
request_method="POST",
renderer="display_agent_jwt.html.j2",
)
@owner_required
def generate_agent_jwt_view(request):
namespace = request.namespace
user = request.user
s = request.dbsession
agent_name = request.POST.get("agent_name", "").strip()
if not agent_name:
return Response("Agent name is required.", status=400)
# Create a user entry for the agent without an email
# Generate UUID for the agent
agent_uuid = uuid.uuid4()
agent_id = str(agent_uuid)
agent_short_id = uuid_to_short_id(agent_uuid)
agent_user = User(
# Create an agent entry
agent = Agent(
id=agent_id,
short_id=agent_short_id,
email=None,
username=agent_name,
is_verified=True, # Agents are verified
name=agent_name,
namespace_id=namespace.id,
token_version=0,
)
s = request.dbsession
s.add(agent_user)
s.add(agent)
s.flush()
# Assign the agent to the namespace with the appropriate role
s.execute(
namespace_user_association.insert().values(
namespace_id=namespace.id,
user_id=agent_user.id,
role='editor', # Agents can be given 'editor' role
)
)
s.flush()
# Generate JWT for the agent
jwt_token = generate_jwt_token(agent_user)
# Generate JWT for the agent including the namespace_id
jwt_token = generate_jwt_token(agent)
return {
"request": request,
@ -852,6 +869,38 @@ def generate_agent_jwt_view(request):
"jwt_token": jwt_token,
}
@view_config(route_name="revoke_agent", request_method="POST")
@owner_required
def revoke_agent_view(request):
namespace = request.namespace
s = request.dbsession
agent_id = request.POST.get("agent_id")
if not agent_id:
return Response("Agent ID is required.", status=400)
# Get the agent
agent = s.query(Agent).filter(
Agent.id == agent_id,
Agent.namespace_id == namespace.id,
).first()
if not agent:
return Response("Agent not found.", status=404)
# Increment the agent's token_version to revoke its token
agent.token_version += 1
s.flush()
request.session.flash(f"Access revoked for agent '{agent.name}'.")
return HTTPFound(
location=request.route_url(
"manage_namespace", namespace_short_id=namespace.short_id
)
)
################################################################################
# Logging Views
################################################################################
@ -865,47 +914,48 @@ def log_webhook_view(request):
auth_header = request.headers.get("Authorization")
if not auth_header or not auth_header.startswith("Bearer "):
return Response("Missing or invalid Authorization header.", status=401)
token = auth_header[len("Bearer "):].strip()
token = auth_header[len("Bearer ") :].strip()
payload = verify_jwt_token(token)
if not payload:
return Response("Invalid or expired token.", status=401)
# Get user and check permissions
user = s.query(User).filter(User.id == payload["user_id"]).first()
if not user:
return Response("User not found.", status=404)
request.user = user
user.dbsession = s # Attach dbsession to user
# Get agent from the JWT
agent_id = payload.get("agent_id")
if not agent_id:
return Response("Agent ID not specified in token.", status=400)
agent = s.query(Agent).filter(Agent.id == agent_id).first()
if not agent:
return Response("Agent not found.", status=404)
# Issue a new JWT if needed and include it in the response
new_token = issue_new_jwt_if_needed(token, user)
if new_token:
request.response.headers['X-New-JWT'] = new_token
# Check token_version
if payload.get("token_version") != agent.token_version:
return Response("Token has been revoked.", status=401)
# Get namespace from header or parameter
namespace_short_id = request.headers.get('X-Namespace') or request.params.get('namespace')
if not namespace_short_id:
return Response("Namespace not specified.", status=400)
namespace = s.query(Namespace).filter(Namespace.short_id == namespace_short_id).first()
# Get namespace from the agent
namespace = s.query(Namespace).filter(Namespace.id == agent.namespace_id).first()
if not namespace:
return Response("Namespace not found.", status=404)
request.namespace = namespace
# Check if user has sufficient permissions
if not check_namespace_permission(user, namespace, 'editor'):
return Response("You do not have permission to write to this namespace.", status=403)
request.namespace = namespace # Attach namespace to request
# No need to issue a new token since tokens do not expire due to time
# Access namespace_dbsession
namespace_dbsession = request.namespace_dbsession
payload = request.json_body
message = payload.get('message', '')
level = payload.get('level', 'INFO')
meta_data = payload.get('metadata', None)
if meta_data:
# Parse payload
try:
json_payload = request.json_body
except json.decoder.JSONDecodeError:
return Response("Invalid JSON payload.", status=400)
message = json_payload.get("message", "")
if not message:
return Response("Log message is required.", status=400)
level = json_payload.get("level", "INFO") # Default to 'INFO' if not provided
meta_data = json_payload.get("metadata", None)
if meta_data is not None:
meta_data = json.dumps(meta_data)
# Create log entry
log_entry = LogEntry(
message=message,
level=level,
@ -917,13 +967,19 @@ def log_webhook_view(request):
return Response("Log entry created.", status=201)
@view_config(route_name="view_namespace_logs", renderer="view_namespace_logs.html.j2")
@view_config(
route_name="view_namespace_logs", renderer="view_namespace_logs.html.j2"
)
@reader_required
def view_namespace_logs_view(request):
namespace = request.namespace
namespace_dbsession = request.namespace_dbsession
logs = namespace_dbsession.query(LogEntry).order_by(LogEntry.timestamp.desc()).all()
logs = (
namespace_dbsession.query(LogEntry)
.order_by(LogEntry.timestamp.desc())
.all()
)
return {
"request": request,
@ -969,11 +1025,13 @@ def main(global_config=None, **settings):
# Set up SQLAlchemy
engine = create_engine(
DB_URL, connect_args={"check_same_thread": False}, poolclass=StaticPool
DB_URL,
connect_args={"check_same_thread": False},
poolclass=StaticPool,
)
session_factory_ = sessionmaker(bind=engine)
Base.metadata.bind = engine
#Base.metadata.create_all(engine)
Base.metadata.create_all(engine)
DBSession = scoped_session(session_factory_)
register(DBSession) # Register with zope.sqlalchemy
@ -991,7 +1049,9 @@ def main(global_config=None, **settings):
config.add_request_method(get_current_namespace, "namespace", reify=True)
# Add namespace_dbsession to requests if namespace is set
config.add_request_method(add_namespace_dbsession, "namespace_dbsession", reify=True)
config.add_request_method(
add_namespace_dbsession, "namespace_dbsession", reify=True
)
# Routes
config.add_route("home", "/")
@ -1004,15 +1064,24 @@ def main(global_config=None, **settings):
# Namespace Management
config.add_route("create_namespace", "/namespace/create")
config.add_route("manage_namespace", "/namespace/{namespace_short_id}/manage")
config.add_route(
"manage_namespace", "/namespace/{namespace_short_id}/manage"
)
config.add_route("update_namespace", "/namespace/{namespace_short_id}/update")
config.add_route("invite_user", "/namespace/{namespace_short_id}/invite")
config.add_route(
"revoke_agent", "/namespace/{namespace_short_id}/revoke_agent"
)
# Route to generate JWTs for agents
config.add_route("generate_agent_jwt", "/namespace/{namespace_short_id}/generate_agent_jwt")
config.add_route(
"generate_agent_jwt", "/namespace/{namespace_short_id}/generate_agent_jwt"
)
# Logging
config.add_route("log_webhook", "/webhook")
config.add_route("view_namespace_logs", "/namespace/{namespace_short_id}/logs")
config.add_route(
"view_namespace_logs", "/namespace/{namespace_short_id}/logs"
)
config.scan()
return config.make_wsgi_app()

17
templates/manage_namespace.html.j2
View file

@ -39,6 +39,23 @@
<p>No users in this namespace.</p>
{% endif %}
<h2>Agents</h2>
{% if agents %}
<ul>
{% for agent in agents %}
<li>
{{ agent.name }}
<form method="POST" action="{{ request.route_url('revoke_agent', namespace_short_id=namespace.short_id) }}" style="display:inline;">
<input type="hidden" name="agent_id" value="{{ agent.id }}">
<button type="submit" onclick="return confirm('Revoke access for {{ agent.name }}?')">Revoke Access</button>
</form>
</li>
{% endfor %}
</ul>
{% else %}
<p>No agents found.</p>
{% endif %}
<h2>Generate JWT for Agent</h2>
<form method="POST" action="{{ request.route_url('generate_agent_jwt', namespace_short_id=namespace.short_id) }}">
<label for="agent_name">Agent Name:</label>

165
test_agent.sh
View file

@ -1,150 +1,33 @@
#!/usr/bin/env bash
#
# A Bash script to demonstrate and test multiple authenticated routes
# in the pyrafiles application (or a similar Pyramid-based app).
#
# This script:
# 1. Checks if cookies.txt is older than a defined threshold (default 1 year).
# - If too old, it deletes cookies.txt so we force re-login.
# 2. If cookies.txt is still valid or absent, proceeds to the next steps:
# - If cookies.txt does not exist or was removed, it does the email + OTP flow.
# - If cookies.txt exists and is fresh, it reuses that session.
# 3. Fetches and prints the Profile page (GET /auth/profile) as a quick test.
# 4. Uploads a sample media file (POST /media/upload), marked public.
# 5. Parses the server's redirect "Location" header to extract user_short_id and media_short_id.
# 6. Views media details (GET /media/<user_short_id>/<media_short_id>/details).
# 7. Edits the newly uploaded media (POST /media/<user_short_id>/<media_short_id>/edit).
# 8. Deletes the media (POST /media/<user_short_id>/<media_short_id>/delete).
# 9. Lists all public media (GET /media/list).
# 10. Logs out (GET /auth/logout).
#
# The script exits on the first command error (set -e).
# Adjust BASE_URL, EMAIL, and file paths as needed.
#
# Usage:
# MAX_COOKIE_AGE=<seconds> ./test_all_routes.sh /path/to/somefile.jpg
# By default, MAX_COOKIE_AGE=31536000 (1 year).
# If cookies.txt is younger than that, we skip re-login. Otherwise, we do the OTP flow.
#
# By default, uses http://localhost:6544 and agent@example.com
# ./test_agent.sh "Your log message"
set -e # Exit on first error
# Description:
# Sends a log message to the PyraLogs server using the provided JWT token.
# Adjust the BASE_URL and JWT_TOKEN variables as needed.
BASE_URL="${BASE_URL:-http://localhost:6544}"
EMAIL="${EMAIL:-agent@example.com}"
MEDIA_FILE="$1"
MAX_COOKIE_AGE="${MAX_COOKIE_AGE:-31536000}" # default ~1 year in seconds
# Variables
BASE_URL="http://127.0.0.1:6544"
JWT_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhZ2VudF9pZCI6IjMzMmM3ZjY2LWMzNTktNDg4Yi05MTIxLWIwN2Y1N2EyMTA3MCIsImFnZW50X25hbWUiOiJ0ZXN0LTkwMDAiLCJuYW1lc3BhY2VfaWQiOiI3YmM3Mjk5Yy1hYTgxLTRhYjQtYmNmMy00OTU5NzNhZjk3NzEiLCJ0b2tlbl92ZXJzaW9uIjowLCJpYXQiOjE3MzY2NDQyNjB9.fTZuLTFARjgTXIUtg2zGkfcZp3sAJYAasSa6MFxK-Gk"
if [[ -z "$MEDIA_FILE" ]]; then
echo "Usage: MAX_COOKIE_AGE=<seconds> $0 /path/to/mediafile"
# Log message from the first argument
LOG_MESSAGE="$1"
# Check if a log message was provided
if [[ -z "$LOG_MESSAGE" ]]; then
echo "Usage: $0 \"Your log message\""
exit 1
fi
echo "=== Checking cookies.txt freshness ==="
# Send the log message using curl
RESPONSE=$(curl -s -X POST "$BASE_URL/webhook" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $JWT_TOKEN" \
-d "{
\"message\": \"$LOG_MESSAGE\",
\"level\": \"INFO\"
}")
COOKIE_FRESH=0
if [[ -f cookies.txt ]]; then
# 'stat -c %Y' returns the file modification time on Linux
# 'stat -f %m' does the same on BSD/macOS
MOD_TIME=$(stat -c %Y cookies.txt 2>/dev/null || stat -f %m cookies.txt 2>/dev/null)
NOW=$(date +%s)
AGE=$((NOW - MOD_TIME))
echo "cookies.txt is $AGE seconds old, threshold is $MAX_COOKIE_AGE."
if (( AGE < MAX_COOKIE_AGE )); then
COOKIE_FRESH=1
echo "Reusing existing cookies.txt (fresh enough)."
else
echo "cookies.txt is too old. Removing it to force re-login."
rm -f cookies.txt
fi
else
echo "No cookies.txt found. A new session will be created."
fi
if [[ "$COOKIE_FRESH" -eq 0 ]]; then
echo ""
echo "=== Starting new session and logging in with OTP flow ==="
curl -s -i -c cookies.txt -b cookies.txt "$BASE_URL/" >/dev/null
echo "Logging in with email: $EMAIL (POST /auth/login)"
curl -s -i -c cookies.txt -b cookies.txt \
-X POST \
-F "email=$EMAIL" \
"$BASE_URL/auth/login"
echo ""
echo "Check your server logs or email for the 6-digit verification code."
read -p "Enter the 6-digit code: " VERIFICATION_CODE
echo "Verifying code (POST /auth/verify)"
curl -s -i -c cookies.txt -b cookies.txt \
-X POST \
-F "code=$VERIFICATION_CODE" \
"$BASE_URL/auth/verify"
echo "Login/OTP flow complete. cookies.txt saved."
fi
echo ""
echo "=== Quick check: Fetching Profile page (GET /auth/profile) ==="
curl -i -c cookies.txt -b cookies.txt "$BASE_URL/auth/profile"
echo ""
echo "=== Uploading media (POST /media/upload) as public ==="
UPLOAD_RESPONSE=$(curl -i -c cookies.txt -b cookies.txt \
-X POST \
-F "title=TestUploadByAgent" \
-F "media_file=@${MEDIA_FILE}" \
-F "is_public=on" \
"$BASE_URL/media/upload")
echo "$UPLOAD_RESPONSE"
echo ""
echo "Parsing 'Location' header from upload response..."
LOCATION_HEADER=$(echo "$UPLOAD_RESPONSE" | grep -i "^Location:" | head -n1 | sed 's/\r//g')
LOCATION_URL="${LOCATION_HEADER#Location: }"
LOCATION_URL=$(echo "$LOCATION_URL" | tr -d '[:space:]')
LOCATION_PATH="${LOCATION_URL#*//*/}"
USER_SHORT_ID=$(echo "$LOCATION_PATH" | cut -d'/' -f2)
MEDIA_SHORT_ID=$(echo "$LOCATION_PATH" | cut -d'/' -f3)
echo "user_short_id=$USER_SHORT_ID"
echo "media_short_id=$MEDIA_SHORT_ID"
if [[ -z "$USER_SHORT_ID" || -z "$MEDIA_SHORT_ID" ]]; then
echo "Failed to parse user or media short ID. Exiting."
exit 1
fi
echo ""
echo "=== Viewing media details (GET /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/details) ==="
curl -i -c cookies.txt -b cookies.txt \
"$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/details"
echo ""
echo "=== Editing media title to 'RenamedByAgent' (POST /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/edit) ==="
curl -i -c cookies.txt -b cookies.txt \
-X POST \
-F "title=RenamedByAgent" \
"$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/edit"
echo ""
echo "=== Deleting the media (POST /media/$USER_SHORT_ID/$MEDIA_SHORT_ID/delete) ==="
curl -i -c cookies.txt -b cookies.txt \
-X POST \
"$BASE_URL/media/$USER_SHORT_ID/$MEDIA_SHORT_ID/delete"
echo ""
echo "=== Listing all public media (GET /media/list) ==="
curl -i -c cookies.txt -b cookies.txt "$BASE_URL/media/list"
echo ""
echo "=== Logging out (GET /auth/logout) ==="
curl -i -c cookies.txt -b cookies.txt "$BASE_URL/auth/logout"
echo ""
echo "Test complete. We exercised multiple routes, including login/verify, profile, "
echo "upload/delete/edit media, and logout. If no errors appeared, all requests succeeded!"
# Output the server's response
echo "Server Response: $RESPONSE"