- is_base64_image() now detects any base64 image (jpeg, png, gif, webp, etc.)
not just the hardcoded jpeg/png patterns
- Added [Vision] debug logging to trace image fetch/save flow
URLs like eBay images contain % and _ which have special meaning in SQL
LIKE patterns. Added escape_like_pattern() helper to escape these chars.
This fixes saved base64 images not being found on refresh.
- Find the most recent image message
- Include full text conversation history for context
- Only send base64 for the most recent image
- Skip older images entirely (no useful text in them)
Multiple large base64 images can overwhelm the vision model. Now we:
1. First pass: identify the 2 most recent image messages
2. Second pass: build chat history, skipping older images
Remove race condition where syncDropdownsAndQueryString() was called before
dropdowns were populated, causing empty values to overwrite localStorage.
Model and voice restoration now happens only in populateModelDropdown() and
populateVoiceDropdown() after async fetch completes.
- Add find_saved_base64_for_url() to lookup existing saved base64 in DB
- Check for existing saved version before fetching external images
- Prevent duplicate saves in save_fetched_image_as_message()
- Use in-memory cache for faster repeated lookups within same session
- Fix Message.query.filter_by() chain mocking in test_bedrock_streaming_protocol
and test_streaming_protocol_backwards_compatibility
- Add is_gevent_patched() helper to detect monkey patching
- Skip bedrock test at runtime when gevent has already patched (avoids RecursionError)
- save_fetched_image_as_message() creates a new message with base64 content
- build_message_content() now accepts room_id to persist fetched images
- Fetched images saved with username "system" (added to SYSTEM_USERS)
- Prevents re-fetching: once fetched, base64 version is in chat history
- extract_external_image_url() to find http/https URLs in img tags
- fetch_external_image_as_base64() fetches via CORS proxy, converts to base64
- build_message_content() now handles both base64 and external image URLs
- Caches fetched images to avoid re-fetching
- Uses cors-proxy.uncloseai.com for robots.txt compliance
- Fetch external images through proxy.unturf.com (respects robots.txt)
- Convert fetched images to base64 for vision API
- Cache both fetched images and descriptions
- Shows "Fetching image..." for external URLs, "Generating description..." for base64
- Handles 403 responses when blocked by robots.txt
Backend:
- Track VISION_MODELS list at startup from available endpoints
- Add is_vision_model() to detect vision-capable models (*-vl*, *vision*, gpt-4o)
- Add extract_base64_from_img_tag() and build_message_content() helpers
- Modify chat_gpt() to include base64 images for vision models
- Add GET /vision endpoint for vision availability status
- Add POST /vision/describe endpoint for image alt-text generation
Frontend:
- Check vision availability on page load via /vision
- Add hover event delegation on chat images
- On hover: call vision model, cache result, set img.title and img.alt
- Shows cursor:wait while loading description
Update code execution error detection to use stderr exclusively
instead of checking both stdout and stderr. This aligns with the
Unsandbox API's proper stream separation where:
- stdout is for program output
- stderr is for errors/warnings
Changes:
- Remove stdout from error detection logic
- Only trigger auto-fix when exit_code != 0 AND stderr is non-empty
- Send only stderr to /api/fix-code endpoint
This prevents false positives where stdout contains normal output
that was previously being treated as error content.
- Use codeBlock.dataset.autoExecAttempt to pass attempt number
- executeCodeBlock transfers attempt to results container
- No longer pre-creates empty results container
- Clean up data attribute after use
- Add null check for classList before calling contains()
- Handles case where nextSibling might be a text node
- Prevents 'classList is undefined' error
- Correctly insert results container after button container
- Use buttonContainer.parentNode.insertBefore() instead of preElement
- Add proper styling when creating results container
- Fixes 'Child to insert before is not a child of this node' error
- Post fixed code to chat so user can see what was changed
- Store fix data in window.pendingAutoExec for message handler
- Auto-trigger execution on newly posted code block
- Preserve attempt counter across fix iterations
- User now sees: original error → fixed code posted → auto-execution → results
- Remove chat message posting that prevented automatic re-execution
- Fixed code now executes directly using same blockElement
- Simplify re-execution flow with direct recursive call
- Results display in same execution results container
Extract infrastructure/proxy debugging information from CLAUDE.md
into unturf-debugging.md to avoid exposing internal network details.
- Created unturf-debugging.md with full proxy chain documentation
- Added unturf-debugging.md to .gitignore
- Simplified CLAUDE.md to reference debugging doc
- Documented Caddy configuration (not nginx)
- Included troubleshooting steps for 502 errors
This keeps sensitive infrastructure details out of the public repo
while maintaining documentation for internal debugging.
Update /api/fix-code endpoint to try MODEL_3 (Qwen Coder) first for
better code generation, with graceful fallback to MODEL_1 (Hermes) if
MODEL_3 is unavailable or returns errors.
This ensures the auto-fix feature works reliably even when specialized
code models are temporarily unavailable (e.g., 502 errors).
Implements an AI-powered auto-fix system that automatically attempts to
repair code execution errors up to 3 times.
Backend changes (app.py):
- Add /api/fix-code endpoint that uses MODEL_1 (Hermes) to analyze
stderr output and generate corrected code
- System prompt instructs AI to output only raw fixed code without
explanations or markdown formatting
- Accepts code, language, stderr, exit_code, and attempt number
Frontend changes (templates/chat.html):
- Modify executeCodeBlock() to detect failed executions (exit_code !== 0)
- Track fix attempts per code block (max 3) using dataset attributes
- Call /api/fix-code when errors are detected
- Display fixed code as new chat message with markdown formatting
- Automatically re-execute the corrected code recursively
- Show progress messages during auto-fix attempts
- Display warning when max attempts (3) are exhausted
Features:
- Fixes common issues: missing imports, syntax errors, type errors
- Posts fixed code to chat for transparency
- Prevents infinite loops with 3-attempt limit
- Graceful error handling with user-friendly status messages
Add inline script at top of base.html <head> to set theme immediately
before any rendering occurs. Prevents white flash when loading chat
and other pages that extend base.html in dark mode.
- Chat messages: logged-in user's name links to /profile
- User lists (sidebar): logged-in user's name is clickable link
- Previous messages: same profile link logic applied
- Other users still link to /profile/username (future feature)
Anywhere the logged-in user's name appears, it now links to their
profile settings page for quick access to logout and username change.
- Add inline script to set theme before page render (browse, profile)
- Prevents white flash when loading pages in dark mode
- Add logout button with confirmation to profile page
- Username already clickable in browse page header (links to profile)
When SMTP_FROM_EMAIL is not set, automatically derive the sender domain from:
1. Flask request.host (if not localhost/127.0.0.1)
2. System FQDN hostname (socket.getfqdn())
3. SMTP_USER or fallback to noreply@opencompletion.local
This allows the app to use the correct sender domain (e.g., noreply@ai.foxhop.net)
when deployed on different hosts, ensuring proper email relay through mx servers.
- Update CODE_EXEC_URL from code.ai.unturf.com to api.unsandbox.com
- Fix response field handling for Unsandbox API format (flat structure)
- Add exit code display with color coding (green=0, red=error)
- Update displayExecutionResults to handle stdout/stderr/exit_code at top level
- Simplify error handling for timeout/cancelled jobs
- Add comprehensive Unsandbox API documentation to CLAUDE.md
* Redesign UI: unified design system across all pages
- Replace scattered inline CSS with comprehensive single style.css
- Implement modern design system:
- System font stack (-apple-system, Segoe UI, etc.)
- Consistent spacing scale (4px base)
- Unified color tokens for light/dark themes
- Reusable component classes (buttons, cards, badges, forms)
- Update all templates to use new CSS classes:
- browse.html: Less chunky, better space usage with room-grid
- index.html: Cleaner centered card layout
- auth.html: Streamlined 4-step flow with gradient background
- profile.html: Modern settings interface
- search.html: Browse-style card layout
- Chat page improvements:
- Tighter layouts (240px/280px sidebars instead of 15%/25%)
- Better message spacing
- Improved code blocks with proper padding
- Cleaner utility belt
- Better responsive design and dark mode support
* modified: app.py
modified: templates/base.html
modified: templates/profile.html
* Revert standalone pages to original design
- Browse, index, auth, profile restored to original inline CSS
- Chat page improvements preserved in style.css
- Search page still uses improved card layout
* Add dark mode support to index and browse pages
- index.html now supports dark mode with CSS variables
- browse.html now supports dark mode with CSS variables
- Theme persists from localStorage across pages
* Remove theme toggle from chat page
- Theme toggle button removed from desktop chat sidebar
- Theme toggle button removed from mobile chat modal
- Theme management now done via profile page only
* Make usernames clickable links to profile pages in chat
* Update search page to match browse page layout and CSS
* Fix chat layout positioning
* Fix room list auto-update when title changes
* Fix new room creation appearing in sidebar
- Add socketio.emit in create_room_api() to broadcast new rooms
- Update socket handler to add new rooms to sidebar dynamically
- Rooms now appear without hard refresh
* Fix /title and /cancel commands being sent to LLM
- Add missing return statements after command handlers
- Commands now properly terminate message processing
- Prevents commands from being interpreted as chat messages
* Make usernames in user lists clickable links to profiles
- Update updateUserLists() to create links for all usernames
- Add hover effect CSS for user list links
- Works for both active and inactive users
- Works for both desktop and mobile views
* Revert user list profile links and update profile page layout
- Remove profile links from user lists (no backend route for other users)
- Update profile page to full-screen layout like browse page
- Add header with navigation buttons
- Remove centered container, use full-width layout
- Add box shadows to sections for visual separation
* Convert all flexbox layouts to CSS grid
- Replace all display: flex with CSS grid equivalents
- Update templates: profile, browse, search, index, auth
- Update static CSS for consistent grid usage
- Use grid-template-columns, grid-auto-flow, and place-items
- Improve layout consistency across all pages
* Fix chatroom horizontal scrolling
- Add overflow-x: hidden to #chat-container and #chat to prevent horizontal scroll
- Add word-break and overflow-wrap to message content for text wrapping
- Change pre tags from overflow: hidden to overflow-x: auto for individual scrolling
- Add min-width: 0 to grid containers to prevent overflow
- Code blocks can now scroll individually while chatroom wraps content
* Remove duplicate CSS variables and fix XSS vulnerability
- search.html: Remove inline styles, link to style.css
- index.html: Remove duplicate CSS variable blocks, link to style.css
- browse.html: Remove duplicate CSS variable blocks, link to style.css
- chat.html: Fix XSS vulnerability in room list updates
- Use textContent/createTextNode instead of innerHTML for user data
- Use DOM methods instead of string concatenation
- Encode URL components with encodeURIComponent
- Extract user count from textContent instead of innerHTML regex
* Merge duplicate CSS rules and replace inline styles with design system
style.css:
- Merge duplicate html, body rules (lines 137-143 and 159-167)
- Consolidate typography and layout properties in single rule
- Remove duplicate BASE LAYOUT section
profile.html:
- Replace style.display mutations with classList API
- Add .availability-indicator.show CSS rule for visibility
- Use classList.add('show') and classList.remove('show')
- Consistent with existing .message.show pattern
browse.html:
- Replace hard-coded gradient colors with CSS variables
- Use var(--gradient-start) and var(--gradient-end) for buttons
- Replace #667eea with var(--button-primary) for tabs and room names
- Remove inline .room-badge styles, use .badge .badge-public/.badge-private
- Apply existing badge classes from style.css for dark mode support
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Russell Ballestrini <russell@unturf.com>
* Add email OTP authentication system and private room management
## Authentication System
- Implement email OTP-based authentication with User and OTPToken models
- Add auth.py module with OTP generation, email sending, and session management
- Create authentication endpoints: /auth/send-otp, /auth/verify-otp, /auth/claim-name, /auth/status, /auth/logout
- Add authentication modal UI with email verification and display name claiming
- Support SMTP configuration via environment variables (optional)
## Room Privacy & Ownership
- Add is_private, is_archived, owner_id, and forked_from_id fields to Room model
- Implement private rooms (single-user, owner-only access)
- Add room forking: users can fork public rooms to public or private
- Add room archive/delete endpoints (owner-only operations)
- Implement access control for private room viewing
## UI Improvements
- Add public/private room tabs in sidebar
- Show authentication prompt in private rooms tab for non-authenticated users
- Add room action buttons (fork, archive, delete) with proper permissions
- Update homepage with statistics (public/private rooms, active users, active rooms)
- Remove model/voice from URL query strings, use localStorage exclusively
## Database Migration
- Create migration 2025011100 for User, OTPToken tables and Room model updates
- Add indexes for email, display_name, is_private, is_archived, owner_id
## Breaking Changes
- URL parameters now only include username (model/voice moved to localStorage)
- Private rooms require authentication to access
- Room creation can now require authentication (for private rooms)
* Update README with authentication and private room documentation
* Simplify README to be less verbose
* Add missing session import to fix linter errors
* Fix migration dependency to resolve multiple heads conflict
* asdf
* Fix SQLAlchemy auto-correlation error in homepage statistics query
* Change tagline from AI-Powered to Machine Learning Powered
* Add dedicated authentication page instead of modal
- Create new /auth route with full-page authentication flow
- Remove modal code from index.html
- Update Sign in link to point to /auth page
- Auth page has 4-step flow: email, OTP, display name, success
- Better UX with gradient background and cleaner design
* Fix migration: remove batch_alter_table to avoid circular dependency
- Use op.add_column() directly instead of batch_alter_table()
- Remove foreign key constraints (defined in models, not needed in migration)
- User and OTPToken tables created by db.create_all() in make init-db
- Fixes CircularDependencyError during migration
* Fix migration: check if columns exist before adding
- Use inspector to check existing columns and indexes
- Only add columns/indexes if they don't already exist
- Handles case where db.create_all() was run before migration
- Fixes 'duplicate column name' error
* Add profile page, room browsing, and updated_at timestamp
Features:
- Profile page with username change and dark/light mode settings
- Browse page for discovering public and private rooms
- Room updated_at timestamp (integer Unix epoch) that updates on new messages
- Dynamic room tabs based on current room type (public/private)
- Fork and delete room actions moved to right sidebar utility belt
- Remove archive feature and success alerts from room actions
Technical changes:
- Add updated_at column to Room model (integer timestamp)
- Add /profile route with authentication requirement
- Add /browse route for room discovery
- Add API endpoints for username availability check and update
- Update room.updated_at on message creation in app.py:1189
- Wider right sidebar (25% instead of 15%) for better button layout
- Profile link on homepage and browse page for authenticated users
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: russell@unturf. <russell@unturf.com>
Document new code execution feature with 30+ language support, isolated sandbox containers, and binary downloads. Also mention text-to-speech capability.
Updated the artifact filename generator to use snake_case (underscores)
instead of kebab-case (dashes) for better consistency with Python
naming conventions.
Changes:
- Updated AI prompt examples to show underscore format
- Modified filename processing to replace spaces with underscores
- Updated validation regex to accept underscores instead of dashes
- Changed docstring to reflect underscore usage
Examples: hello_world, prime_checker, array_sort (instead of hello-world, etc.)
Co-authored-by: Claude <noreply@anthropic.com>
* Add AI-powered artifact naming for compiled binaries
Implements intelligent filename generation for downloaded binaries using
Hermes AI to analyze code and generate meaningful 1-3 word filenames.
Changes:
- Add /api/generate-artifact-name endpoint that uses MODEL_1 (Hermes)
- Modify frontend to call naming API before download
- Add ENABLE_AI_ARTIFACT_NAMING environment variable (enabled by default)
- Filenames are descriptive (e.g., "fizzbuzz", "hello-world", "prime-checker")
- Graceful fallback to "compiled_binary" if naming fails or is disabled
The feature can be disabled by setting ENABLE_AI_ARTIFACT_NAMING="false"
in environment variables.
* Rename env var to ENABLE_CODE_GEN_FILENAMES
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Reposition download button from output area to button container
- Place next to Copy and Run buttons with consistent styling
- Button hidden by default, shown only when artifact available
- Remove colored background to match other action buttons
Co-authored-by: Claude <noreply@anthropic.com>
* Enable binary downloads on timeout/cancellation
When code execution times out or is cancelled, the compiled binary
may still be available. This change ensures displayExecutionResults()
is called for timeout/cancelled jobs, allowing users to download
the binary artifact even when execution doesn't complete normally.
* Fix partial output display for timeout/cancellation
Previous commit broke partial output display by passing job.result
directly to displayExecutionResults(), but for timeout/cancelled jobs
the output is in partial_output field, not stdout.
Now properly maps partial_output to stdout before displaying, so users
see both the error message and any output that was captured before
timeout/cancellation, plus binary downloads if available.
* Add debugging for missing artifact on timeout/cancel
Check multiple possible locations for artifact:
- job.artifact (top level)
- job.result.artifact (nested)
Add console logging to see full job structure when timeout/cancel
occurs so we can understand why the binary isn't appearing.
* Try fetching artifact from separate endpoint on timeout/cancel
When timeout/cancel occurs, the artifact isn't in the job response.
Try fetching from /jobs/{job_id}/artifact endpoint as a fallback.
This explores whether the executor service has a separate artifact
endpoint that we can use to retrieve compiled binaries even when
execution is cancelled or times out.
* Remove debug logging, document artifact limitation
Removed console.log debugging statements now that we've confirmed
the executor service doesn't include artifacts in timeout/cancelled
responses and doesn't have a /jobs/{job_id}/artifact endpoint.
Kept the artifact fetching code with comments for future compatibility
if the executor service adds this feature.
Current limitation: Binary downloads only work for completed executions,
not for timeout/cancelled ones. The binary exists but the executor
service doesn't return it.
* Try multiple artifact endpoint patterns for timeout/cancel
When artifact isn't in the job response, try fetching from:
- /artifacts/{job_id}
- /jobs/{job_id}/artifact
- /jobs/{job_id}/download
- /jobs/{job_id}/binary
- /download/{job_id}
- /binary/{job_id}
Handles both JSON responses and direct binary responses. Logs
each attempt to console so we can see which endpoint (if any) works.
* Revert endpoint searching - artifact should be in /jobs/{id}
According to OpenAPI spec, there are no separate artifact endpoints.
The artifact should be included in GET /jobs/{id} response for ALL
job statuses (completed, cancelled, timeout).
Current limitation: The executor service only includes result.artifact
for "completed" status, not for "cancelled" or "timeout" status.
The frontend code is correct - it checks job.artifact and
job.result.artifact. The issue is the executor service needs to
include the artifact in cancelled/timeout responses.
* Add debug logging for cancelled/timeout artifact checks
Since the executor service was supposedly patched to include artifacts
in GET /jobs/{id} responses even for cancelled/timeout jobs, add
detailed logging to verify:
1. What the full job response looks like
2. Whether artifact is at job.artifact or job.result.artifact
3. Artifact details if found
This will help determine if the patch is deployed and working.
* Add test-artifact Makefile target for testing executor API
Tests binary artifact retrieval from code executor service:
- Compiles C code with return_artifact=true
- Extracts base64 artifact from response
- Decodes and executes the binary
Can test against different URLs:
make test-artifact URL=https://code.ai.unturf.com
Tested against production and confirmed:
- Artifacts ARE included for completed jobs
- Artifacts are NOT included for cancelled/timeout jobs (even with
return_artifact=true). Exit code 137 indicates SIGKILL.
* Document confirmed limitation - no artifacts for cancelled jobs
Tested against production executor API (make test-artifact) and confirmed:
- Cancelled jobs return exit_code 137 (SIGKILL)
- NO artifact field in response (neither job.artifact nor job.result.artifact)
- Artifacts only returned for fully completed jobs
Code still checks for artifacts in case this limitation is fixed
in the future, but currently binary downloads will not work for
cancelled/timeout executions.
To fix: Executor service needs to include compiled binary in
response even when execution is killed (compilation succeeded).
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Request compiled binaries via return_artifact parameter
- Add "Download Binary" button when artifact is available
- Support base64 decoding and browser download
- Handle artifact errors gracefully
- Works with C, C++, Rust, Go, Java, and other compiled languages
Co-authored-by: Claude <noreply@anthropic.com>
Problem:
- Buttons were being inserted as children of <pre> element
- This caused buttons to appear inside code blocks with wrong styling
- Template caching made changes appear to require "two commits"
Solution:
- Change insertion point from block.parentNode to preElement.parentNode
- This places buttons as siblings of <pre>, not children
- Add TEMPLATES_AUTO_RELOAD=True to prevent Flask template caching
Technical Details:
- block is the <code> element
- block.parentNode is the <pre> element
- preElement.parentNode.insertBefore puts buttons after <pre>
- Previous code put buttons inside <pre> after <code>
DOM Structure Before:
<pre>
<code>...</code>
<buttons> <!-- Wrong: inside pre -->
</pre>
DOM Structure After:
<pre>
<code>...</code>
</pre>
<buttons> <!-- Correct: after pre -->
Co-authored-by: Claude <noreply@anthropic.com>
* Move code block buttons below code instead of above
- Modified addCopyButtonToCodeBlock to insert button container after code block
- Updated truncateCodeBlock to remove duplicate buttons before adding its own
- Ensures clean button placement for both regular and truncated code blocks
* Refactor code block button rendering for efficiency
- Process blocks in optimal order: truncate → highlight → line numbers → buttons
- Eliminate redundant button creation/removal cycle
- truncateCodeBlock now only truncates and returns boolean
- addCopyButtonToCodeBlock handles all button creation (including Show More)
- Buttons always appear below code blocks after full processing
This prevents wasteful creation and immediate deletion of buttons for truncated blocks.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add download button for TTS audio
- Add download button next to Play button for all messages
- Button is initially hidden and appears after TTS audio is generated
- Works for both manual play and auto-play modes
- Works for both regular and streaming messages
- Handles cached audio properly
- Download filename includes message ID and voice name
* Refactor: Extract download button logic into helper function
- Create enableDownloadButton() helper to eliminate code duplication
- Replace 4 identical blocks (56 lines) with 4 function calls (4 lines)
- Improves maintainability and follows DRY principle
- Handles both cached and fresh audio in both speakText functions
* Remove hardcoded voice fallbacks, use API or empty list
- Remove hardcoded voice options from HTML dropdown
- Remove all fallbacks to default voices (tts-1:onyx)
- If voices API fails, leave dropdown empty instead of falling back
- localStorage persistence for voice selection already implemented
- Voices API caching already working (1-minute cache like models)
- Voice selection now purely driven by API response
* Fix: Make download button visible after TTS audio loads
- Add download button to previous_messages handler (was missing)
- Change display from "" to "inline-block" for visibility
- Download button now appears properly after TTS processes
* Add debug logging for download button issue
- Add console.log to trace enableDownloadButton execution
- Change === to == for messageId comparison (handle type coercion)
- Log wrapper status, button status, and ID matching
- This will help identify why download button doesn't appear
* Remove debug logging, keep type coercion fix
- Remove console.log statements now that issue is identified
- Keep == comparison (was the actual fix)
- Add comment explaining why == instead of ===
- dataset.messageId is string, messageId param is number
---------
Co-authored-by: Claude <noreply@anthropic.com>
The integration tests were failing in GitHub Actions with 'unable to open
database file' errors because the Flask instance directory didn't exist.
The app.py code at line 40 creates a database URI using app.instance_path,
which requires that directory to exist. In GitHub Actions, this directory
doesn't exist by default, causing SQLite to fail when trying to create
the database file (even though tests override to use :memory:).
Solution: Create instance directory in setUp() before app context is pushed.
Changes:
- Add os.makedirs(app.app.instance_path, exist_ok=True) in setUp()
- Also fixed temp file paths to use absolute paths for research directory
- All 9 integration tests now pass locally
This ensures tests work in both local and GitHub Actions environments.
Fixed the integration tests to use absolute paths when creating
temporary activity YAML files in the research directory.
The tests were failing in GitHub Actions with "unable to open database
file" errors because they used relative paths (Path("research")) which
didn't work correctly in the GitHub Actions working directory.
Changes:
- Use Path(__file__).parent.parent.parent to get absolute base directory
- Apply absolute path to both file creation and cleanup operations
- All 9 integration tests pass locally
This ensures tests work consistently across local development and
GitHub Actions environments.
