Commit graph

6 commits

Author SHA1 Message Date
a260b5fa02
Handle cancellation and timeout recovery (#37)
* Enable binary downloads on timeout/cancellation

When code execution times out or is cancelled, the compiled binary
may still be available. This change ensures displayExecutionResults()
is called for timeout/cancelled jobs, allowing users to download
the binary artifact even when execution doesn't complete normally.

* Fix partial output display for timeout/cancellation

Previous commit broke partial output display by passing job.result
directly to displayExecutionResults(), but for timeout/cancelled jobs
the output is in partial_output field, not stdout.

Now properly maps partial_output to stdout before displaying, so users
see both the error message and any output that was captured before
timeout/cancellation, plus binary downloads if available.

* Add debugging for missing artifact on timeout/cancel

Check multiple possible locations for artifact:
- job.artifact (top level)
- job.result.artifact (nested)

Add console logging to see full job structure when timeout/cancel
occurs so we can understand why the binary isn't appearing.

* Try fetching artifact from separate endpoint on timeout/cancel

When timeout/cancel occurs, the artifact isn't in the job response.
Try fetching from /jobs/{job_id}/artifact endpoint as a fallback.

This explores whether the executor service has a separate artifact
endpoint that we can use to retrieve compiled binaries even when
execution is cancelled or times out.

* Remove debug logging, document artifact limitation

Removed console.log debugging statements now that we've confirmed
the executor service doesn't include artifacts in timeout/cancelled
responses and doesn't have a /jobs/{job_id}/artifact endpoint.

Kept the artifact fetching code with comments for future compatibility
if the executor service adds this feature.

Current limitation: Binary downloads only work for completed executions,
not for timeout/cancelled ones. The binary exists but the executor
service doesn't return it.

* Try multiple artifact endpoint patterns for timeout/cancel

When artifact isn't in the job response, try fetching from:
- /artifacts/{job_id}
- /jobs/{job_id}/artifact
- /jobs/{job_id}/download
- /jobs/{job_id}/binary
- /download/{job_id}
- /binary/{job_id}

Handles both JSON responses and direct binary responses. Logs
each attempt to console so we can see which endpoint (if any) works.

* Revert endpoint searching - artifact should be in /jobs/{id}

According to OpenAPI spec, there are no separate artifact endpoints.
The artifact should be included in GET /jobs/{id} response for ALL
job statuses (completed, cancelled, timeout).

Current limitation: The executor service only includes result.artifact
for "completed" status, not for "cancelled" or "timeout" status.

The frontend code is correct - it checks job.artifact and
job.result.artifact. The issue is the executor service needs to
include the artifact in cancelled/timeout responses.

* Add debug logging for cancelled/timeout artifact checks

Since the executor service was supposedly patched to include artifacts
in GET /jobs/{id} responses even for cancelled/timeout jobs, add
detailed logging to verify:

1. What the full job response looks like
2. Whether artifact is at job.artifact or job.result.artifact
3. Artifact details if found

This will help determine if the patch is deployed and working.

* Add test-artifact Makefile target for testing executor API

Tests binary artifact retrieval from code executor service:
- Compiles C code with return_artifact=true
- Extracts base64 artifact from response
- Decodes and executes the binary

Can test against different URLs:
  make test-artifact URL=https://code.ai.unturf.com

Tested against production and confirmed:
- Artifacts ARE included for completed jobs
- Artifacts are NOT included for cancelled/timeout jobs (even with
  return_artifact=true). Exit code 137 indicates SIGKILL.

* Document confirmed limitation - no artifacts for cancelled jobs

Tested against production executor API (make test-artifact) and confirmed:
- Cancelled jobs return exit_code 137 (SIGKILL)
- NO artifact field in response (neither job.artifact nor job.result.artifact)
- Artifacts only returned for fully completed jobs

Code still checks for artifacts in case this limitation is fixed
in the future, but currently binary downloads will not work for
cancelled/timeout executions.

To fix: Executor service needs to include compiled binary in
response even when execution is killed (compilation succeeded).

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-11 16:34:05 -05:00
e28dc11f04 Improve user experience with battleship feedback and auto-play TTS
- Fix battleship feedback perspective confusion with better Hermes prompting
- Add auto-play TTS button with localStorage persistence and queueing system
- Move activity controls below model/voice selectors in sidebar
- Add activity controls to mobile hamburger menu
- Fix model/activity dropdowns to stay within container bounds
- Filter activities API to only show .yaml/.yml files
- Clean up system message labels by moving to usernames (System (Feedback), System (Question))
- Apply black formatting to app.py
2025-08-11 09:34:29 -04:00
092ebd0ee0
Update Makefile
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-08-10 20:54:43 -04:00
45a8f60cd2 Significantly improve test coverage with comprehensive integration tests
Major improvements:
- app.py coverage: 15% → 25% (+10 percentage points)
- research/guarded_ai.py coverage: 68% → 81% (+13 percentage points)
- Overall project coverage: 68% → 72% (+4 percentage points)

Key changes:
- Add comprehensive Flask integration tests for app.py activity functions
- Test real database operations with in-memory SQLite
- Add extensive guarded_ai.py error handling and client management tests
- Enhanced Makefile with comprehensive test targets
- Updated requirements-test.txt with flake8
- All 135 tests now passing with proper test coverage

The integration tests use real Flask environment, actual YAML processing,
and genuine database operations instead of mocks for accurate coverage.
2025-08-10 20:52:56 -04:00
1b44c2d66b Integrate comprehensive testing framework with Makefile
- Added unit tests for YAML loading and parsing functionality
- Created integration tests for multiple activity files validation
- Implemented functional tests for complete activity workflows
- Added battleship pre_script functionality tests
- Integrated all test types into comprehensive Makefile
- Fixed CLI validator test with proper failing fixture
- Applied black formatting to all Python files
- Removed problematic hardcoded targets from Makefile
- Added proper venv dependency management

Test coverage includes:
- Unit: YAML loading, validator functionality
- Integration: Cross-file validation, metadata operations
- Functional: End-to-end activity flows, pre_script execution
- All 30 activity files validated and tested
2025-08-10 19:38:50 -04:00
d4a075ac9a Complete testing framework with comprehensive test coverage
- Add comprehensive testing framework with 67 test cases covering unit, integration, and functional testing
- Create universal YAML validator supporting all activity types with validation for metadata operations, terminal steps, and Python syntax
- Implement proper Makefile with venv management and test runners following unDRY principles for copy-paste engineering
- Add requirements-test.txt for test dependencies separation
- Configure pytest with conftest.py for proper environment variable management
- Update CLAUDE.md with Makefile best practices
- All 67 tests passing with proper mocking of external dependencies

Testing coverage includes:
• Unit tests (37): Core app functions, utilities, navigation, response handling
• Integration tests (20): Complete activity workflows and error handling
• Functional tests (9): Full battleship game scenarios and edge cases
• YAML validator (17): Universal validation for all activity configurations
2025-08-10 19:38:47 -04:00