Document classifier_model and feedback_model in CLAUDE.md
Added comprehensive Activity YAML Schema section covering: - Model Configuration feature (classifier_model and feedback_model) - Why separate models (speed, quality, cost, flexibility) - Model defaults (MODEL_1/Hermes as universal default) - Recommended model combinations table - Environment variable configuration - Example programming activity with dual models - Activity YAML validation instructions - CLI testing with model configuration - Qwen3-Coder-30B setup guide (llama.cpp and ollama) This documents the new dual-model architecture that allows: - Fast classification with Hermes (8B) - Specialized feedback with domain models (e.g., Qwen3-Coder 30B) - Activity and step-level model overrides
This commit is contained in:
parent
1c5a4960fd
commit
82aeeab094
1 changed files with 139 additions and 0 deletions
139
CLAUDE.md
139
CLAUDE.md
|
|
@ -75,3 +75,142 @@
|
||||||
- `/run` - Auto-detect language and execute
|
- `/run` - Auto-detect language and execute
|
||||||
- Add play button next to copy button for code blocks
|
- Add play button next to copy button for code blocks
|
||||||
- Display execution results inline below code blocks
|
- Display execution results inline below code blocks
|
||||||
|
|
||||||
|
## Activity YAML Schema
|
||||||
|
|
||||||
|
### Model Configuration (New Feature)
|
||||||
|
|
||||||
|
Activities can specify separate models for classification and feedback generation:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Activity-level defaults (optional)
|
||||||
|
classifier_model: "MODEL_1" # For categorizing user responses into buckets
|
||||||
|
feedback_model: "MODEL_1" # For generating AI feedback and translations
|
||||||
|
|
||||||
|
# Step-level overrides (optional)
|
||||||
|
sections:
|
||||||
|
- section_id: "coding"
|
||||||
|
steps:
|
||||||
|
- step_id: "code_review"
|
||||||
|
classifier_model: "MODEL_1" # Keep fast classification
|
||||||
|
feedback_model: "MODEL_3" # Use specialized code model
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why Separate Models?**
|
||||||
|
|
||||||
|
1. **Speed**: Use fast 8B models for classification → instant bucketing
|
||||||
|
2. **Quality**: Use specialized models for feedback → better explanations
|
||||||
|
3. **Cost Efficiency**: Don't waste tokens on simple categorization
|
||||||
|
4. **Flexibility**: Override per-step for specific needs
|
||||||
|
|
||||||
|
**Model Defaults**
|
||||||
|
|
||||||
|
If not specified, both default to `MODEL_1` (Hermes-3-Llama-3.1-8B):
|
||||||
|
- Always available in base install
|
||||||
|
- Fast and accurate
|
||||||
|
- Excellent for role-playing and general tasks
|
||||||
|
- Great classifier and feedback generator
|
||||||
|
|
||||||
|
**Recommended Model Combinations**
|
||||||
|
|
||||||
|
| Activity Type | Classifier | Feedback | Rationale |
|
||||||
|
|--------------|------------|----------|-----------|
|
||||||
|
| General Education | MODEL_1 | MODEL_1 | Fast, accurate, always available |
|
||||||
|
| Programming | MODEL_1 | MODEL_3 | Fast bucketing + code specialist (Qwen3-Coder) |
|
||||||
|
| Role-Playing | MODEL_1 | MODEL_1 | Hermes excels at character consistency |
|
||||||
|
| Advanced Topics | MODEL_1 | MODEL_2 | Fast bucketing + larger model for depth |
|
||||||
|
|
||||||
|
**Environment Variables**
|
||||||
|
|
||||||
|
Models are configured via environment variables in `vars.sh`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# MODEL_1 - Hermes (always available, default)
|
||||||
|
export MODEL_ENDPOINT_1=http://localhost:8080/v1
|
||||||
|
export MODEL_API_KEY_1=your-api-key
|
||||||
|
|
||||||
|
# MODEL_2 - Additional model (optional)
|
||||||
|
export MODEL_ENDPOINT_2=http://localhost:8081/v1
|
||||||
|
export MODEL_API_KEY_2=your-api-key
|
||||||
|
|
||||||
|
# MODEL_3 - Qwen3-Coder (recommended for programming)
|
||||||
|
export MODEL_ENDPOINT_3=http://localhost:8082/v1
|
||||||
|
export MODEL_API_KEY_3=your-api-key
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example: Programming Activity**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# research/activity37-programming-languages.yaml
|
||||||
|
classifier_model: "MODEL_1" # Hermes for fast classification
|
||||||
|
feedback_model: "MODEL_3" # Qwen3-Coder-30B for code generation
|
||||||
|
|
||||||
|
sections:
|
||||||
|
- section_id: "hello_world"
|
||||||
|
steps:
|
||||||
|
- step_id: "write_hello"
|
||||||
|
question: "Write a Hello World program in your chosen language"
|
||||||
|
tokens_for_ai: |
|
||||||
|
Get the student's chosen language from metadata (programming_language).
|
||||||
|
Evaluate their code in THAT specific language.
|
||||||
|
feedback_tokens_for_ai: |
|
||||||
|
Provide detailed feedback on their code syntax and style.
|
||||||
|
Generate example code if they need help.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Activity YAML Validation
|
||||||
|
|
||||||
|
**Validator Location**: `activity_yaml_validator.py`
|
||||||
|
|
||||||
|
**Validate Activities**:
|
||||||
|
```bash
|
||||||
|
python activity_yaml_validator.py research/activity*.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
**Model Field Validation**:
|
||||||
|
- `classifier_model` (optional, string): Activity or step-level
|
||||||
|
- `feedback_model` (optional, string): Activity or step-level
|
||||||
|
- Both default to "MODEL_1" if not specified
|
||||||
|
- Can reference MODEL_1, MODEL_2, MODEL_3, etc.
|
||||||
|
|
||||||
|
**Testing Activities**
|
||||||
|
|
||||||
|
CLI simulation tool supports model configuration:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source vars.sh
|
||||||
|
python research/guarded_ai.py research/activity37-programming-languages.yaml
|
||||||
|
# Uses MODEL_1 for classification, MODEL_3 for code feedback
|
||||||
|
```
|
||||||
|
|
||||||
|
### Model Setup: Qwen3-Coder-30B (MODEL_3)
|
||||||
|
|
||||||
|
**Why Qwen3-Coder?**
|
||||||
|
- 30B parameters (much smarter for code)
|
||||||
|
- Trained on 100+ programming languages
|
||||||
|
- Q4_K_M quantization (~20GB RAM)
|
||||||
|
- Perfect for activity37 (universal programming activity)
|
||||||
|
|
||||||
|
**Setup with llama.cpp**:
|
||||||
|
```bash
|
||||||
|
# Download
|
||||||
|
huggingface-cli download unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF \
|
||||||
|
Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
|
||||||
|
|
||||||
|
# Run server (GPU acceleration)
|
||||||
|
llama-server -m Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf \
|
||||||
|
--host 0.0.0.0 --port 8082 -ngl 99
|
||||||
|
|
||||||
|
# Configure in vars.sh
|
||||||
|
export MODEL_ENDPOINT_3=http://localhost:8082/v1
|
||||||
|
export MODEL_API_KEY_3=dummy
|
||||||
|
```
|
||||||
|
|
||||||
|
**Setup with ollama**:
|
||||||
|
```bash
|
||||||
|
ollama run unsloth/qwen3-coder:30b-instruct-q4_K_M
|
||||||
|
|
||||||
|
# Configure in vars.sh
|
||||||
|
export MODEL_ENDPOINT_3=http://localhost:11434/v1
|
||||||
|
export MODEL_API_KEY_3=dummy
|
||||||
|
```
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue