Commit f2a9f00f authored by Kaifmohd's avatar Kaifmohd
Browse files

LLM-ASYST

parents
# Python
__pycache__/
*.py[cod]
*.pyo
*.pyd
*.so
*.egg-info/
.venv/
.env
.env/
.env.*
!.env.example
!**/.env.example
.venv.bak/
.pytest_cache/
.pytest-tmp-*/
.mypy_cache/
.ruff_cache/
.pyre/
# Node/Vite
node_modules/
dist/
.vite/
.npm/
.pnpm-store/
yarn.lock
pnpm-lock.yaml
# Logs and temp
*.log
*.tmp
*.swp
*.bak
*.db
/.tmp-*/
/outputs/
/asyst-backend/docs/uploads/*
!/asyst-backend/docs/uploads/.gitkeep
# OS/editor
.DS_Store
Thumbs.db
.idea/
.vscode/
# Docker buildx cache
.docker-config/
# Local models and artifacts
/models/*
!/models/.gitkeep
# LLM-ASYST
LLM-ASYST is a human-in-the-loop system for grading and evaluating short free-text answers. An LLM proposes scores, while the professor remains responsible for reviewing, accepting, or overriding the results.
The user-facing application is branded **Asyst**.
## Workflows
- **Grade answers:** upload a CSV/XLSX file, configure the grading method and model provider, generate AI scores, review answers by question and predicted-grade category, and inspect student results.
- **Evaluate model:** upload professor-scored answers, run the same AI grading configuration, and compare AI predictions with professor grades using per-label precision, recall, and support. Results can be viewed on the original 0.5-point scale or a rounded whole-number scale.
Both workflows support these grading methods:
- Without reference answer
- Reference answer
- Reference answer with grading rules
- Example-guided grading
## Technology
- React 19, TypeScript, Vite, and Tailwind CSS
- FastAPI, Pydantic, SQLAlchemy, pandas, and SQLite
- OpenAI-compatible access for a local LLM, Academic Cloud Chat AI, or custom OpenAI credentials
- Docker Compose with an optional bundled llama.cpp server
## Repository Structure
```text
LLM-ASYST/
|-- asyst-backend/ FastAPI application, grading services, and tests
|-- asyst-ui/ React frontend and tests
|-- diagrams/ Architecture and workflow diagrams
|-- models/ Local GGUF model directory (model files are ignored)
|-- docker-compose.yml CPU-based local stack
|-- docker-compose.cuda.yml CUDA override for the bundled LLM
`-- docker-compose.host-llm.yml
Override for an LLM server running on the host
```
## Prerequisites
Choose either:
- Docker Desktop with Docker Compose; or
- Python 3.11 or later and Node.js 22 or later for local development.
You also need access to at least one supported LLM provider.
## Run with Docker Compose
### Bundled local LLM on CPU
Place this model file in `models/`:
```text
gemma-4-E4B-it-Q4_K_M.gguf
```
Then start the stack:
```sh
docker compose up --build
```
### Bundled local LLM with CUDA
```sh
docker compose -f docker-compose.yml -f docker-compose.cuda.yml up --build
```
An NVIDIA GPU, compatible drivers, and the NVIDIA Container Toolkit are required.
### LLM server running on the host
```sh
docker compose -f docker-compose.yml -f docker-compose.host-llm.yml up --build
```
The default host endpoint is `http://host.docker.internal:8080/v1`. Override it when necessary:
```powershell
$env:HOST_LLM_BASE_URL = "http://host.docker.internal:8080/v1"
$env:HOST_LLM_MODEL = "your-model-name"
```
After startup:
- Frontend: `http://localhost:5173`
- Backend API: `http://localhost:8000`
- Interactive API documentation: `http://localhost:8000/docs`
Docker stores uploaded files and the SQLite database in the named `asyst-backend-data` volume.
## Run Locally
### Backend
Create `asyst-backend/.env` when local provider configuration is required. This file is ignored by Git.
Example configuration for Academic Cloud Chat AI:
```dotenv
UPLOAD_DIR=uploads
DATABASE_URL=sqlite:///./asyst.db
CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
CHAT_AI_BASE_URL=https://chat-ai.academiccloud.de/v1
CHAT_AI_MODEL=gemma-4-31b-it
CHAT_AI_API_KEY=replace-with-your-key
LLM_MAX_TOKENS=1024
LLM_TEMPERATURE=0
LLM_REQUESTS_PER_MINUTE=10
```
Never commit a real API key.
Windows PowerShell:
```powershell
cd asyst-backend
py -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload
```
macOS/Linux:
```sh
cd asyst-backend
python3 -m venv .venv
./.venv/bin/python -m pip install -r requirements.txt
./.venv/bin/python -m uvicorn app.main:app --reload
```
### Frontend
In a second terminal:
```sh
cd asyst-ui
npm ci
npm run dev
```
The frontend uses `http://localhost:8000` by default. Set `VITE_API_BASE_URL` before starting Vite if the backend uses another address.
## Tests and Build
Backend:
```powershell
cd asyst-backend
.\.venv\Scripts\python.exe -m pytest
```
Frontend:
```sh
cd asyst-ui
npm test
npm run lint
npm run build
```
## Configuration and Secrets
Do not commit API keys, local `.env` files, SQLite databases, uploaded student files, virtual environments, dependency directories, build output, or model weights. The included `.gitignore` excludes these files.
Before the first GitLab commit, verify the repository contents with:
```sh
git status
```
.venv/
venv/
__pycache__/
*.py[cod]
*.log
.env
uploads/
asyst.db
__pycache__/
*.py[cod]
*.log
*.db
.venv/
venv/
.pytest_cache/
.pytest-tmp*/
.env
uploads/
.vscode/
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
"""Asyst backend package."""
from __future__ import annotations
from fastapi import APIRouter
from app.api.routes.evaluation import router as evaluation_router
from app.api.routes.grading import router as grading_router
from app.api.routes.review import router as review_router
from app.api.routes.upload import router as upload_router
api_router = APIRouter()
api_router.include_router(upload_router)
api_router.include_router(grading_router)
api_router.include_router(review_router)
api_router.include_router(evaluation_router)
from __future__ import annotations
from fastapi import APIRouter, HTTPException, status
from app.db import SessionLocal
from app.schemas.label_evaluation import LabelEvaluationResponse
from app.services.label_evaluation_service import LabelEvaluationError, get_label_evaluation
router = APIRouter(tags=["evaluation"])
@router.get("/runs/{run_id}/label-evaluation", response_model=LabelEvaluationResponse)
def run_label_evaluation(run_id: str) -> LabelEvaluationResponse:
db = SessionLocal()
try:
try:
report = get_label_evaluation(db, run_id)
except LabelEvaluationError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return LabelEvaluationResponse(**report)
finally:
db.close()
from __future__ import annotations
from fastapi import APIRouter, Body, HTTPException, Query
from app.schemas.grading import GradingStartRequest
from app.services.grading import get_grade_run_status, start_grade_run
from app.services.llm.profiles import build_runtime_config_for_profile
router = APIRouter(prefix="/grade", tags=["grading"])
@router.post("/{run_id}/start")
def start_grading_for_run(
run_id: str,
limit: int | None = Query(default=None, ge=1),
strategy: str | None = Query(default=None),
payload: GradingStartRequest | None = Body(default=None),
) -> dict[str, str]:
runtime_config = None
if payload is not None:
runtime_config = build_runtime_config_for_profile(
llm_profile=payload.llm_profile,
provider=payload.provider,
model=payload.model,
base_url=payload.base_url,
api_key=payload.api_key.get_secret_value() if payload.api_key else None,
timeout_seconds=payload.timeout_seconds,
retry_max_attempts=payload.retry_max_attempts,
retry_initial_seconds=payload.retry_initial_seconds,
retry_max_seconds=payload.retry_max_seconds,
max_tokens=payload.max_tokens,
temperature=payload.temperature,
)
selected_strategy = payload.strategy if payload and payload.strategy else strategy
try:
return start_grade_run(
run_id,
limit=limit,
strategy=selected_strategy,
runtime_config=runtime_config,
custom_grading_rules=payload.custom_grading_rules if payload else None,
examples=payload.examples if payload else None,
include_feedback=payload.include_feedback if payload else False,
)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
@router.get("/{run_id}/status")
def get_grading_status(run_id: str) -> dict[str, int | str | None]:
return get_grade_run_status(run_id)
from __future__ import annotations
from io import BytesIO
from typing import Literal
from fastapi import APIRouter, HTTPException, status
from fastapi import Query
from fastapi.responses import StreamingResponse
from app.db import SessionLocal
from app.models.grading import Answer, GradingRun, Question, Student
from app.schemas.review import (
AnswerReviewRequest,
AnswerReviewResponse,
QuestionAnswerReviewResponse,
QuestionReviewListResponse,
RunDeleteResponse,
RunHistoryItem,
RunHistoryResponse,
RunReviewSummaryResponse,
StudentAnswerReviewResponse,
StudentResultListResponse,
StudentReviewListResponse,
)
from app.services.export_service import ExportError, build_moodle_export_csv
from app.services.result_service import ResultError, get_student_result_rows
from app.services.run_delete_service import (
RunDeleteConflict,
RunDeleteError,
RunDeleteNotFound,
delete_run,
)
from app.services.review_service import (
ReviewError,
apply_answer_review,
get_question_answers,
get_run_questions,
get_run_review_summary,
get_run_students,
get_student_answers,
)
router = APIRouter(tags=["review"])
@router.post("/answers/{answer_id}/review", response_model=AnswerReviewResponse)
def review_answer(answer_id: int, payload: AnswerReviewRequest) -> AnswerReviewResponse:
db = SessionLocal()
try:
try:
answer = apply_answer_review(db, answer_id=answer_id, payload=payload)
except ReviewError as exc:
message = str(exc)
status_code = status.HTTP_404_NOT_FOUND if message == "Answer not found" else status.HTTP_422_UNPROCESSABLE_ENTITY
raise HTTPException(status_code=status_code, detail=message) from exc
return AnswerReviewResponse(
answer_id=answer.id,
run_id=answer.run_id,
student_id=answer.student_id,
question_id=answer.question_id,
processing_status=answer.processing_status,
review_status=answer.review_status,
review_required=answer.review_required,
llm_score=answer.llm_score,
human_score=answer.human_score,
final_score=answer.final_score,
accepted_llm=answer.accepted_llm,
review_note=answer.review_note,
reviewed_by=answer.reviewed_by,
reviewed_at=answer.reviewed_at,
graded_at=answer.graded_at,
)
finally:
db.close()
@router.get("/runs", response_model=RunHistoryResponse)
def list_grading_runs(
run_type: Literal["grading", "evaluation"] | None = Query(default=None),
) -> RunHistoryResponse:
db = SessionLocal()
try:
query = db.query(GradingRun)
if run_type is not None:
query = query.filter(GradingRun.run_type == run_type)
runs = query.order_by(GradingRun.created_at.desc(), GradingRun.id.desc()).all()
items = [
RunHistoryItem(
run_id=run.run_id,
filename=run.filename,
run_type=run.run_type,
status=run.status,
created_at=run.created_at,
total_students=db.query(Student).filter(Student.run_id == run.run_id).count(),
total_questions=db.query(Question).filter(Question.run_id == run.run_id).count(),
total_answers=db.query(Answer).filter(Answer.run_id == run.run_id).count(),
)
for run in runs
]
return RunHistoryResponse(items=items)
finally:
db.close()
@router.get("/runs/{run_id}/review-summary", response_model=RunReviewSummaryResponse)
def run_review_summary(run_id: str) -> RunReviewSummaryResponse:
db = SessionLocal()
try:
try:
summary = get_run_review_summary(db, run_id)
except ReviewError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return RunReviewSummaryResponse(**summary)
finally:
db.close()
@router.get("/runs/{run_id}/students", response_model=StudentReviewListResponse)
def list_run_students(
run_id: str,
review_status: Literal["pending", "accepted", "overridden"] | None = Query(default=None),
search: str | None = Query(default=None),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=50, ge=1, le=500),
) -> StudentReviewListResponse:
db = SessionLocal()
try:
try:
data = get_run_students(
db,
run_id,
review_status=review_status,
search=search,
page=page,
page_size=page_size,
)
except ReviewError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return StudentReviewListResponse(**data)
finally:
db.close()
@router.get("/runs/{run_id}/students/{student_id}/answers", response_model=StudentAnswerReviewResponse)
def get_student_answer_view(run_id: str, student_id: int) -> StudentAnswerReviewResponse:
db = SessionLocal()
try:
try:
data = get_student_answers(db, run_id=run_id, student_id=student_id)
except ReviewError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return StudentAnswerReviewResponse(**data)
finally:
db.close()
@router.get("/runs/{run_id}/questions", response_model=QuestionReviewListResponse)
def list_run_questions(run_id: str) -> QuestionReviewListResponse:
db = SessionLocal()
try:
try:
data = get_run_questions(db, run_id=run_id)
except ReviewError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return QuestionReviewListResponse(**data)
finally:
db.close()
@router.get("/runs/{run_id}/questions/{question_id}/answers", response_model=QuestionAnswerReviewResponse)
def get_question_answer_view(
run_id: str,
question_id: int,
review_status: Literal["pending", "accepted", "overridden"] | None = Query(default=None),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=50, ge=1, le=500),
sort: Literal["student_name", "llm_score", "final_score", "review_status"] | None = Query(default=None),
predicted_score: list[float] | None = Query(default=None),
score_scale: Literal["full", "whole"] = Query(default="full"),
) -> QuestionAnswerReviewResponse:
db = SessionLocal()
try:
try:
data = get_question_answers(
db,
run_id=run_id,
question_id=question_id,
review_status=review_status,
page=page,
page_size=page_size,
sort=sort,
predicted_scores=predicted_score,
score_scale=score_scale,
)
except ReviewError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return QuestionAnswerReviewResponse(**data)
finally:
db.close()
@router.get("/runs/{run_id}/results/students", response_model=StudentResultListResponse)
def get_student_results(run_id: str) -> StudentResultListResponse:
db = SessionLocal()
try:
try:
items = get_student_result_rows(db, run_id=run_id)
except ResultError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
return StudentResultListResponse(run_id=run_id, items=items)
finally:
db.close()
@router.get("/runs/{run_id}/export/moodle")
def export_moodle(run_id: str) -> StreamingResponse:
db = SessionLocal()
try:
try:
csv_text = build_moodle_export_csv(db, run_id=run_id)
except ExportError as exc:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from exc
payload = BytesIO(csv_text.encode("utf-8"))
headers = {"Content-Disposition": f'attachment; filename="moodle_export_{run_id}.csv"'}
return StreamingResponse(payload, media_type="text/csv", headers=headers)
finally:
db.close()
@router.delete("/runs/{run_id}", response_model=RunDeleteResponse)
def delete_grading_run(run_id: str) -> RunDeleteResponse:
db = SessionLocal()
try:
try:
summary = delete_run(db, run_id=run_id)
except RunDeleteNotFound as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
except RunDeleteConflict as exc:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from exc
except RunDeleteError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
return RunDeleteResponse(**summary.__dict__)
finally:
db.close()
from __future__ import annotations
from typing import Literal
from fastapi import APIRouter, File, HTTPException, Query, UploadFile, status
from app.schemas.upload import UploadResponse
from app.services.file_storage import FileStorageService
from app.services.parser import ParsingError, process_uploaded_file
from app.utils.file_utils import extract_extension
router = APIRouter()
_ALLOWED_EXTENSIONS = {"csv", "xlsx"}
@router.post("/upload", response_model=UploadResponse)
async def upload_file(
file: UploadFile = File(...),
run_type: Literal["grading", "evaluation"] = Query(default="grading"),
) -> UploadResponse:
if not file.filename:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No file name provided.",
)
extension = extract_extension(file.filename)
if extension not in _ALLOWED_EXTENSIONS:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Unsupported file type. Only .csv and .xlsx are allowed.",
)
try:
content = await file.read()
except Exception as exc: # pragma: no cover
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Could not read uploaded file.",
) from exc
if not content:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Uploaded file is empty.",
)
metadata = FileStorageService.store_temp_file(
original_filename=file.filename,
content=content,
file_extension=extension,
)
try:
process_uploaded_file(
upload_id=metadata["upload_id"],
original_filename=metadata["original_filename"],
relative_path=metadata["relative_path"],
file_type=metadata["file_type"],
run_type=run_type,
)
except ParsingError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"File uploaded but parsing failed: {exc}",
) from exc
return UploadResponse(**metadata)
"""Core configuration package."""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
from dotenv import load_dotenv
_BACKEND_ROOT = Path(__file__).resolve().parents[2]
load_dotenv(_BACKEND_ROOT / ".env")
def _parse_cors_origins(value: str | None) -> list[str]:
raw = value or "http://localhost:5173,http://127.0.0.1:5173"
return [origin.strip() for origin in raw.split(",") if origin.strip()]
@dataclass
class Settings:
upload_dir: Path
cors_origins: list[str]
database_url: str
llm_provider: str
llm_api_key: str | None
llm_base_url: str
llm_model: str
llm_timeout_seconds: float
llm_retry_max_attempts: int
llm_retry_initial_seconds: float
llm_retry_max_seconds: float
llm_max_tokens: int
llm_temperature: float
llm_requests_per_minute: int
llm_prompt_strategy: str
def get_settings() -> Settings:
upload_dir = Path(os.getenv("UPLOAD_DIR", "uploads"))
cors_origins = _parse_cors_origins(os.getenv("CORS_ORIGINS"))
database_url = os.getenv("DATABASE_URL", "sqlite:///./asyst.db")
llm_provider = os.getenv("LLM_PROVIDER", "openai_compatible").strip().lower()
llm_api_key = os.getenv("LLM_API_KEY") or os.getenv("API_KEY")
llm_base_url = os.getenv("LLM_BASE_URL", "https://chat-ai.academiccloud.de/v1")
llm_model = os.getenv("LLM_MODEL", "gemma-3-27b-it")
llm_timeout_seconds = float(os.getenv("LLM_TIMEOUT_SECONDS", "30"))
llm_retry_max_attempts = int(os.getenv("LLM_RETRY_MAX_ATTEMPTS", "3"))
llm_retry_initial_seconds = float(os.getenv("LLM_RETRY_INITIAL_SECONDS", "1"))
llm_retry_max_seconds = float(os.getenv("LLM_RETRY_MAX_SECONDS", "10"))
llm_max_tokens = int(os.getenv("LLM_MAX_TOKENS", "120"))
llm_temperature = float(os.getenv("LLM_TEMPERATURE", "0"))
llm_requests_per_minute = int(os.getenv("LLM_REQUESTS_PER_MINUTE", "10"))
llm_prompt_strategy = os.getenv("LLM_PROMPT_STRATEGY", "p1_reference_zero_shot").strip().lower()
return Settings(
upload_dir=upload_dir,
cors_origins=cors_origins,
database_url=database_url,
llm_provider=llm_provider,
llm_api_key=llm_api_key,
llm_base_url=llm_base_url,
llm_model=llm_model,
llm_timeout_seconds=llm_timeout_seconds,
llm_retry_max_attempts=llm_retry_max_attempts,
llm_retry_initial_seconds=llm_retry_initial_seconds,
llm_retry_max_seconds=llm_retry_max_seconds,
llm_max_tokens=llm_max_tokens,
llm_temperature=llm_temperature,
llm_requests_per_minute=llm_requests_per_minute,
llm_prompt_strategy=llm_prompt_strategy,
)
settings = get_settings()
from __future__ import annotations
from sqlalchemy import create_engine
from sqlalchemy import inspect, text
from sqlalchemy.orm import sessionmaker
from app.core.config import settings
from app.models.base import Base
def _sqlite_connect_args(database_url: str) -> dict[str, bool]:
if database_url.startswith("sqlite"):
return {"check_same_thread": False}
return {}
engine = create_engine(
settings.database_url,
future=True,
connect_args=_sqlite_connect_args(settings.database_url),
)
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, expire_on_commit=False)
def init_db() -> None:
import app.models.grading # noqa: F401
Base.metadata.create_all(bind=engine)
_ensure_sqlite_compat_columns()
def _ensure_sqlite_compat_columns() -> None:
if not settings.database_url.startswith("sqlite"):
return
additions = {
"grading_runs": {
"run_type": "VARCHAR(32) NOT NULL DEFAULT 'grading'",
},
"answers": {
"llm_raw_response": "TEXT",
"llm_parse_warning": "TEXT",
"observed_score": "FLOAT",
},
}
with engine.begin() as conn:
inspector = inspect(conn)
for table_name, columns in additions.items():
if not inspector.has_table(table_name):
continue
existing = {column["name"] for column in inspector.get_columns(table_name)}
for column_name, ddl in columns.items():
if column_name not in existing:
conn.execute(text(f"ALTER TABLE {table_name} ADD COLUMN {column_name} {ddl}"))
from __future__ import annotations
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.api.router import api_router
from app.core.config import settings
from app.db import init_db
app = FastAPI(title="LLM-ASYST Backend", version="0.1.0")
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(api_router)
@app.on_event("startup")
def startup() -> None:
init_db()
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok"}
from app.models.base import Base
from app.models.grading import Answer, GradingRun, Question, Student
__all__ = ["Base", "GradingRun", "Student", "Question", "Answer"]
from __future__ import annotations
from sqlalchemy.orm import DeclarativeBase
class Base(DeclarativeBase):
pass
from __future__ import annotations
from datetime import datetime, timezone
from sqlalchemy import DateTime, Float, ForeignKey, Index, Integer, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
def utc_now() -> datetime:
return datetime.now(timezone.utc)
class GradingRun(Base):
__tablename__ = "grading_runs"
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
run_id: Mapped[str] = mapped_column(String(64), unique=True, index=True, nullable=False)
filename: Mapped[str] = mapped_column(String(255), nullable=False)
file_path: Mapped[str] = mapped_column(String(512), nullable=False)
file_type: Mapped[str] = mapped_column(String(16), nullable=False)
run_type: Mapped[str] = mapped_column(String(32), nullable=False, default="grading")
status: Mapped[str] = mapped_column(String(32), nullable=False, default="uploaded")
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, nullable=False)
updated_at: Mapped[datetime] = mapped_column(
DateTime,
default=utc_now,
onupdate=utc_now,
nullable=False,
)
class Student(Base):
__tablename__ = "students"
__table_args__ = (
Index("ix_students_run_name", "run_id", "last_name", "first_name"),
Index("ix_students_run_email", "run_id", "email"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
run_id: Mapped[str] = mapped_column(ForeignKey("grading_runs.run_id"), index=True, nullable=False)
student_code: Mapped[str] = mapped_column(String(64), nullable=False)
first_name: Mapped[str] = mapped_column(String(128), nullable=False)
last_name: Mapped[str] = mapped_column(String(128), nullable=False)
email: Mapped[str] = mapped_column(String(255), nullable=False)
status: Mapped[str | None] = mapped_column(String(64), nullable=True)
row_index: Mapped[int] = mapped_column(Integer, nullable=False)
class Question(Base):
__tablename__ = "questions"
__table_args__ = (
UniqueConstraint("run_id", "question_number", name="uq_questions_run_qnum"),
Index("ix_questions_run_number", "run_id", "question_number"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
run_id: Mapped[str] = mapped_column(ForeignKey("grading_runs.run_id"), index=True, nullable=False)
question_number: Mapped[int] = mapped_column(Integer, nullable=False)
question_text: Mapped[str] = mapped_column(Text, nullable=False)
reference_answer: Mapped[str | None] = mapped_column(Text, nullable=True)
max_points: Mapped[float | None] = mapped_column(Float, nullable=True)
class Answer(Base):
__tablename__ = "answers"
__table_args__ = (
Index("ix_answers_run_review_status", "run_id", "review_status"),
Index("ix_answers_run_processing_status", "run_id", "processing_status"),
Index("ix_answers_run_student", "run_id", "student_id"),
Index("ix_answers_run_question", "run_id", "question_id"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True)
run_id: Mapped[str] = mapped_column(ForeignKey("grading_runs.run_id"), index=True, nullable=False)
student_id: Mapped[int] = mapped_column(ForeignKey("students.id"), index=True, nullable=False)
question_id: Mapped[int] = mapped_column(ForeignKey("questions.id"), index=True, nullable=False)
answer_text: Mapped[str] = mapped_column(Text, nullable=False)
processing_status: Mapped[str] = mapped_column(String(32), nullable=False, default="pending")
review_status: Mapped[str] = mapped_column(String(32), nullable=False, default="pending")
llm_score: Mapped[float | None] = mapped_column(Float, nullable=True)
llm_feedback: Mapped[str | None] = mapped_column(Text, nullable=True)
llm_raw_response: Mapped[str | None] = mapped_column(Text, nullable=True)
llm_parse_warning: Mapped[str | None] = mapped_column(Text, nullable=True)
observed_score: Mapped[float | None] = mapped_column(Float, nullable=True)
human_score: Mapped[float | None] = mapped_column(Float, nullable=True)
final_score: Mapped[float | None] = mapped_column(Float, nullable=True)
accepted_llm: Mapped[bool] = mapped_column(nullable=False, default=False)
reviewed_by: Mapped[str | None] = mapped_column(String(255), nullable=True)
review_note: Mapped[str | None] = mapped_column(Text, nullable=True)
attempt_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
graded_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
review_required: Mapped[bool] = mapped_column(nullable=False, default=False)
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, nullable=False)
updated_at: Mapped[datetime] = mapped_column(
DateTime,
default=utc_now,
onupdate=utc_now,
nullable=False,
)
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment