9 Commits

Author SHA1 Message Date
5f554ea769 refactor: environment variable configuration for all pipeline settings
- Add config.py with dataclass-based configuration from env vars
- Remove hardcoded RunPod endpoint and credentials
- Consolidate duplicate training components into single reusable function
- Add .env.example with all configurable options
- Update README with environment variable documentation
- Add Kubernetes secrets example for production deployments
- Add timeout and error handling improvements

BREAKING: Pipeline parameters now use env vars by default.
Set RUNPOD_API_KEY, RUNPOD_ENDPOINT, S3_BUCKET, and AWS creds.
2026-02-03 20:47:27 +00:00
0bf3837e78 feat: add ADE, Triage, and Symptom-Disease training pipelines
New tasks supported:
- task=ade: Adverse Drug Event classification (ADE Corpus V2, 30K samples)
- task=triage: Medical Triage classification (urgency levels)
- task=symptom_disease: Symptom-to-Disease prediction (40+ diseases)

All use HuggingFace datasets, Bio_ClinicalBERT, and S3 model storage.
2026-02-03 16:20:55 +00:00
2e479fc61b chore: clean up repo structure
- Remove compiled YAML files (can be regenerated)
- Remove example pipelines
- Remove unused med_rx_training.py
- Update README with comprehensive docs
- Clean up .gitignore
2026-02-03 16:11:40 +00:00
b086239c52 Fix Python/transformers version compatibility 2026-02-03 01:11:20 +00:00
297a75da4d Remove internal domains from README 2026-02-03 00:45:27 +00:00
222be0fb68 Use Tailscale endpoints, add RunPod Docker build files 2026-02-03 00:23:16 +00:00
07bb8aa6bb Fix MinIO endpoint to use internal cluster service 2026-02-03 00:15:26 +00:00
9ca3d6c195 Add DDI training pipeline with RunPod serverless GPU support 2026-02-02 23:56:05 +00:00
591a312399 Initial Kubeflow GitOps setup with example pipelines 2026-02-02 23:37:53 +00:00