Python · PyTorch · Hugging Face Transformers · Weights & Biases
SEEK
Ensemble knowledge distillation for domain-specific QA.
Summary
Stacked Ensemble of Expert Models — an NLP pipeline that fine-tunes specialist models on domain-specific datasets and uses a learned routing mechanism to direct queries to the right expert.
Role
Team member (3-person project).
Highlights
- —Fine-tuned domain-specific expert models on ARC, SciQ, MedMCQA, and AquaRat benchmarks.
- —Built a query router that maps inputs to the most relevant specialist using learned routing maps.
- —Applied knowledge distillation across the ensemble to compress multi-domain QA performance.