Senior ML Engineer (LLM/VLM)

Senior ML Engineer (LLM/VLM)

Position summary

Senior ML Engineer owns the in-house LLM/VLM lifecycle: datasets, weight training, evaluation, and self-hosted production. The role is weighted toward fine-tuning and serving on our own infrastructure, not prompt assembly, RAG integration, or third-party LLM APIs.

Personal and sensitive data are processed only on-premises. External cloud LLM APIs are not used for workloads that include such data.

You are expected to run the loop end to end, decide what to retrain from production failures or eval gaps, and keep datasets, eval configs, and release criteria documented so the pipeline is not tied to one engineer.

Key responsibilities

Model training and fine-tuning

• Fine-tune LLM and VLM models for product use cases using LoRA/QLoRA, distillation, and related methods through production-ready weights
• Adapt models to domain-specific tasks without relying on prompt-only or RAG-only approaches as the primary delivery path
• Maintain reproducible training configs and document model versions bound to releases

Evaluation and data

• Build evaluation as a release gate: field-level metrics (exact-match, precision, recall, F1, CER, WER), held-out sets, and train/test leakage checks before rollout
• Own data workflows: collection, teacher-labeling, versioned reproducible datasets, and synthetic data generation with validation
• Define acceptance criteria tied to eval results for production-bound model versions

Production inference and operations

• Deploy and operate self-hosted inference under load: vLLM, batching, VRAM management, quantization, structured and guided JSON outputs
• Monitor production model behavior and initiate retraining from eval failures or production error patterns
• Run containerized deployments (Docker, Kubernetes), CI/CD for model releases, and quality logging for model outputs
• Keep personal and sensitive data within the in-house environment

Required qualifications

• 5+ years of experience delivering ML systems to production, including high-load AI systems
• Hands-on fine-tuning of LLM or VLM models: you train weights (LoRA/QLoRA, distillation), not only prompt design or RAG wiring
• Evaluation practice as a release discipline: held-out metrics by field, leakage detection, and clear pass/fail gates before production
• End-to-end data ownership: teacher-labeling, reproducible versioned datasets, validated synthetic data
• Production experience with self-hosted LLM serving (vLLM or equivalent): batching, quantization, structured outputs under load
• Ownership of at least one full cycle to production traffic: data preparation, fine-tuning, evaluation, deployment, and iteration from prod or eval signals
• Strong Python and production backend skills (e.g. FastAPI); PostgreSQL, Redis where applicable
• Docker, Kubernetes, and CI/CD for model and service releases
• In-house / self-hosted ML operation; no sending personal or sensitive data to third-party LLM APIs

Preferred qualifications

• OCR on multilingual documents: text and structure extraction from mixed languages within a single file
• Experience with Hugging Face Transformers, PEFT, and training tooling in production contexts
• MLOps practice: output quality monitoring, drift signals, model versioning
• STEM degree

Technology stack

• Training: PyTorch, Hugging Face Transformers, PEFT (LoRA/QLoRA), distillation workflows
• Serving: vLLM, quantization, guided/structured JSON generation
• Data and eval: held-out splits, field-level metrics, dataset versioning, teacher-labeling pipelines
• Backend: Python, FastAPI; PostgreSQL, Redis
• Infra: Docker, Kubernetes; CI/CD (GitHub Actions, GitLab CI, or similar)
• Observability: logging, model output quality monitoring (Prometheus/Grafana or equivalent)
• Testing: pytest, Pydantic