5 дней назад
Member of Technical Staff, LLM & Production (AI)
180 000 - 450 000$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Member of Technical Staff, LLM & Production (AI) (LLM evaluation and training infrastructure): Building production-grade evaluation suites, data pipelines, and reinforcement-learning environments for real-time multimodal intelligence with an accent on reproducibility, distributed execution, and model quality measurement. Focus on designing agentic evaluation systems, scaling training environments across workers, detecting data contamination and metric regressions, and moving product capabilities into native model behavior.
Location: San Jose, United States
US base salary: $180,000–$450,000 annually
Company
is an artificial intelligence company developing personalized, proactive multimodal intelligence and next-generation hardware interfaces for interacting with the world through speech, text, vision, and persistent memory.
What you will do
- Build and own end-to-end evaluation suites for LLM capabilities, including task specifications, grading logic, and researcher dashboards.
- Operate evaluation harnesses as production services with sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching.
- Own data pipelines from collection and filtering through deduplication, labeling, synthetic data generation, and quality gates.
- Build reinforcement-learning environments, including reward functions, simulators, verifiers, and task generators, and scale them across workers.
- Work with researchers to turn hypotheses into measurable evaluations and environments, then interpret the results.
- Improve research engineering standards through testing, versioned datasets and evaluations, and clear boundaries between training, inference, and evaluation.
Requirements
- 3+ years of software engineering experience, including 1–2 years of hands-on experience with LLM training, evaluation, or data infrastructure.
- Deep Python fluency and practical distributed-systems experience.
- Experience building evaluation infrastructure, including tasks, graders, and supporting systems.
- Working knowledge of supervised fine-tuning and reinforcement learning for language models.
- Production engineering practices including testing, reproducibility, and dependable research tooling.
- Ability to work with ambiguous research ideas and make them measurable.
Nice to have
- Experience with agentic evaluations, tool use, code execution, sandbox environments, and multi-turn tasks.
- Experience with PyTorch, JAX, vLLM, SGLang, or equivalent training and inference stacks.
- Experience running data-labeling operations or collecting human preferences at scale.
- Open-source contributions to evaluation frameworks or public benchmarks.
- Experience with verifiers, code sandboxes, or simulation environments.
Culture & Benefits
- Hands-on individual-contributor role at the intersection of model research and production systems.
- Close collaboration with researchers on multimodal intelligence and AI hardware.
- Full-time employment with a US base salary range of $180,000–$450,000 annually.
- Total compensation may include additional components and benefits depending on the role.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
10 дней назад
Principal Machine Learning Engineer, Applied AI
252 000 - 336 000$
11 дней назад
Software Engineer (AI)
230 000 - 390 000$
12 дней назад
Technical Lead, Early Stage Supply Chain Project (AI)
262 000 - 369 000$
Cohere
7 дней назад
Member Of Technical Staff (AI)
295 000 - 535 000CAD
Cohere
7 дней назад
Member Of Technical Staff (AI)
295 000 - 535 000CAD
11 дней назад
AI Applications Engineer
100 000 - 175 000$