Member Of Technical Staff, Evals & Post-Training Product (AI)

TL;DR

Member of Technical Staff, Evals & Post-Training Product (AI): Building products and workflows that connect evaluation and post-training into a continuous loop with an accent on product engineering, developer experience, and model quality. Focus on enabling external developers through our open-source Eval Protocol SDK and owning key product experiences for fine-tuning custom models.

Location: San Mateo, CA

Company

Fireworks is building the future of generative AI infrastructure, delivering high-quality models with fast and scalable inference.

What you will do

• Build internal eval workflows to measure model quality and inform post-training decisions.
• Own user-facing product workflows for post-training, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
• Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs.
• Support high-priority engagements, triage issues, and convert bespoke workflows into productized solutions.

Requirements

• 1 - 7 years of software engineering experience.
• Hands-on experience with LLM evaluations and/or post-training methods.
• Product Engineering Skills: The ability to work across backend systems and developer-facing product surfaces.
• Understanding of the GenAI Lifecycle.
• User-Centric Mindset: Willing to talk to users, triage GitHub issues for open-source projects, and build products from scratch to serve emerging needs.

Nice to have

• 3+ years of software engineering experience.
• Strong familiarity with designing and running evaluations for domain-specific use cases.
• Prior contributions to developer tools or AI/ML repositories.
• Interest in the hardware side of AI—understanding GPU constraints, inference optimization techniques, and how they relate to model performance.
• Experience in fast-paced environments where you own features end-to-end.

Culture & Benefits

• Solve hard problems at the forefront of AI infrastructure.
• Work with bleeding-edge technology that impacts how businesses and developers harness AI globally.
• Join a fast-growing, passionate team where your work directly shapes the future of AI.
• Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation.