Staff Software Engineer - ML Observability (AI)

Location: Hybrid position based in Boston, Massachusetts, USA, or New York, New York, USA

Salary: $234,000–$300,000 USD per year, plus equity and possible variable compensation

Company

Datadog builds software observability products, including tools for monitoring AI workloads and LLM-powered applications in production.

What you will do

• Design and implement features for the LLM Observability product.
• Prototype and scale capabilities that provide insights into generative AI systems and improve their performance.
• Develop tools for tracing, evaluating, and debugging LLM applications.
• Shape architecture decisions and build resilient, high-performance distributed systems.
• Collaborate with engineering, product, UX, and applied science teams to refine product direction.
• Use customer pain points and industry developments to guide engineering priorities and innovation.

Requirements

• BS, MS, or PhD in Computer Science, Engineering, or a related scientific field, or equivalent experience.
• Deep understanding of distributed systems and scalable backend architectures.
• Hands-on experience building and shipping LLM-powered or generative AI applications.
• Understanding of model internals, inference pipelines, evaluation techniques, and prompt engineering.
• Experience with observability tools or platforms.
• Ability to communicate clearly, work through ambiguity, and write clean, maintainable code.

Culture & Benefits

• Hybrid workplace emphasizing office collaboration and work-life harmony.
• Opportunity to influence product direction and business impact.
• Collaboration with knowledgeable and supportive colleagues.
• Competitive global benefits, varying by country of employment and employment type.
• Healthcare, dental, parental planning, mental health benefits, 401(k) plan and match, paid time off, fitness reimbursements, and discounted employee stock purchase plan.
• Continuous professional development opportunities.