Senior AI Reliability Engineer (Platform) (AI)

TL;DR

Senior AI Reliability Engineer (Platform) (AI): Designing and scaling reliability frameworks and evaluation pipelines for AI-enabled workflows with an accent on LLMs, RAG, and multi-agent systems. Focus on building observability, guardrails, and automated testing to ensure the safety and production-readiness of AI systems.

Location: Berlin Office

Company

Healthtech company transforming cancer patient data into real-world evidence to improve oncology care worldwide.

What you will do

• Design and improve evaluation frameworks and automated testing pipelines for LLM and agentic workflows.
• Define and monitor quality, safety, and performance metrics, including drift detection and hallucination risk.
• Develop SRE practices (SLOs, SLIs, alerting, and incident response) specifically for AI failure modes.
• Design orchestration and governance for multi-agent systems, including auditability and human oversight.
• Collaborate with global teams to establish AI standards and guide model selection decisions.
• Participate in on-call rotations to support production AI systems.

Requirements

• 5+ years of experience in platform engineering, SRE, or MLOps.
• Strong proficiency in Python and experience building production-quality systems.
• Experience with LLMs, RAG, AI agents, and prompt evaluation.
• Proficiency with AWS, Kubernetes, CI/CD, and workflow orchestration.
• Knowledge of agentic systems and state management.
• English: Fluent (C1) required

Nice to have

• Experience with red-teaming, AI safety, or adversarial testing.
• Hands-on experience with Datadog, Splunk, OpenTelemetry, Databricks, Spark, or Airflow.
• Background in healthcare or other regulated, privacy-sensitive environments.

Culture & Benefits

• Culture centered on empowerment, compassion, and clarity.
• Opportunity to work in a multidisciplinary team of oncologists, data scientists, and engineers.
• Mission-driven work focused on improving cancer care.
• Collaborative global environment.