Associate Director, Evaluation and Quality Standards (Generative AI)

Location: Cornellà de Llobregat, Barcelona, Spain; Madrid, Spain

Base pay: €75,000–€129,260 annually, plus an annual bonus or sales commissions depending on the role and location.

Company

Johnson & Johnson MedTech develops healthcare and medical technology solutions across prevention, treatment, and care.

What you will do

• Design and maintain evaluation frameworks for generative AI solutions covering LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.
• Develop therapeutic-area-specific criteria and benchmarks for scientific validity, regulatory compliance, data quality, and operational reliability.
• Establish validation datasets, automated evaluation pipelines, evaluation gates, and production-readiness standards across GenAI development stages.
• Lead independent vendor and solution assessments, including benchmarking, integration complexity, total cost of ownership, regulatory posture, and vendor lock-in risk.
• Present findings to governance and portfolio committees and collaborate with QMS/MLOps, therapeutic area teams, Data Strategy, and architecture teams.
• Build and lead the GenAI Evaluation & Standards team and develop internal capability through training, documentation, and evaluation tooling.

Requirements

• Advanced degree, preferably a PhD, in computational biology, bioinformatics, data science, computer science, AI/ML, biomedical engineering, applied mathematics, or a related discipline.
• At least 8 years of post-academic industry experience, including significant experience in pharmaceutical or biotechnology R&D.
• Hands-on expertise with generative AI systems, including large language models, retrieval-augmented generation, agentic frameworks, and prompt engineering.
• Demonstrated experience designing evaluation frameworks, scientific benchmarks, or quality standards for AI/ML systems.
• Strong people leadership experience, including building and managing technical teams in a matrixed organization.
• Excellent communication, strategic thinking, data science, data quality, compliance, and critical-thinking skills.

Culture & Benefits

• Inclusive work environment based on respect for diversity, dignity, and individual contribution.
• Vacation days, parental leave of at least 12 weeks, bereavement leave, caregiver leave, and volunteer leave.
• Well-being reimbursement and programs supporting financial, physical, and mental health.
• Insurance plans, service anniversary awards, recognition awards, and an annual performance-based bonus, subject to applicable plan terms.