Manager, AI Benchmarking and Evaluation Research (Remote, ROU)

Location: Romania — Remote

Company

Cybersecurity company developing an AI-native platform to protect organizations from breaches and process large-scale security event data.

What you will do

• Lead, mentor, and grow a team of researchers and engineers evaluating AI models for cybersecurity tasks.
• Define the strategy, roadmap, and success metrics for AI, LLM, and agentic-system evaluations across incident response, threat hunting, alert triage, and investigation.
• Design standardized methodologies, benchmark datasets, and reproducible testing pipelines based on real SOC workflows.
• Assess model accuracy, robustness, reliability, and operational effectiveness in security-analyst scenarios.
• Establish quantitative and qualitative metrics for AI performance against threat-detection and response challenges.
• Collaborate with engineering, product, and threat-research teams to turn findings into scalable improvements and communicate recommendations to technical and executive audiences.

Requirements

• Hands-on SOC experience and strong knowledge of day-to-day security operations.
• At least 3 years of management or team-leadership experience, including mentoring and growing technical teams.
• Strong knowledge of incident response, threat hunting, detection, investigation, and remediation workflows.
• Practical experience with AI, including AI/LLM capabilities and agentic systems.
• Ability to define and standardize evaluation methodologies and reproducible testing pipelines.
• Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes.

Nice to have

• Security certifications such as GCIA, GCIH, GCFA, or OSCP.
• Experience with LLM or agentic evaluation frameworks and benchmarking pipelines.
• Python programming and data-science tooling experience.
• Familiarity with SOC platforms, SIEM/SOAR tools, detection engineering, adversarial testing, or red-teaming.
• Understanding of MITRE ATT&CK and its mapping to detection and response workflows.

Culture & Benefits

• Remote work with flexibility and autonomy.
• Competitive salary, stock options, private healthcare, and life insurance.
• Training budget and professional development opportunities.
• Flexible time off, competitive vacation and holidays, and paid parental and adoption leave.
• Wellness programs, employee networks, volunteer opportunities, and team hangouts.