AI Research Peer Review Evaluator (ML/AI)

Location: Fully remote; work from anywhere

Company

Zurich-based AI company and ETH/HSG spin-off developing machine learning and computer vision technology for autonomous driving, medical imaging, and visual inspection.

What you will do

• Read ML/AI research papers and assess their contributions, methodology, experiments, claims, and limitations.
• Review original human peer reviews to establish an expert baseline.
• Evaluate AI-generated peer reviews using a structured scoring rubric.
• Assess technical accuracy, analytical depth, constructive value, and novelty or significance assessments.
• Identify hallucinations, unsupported claims, missed technical issues, and valuable insights.
• Compare AI-generated reviews and provide concise, evidence-based rationales.

Requirements

• Master’s degree, PhD, or current graduate study in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a related technical field.
• Contribution to at least one scientific or research paper, ideally as a first author.
• Experience critically reading and evaluating ML/AI research papers, including methodology, experiments, results, limitations, and scientific claims.
• Familiarity with major ML/AI venues such as NeurIPS, ICML, ICLR, ACL, or CVPR.
• Strong analytical, written communication, and attention-to-detail skills, including the ability to identify factual inaccuracies and hallucinated technical claims.
• Ability to conduct academic literature searches, verify citations and publication dates, and consistently apply detailed evaluation guidelines.

Nice to have

• Prior peer-review experience for an ML/AI conference, journal, workshop, or similar academic venue.
• First-author research publications and expertise in specific ML/AI research areas.

Culture & Benefits

• Remote, project-based contractor role with flexible working hours.
• Part-time schedule with the ability to work from anywhere.
• Direct involvement in evaluating cutting-edge agentic AI systems for scientific research.
• Opportunity to apply ML/AI research expertise to improve AI-generated scientific peer review.

Hiring process

• Submit a CV and a brief note describing research background and areas of expertise.
• Include relevant publications and previous peer-review experience.
• Optionally provide familiar ML/AI research areas and conferences.