Data Scientist (AI)

TL;DR

Data Scientist (AI): Focusing on evaluation and benchmarking data development for Quick Suite features, with an accent on Research and other generative AI capabilities. Focus on engineering high-quality datasets that are essential to the success of hirify.global Quick Suite.

Location: Hybrid work model near one of our U.S. hirify.global offices.

Salary: USA, CA, Santa Clara - 157,300.00 - 212,800.00 USD annually; USA, NY, New York - 153,400.00 - 207,500.00 USD annually; USA, WA, Seattle - 136,000.00 - 184,000.00 USD annually

Company

hirify.global Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform.

What you will do

• Design and develop comprehensive evaluation and benchmarking datasets for Quick Suite AI-powered features.
• Leverage LLMs for synthetic data corpora generation; data evaluation and quality assessment using LLM-as-a-judge settings.
• Create ground truth datasets with high-quality question-answer pairs across diverse domains and use cases.
• Lead human annotation initiatives and model evaluation audits to ensure data quality and relevance.
• Conduct statistical analysis to measure model performance, identify failure patterns, and guide improvement strategies.
• Build scalable data pipelines and tools to support continuous evaluation and benchmarking efforts.

Requirements

• 2+ years of data scientist experience.
• 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience.
• 3+ years of machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance experience.
• 1+ years of working with or evaluating AI systems experience.
• 1+ years of creating or contributing to mathematical textbooks, research papers, or educational content experience.
• Master's degree in Science, Technology, Engineering, or Mathematics (STEM), or experience working in Science, Technology, Engineering, or Mathematics (STEM).

Nice to have

• Ph.D. in Science, Technology, Engineering, or Mathematics (STEM).
• Knowledge of machine learning concepts and their application to reasoning and problem-solving.
• Experience in a ML or data scientist role with a large technology company.
• Experience in defining and creating benchmarks for assessing GenAI model performance.

Culture & Benefits

• Employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences.
• Endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
• We value work-life harmony and strive for flexibility as part of our working culture.