Вакансия из Hirify Global, списка международных tech-компаний Для мэтча и отклика нужен Plus
Пожаловаться
Зарплата и рынок
развернуть
$155к/год
$150к$190кмедиана$210к
По похожим ролям (18) ~ $190к/год
8
Не очень вакансия
развернуть
В описание попал длинный список других, не связанных с вакансией позиций, поэтому нельзя понять, насколько это чистое и надежное объявление именно о роли специалиста по оценке ИИ.
Список чужих вакансий
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
AI Evaluation Specialist (AI/Generative AI): Evaluating enterprise AI tools, models, agents, and AI-enabled applications with an accent on quality, accuracy, safety, privacy, security, usability, and business readiness. Focus on designing repeatable evaluation frameworks, testing RAG and tool-enabled workflows, comparing vendors, and analyzing model behavior and failure modes.
Location: Atlanta, Georgia, United States; hybrid work model
Salary: $155,000
Company
hirify.global delivers outsourced services and workforce solutions across North America, supporting clients with efficient, quality-focused delivery.
What you will do
Plan and execute hands-on evaluations of enterprise generative AI tools, models, agents, and AI-enabled applications using representative business scenarios, workflows, prompts, datasets, and user personas.
Design and maintain evaluation frameworks, scorecards, rubrics, benchmarks, and measurable acceptance criteria.
Evaluate safety, privacy, security, data handling, governance, responsible AI, and compliance considerations with relevant stakeholders.
Compare AI products, models, configurations, and vendors, including functionality, integrations, performance, cost, and proof-of-concept results.
Evaluate RAG, enterprise search, file and data analysis, connectors, APIs, and Model Context Protocol-enabled workflows while sharing findings and recommendations.
Requirements
5+ years of experience in model evaluation or model quality assurance, including hands-on evaluation of AI-enabled products or services.
Experience developing structured evaluation plans, test cases, scoring rubrics, benchmarks, and measurable acceptance criteria.
Hands-on experience with generative AI tools and large language models, including prompt design, systematic testing, output analysis, hallucination identification, and analysis of other failure modes.
Experience assessing AI systems for accuracy, relevance, consistency, safety, bias, privacy, security, usability, performance, and cost.
Familiarity with large language models, retrieval-augmented generation, AI agents, APIs, enterprise search, and tool or connector integrations.
Bachelor’s degree.
Must be a U.S. Citizen or Lawful Permanent Resident (Green Card Holder).
Must be able to obtain Public Trust clearance.
Nice to have
Experience evaluating enterprise AI platforms or assistants, including OpenAI products and life sciences models such as Rosalind.
Experience with automated LLM evaluation frameworks, model-as-a-judge methods, human-in-the-loop evaluation, red teaming, or adversarial testing.
Experience evaluating RAG systems, AI agents, MCP-enabled tools, enterprise connectors, or multi-step AI workflows.
Experience developing dashboards or reporting mechanisms for evaluation metrics, quality trends, risk findings, and regression results.
Ability to analyze datasets and evaluation results using Python, SQL, spreadsheets, notebooks, or analytics platforms.
Industry-recognized certification in AI, machine learning, data science, cybersecurity, cloud technologies, or responsible AI.
Master’s degree preferred.
Culture & Benefits
Direct-hire position with a hybrid work model.
Collaborative environment involving product, data science, engineering, security, privacy, legal, governance, and business stakeholders.
Eligible employees may receive medical, dental, vision, spending account, life insurance, voluntary plan, and 401(k) benefits.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →