Senior Machine Learning Engineer (Search Quality/AI)

TL;DR

Senior Machine Learning Engineer (AI/Ecommerce): Building and optimizing e-commerce search and discovery systems with an accent on LLM-based relevance evaluation and real-time quality monitoring. Focus on fine-tuning LLMs for relevance judgment, optimizing inference at scale, and designing automated quality pipelines across 40+ languages and 20+ domains.

Location: Remote (Global)

Salary: $80,000–$120,000 USD per year

Company

Constructor.io is an AI-first e-commerce search and discovery platform that helps global brands drive revenue and conversion gains.

What you will do

• Develop and fine-tune LLM-based evaluation systems to assess search relevance and align model judgments with real user behavior.
• Implement real-time quality improvements in production, including re-ranking, filtering, and signal computation under strict latency requirements.
• Build automated quality monitoring pipelines and agentic systems to detect degradations and generate actionable recommendations.
• Optimize and scale LLM inference across a diverse customer base spanning 40+ languages and 20+ domains.
• Collaborate within cross-functional teams consisting of ranking, product, and data engineering specialists.

Requirements

• 4+ years of experience shipping production ML systems.
• Professional experience with search, information retrieval, or recommendation systems.
• Hands-on expertise in fine-tuning, evaluation frameworks, and scaling LLM deployments.
• Strong proficiency in Python, PyTorch, SQL, and data orchestration tools like Spark and Airflow.
• Experience designing and running A/B tests to validate the impact of models.
• Excellent English communication skills (B2+).

Culture & Benefits

• Fully remote work environment with a dedicated work-from-home stipend.
• Unlimited vacation time, with a strong encouragement to take at least 3 weeks per year.
• Annual training and development budget for every employee.
• Provision of Apple laptops for new hires and stock options.
• Maternity and paternity leave for qualified employees.
• Regular team offsite events for connection and collaboration.