Member of Technical Staff - Fullstack Software Engineer (AI)

TL;DR

Member of Technical Staff – Fullstack Software Engineer (AI): Building evaluation platform for Copilot across data sampling, AI data collection and processing, analysis, and insight generation with an accent on fullstack development from backend databases and services to browser/mobile automation and dashboards. Focus on designing scalable pipelines, prompt engineering, statistical analysis, and experimentation methodologies to assess AI performance, relevance, and trustworthiness.

Location: Must be local to Redmond, WA or Mountain View, CA and in office 4 days a week.

Salary: USD $100,600 – $258,000 per year (depending on level IC3/IC4 and location).

Company

hirify.global organization dedicated to advancing Copilot and consumer AI products in collaboration with Bing, Edge, MSN, and Shopping.

What you will do

• Work across full stack of evaluation platform: data sampling, AI data collection/processing, analysis, and insight generation.
• Design pipeline architecture for rapid iteration and scalability.
• Conduct post-analysis of labeled data and develop dashboards for insights visualization.
• Collaborate with product team to enhance quality and address gaps using experimentation methodologies like A/B testing.
• Leverage prompt engineering and AI tools for desktop, web, and mobile scenarios including multi-turn voice conversations.

Requirements

• Must be local to Redmond, WA or Mountain View, CA and work in office 4 days a week.
• Bachelor’s Degree in Computer Science or related AND 4+ years technical engineering experience with coding in C#, Java, or Python (or equivalent).
• Expertise in experimentation methodologies, A/B evaluation, data sampling, measurement, and analysis.
• Experience translating metrics into actionable insights and strategic vision for product quality.

Nice to have

• 2+ years with SQL, PostgreSQL, or MySQL.
• 2+ years building scalable services on Azure, AWS, or GCP with datastores.
• HTML, CSS, JavaScript, ASP.NET, REST, jQuery.
• Browser automation (Selenium, Puppeteer, Playwright) and mobile automation (Appium).
• Familiarity with LLMs, AI ChatBots, Prompt Engineering, Azure OpenAI.
• Azure DevOps, GIT, leading technical projects.

Culture & Benefits

• Growth mindset, innovation, collaboration with respect, integrity, and accountability.
• Inclusive culture where everyone can thrive.
• Benefits and other compensation available (details on careers site).