Software Quality Engineer (AI)

Location: Hybrid, based out of the London, San Francisco, Seattle, or New York City office hubs

Company

Writer develops an enterprise AI platform for building and deploying AI agents powered by enterprise-grade large language models.

What you will do

• Define quality assurance strategies, test plans, and validation criteria for AI agents and LLM-powered applications.
• Design, develop, and maintain scalable automated test frameworks using Python or TypeScript and modern testing tools.
• Collaborate with product managers, machine learning engineers, and data scientists to test complex AI features and model behavior.
• Integrate automated checks into CI/CD pipelines and improve testing infrastructure and release quality.
• Monitor production systems and AI model performance, investigate defects, and perform root cause analysis.
• Manage test data, mock services, customer feedback, debugging logs, and the full software defect lifecycle.

Requirements

• 5+ years of experience in software quality assurance or engineering, focused on complex distributed systems or AI/ML applications.
• 5+ years of experience working with GitHub Actions.
• Proficiency in Python or TypeScript for test automation and experience with Playwright.
• Understanding of AI/ML concepts, model evaluation metrics, data pipelines, and generative AI testing challenges.
• Strong analytical, debugging, collaboration, and independent ownership skills.
• Experience with agentic AI systems, AI skills, and the Model Context Protocol (MCP).

Nice to have

• Experience with AWS, Azure, or GCP.
• Experience with Docker and Kubernetes in a CI/CD environment.
• Experience testing cloud-based distributed systems.

Culture & Benefits

• Hybrid work from one of the listed office hubs.
• Paid time off and company holidays.
• Medical and dental insurance, paid parental leave, and fertility and family planning support.
• Competitive pension scheme and company contribution for UK full-time employees.
• Wellness, learning and development stipends, company stock options, and team off-sites.