Software Test Engineer (AI)

Location: Remote in the USA

Base salary: $150,000–$220,000 annually, plus equity, bonus, and a 10% annual bonus.

Company

Deepgram provides real-time voice AI APIs and self-hosted software for speech-to-text, text-to-speech, and production-grade voice agents.

What you will do

• Define and execute test plans across products, APIs, SDKs, model-powered features, and data platforms.
• Build and maintain automated functional, integration, end-to-end, regression, API, browser, and service-level test suites.
• Translate product requirements and customer acceptance criteria into test strategies, test cases, and release gates.
• Create customer-focused and adversarial datasets, fixtures, test environments, test harnesses, dashboards, and result-aggregation pipelines.
• Validate speech-to-text, text-to-speech, and other AI features using metrics, expected outputs, human review, and regression coverage.
• Integrate tests, quality checks, canaries, load testing, and release validation into CI/CD while partnering with Engineering, Product, Research, Data, Infrastructure, DevOps, and QA.

Requirements

• BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
• 5+ years of professional software or QA engineering experience, including shipped test infrastructure or evaluation systems.
• Backend or scripting experience with Python, Rust, Go, or a similar language.
• Experience designing automated test pipelines, evaluation frameworks, or data-processing systems.
• Strong analytical skills for reasoning about metrics, thresholds, statistical variation, and regression signals.
• Ability to lead ambiguous technical work and communicate effectively across research, engineering, and product teams.

Nice to have

• Experience evaluating LLMs, RAG pipelines, agents, or multimodal AI systems.
• Experience with voice, audio, speech recognition, or real-time systems and metrics such as WER, MOS, latency, and time-to-first-byte.
• Experience with cloud infrastructure, containers, ephemeral environments, CI/CD, Grafana, canaries, or anomaly detection.
• Experience testing React Native, mobile, or other cross-platform applications.
• Open-source contributions or experience bridging product, QA, evaluation, training, inference, data, or agent-framework teams.

Culture & Benefits

• AI-first operating model requiring active use and experimentation with advanced AI tools.
• Fast-changing environment focused on experimentation, adaptation, and continuous learning.
• Equity and bonus compensation in addition to base salary.