Testing Engineer (AI/ML)

Location: Colombia

Company

Applied AI engineering firm building and deploying AI co-workers and enterprise software solutions across multiple industries, including Consumer Products and Retail, Education, Energy, Financial Services, Healthcare, and Manufacturing.

What you will do

• Write and maintain automated tests for UI, API, and integration layers using tools such as Selenium, Cypress, Playwright, and PyTest.
• Validate ML model outputs for accuracy, consistency, expected behavior, and defined evaluation criteria.
• Test data pipelines and AI workflows, including dataset validation and data quality checks.
• Execute test suites, analyze results, document and track defects using tools such as Jira and TestRail.
• Integrate automated tests into CI/CD pipelines and support end-to-end model evaluation.
• Collaborate with developers, ML engineers, and data scientists; participate in sprint ceremonies, quality strategy discussions, reporting, and QA process improvements.

Requirements

• 2+ years of experience in test automation or AI/ML testing, with comfort in at least one track and working knowledge of the other.
• Proficiency in Python, JavaScript, Java, or a similar programming language, with the ability to write and maintain test code independently.
• Experience with test automation frameworks for UI, API, or ML workflows.
• Understanding of AI/ML concepts and the ML lifecycle, including validation of model outputs against defined criteria.
• Experience with CI/CD pipelines and version control.
• Working knowledge of API testing, data validation, SDLC, STLC, and Agile methodologies.
• Proficiency with bug-tracking and test management tools.
• Familiarity with AI/ML testing, including LLM evaluation for accuracy, hallucination, toxicity, and bias; RAG retrieval quality and grounding; model drift monitoring; prompt regression testing; AI safety; and red-teaming.
• Demonstrable day-to-day use of AI-forward tools such as Claude and Cursor, plus strong problem-solving skills and sound judgment in ambiguous testing situations.

Nice to have

• Experience with mobile automation, performance testing, MLOps pipelines, or basic statistics.

Culture & Benefits

• Work on production-ready AI systems designed for real enterprise workflows and measurable client impact.
• Collaborate in cross-functional teams with experienced engineers, ML engineers, and data scientists.
• Take end-to-end ownership of testing responsibilities and contribute directly to quality strategy and process improvements.
• Operate in a fast-paced environment where delivery speed, craft, direct feedback, curiosity, and accountability are valued.
• Work alongside teams averaging more than 15 years of experience, with an emphasis on learning and continuous improvement.