TL;DR

AI Engineer (AI Tests Generation Startup): Develop and optimize an AI-powered Chrome extension for converting manual web test journeys into production-ready end-to-end test code with an accent on continuous evaluation pipelines and advanced LLM techniques. Focus on refining evaluation metrics, shipping core agent improvements, and building deterministic feedback loops to enhance test generation quality.

Location: Limassol, Cyprus; Paphos, Cyprus

Company

%hirify_global%-backed incubator startup focused on AI-powered web testing solutions with a small, fast-moving team and no bureaucracy.

What you will do

Own experimentation, evaluation pipeline, and metrics for the code generation agent.
Refine and rebuild evaluation systems to ensure metrics correlate with user value.
Prototype, evaluate, and productionize features like agent planning, context reduction, and RAG.
Build deterministic feedback loops integrating code analysis and test execution.
Collaborate closely with experts to improve AI test generation quality.

Requirements

Location: Based in Limassol or Paphos, Cyprus
Senior AI Engineer with hands-on experience shipping LLM or ML systems.
Experience building evaluation pipelines and defining user-value metrics.
Proficient in production-quality Python development.
At least one year experience in a startup or fast-moving team.
English: Required, at least B2 level

Nice to have

Experience with LLM agents involving tool-calling, planning, or multi-step reasoning.
Familiarity with code generation benchmarks like SWE-bench or HumanEval.
Knowledge of web UI automation frameworks such as Selenium or Playwright.
Experience with static program analysis and AST parsing.