Senior AI Engineer (AI)

TL;DR

Senior AI Engineer (AI): Developing an AI-powered Student Journey Platform with an accent on RAG, multi-agent architectures, and LLM evaluation frameworks. Focus on designing structured evaluation systems, improving production-grade reliability, and building observable AI workflows.

Location: Must be based in the US (Remote)

Company

An education technology company that provides support and expertise to universities to develop and grow workforce-focused online degree programs.

What you will do

• Build and maintain evaluation frameworks (LLM-as-Judge, rubric-based scoring) to measure output quality and reliability.
• Architect and implement multi-agent workflows with clear coordination, tool usage, and failure handling.
• Implement structured observability into AI systems, including tracing, prompt tracking, and cost/latency monitoring.
• Design and implement Retrieval-Augmented Generation (RAG) systems and Model Context Protocol (MCP) servers.
• Develop and manage fine-tuning workflows (SFT, preference optimization) and handle dataset preparation and validation.
• Collaborate with product and engineering teams to translate business requirements into testable AI designs.

Requirements

• 3-5 years of full-stack engineering experience with strong OOP and AI-focused system design fundamentals.
• Professional experience in Python, C#, Java, or similar production languages.
• Experience with LLM evaluation and observability tooling such as Langfuse, LangSmith, or OpenTelemetry.
• Experience implementing guardrails, policy enforcement, and safety layers in AI systems.
• Must be based in the US.

Nice to have

• Experience building production-grade RAG systems (chunking strategies, embeddings, reranking).
• Familiarity with LLM performance optimization techniques including caching, routing, and batching.
• Experience deploying AI systems in cloud environments (AWS, Azure, GCP) or using Databricks (ML Flow).
• Experience contributing to internal AI standards or reusable platform-level tooling.

Culture & Benefits

• Remote-first work environment within the US.
• Opportunity to contribute to a platform increasing access to affordable education.
• Inclusive and diverse workforce environment as an equal-opportunity employer.