Backend Developer: Ai Api Router (Python/Node.js)

TL;DR

Backend Developer (Python/Node.js): Designing and developing a microservice that intelligently routes requests between AI providers (OpenAI, Anthropic, etc.), focusing on API gateway development, request classification, and routing table management. Focus on optimizing costs, latency, and automatic fallback mechanisms, ensuring a balance between speed, cost, and quality for real-time request routing.

Location: Remote

Salary: 200 000 – 350 000 ₽

Компания

Team developing an AI service with tens of thousands of users and a high volume of requests to AI providers, developing a smart request router between AI providers (OpenAI, Anthropic, DeepSeek, Google, etc.) with intelligent routing and cost optimization.

Что делать

• Develop a microservice that accepts requests in OpenAI-compatible format and intelligently routes them between AI providers.
• Develop an API gateway with a single endpoint compatible with OpenAI API (Chat Completions format).
• Create a classifier of incoming requests by complexity (simple / medium / complex / code / image) based on heuristics and ML models.
• Manage a routing table to determine which provider and model is used for each type of request.
• Implement request proxying to providers (OpenAI, Anthropic, Google, DeepSeek, etc.) with streaming support.
• Log metrics: cost of each request, latency, selected model, tokens input/output.

Требования

• Experience with OpenAI API (or similar LLM API) is required.
• Understanding of streaming (SSE) is required.
• Experience in designing and developing microservices.
• Understanding of REST API and working with multiple external APIs in parallel.
• Ability to write clean, maintainable code.
• Self-sufficiency: tasks are set at the level of "what needs to be obtained" rather than "how exactly to do it".

Хорошо, если есть

• Experience with LLM routing or model orchestration.
• Experience with billing systems and ЮKassa.
• Experience with Redis for caching and rate limiting.
• Understanding of tokenization and pricing of LLM models.
• Experience with ML (LightGBM/CatBoost) for developing a classifier.

Условия

• Remote work.
• Project work with the prospect of permanent cooperation.
• Payment is discussed individually, depending on experience.
• Direct communication with the founders, without bureaucracy.