Principal Software Engineer (AI)

TL;DR

Principal Software Engineer (AI): Building the backend and orchestration layer for an AI-powered audit and accounting platform with an accent on turning frontier models into production-grade workflows at scale. Focus on architecting high-throughput services, designing AI agents, and managing large-scale data retrieval and indexing strategies.

Location: New York

Company

Partnered with an AI lab building the next generation of audit and accounting products used by professionals in real client work.

What you will do

• Own the AI backbone, including the backend and orchestration layer for agents, conversations, workspaces, and knowledge bases.
• Architect backend services using Python, FastAPI, PostgreSQL, and AWS to power generative AI agents and document-centric workflows.
• Develop AI orchestration systems covering routing, tool calling, MCP servers, multi-step workflows, and guardrails for LLMs.
• Optimize systems for high throughput and low latency via caching, queuing, rate limiting, and model failover strategies.
• Manage large-scale data processing, including vector stores and indexing strategies for millions of documents.
• Lead multi-quarter initiatives across AI, product, and infrastructure, while mentoring staff and senior engineers.

Requirements

• 8-10+ years of backend development experience building scalable web services and APIs.
• Deep expertise in Python, with production experience in FastAPI, PostgreSQL, and AWS.
• Strong background in distributed systems, including data modeling, API contracts, and performance tuning under load.
• Proven track record of owning large, complex projects end-to-end from architecture to long-term operation.
• Genuine interest in AI-native software, LLMs, and retrieval systems.
• Location: Must be based in New York

Nice to have

• Hands-on experience with LLM API integration (OpenAI, Anthropic).
• Experience with vector databases, embeddings, or semantic search.
• Prior work in regulated domains such as fintech, tax, audit, or legal.