Назад
1 день назад

Software Engineer (AI Inference Product)

200 000 - 400 000$
Формат работы
hybrid
Тип работы
fulltime
Грейд
middle
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Software Engineer (AI Inference Product) (Backend/Distributed Systems): Building and operating the API layer for hosted model endpoints, including authentication, rate limiting, routing, metering, and billing, with an accent on low latency, reliability, and high-scale inference infrastructure. Focus on designing real-time capacity systems, improving observability, and shipping developer-facing products across a global GPU fleet.

Location: Hybrid in San Francisco or New York, United States

Salary: $200K–$400K plus equity

Company

Baseten provides AI inference infrastructure, hosted model APIs, and developer tooling for companies bringing machine-learning models into production.

What you will do

  • Build and operate the API layer behind Model APIs, including request handling, authentication, rate limiting, routing, and low-latency high-availability services.
  • Own real-time usage metering, billing, pricing, and packaging infrastructure.
  • Design rate-limiting and capacity systems that balance customer throughput with global GPU fleet utilization.
  • Ship developer-facing API products, including model routing, usage visibility, self-serve controls, and predictable API behavior.
  • Implement deep observability and alerting, and partner with inference, Model APIs, and product teams to deliver capabilities safely.
  • Own projects end to end from design through deployment, on-call, and iteration based on customer feedback.

Requirements

  • Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience.
  • 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs with reliability, latency, and scale requirements.
  • Proven ownership of low-latency, reliable backend services, including rate limiting, authentication, quotas, metering, and migrations.
  • Infrastructure and performance skills covering profiling, tracing, capacity planning, and SLO management.
  • Ability to debug performance and reliability issues across application, runtime, and infrastructure layers.
  • Strong developer-experience focus and interest in how systems are used.
  • Willingness to learn new languages, frameworks, and systems, with an interest in inference engineering; prior ML or LLM experience is not required.
  • Excellent written communication and collaboration skills, including design documentation and cross-functional work.

Nice to have

  • Deep Kubernetes experience, including operators and custom resources, plus service meshes or API gateways.
  • Experience with distributed scheduling, autoscaling, or service orchestration.
  • Developer-facing infrastructure or API experience, or contributions to open-source infrastructure or ML systems.
  • Familiarity with observability tooling, CI/CD systems, or release automation.
  • Experience with model serving or inference systems.

Culture & Benefits

  • Meaningful equity and competitive compensation.
  • For U.S. employees, 100% medical, dental, and vision insurance coverage for employees and dependents.
  • Flexible PTO and a company-wide Winter Break from Christmas Eve through New Year’s Day.
  • Paid parental leave and a fertility and family-building stipend through Carrot.
  • For U.S. employees, company-facilitated 401(k).
  • Exposure to ML startups and opportunities for learning and networking.

Hiring process

  • Apply with relevant information demonstrating backend, distributed-systems, and infrastructure experience.
  • Baseten is an equal opportunity employer and considers qualified applicants with criminal histories consistently with applicable law, including the San Francisco Fair Chance Ordinance where applicable.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →