Software Engineer, Inference (AI)

Location: Sunnyvale, CA or Bellevue, WA, United States

Base salary: $120,000–$160,000 per year, plus discretionary bonus, equity awards, and benefits subject to eligibility.

Company

CoreWeave provides high-performance cloud infrastructure for GPU workloads across AI/ML, VFX, rendering, and real-time inference.

What you will do

• Own a model-serving service end to end, including design, implementation, testing, deployment, and operations.
• Build dashboards, alerts, traces, structured observability, and automated runbook steps.
• Improve inference latency and throughput through micro-batching, concurrency tuning, and caching.
• Participate in on-call rotations and lead post-incident action items for the service.
• Review pull requests and provide design feedback in team reviews.

Requirements

• 0–3 years of software engineering experience, or equivalent experience building cloud or distributed services in production.
• Proficiency in Python or Go, with familiarity with gRPC/HTTP APIs and containerized deployments.
• Working knowledge of Kubernetes, Slurm, or Ray and CI/CD pipelines.
• Experience with metrics and alerting using Prometheus, Grafana, or OpenTelemetry, plus structured logging.
• Basic understanding of GPU inference concepts, including KV caching, mixed precision, and streaming tokens.
• Must satisfy U.S. export-control access requirements as a U.S. person, an individual eligible to access the information without required authorization, or an individual eligible and reasonably likely to obtain the required authorization.

Nice to have

• Experience with inference frameworks such as Triton, vLLM, TensorRT-LLM, Ray Serve, or TorchServe.
• Hands-on performance profiling or benchmarking with tools such as wrk, vegeta, perf, or Nsight.
• Contributions to reliability practices, including SLI/SLOs or postmortem improvements.

Culture & Benefits

• Remote-friendly culture with a focus on curiosity, ownership, innovation, and disruptive technical work.
• 100% employer-paid medical, dental, and vision insurance.
• 401(k) with a generous employer match, life and disability insurance, and flexible PTO.
• Tuition reimbursement, ESPP participation, mental wellness support, family-forming support, paid parental leave, and childcare support.
• Catered lunch at office and data center locations and a casual work environment.