Infrastructure MLOps Software Engineer (ML/LLM)

Location: San Francisco, Brooklyn, Cambridge, or remote within the United States

Annual salary: $195,000–$250,000 in San Francisco or New York City; $185,000–$240,000 in Boston, DC Metro, Los Angeles, or Seattle; $175,000–$230,000 in Denver, Atlanta, Austin, Las Vegas, Sacramento, or Chicago; $165,000–$220,000 in Texas or Florida.

Company

hirify.global builds a cloud-delivered television streaming product with live and on-demand content, native applications, machine learning, and multi-screen playback experiences.

What you will do

• Manage core cloud infrastructure supporting ML and LLM workloads across the company.
• Design, build, and maintain AWS and Kubernetes infrastructure, including GPU scheduling, capacity, quota, and cost management.
• Build scalable model-serving and inference systems for recommendation models and LLMs, including autoscaling and latency/throughput optimization.
• Develop monitoring and observability for system health, model quality, drift, evaluation, and online performance.
• Build offline and online evaluation pipelines, regression datasets, LLM-as-judge harnesses, A/B testing, and feedback loops.
• Maintain CI/CD systems, Infrastructure as Code practices, platform reliability and security, and integrations with third-party vendors.

Requirements

• 8+ years of software development and infrastructure management experience, including 4+ years developing and deploying ML systems at scale across training and inference.
• Experience with distributed systems, microservices architecture, Linux, Docker, and containerized environments.
• Experience managing AWS or GCP environments and services including CloudWatch, Route 53, RDS, ElastiCache, SQS, load balancers, VPC networking, and IAM security.
• Experience with Kubernetes and GPU or accelerated workloads, including NVIDIA device plugin, MIG, or timeslicing.
• Strong understanding of TCP/IP, DNS, HTTP, networking, and application security principles.
• Familiarity or hands-on experience with Terraform or CloudFormation, monitoring and observability systems, and CI/CD automation systems such as Jenkins or GitLab.
• Experience with relational and non-relational databases, modern data warehousing, and querying.
• Proficiency writing, testing, and profiling software in Golang, JavaScript/TypeScript, C++, Ruby, Python, or similar languages.
• Ability to collaborate and communicate with internal and external stakeholders in business and technical roles.

Nice to have

• Experience optimizing inference through batching, quantization, KV-cache management, or GPU memory tuning.
• Experience with ML platform tooling, model registries, experiment tracking, feature stores, and workflow orchestration.
• Experience building model evaluation, monitoring, experimentation, drift detection, LLM-as-judge, or model A/B testing systems.
• Experience with LLMOps, inference gateways, hosted model APIs, RAG pipelines, vector databases, and prompt or version management.
• Experience operating agent harnesses or tool-using LLM applications in production.
• Experience with data and model versioning, data or labeling pipelines, and FinOps for GPU and inference workloads.

Culture & Benefits

• People-first, pragmatic, transparent, and inclusive culture with a focus on collaboration and quality.
• Flexible working hours and remote work availability; an additional bonus is available for employees working at least three days per week in an office.
• Health, dental, and vision coverage; 401(k) contributions; unlimited paid vacation and sick leave; and up to 20 weeks of paid parental leave.
• Professional development support of $5,250 annually, plus home office and TV stipends.
• Additional vacation bonus, commuter benefits, Gympass membership, and dog-friendly offices.

Hiring process

• Applications are managed with automated tools that assess relevant skills and qualifications and flag potentially fraudulent or duplicate submissions.
• Candidates may request human review of an automated decision or a reasonable accommodation by contacting recruiting@hirify.global.com.