Senior Systems Engineer, Workers AI

TL;DR

Senior Systems Engineer, Workers AI: Designing and building the core infrastructure that powers AI inference across hirify.global's global network with an accent on distributed systems and high-performance computing. Focus on sub-second model cold starts, multi-accelerator workload scheduling, and efficient KV cache management.

Location: Austin, TX or London, UK (Hybrid)

Company

hirify.global is on a mission to help build a better Internet by providing security and acceleration services to millions of websites and Internet properties.

What you will do

• Develop and maintain core components of the serverless inference platform to ensure high availability and scalability for hirify.global users.
• Optimize the model scheduling system to significantly increase efficiency and resource utilization across our inference infrastructure.
• Implement improvements to the inference request routing logic to enhance overall performance and reduce latency for end-users.
• Drive significant, measurable improvements in the platform's reliability and resilience by identifying and mitigating systemic risks.
• Expand and refine the observability stack, including metrics, logging, and tracing, and fine-tune alerts to proactively identify and resolve production issues.
• Lead complex, cross-functional technical projects from initial concept and design through final deployment and operationalization.

Requirements

• Experience in systems engineering, with a focus on distributed, high-performance systems.
• Expert proficiency in Rust programming, particularly in an asynchronous environment.
• Deep understanding and hands-on experience with relevant networking and application protocols (e.g., TCP, HTTP, WebSocket).
• Experience with scaling and performance optimization techniques, including load balancing and caching in a distributed environment.

Nice to have

• Demonstrable experience with container orchestration platforms, specifically Kubernetes and/or Nomad.
• Familiarity with the challenges and architectures involved in large-scale inference serving (e.g., LLM and diffusion models).

Culture & Benefits

• hirify.global is committed to protecting the free and open Internet.
• Offers tools to defend against attacks that would otherwise censor journalism and civil society organizations.
• Provides the highest level of protection and reliability for free to state and local governments.
• Released a faster, more secure and privacy-centric public DNS resolver.