Senior Software Development Engineer (AI Observability)

Location: Pleasanton, California, United States; Flex Work requires spending at least 50% of each quarter in the office or in the field with customers, prospects, and partners.

Salary: $176,000–$264,000 annual base pay for the primary location; $148,200–$264,000 for additional US locations. The role may also include bonus or commission eligibility and annual refresh stock grants.

Company

hirify.global is a Fortune 500 company and AI platform for managing people, money, and agents.

What you will do

• Design and build scalable, fault-tolerant microservices and high-throughput ingestion pipelines supporting sub-second query execution.
• Extend distributed tracing and telemetry with OpenTelemetry to cover agent execution, tool calls, LLM reasoning steps, and prompt/response chains.
• Optimize ClickHouse, Grafana Tempo, and Apache Iceberg query performance and data-retention strategies for platform availability.
• Build gRPC and REST APIs integrating the Model Context Protocol framework and Agent Ready Tools.
• Collaborate with Product, Data Engineering, SRE, Legal/Compliance, QA automation leads, and cross-functional partners on critical releases.
• Produce architecture specifications, runbooks, and system documentation while contributing to CI/CD, load testing, and multi-region failover strategies.

Requirements

• 8+ years of software development engineering experience, including 5+ years designing, building, and operating complex distributed-system architectures with proven 99.9% uptime and fault tolerance.
• 5+ years writing production-grade code in at least two of Java, Go, Scala, or Python.
• Experience with data partitioning, dynamic sharding, high-dimensional vector retrieval, caching, high-throughput indexing, and sub-second queries over massive datasets.
• Experience with gRPC, REST, OTLP, object-oriented design, clean abstractions, domain-driven design, and microservices patterns.
• Knowledge of distributed load and integration testing, telemetry-driven regression checks, CI/CD automation, and Git, GitHub, or Bitbucket workflows.
• Experience with enterprise security practices including mTLS, RBAC/ABAC, and encryption at rest and in transit.
• Ability to mentor engineers, technically lead cross-functional pods, and write architecture specifications, API contracts, runbooks, and LLM/agentic tracing documentation.
• Bachelor’s degree in Computer Science, Software Engineering, or a related field, or equivalent experience.

Nice to have

• Master’s degree in Computer Science or Distributed Systems.
• Experience with LangChain and LangSmith for agentic tracing and observability.

Culture & Benefits

• Flexible work combines in-person collaboration with remote work, with scheduling adapted to the business, team, and individual needs.
• Benefits may include hirify.global’s bonus or role-specific commission/bonus plan and annual refresh stock grants.
• hirify.global provides reasonable accommodations during the application process and considers qualified applicants with arrest and conviction records where required by law.
• hirify.global is an equal opportunity employer, including for individuals with disabilities and protected veterans.

Hiring process

• Applications must be submitted through hirify.global Careers; hirify.global does not charge recruiting, consulting, or coaching fees.
• Candidates needing reasonable accommodation may email accommodations@hirify.global.com as far in advance as possible.
• Employee referrals should follow hirify.global’s Employee Referral process through the referring connection.