Staff Software Engineer (AIOps)

Location: Hybrid work in Scottsdale, San Francisco, Chicago, or New York; U.S. work eligibility required. The compensation range is listed for Phoenix and Chicago.

Base pay: $124,000–$165,000 USD per year, plus discretionary incentive eligibility and benefits.

Company

Financial technology services support the U.S. payments system through solutions including Zelle and Paze.

What you will do

• Design, build, test, and operate the production AIOps agent runtime, including event ingestion, context assembly, planning, tool invocation, verification, escalation, and response handling.
• Develop typed, scoped, versioned, and auditable tool interfaces for CI/CD, infrastructure automation, Kubernetes, network services, ITSM, observability, and secrets management.
• Implement action controls such as human approvals, permission boundaries, blast-radius limits, dry runs, reversibility, and emergency shutdowns.
• Build event routing, telemetry, model routing, session management, retries, timeouts, and cost and latency controls.
• Develop auditability, historical incident replay, shadow-mode evaluation, regression testing, and feedback mechanisms for agent decisions.
• Partner with Engineering, Operations, Architecture, Security, Risk, and Compliance to establish reusable platform patterns and guardrails.

Requirements

• Eight or more years of experience in software engineering, platform engineering, distributed systems, DevOps, AIOps, or a related discipline.
• Strong production software engineering and distributed-systems design experience, including interfaces, state, retries, timeouts, idempotency, failure modes, and operational risk.
• Strong production development experience with Python for services, APIs, automation, orchestration systems, or agent runtimes.
• Hands-on experience building production agentic systems, tool-using AI applications, or autonomous and semi-autonomous control loops.
• Experience with AI agent tool-use or function-calling interfaces using MCP or comparable integration patterns, plus production guardrails, monitoring, evaluation, and cost and latency management.
• Experience with event-driven architectures, observability, and modern frontend and backend development using technologies such as Kafka, OpenTelemetry, Prometheus, Grafana, React, TypeScript, or Python.

Nice to have

• LangGraph or comparable agent orchestration experience.
• Go, CI/CD, infrastructure automation, Kubernetes operations, or ITSM experience.
• Anomaly detection, time-series analysis, chaos engineering, simulation, incident replay, or game-day testing experience.
• Experience with PCI DSS, SOX ITGC, NYDFS Part 500, FFIEC guidance, or regulated fintech, banking, and payments environments.
• AWS, Kubernetes, AI engineering, security, or related technical certification.

Culture & Benefits

• Hybrid collaboration model for the listed U.S. locations.
• Medical, dental, and vision coverage with HSA and FSA options.
• 401(k) plan with a 100% company safe-harbor match on the first 6% deferred.
• Flexible time off or generous PTO, 11 paid company holidays, and a paid volunteer day.
• 12 weeks of paid parental leave and family-planning support.