Principal Machine Learning Engineer, SecureAI (AI)

Location: Toronto, Ontario, Canada; hybrid work arrangement indicated.

Salary: CAD 192,000–264,000 annual base salary.

Company

Okta builds identity and access management infrastructure, including an Identity Security Fabric for AI agents and the agentic era.

What you will do

• Implement intent-based enforcement to verify that agent runtime requests match their intended purpose.
• Apply LLM reasoning, prompt parsing, embeddings, vector search, and zero-shot classification to evaluate agent intent and executed actions in real time.
• Integrate low-latency inference and semantic evaluation engines into API gateway request paths and policy frameworks such as Cedar.
• Architect scalable ML and Generative AI systems with retrieval, inference, evaluation, and RAG pipelines.
• Build prompt-injection defenses, guardrails, schema validation, structured-output enforcement, and automated production evaluation pipelines.
• Mentor and coach engineers and collaborate with product and engineering teams on greenfield initiatives.

Requirements

• 10+ years of software development experience and strong programming expertise in Python.
• Hands-on applied machine learning experience, including feature engineering, model training, and fine-tuning.
• Hands-on experience with Generative AI platforms such as AWS Bedrock, OpenAI, or Anthropic.
• Deep understanding of retrieval-augmented generation, embeddings, and knowledge-base workflows.
• Hands-on experience with LiteLLM, LangGraph, LangChain, LlamaIndex, MCP, or related AI agent frameworks.
• Familiarity with FastAPI, PyTorch, TensorFlow, Spark ML, and workflow orchestration tools such as Airflow.
• Experience defining evaluation metrics, pipelines, and feedback loops for ML and Generative AI systems.
• Experience building tools or infrastructure for AI/ML applications and understanding the developer lifecycle in an AI-native environment.
• Ability to collaborate with product and engineering teams, drive greenfield initiatives, navigate unknowns, and iterate quickly.
• Bachelor’s or Master’s degree in Computer Science or a related field.

Nice to have

• Familiarity with Go or TypeScript.
• Experience integrating AI-driven systems with identity, authentication, or security products.
• Exposure to ethical AI, model risk, or compliance frameworks.
• Familiarity with evaluation datasets, synthetic data generation, or LLM-as-a-judge methods.

Culture & Benefits

• Immersive in-person onboarding designed to accelerate impact and build connections.
• Global community spanning more than 20 offices worldwide.
• Health, dental, and vision insurance, 401(k), flexible spending account, paid leave, PTO, and parental leave.
• Equity and bonus opportunities may be available according to applicable plans and policies.
• Focus on well-being, social impact, talent development, connection, and community.