Data Scientist (AI)

TL;DR

Data Scientist (AI): Building evaluation and benchmarking infrastructure for an agentic AI platform with an accent on evaluation frameworks and knowledge graph validation. Focus on designing rigorous pipelines for LLMs, measuring agent performance, and curating high-quality synthetic datasets.

Location: Must be based in Frisco, Texas, US

Company

hirify.global is a global cybersecurity company redefining the future of protection through an open, native platform driven by AI, automation, and analytics.

What you will do

• Architect and implement rigorous evaluation pipelines for agentic AI systems, including reasoning agents and RAG pipelines.
• Design benchmarks to assess accuracy, reliability, latency, and safety across LLMs tailored to cybersecurity use cases.
• Develop methods to validate knowledge graph quality, covering entity resolution, relationship accuracy, and completeness.
• Build and maintain high-quality synthetic and real-world datasets for training, fine-tuning, and testing.
• Define evaluation metrics and surface results through dashboards for engineering and product teams.
• Research and adapt latest evaluation methodologies such as LLM-as-judge, RAGAS, and MT-Bench.

Requirements

• 5+ years of professional experience in data science, ML engineering, or AI research.
• Strong proficiency in Python (pandas, NumPy, scikit-learn) and statistical experimental design.
• Hands-on experience evaluating LLMs using frameworks like RAGAS, HELM, or EleutherAI.
• Experience testing agentic AI systems, including tool use and multi-agent coordination patterns.
• Experience with knowledge graphs (NebulaGraph, Neo4j) and vector databases (Qdrant preferred).
• Location: Must be based in Frisco, Texas, US

Nice to have

• Familiarity with the cybersecurity domain, SOC workflows, and threat detection.
• Experience with AWS and experiment tracking tools like MLflow, Weights & Biases, or Langfuse.
• Experience evaluating AI systems in high-stakes or regulated environments.

Culture & Benefits

• Comprehensive health coverage including medical, dental, and vision.
• Retirement plans and paid parental leave.
• Paid time off and flexible work hours.
• Support for community involvement and a commitment to a diverse, inclusive workplace.