Senior Software Engineer (Data Infrastructure)

TL;DR

Senior Software Engineer (Data Infrastructure): Building robust data pipelines and infrastructure for evaluating real-world AI performance with an accent on processing and analyzing massive volumes of human vote data. Focus on designing scalable systems, ensuring data integrity, and collaborating with researchers to derive actionable insights from user feedback.

Location: Must be based in the Bay Area

Company

hirify.global is an open platform dedicated to evaluating AI model performance in the real world through community feedback and rigorous data analysis.

What you will do

• Design and build robust data pipelines to ingest, process, and transform user vote data.
• Collaborate with researchers and product leadership to define and meet data-driven product goals.
• Implement solutions for result dashboards and reports to inform the public and model developers.
• Ensure the reliability, quality, and integrity of production data pipelines.
• Scale data infrastructure to manage increasing volumes of analytical data.

Requirements

• 5+ years of software engineering experience with a focus on data engineering and big data technologies.
• Proficiency in SQL and at least one language like Python, Scala, or R.
• Hands-on experience with data processing frameworks such as Apache Spark or Ray Data.
• Experience with big data platforms like Databricks or Snowflake.
• Demonstrated history of designing, optimizing, and debugging production data pipelines.

Nice to have

• Prior experience in data analytics or data lake platforms.
• Familiarity with Delta lake or streaming tables.
• Exposure to machine learning concepts.

Culture & Benefits

• Competitive compensation and equity packages.
• Comprehensive health and wellness benefits including medical, dental, and vision coverage.
• Opportunity to contribute to cutting-edge AI research with a mission-driven team.
• Focus on transparency, craftsmanship, and community impact.