Senior Software Engineer (Big Data)

TL;DR

Senior Software Engineer (Big Data): Process and analyze large datasets to produce housing market indicators and forecasts with an accent on designing robust data pipelines and optimizing infrastructure for scalability. Focus on collaborating with data science teams, developing forecasting models, and enhancing data products for real-time and batch processing.

Location: Work remotely from any location within the U.S.

Company

Company impacting the housing market through advanced data engineering and analysis for better policy and economic decision-making.

What you will do

• Process and analyze large datasets to produce housing market indicators and forecasts.
• Collaborate with data science teams to enhance data products.
• Design and implement robust data pipelines for real-time and batch processing.
• Optimize data delivery processes and infrastructure for scalability.
• Develop and maintain internal forecasting models and frameworks.
• Assist stakeholders in data investigation and model support.
• Create and maintain documentation for data systems and processes.

Requirements

• 5+ years of experience in Big Data Engineering or Machine Learning Engineering.
• Proficiency in Python programming language.
• Hands-on experience with big data ecosystems and AWS cloud services.
• Strong ability to design and orchestrate data pipelines.
• Familiarity with SparkSQL, Spark Streaming, PostgreSQL, and Kubernetes.
• Understanding of Agile/DevOps processes and tools like GitLab.
• Strong problem-solving, analytical, and communication skills.
• Familiarity with machine learning algorithms and concepts.

Nice to have

• Experience with Databricks.

Culture & Benefits

• Collaborative and supportive team environment.
• Access to cutting-edge technologies and resources.
• Opportunities for professional development and growth.
• Flexible work hours and autonomy in your role.
• Inclusive and innovative company culture.
• Competitive salary with opportunities for equity awards.