Lead Data Engineer (AWS/Spark)

Location: Brazil-based; home-based role with availability to travel to São Carlos/SP when needed

Company

Global data and technology company operating across financial services, healthcare, automotive, agribusiness, insurance, and other markets.

What you will do

• Design, implement, and evolve a petabyte-scale data platform on AWS.
• Build and optimize scalable, resilient data pipelines and services using JVM languages, Apache Spark, Python, and cloud-native services.
• Lead technical initiatives, contribute to architecture and design decisions, and align platform evolution with business strategy.
• Mentor engineers, conduct code reviews, and promote engineering standards and best practices.
• Ensure operational readiness through testing, monitoring, release procedures, and production support.
• Evaluate GraphQL integrations, vector databases, and AI/LLM-based techniques for data access and development efficiency.

Requirements

• Expert-level software and data engineering experience with large-scale data platforms.
• Deep experience with petabyte-scale systems, Java-based languages such as Scala, and Apache Spark.
• Strong expertise in AWS data services, including Glue, S3, Athena, Managed Airflow, and Iceberg.
• Advanced understanding of distributed systems, parallel workloads, query optimization, data partitioning, and efficient storage patterns.
• Experience influencing architecture, mentoring engineers, working in agile environments, and collaborating across engineering teams.
• Advanced English and availability to travel to São Carlos/SP when needed. Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.

Nice to have

• Experience with DBT or modern transformation frameworks.
• Knowledge of concurrent and parallel programming.
• Familiarity with Python, Angular, and TypeScript.
• Experience with vector databases such as pgvector or Redis vector fields.
• Practical experience applying AI/LLM techniques to data platforms, data access, or software quality.

Culture & Benefits

• People-centric, inclusive, and purpose-driven work environment.
• Focus on diversity, inclusive recruitment, and professional development.
• Affinity groups supporting LGBTQIAPN+ employees, racial equity, gender equity, people with disabilities, and generational inclusion.
• Work with an international organization employing more than 25,000 people across 32 countries.