5 дней назад
Tech Lead Data (Databricks)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Tech Lead Data (Databricks): Leading the development of scalable, reliable data pipelines on Azure Databricks using PySpark and Spark SQL across a Medallion Lakehouse architecture with an accent on DataOps, governance, performance, and security. Focus on defining technical standards with the Data Architect, optimizing Spark workloads, implementing automated testing and observability, and mentoring Data Engineering squads.
Location: Remote in Brazil
Company
CI&T helps large enterprises turn AI and technology transformation into real business impact through AI deployment, application modernization, Data & AI, Martech, and business strategy.
What you will do
- Provide technical leadership to Data Engineering squads through architecture, code reviews, technical decisions, and delivery support.
- Define Lakehouse standards, coding practices, data modeling conventions, and technical references with the Data Architect.
- Design and oversee Databricks pipelines across Bronze, Silver, and Gold layers using PySpark and Spark SQL.
- Implement DataOps practices including Git, CI/CD, automated testing, data quality checks, and documentation.
- Optimize Spark jobs and pipelines for partitioning, performance, cost, reliability, observability, and alerting.
- Mentor engineers and collaborate with business and analytics stakeholders to deliver scalable data solutions.
Requirements
- Strong Data Engineering experience with prior technical leadership of teams and projects.
- Hands-on expertise in Databricks, PySpark, Spark SQL, batch pipelines, Delta Lake, and Medallion architecture.
- Experience with Azure data services, including Azure Databricks, Azure Data Lake Storage, and identity and security services.
- Strong Python and SQL skills, with knowledge of dimensional modeling, data contracts, and schemas.
- Experience with Git, CI/CD, automated testing, documentation, design and code reviews, and performance troubleshooting.
- Advanced English is required for stakeholder collaboration and technical documentation.
Nice to have
- Unity Catalog, Databricks Workflows, Databricks Repos, or Delta Live Tables.
- Great Expectations, Soda, dbt, Airflow, or observability practices.
- CDC or streaming ingestion with Event Hubs or Kafka.
- Consulting experience, international client exposure, or Databricks/Azure certifications.
Culture & Benefits
- Remote work arrangement in Brazil.
- Collaboration across a global organization operating in more than 25 countries.
- Benefits are provided through CI&T's Brazil careers program.
Hiring process
- Present an assessment during the selection process.
- Support is available for obtaining the assessment if needed.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
6 дней назад
Senior Big Data Engineer (Databricks)
9 дней назад
Databricks Data Engineer, Risk Technology (Azure Databricks)
120 000 - 160 000$
9 дней назад
Senior Data Engineer (Databricks)
8 дней назад
Senior (Lead) Data Platform Engineer (Databricks)
5 300 - 6 500€
9 дней назад
Senior Data Engineer (AI/Data Platforms)
9 дней назад