Senior Data Developer (Databricks)

Location: Brazil; Workplace: Hybrid

Company

CI&T helps large enterprises turn AI potential into business impact through AI deployment, AI-native execution, and technology-integrated business solutions.

What you will do

• Build Databricks ingestion pipelines using medallion architecture with bronze, silver, and gold layers, Auto Loader, and Delta Lake.
• Implement before-and-after reconciliation pipelines with record counts, field-level comparisons, key and table mapping, and large-scale validation rules.
• Model the gold layer for reporting on approval and rejection rates, outcome reasons, and views segmented by cluster and object.
• Expose results through an API for a web application using Databricks SQL Warehouse with the Statement Execution API or a custom service, and document the API contract.
• Configure Unity Catalog governance, including permissions, data lineage, and auditing for client production data.
• Orchestrate workflows with Workflows/Jobs, implement data quality checks such as DLT expectations, and manage version control with Databricks Asset Bundles.

Requirements

• At least 2 years of experience specifically using Databricks, including PySpark, Delta Lake, Unity Catalog, and Workflows.
• Experience in data engineering.
• Advanced proficiency in SQL and Python.
• Experience with data reconciliation or validation in migration projects.
• Knowledge of SAP data structures.
• Experience developing reports based on system data.
• Advanced English for conversation.

Culture & Benefits

• Hybrid work in Brazil.
• Work in a global organization of 8,000 employees across more than 25 countries.
• Benefits are provided as part of the CI&T employment offering.
• An assessment presentation is part of the selection process; support is available for obtaining the assessment if needed.

Hiring process

• Present an assessment during the selection process.
• Support is available for candidates who do not yet have the assessment.