Middle Data Engineer (Databricks/Python/Terraform)

#9338
Middle+ Data Engineer id 101
Грейд: Middle+
Языки: разговорный английский
Старт: asap, до конца года
Локация: РБ под ВПНом, ЕС + обязательное наличие мака для работы

Responsibilities:
• Develop, deploy, and maintain data pipelines and data governance systems on Databricks (Unity Catalog, Delta Lake, Workflows)
• Conduct in-depth data analysis, profiling, and processing to classify data by domain and detect personal data (PII)
• Contribute to the design of data access systems, focusing on data contracts, catalog access, and grant/rollback automation managed as Infrastructure-as-Code with Terraform
• Collaborate with peers to review and ensure access architectures and contracts are effective, flexible, and reusable
• Ensure that data products and access controls are deployed according to contracts and operate without unintended side effects
• Create and maintain comprehensive documentation for data contracts, access models, and governance processes
• Monitor and support deployed pipelines and quality checks to ensure they meet data quality and performance expectations
• Actively seek out information and internal solutions to ensure reusability and adoption of common data platform technologies

Requirements:
• 3+ years of experience in data engineering. Previous software engineering experience is preferable
• Strong proficiency in Python and its common data processing frameworks (e.g., Spark/PySpark for streaming, batch, and asynchronous data)
• Strong foundational knowledge in data modeling, distributed data processing, SQL, and data warehouse/lakehouse architecture
• Hands-on experience provisioning cloud data platforms and infrastructure (Databricks or GCP or AWS) with Terraform is a must, along with a willingness to learn new technologies for integration purposes
• The ability to design data access and governance solutions using common design patterns and various design tools
• A solid understanding of the business value of delivered data products and governance controls
• Strong problem-solving skills, with an emphasis on using technical facts and reasoning
• Excellent communication and collaboration skills, including active listening and the ability to build consensus

Писать Показать контакты
Канал Аутстаф запросов @outstaff_requests_phpdev