Senior Data Engineer (Azure Databricks)

TL;DR

Senior Data Engineer (Azure Databricks/Python): Designing and maintaining scalable Spark applications and data pipelines for a global professional services network with an accent on performance optimization and microservices architecture. Focus on building robust data processing workflows, implementing Delta Tables, and ensuring high code quality within an AI-driven tool ecosystem.

Location: Fully Remote. Must be based in Kazakhstan, Uzbekistan, Romania, Poland, Georgia, or Bulgaria

Company

hirify.global is a talent gateway providing career opportunities within world-leading professional services networks, currently recruiting for one of the Big Four accounting firms.

What you will do

• Design, develop, and maintain robust and scalable Spark applications.
• Define and enforce coding standards and conduct thorough code reviews to ensure high quality.
• Optimize code for performance and scalability, ensuring efficient data handling.
• Collaborate with cross-functional teams to deliver high-quality software solutions.
• Communicate regularly with the client and manage the flow of day-to-day operations.
• Create and maintain comprehensive documentation for code, processes, and workflows.

Requirements

• 5+ years of hands-on experience in software development.
• Extensive experience with Apache Spark, including Databricks and/or Azure Synapse/Fabric.
• Proficiency in Python, with strong skills in data manipulation using Pandas or Polars.
• Solid understanding of columnar storage formats (Parquet) and practical experience with Delta Tables.
• Solid understanding of microservices architecture and its implementation in scalable systems.
• English: Intermediate+ (B2) proficiency required.

Nice to have

• Experience with Azure Cloud services (Service Bus, Data Lake, Blob Storage, Redis).
• Familiarity with FastAPI.
• Expertise in containerization and orchestration tools such as Docker and Kubernetes.

Culture & Benefits

• Fully remote work arrangement.
• Opportunity to work with a modern tech stack including LangChain, RAG pipelines, and Multi-modal LLMs.
• Engagement with a world-class professional services network.
• Collaborative environment with a focus on excellence and innovation.