Senior Azure Data Engineer

Senior Azure Data Engineer

 Abu Dhabi, United Arab Emirates

 

Contract

Seeking a highly skilled Senior Engineer with extensive experience in data engineering to join our team.

The ideal candidate will have a strong foundation in Databricks, PySpark, Azure Data Factory (ADF), and query optimization techniques, with a particular focus on building metadata-driven ETL pipelines.
This role will be instrumental in designing, developing, and maintaining robust data pipelines that can adapt to evolving data landscapes.
 

• Design, develop, and maintain scalable, metadata-driven ETL pipelines using Databricks, PySpark, and ADF.
• Create and manage data catalogues and metadata repositories.
• Implement automated data lineage and impact analysis to ensure data quality and traceability.
• Optimize query performance and data processing efficiency to meet stringent SLAs.
• Collaborate with data analysts and scientists to understand data requirements and translate them into technical solutions.
• Troubleshoot and resolve data-related issues promptly.
• Mentor and guide junior team members in data engineering best practices.

Essential Skills

• Azure Databricks, PySpark, ETL
• Nice to Have Skills.
• ADF, Data Modelling