Data Engineer

Data Engineer

 Abu Dhabi, United Arab Emirates

 

Permanent

Job Title: Data Engineer

About the Role:

We are seeking an experienced Data Engineer to join a dynamic team responsible for developing and optimizing data processing pipelines, analyzing new data sources, and building a centralized data platform. You will be working with complex biological, bioinformatics, and scientific data, and collaborating with experts to ensure the integration of AI and ML into the data pipelines.

Key Responsibilities:

• Develop and maintain data processing pipelines to integrate multiple data sources.
• Collaborate with subject matter experts to model and load data into Data Vault entities.
• Conduct exploratory data analysis to support insights and decision-making.
• Research and develop pilots and MVPs for new technologies and solutions.
• Create and manage data marts to facilitate easier access and analysis.
• Ensure data quality, data lineage, and overall platform optimization.
• Integrate NLP for the markup and linking of scientific data.

Requirements:

Education:

• Bachelor’s, Master’s, or PhD in Computer Science or a related field.

Experience and Skills:

• Strong Python skills for data engineering and data science tasks.
• Experience with ETL pipelines and data integration.
• Familiarity with Continuous Integration and Continuous Delivery tools (e.g., GitLab).
• Experience integrating ML models into pipelines, including human-in-the-loop solutions.
• Strong experience creating complex orchestration patterns in Airflow.
• Knowledge of data warehouse methodologies, specifically Data Vault 2.
• Proficiency with SQL and experience with both relational and NoSQL databases (e.g., PostgreSQL, MongoDB, ElasticSearch, S3).
• Familiarity with Linux environments and Amazon Web Services (AWS).
• Strong communication skills (both written and oral) in English.

Bonus Points:

• Bioinformatics background.
• Experience with NLP techniques.