Senior Data Engineer

Standort: Manchester, Greater Manchester Gehalt: £350 - £375 per day
Bereich: Consultancy Bereich: Freiberufler
Reference #: CR/076086_1607685479

Senior Data Engineer / Manchester / 4-6 months / Start ASAP


* Design, Develop, Test and maintain efficient, scalable data pipelines and ETL process
* Lead the tuning and optimisation of existing data pipelines and ETL processes
* Conduct code reviews and make sure the best practices are followed and the pipelines are aligned to standards
* Translate customer requirements to develop effective, robust data solutions in line with customer Data Engineering standards
* Own the code base, release management process and planning to ensure confidence and coverage
* Continuously implement, promote and improve our ways of working, defining best practice, frameworks and processes, working in close collaboration with other team members and Architects

Key Skills/Knowledge:

* Azure Data Factory, Azure Databricks, Azure SQL Dataware house / Azure Synapse
* Performance Tuning and ETL optimisation
* Teradata

Experience required:

* Must have experience in developing, building and deploying solutions based on Microsoft Azure SQLDW , TSQL, Databricks, Data Lake, Azure Data Factory and Blob Storage
* Experience in building pipelines and migrating large volumes of data into Azure SQL Data Warehouse / Azure Synapse or SQL Server BI Toolset (SRSS,SSIS,SSAS) and resolving performance issues.
* Good understanding of Full Databricks Knowledge and Understanding for ETL. (PySpark, SparkSQL), Databricks Delta, Cluster Creation and Maintenance
* Knowledge of Access Controls (Data Lake or Blob, RBAC, ACLs)
* Very good understanding of capabilities of Azure SQL Data Warehouse/Azure Synapse and best practices to be followed
* Demonstrable experience and understanding of end to end data delivery techniques including data modelling, ETL building, Quality Assurance and performance tuning
* Experience in designing and building cloud based ETL/ELT pipelines and data migrations and dealing with high volume systems
* Good understanding of monitoring and tuning data pipeline workloads, preferably in Azure Synapse
* Experience in dealing with different data schemas (including Microsoft SQL Server i.e. MSSQL, NoSQL, API Based) is required
* Experience in agile delivery methodologies, CI/CD practices