Verbinden...

W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9yzwqty29tbwvyy2uvanbnl2jhbm5lci1kzwzhdwx0lwrllmpwzyjdxq

PySpark Developer (Inside IR35)

Standort: London, England Gehalt: £350 - £400 per day
Bereich: Consultancy Bereich: Freiberufler
Reference #: CR/080963_1623052733

PySpark Developer (Inside IR35) / London or Dublin / 6 months / Start ASAP

Key responsibilities:

* Fundamentals of Spark using the Dataframe API
* Understanding partitioning of data
* Analysing and performance tuning Spark queries e.g. looking at the DAG
* Knowledge of Hadoop and its ecosystem of technologies especially Hive Python, OOP concepts using Python
* Knowledge of Conditional Statements & Loops: If-else Control Structures, For/While Loops
* Demonstrate a comprehensive understanding of Complex Data Types: Shallow & Deep Copies, Working with Lists & Tuples, Dictionaries & Sets
* Understand Fundamental Data Structures & their Implementation
* Good knowledge of Exceptions & Command Line Arguments
* Contributes to quality assurance by writing unit and functional tests.
* Ensures development happens for all Software Components in accordance with Detailed Software Requirements specification, the functional design and the technical design document.
* Basic knowledge of UNIX
* Demonstrate source control knowledge (preferably GIT)
* Ability to analyse databases directly using query language tools such as SQL
* Experience on ETL process on Big Data
* Have an understanding of data relationships, normalisation