Lead Data Engineer — Melbourne, Melbourne Region

Lead Data Engineer required to support the development of an Azure Databricks platform for a leading financial services business. Responsibilities: design and development of pipelines using Databricks and Lakehouse optimisation and maintenance of data workflows, ensuring quality and integrity performance tuning and monitoring Notebook development using Python and/or Pyspark implementation of best practices for data engineering, including governance and security mentoring and coaching of junior Data Engineers Requirements: extensive experience building data pipelines in a Databricks Lakehouse environment well-versed with Spark and other big data technologies excellent coding skills using SQL and Python strong background developing solutions in an Azure Cloud environment, including strong ADF development skills Click on the 'Apply' button to submit your CV.