Lead Data Engineer — The Rocks, Sydney

Expired

Lead Data Engineer Sydney, Australia $220,000 (including Super) Bonus About the Role: Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency. Key Responsibilities: Lead the development and transformation of the regulatory reporting platform. Collaborate across business and technical teams to deliver complex data solutions. Implement and optimize data integration pipelines and data warehouse solutions. Oversee the design and architecture of large-scale data solutions using AWS services. Drive platform efficiency through automation and AI enhancements. Ensure data governance and compliance with internal and external regulations. Provide leadership and mentorship to data engineers, fostering a culture of innovation and excellence. Required Skills and Experience: Proficient in Spark, Hive, Presto, and big data storage formats (Parquet, ORC, Avro). Experience with data pipeline orchestration tools such as Airflow. Hands-on experience with data integration tools (Talend, DataStage, or Informatica). Strong AWS experience, including EC2, S3, RDS, and EMR. Expertise in data modeling techniques and data architecture best practices. Knowledge of data lineage, technical metadata, data quality, and reconciliation processes. Advanced skills in Spark SQL and Python for data automation. Ability to tune and optimize data queries for performance. Strong communication and negotiation skills. Proven experience in leading cross-platform data solutions and complex projects. Ability to mentor and lead technical teams. If you feel you possess the relevant skills to the above, please respond with most updated CV