Lead Data Engineer — Sydney CBD, Sydney

Expired

Lead Data Engineer | Permanent | Databricks | AWS Permanent hybrid Databricks and AWS Hybrid Excellent opportunity for a Senior/Lead Data Engineer to join our growing data consultancy with reputable clients, providing a collaborative and inclusive culture, and clear pathways for growth. You will have strong hands-on experience with Databricks, AWS, and associated migrations and integration of data. Along with, a thorough understanding of data warehousing methodologies and the ability to enhance data quality and documentation processes. Experience Proven experience with Databricks, particularly in migration and integration projects. Strong background with AWS and cloud-native data architectures. Solid experience working with Microsoft stack technologies (SSIS, SSRS, SSAS, Power BI). Experience with CI/CD practices and tools for data pipeline automation ideally using GitLab or similar. Familiarity with data lakes, data warehousing, and orchestration tools (e.g., Apache Airflow). Expertise in Apache Spark (or similar) and scalable data processing. Strong understanding of data governance, security, and compliance. Excellent problem-solving skills and ability to lead complex projects. Experience in data mapping activities, documentation, and coaching peers. Strong collaboration skills to work with architecture teams and other stakeholders. Terraform experience is nice to have Experience in C Data Sync is nice to have Experience with Kimball methodology and Data Mesh is nice to have Responsibilities Lead the migration of data and processes to Databricks on AWS. Coach & Mentor current team – new stack (Databrick/AWS) Collaborate with architecture teams to support design activities and ensure a smooth transition. Apply the Kimball methodology to data warehousing and enhance data quality. Execute and manage data mapping activities to ensure accurate data migration. Develop and maintain comprehensive documentation and provide coaching/guidance to peers. Work on maturing data quality processes and implementing best practices within the organization. Troubleshoot and resolve issues related to data integration and migration. Benefits Inclusive work culture with a strong focus on employee contributions Ongoing professional development, including free technical certifications Get 20% more annual leave Your birthday day off Anniversary gifts