This is a Engineer - Data Platforms role with one of the leading companies in AU right now Westpac with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off. More About the Role at Westpac The Data Platforms (DP) function comprises of a team of data professionals dedicated to building game-changing data capabilities for Westpac Group and shaping the future of banking. We understand the big picture and focus on and deliver initiatives that will significantly and positively impact the Westpac Group. In this hands-on role, you will be a strong technical expert, responsible for building real-time and batch data solutions. You will handling multiple programs concurrently and be a go-to SME for the development of engineering solutions. Responsibilities include: - Enhancement, maintenance, and support of the technical deliverables including hands-on coding, ensuring the quality of team outputs, representing the team in various forums, and aligning team contributions with the overall strategic roadmap. - Design and implement scalable, reliable, and secure solutions, and collaborating with cross-functional teams to gather requirements and ensure successful delivery. - Collaborate with fellow Engineers across the domain to define technical direction for service offerings, and work with Service Owners and key business stakeholders to shape the delivery roadmap and coordinate with support teams to ensure smooth delivery. What do I need? - Solid engineering experience developing large-scale applications using Scala and Python, plus handling large datasets using PySpark in numerous production workloads. - Proven expertise in real-time streaming jobs, time data processing and pipelines in production environments. - Extensive experience in optimising Spark code using Directed Acyclic Graphs (DAGs) involving several strategies to enhance performance and efficiency. - Experienced working with HDFS, pig, hive, HBase, Phoenix etc. - Proven experience and understanding of cloud platforms, particularly within Azure, having worked with ALDS, EventHub, Stream Analytics, HDInsight, Synapse Analytics, Data Factory, Cosmos DB, Kubernetes. Similar experience with AWS/GCP is also fine. - Strong functional programming and ability with complex data structures, with understanding and experience in data warehousing concepts. - Expertise in DevOps functions, contributing to CI/CD pipelines, and experience in implementing MLOps pipelines in production is advantageous. - Proficient in using source control tools, preferably Bitbucket, with a good understanding of branching and merging strategies. This role is part in the UNITE program which will be Westpac’s biggest transformation project ever. It is a business-led, tech-enabled transformation - delivering simplification and benefits for our customers, employees and shareholders. A major priority for the Group over the next 3 to 4 years, UNITE is about delivering ‘one best way’ to make our processes, systems and tech simpler so we can deliver a better experience for customers, make things easier for our people, and reduce risk for generations to come. If you don’t think you're a perfect fit, you should still sign up to Hatch and create a profile, we'll match you to other roles that suit your profile. Hatch exists to level the playing field for people as they discover a career that’s right for them. We model this in our hiring process for our partners like Westpac. ✅ Applying here is the first step in the hiring process for this role at Westpac. We do not discriminate on the basis of gender identity, sexual orientation, cultural identity, disability, age, or any other non-merit factors. To put it simply, Hatch is for everyone.
Visit engineeringsuper.com.au to find our PDS and TMD.