Senior Data Engineer — Sydney CBD, Sydney

Role overview: You will be joining one of Australia's largest financial institutions, within the fastest paced financial markets area of the business. Working as part of a new team of highly skilled engineers, you will be working on building a new regulatory audit response capability, building controls around validating rates in trading. As a Senior Data Engineer, you will play a crucial role in developing and maintaining the data infrastructure, ensuring the seamless integration and analysis of vast amounts of data. Your expertise will directly contribute to market-leading strategies and decision-making processes. Day to day you will Develop and maintain scalable, reliable, and secure data pipelines and architectures. Design, implement, and optimize complex SQL queries and Data Definition Language (DDL) operations. Utilize your strong hands-on coding ability in Python to build robust data solutions. Deploy and manage applications in a clustered environment using Docker and Kubernetes. Leverage AWS services to enhance our data processing and storage capabilities. Ensure seamless integration with modern systems through API-based solutions. Query and analyze big data using tools such as Redshift, Hive, Spark, and Presto. Implement and manage data pipeline orchestration tools like Airflow or Argo Workflows. Utilize source control tools like Bitbucket to manage and collaborate on codebase effectively. Apply your knowledge of Linux/Unix systems to maintain and optimize our data infrastructure. We are looking for 7 years of hands on Python coding experience working on complex applications Experience querying big data with tools like Redshift, Hive, Spark, and Presto, orchestrating data pipelines with Airflow or Argo Workflows Strong SQL experience, authoring complex queries and Data Definition Language (DDL) Experience within Linux/Unix environments Experience working in cloud environments, ideally AWS Experience with CI/CD tools Working knowledge of DevOps principals and best practices AU PR or Citizenship candidates only Sydney based candidates only, Hybrid model (3 days a week in CBD office) Sound like an interesting next move, and have the skills to suit? Apply now