Data Engineer — Canberra Region, Australian Capital Territory

Expired

12 12 12 month contract $120 - $145 per hour (inc-Super) ACT, NSW, VIC, QLD or SA office location NV1 Security Clearance Mandatory Our client, a Federal Government agency, who focuses on law enforcement and criminal treat intelligence, are seeking multiple Data Engineer (s) for long 121212 month contracts based in ACT, NSW, VIC, QLD or SA. Duties of the Data Engineer Design, build, and maintain scalable and efficient data pipelines to process large datasets from multiple sources. Implement and manage databases that store and organise data, optimising performance and ensuring data integrity. Integrate data from various sources, including APIs, flat files, and databases, into a centralised data repository. Create and maintain data models to support business needs, defining relationships between different data entities. Implement processes to ensure data accuracy, consistency, and security, including data cleansing and validation. Identify and address performance bottlenecks in data processing systems, optimising queries and pipelines. Leverage cloud-based platforms and big data technologies to build and manage large-scale data processing systems. Develop scripts to automate data processing tasks, streamline workflows and reduce manual intervention. Maintain detailed documentation of data systems, processes, and pipelines to ensure they are easily understood and maintained. Skills and Experience Required Demonstrated experience in designing, implementing and maintaining scalable and efficient data pipelines to process and integrate large datasets from multiple sources, including experience with performance optimisation. Demonstrated experience in creating and maintaining data models to support business needs, and ability to define relationships between different data entities. Experience with designing, building and maintaining a data analytics ecosystem / platform to support analytics and data science teams. Experience with Relational, Graph, Document, and Big Data Database Management Systems (DBMS). Experience with data modelling concepts for the various categories of DBMS. Proficiency in programming languages such as Python, SQL, Java, C#, or Scala. Experience with Extract, Transform, Load (ETL) tools, Data Warehousing, and Analytics. Experience with any of the following: Containerisation and orchestration tools (e.g., Docker, Kubernetes). Real-time data processing and streaming technologies, and data integration tools (e.g. Apache Kafka, Apache Nifi). Cloud-based platforms. Big Data technologies (e.g., Elasticsearch, Apache Spark, Databricks). CI/CD pipelines and infrastructure-as-code tools (e.g., Ansible). DataOps and MLOps. How to Apply Please forward your resume to Ben Cousins at Recruitment Hive by clicking the Apply button, or email your resume to info RecruitmentHive.com.au citing Job ID: BC9896. Alternatively you may call Ben on (02) 6299 1006. Applications close 18/9/2024.