NXT GIG is seeking a skilled Big Data Engineer to join our passionate data team. In this role, you will be responsible for designing and implementing scalable big data solutions that enable comprehensive data analysis and insights. You will work with large datasets using technologies such as Hadoop, Spark, and Kafka to create robust data pipelines and ensure efficient data processing workflows. Collaborating closely with data scientists and analysts, you will translate complex business requirements into effective data solutions. If you are excited about working with big data technologies and driving impactful data-driven decisions, we encourage you to apply and join us at NXT GIG. Responsibilities Design, develop, and maintain scalable big data systems and architecture using Hadoop, Spark, and related technologies. Build and optimize ETL processes to facilitate data extraction, transformation, and loading from various sources. Collaborate with stakeholders to gather requirements and develop big data strategies that meet business needs. Monitor and troubleshoot data processing and ensure data quality throughout the pipeline. Develop documentation for data models, data sources, and data processing workflows. Stay informed about new technologies in the big data ecosystem and advocate for best practices. Provide support and training for data analysts and other users in utilizing big data tools and solutions. Requirements Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field. Proven experience as a Big Data Engineer or in a similar role with a strong focus on big data technologies. Strong knowledge of Hadoop ecosystem components (HDFS, MapReduce, Hive, Pig) and Spark. Proficiency in programming languages such as Java, Scala, or Python. Experience with data modeling, ETL processes, and data warehousing concepts. Familiarity with real-time data processing tools such as Apache Kafka or Flink. Excellent analytical skills, attention to detail, and ability to work collaboratively in a team environment.
Visit engineeringsuper.com.au to find our PDS and TMD.