Senior Data Engineer (IT) — Brisbane, Brisbane Region

Expired

Our esteemed client has embarked on a large program of work, that will see a transformational change for the business across many aspects of their technology landscape. One of the streams is for data, analytics and reporting and is responsible for implementing a modern data platform. Business intelligence solutions and associated technologies to enable integration, reporting and analytic outcomes. The Senior Data Engineer will be responsible for all aspects of the development lifecycle for the new Data Platform. The role will assist in providing technical leadership within the project team and work collaboratively with operations teams and other data engineering teams in Digital as well as stakeholders across the Retail business. The successful applicant will work closely with the platform Solution Architect on a modern data and analytics solution using technology powered by Databricks within the AWS Cloud. Role Objectives Provide technical leadership and coaching to the data engineering team; Lead in the transition of technical solutions to operations teams of varying experience; Work with Business Analysts, Solution Architects and Data Modellers to understand data platform requirements; Contribute to the development of frameworks to support automation architecture for the platform; Develop metadata driven development pipelines within the Databricks platform using Python and other related tools and technologies; Configure and maintain Databricks resources such as clusters, catalogues, schemas and security models; Conduct performance analysis and undertake required optimisation to enhance user experience and system-wide performance; Undertake unit testing for their own development and provide peer review to others as part of internal quality assurance; Develop and maintain CI/CD across all environments for fully automated deployments; Prepare, maintain and review technical documentation as well as maintain best practices in code management and code branching; Actively participate in team design and planning sessions as part of the agile sprint process and proactively provide updates and communicate risks or impediments; and Provide guidance to business users on best ways to utilise platform features. The role requires someone with extensive experience in developing data and analytics solutions in Databricks with integrated data management and data governance capability. Technical Requirements The role is suited to someone who has: At least 8 years' experience in a data related role, with at least 5 years using Databricks in a data warehousing or analytics capacity; Extensive experience in metadata driven development methodologies and automation frameworks; Experience in data warehouse performance optimisation; Experience in working with APIs for data ingestion and sharing; Experience at developing data quality frameworks; and Experience at designing and implementing CI/CD for automated deployments. Reporting Structure The role will report to Digital Lead - Project Phoenix If this seems like an opportunity you might be interested in, please call Amy McAuley at u&u on 07 3232 9116 quoting reference number 34111. Alternatively, to submit an application, please click below. Please submit your resume in Word format only.