The Opportunity
As a Senior Data Engineer you will drive the design and implementation of solutions for key data-centric projects and ongoing business conducted on our client's modern Data and Analytics Platform. Our client's work heavily relies on data, and data engineering is a critical part of their work. Their platform has been built with top tier tools and includes Azure Data Lake, Azure Data Factory, Databricks, Apache Spark, Aristotle, RStudio, Power BI and SAS.
The Role
The role’s responsibilities will include:
• Contribute your expertise to development of data engineering solutions for ingestion and ETL of data assets critical for our client's projects and ongoing business
• Establish a streamlined process for the formation of joined data and manipulation to deliver business ready data for use by the many teams
• Develop and lead data engineering solutions to join and transform data assets critical to delivery of their projects
• Review and refine existing engineering solutions on the their data and analytics platform
• Provide advice and guidance on risks and issues to non-technical managers and stakeholders
• Provide ad hoc advice on data governance issues to data governance team members
• Assist with data modelling on existing and business intelligence data assets to inform data architecture
• Develop, implement and review data-related infrastructure, processes, and procedures
• Work closely with our application, infrastructure and networking partners on development of infrastructure to support data and analytics projects, and integration between cloud and on-premise hosted services
• Work under limited direction and be accountable for undertaking planning, analysis, design, development, and delivery activities within tight timeframes
• Lead a small, technically-focussed team of Data Engineers (note: this team has not yet been established, however the additional resources are likely to be engaged as their uptake of the Data and Analytics platform increases.)
Your Background and Experience
The ideal candidate will have:
• Bachelor's degree or equivalent combination of education and experience.
• Understanding of Machine Learning Concepts
• Strong knowledge of database structure security systems, data analytics, data mining and data modelling techniques
• Demonstrated ability to combine multiple large and complex datasets to deliver high-quality datasets
• An applied understanding of UI design concepts.
• Well-developed team collaboration and communication skills
• Demonstrated ability to provide advice and develop solutions within multi-disciplinary work environments
• Demonstrated ability to communicate complex technical concepts to non-technical staff
• Demonstrated experience in:
o Developing, configuring, validating, managing Microsoft cloud platforms, Data Factory, SQL Server, Databricks
o DevOps and/or DataOps
o Documented, Hands-on ETL development using tools like SQL Server Integration Services or Azure Data Factory
o Data modelling
o Data analysis and management, with excellent analytical problem-solving abilities
o Coding languages (such as SQL, Python, R, C Sharp/C++, Powershell)
o System administration
o Data lake management from design, development, delivery, validation and ongoing maintenance
o Using and developing engineering solutions for data visualisation tools such as RStudio, Power BI, SAS, Qlik, Tableau.
o Deployment and maintenance of machine learning models
o Developing datasets API delivery for a variety of scenarios
o Machine learning technologies
o Apache Spark (Scala or Python)
o Spark SQL coding
o Automation
o End-to-end Machine Learning pipeline
o Proven experience working on Data Science projects
Salary / Rate
Hourly rate negotiable
How To Apply
To apply online, please click on the appropriate link. If you have already filled out our Candidate Registration Form for another role previously, please contact us on careers@xaana.ai.
Alternatively, for a confidential discussion, please contact Rita Zhang on 0493 364 203 or rita.zhang@xaana.ai.
Xaana ARIMA is committed to building a diverse and inclusive workforce and we encourage applications from people of all cultures, capabilities and backgrounds."
