Data Engineering Lead

Summary

Join Lendi Group, a leading digital platform revolutionizing the property journey, as a Data Engineering Lead in Sydney. Lead a team of data engineers, developing and managing a modern data stack using technologies like Snowflake, Databricks, Python, and dbt. Responsibilities include data pipeline development, cloud data warehouse management, Power BI semantic model development, and ensuring data governance and security. You will collaborate with cross-functional teams, optimize data platforms, and implement best practices. The ideal candidate possesses 8+ years of data engineering experience with modern development practices and proficiency in SQL and Python. This role offers a flexible hybrid work structure, extended paid parental leave, and a people-first company culture.

Requirements

• Minimum of 8 years of data engineering experience with modern development practices
• Proficiency in SQL with a cloud data warehouse (Snowflake, AWS RDS, Databricks) and experience with ETL tools and processes
• Experience with programming languages such as Python
• Strong understanding of data warehousing concepts and data modelling techniques
• Knowledge of data governance and data quality best practices
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills
• Ability to work independently and manage multiple priorities in a fast-paced environment

Responsibilities

• Work closely with a cross-functional agile team to deliver high-quality data solutions
• Work closely with data analysts, data scientists, and other stakeholders to understand business requirements and translate them into technical solutions
• Mentor and guide a team of data engineers and analytic engineers (local and offshore), fostering a culture of innovation, collaboration, and engineering excellence
• Data pipeline development, maintenance and engineering support with modern data stack in a rapid, agile development practices
• Cloud data warehouse – development, testing and production deployments; Python, Databricks, Snowflake SQL and administration (mandatory), dbt (mandatory), Airflow (mandatory), Kafka (desirable), Segment (desirable), Terraform (desirable) AWS fundamentals (mandatory), Buildkite (or equivalent CI/CD platform)
• Lead the development and management of PowerBl semantic models, ensuring strong governance, repeatability, and scalability
• Provide standards, guidelines and collaborate in design of logical data modelling and the implementation of physical data models into the platform to ensure consistency, interoperability, maintainability and efficiency
• Management of data platform budget, ensuring optimisation of costs and vendor management
• Optimize data pipelines, build pipelines & BI solutions for performance and scalability
• Implement data engineering best practices, design, testing, documentation and maintenance
• Management of the data platform access processes, security access controls, role & attribute access controls and data security controls in collaboration with Cyber and Platform infrastructure teams to ensure that the platform is secure by design
• Management and provisioning of AI and BI services on the data platform
• Ensure compliance with data governance policies and procedures
• Monitor data quality and address any issues related to data completeness and accuracy
• Provide input to solution design & technical support of pipelines
• Maintain comprehensive documentation of data engineering solutions, data models, and processes
• Contribute to design guides and training materials as needed
• Provide support for issue and incident resolution

Preferred Qualifications

Kafka (desirable), Segment (desirable), Terraform (desirable)

Benefits

• Enjoy the flexibility of remote working arrangements
• We offer a highly flexible hybrid structure to suit your individual needs
• We offer extended Paid Parental Leave: we celebrate our growing Lendi Group family with up to 26 weeks leave for primary carers and up to 4 weeks for secondary carers
• An additional week’s Loyalty Leave each year after reaching 3 years’ service
• We are a people-first company that invests in initiatives to promote a healthy work environment, such as offering an Employee Assistance Program and social groups such as our industry-leading Women in Technology group