Data Engineer

Summary

Join our team as a Cloud Data Engineer to develop on our data lake platform and applications on the Azure cloud. This role requires strong data engineering, data modeling, and SQL skills, along with Python programming experience. You will design and develop solutions for cloud applications, ensuring data integration with various services. The position demands hands-on experience with Azure data services, Python, SQL, and ETL processes. You will participate in Agile ceremonies and collaborate with the team to deliver cross-functional products. Documenting project artifacts and maintaining knowledge of industry best practices are also key responsibilities.

Requirements

• Bachelor's degree in CS
• 7 years of hands-on experience in designing and developing distributed data pipelines
• 5 years of hands-on experience in Azure data service technologies
• 5 years of hands-on experience in Python, SQL, Object oriented programming, ETL and unit testing
• Experience with data integration with APIs, Web services, Queues
• Experience with Azure DevOps and CI/CD as well as agile tools and processes including JIRA, confluence

Responsibilities

• Understand requirements and engage with team to design and deliver projects
• Design and implement data lake house projects within azure
• Design and develop application lifecycle utilizing Microsoft Azure technologies
• Participate in design and planning and necessary documentation
• Participate in Agile ceremonies including daily standups, scrum, retrospectives, demos, code reviews
• Hands on with Python/SQL development and Azure data pipelines
• Engage with team to develop and deliver cross functional products
• Document and maintain project artifacts
• Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices
• Complete training as required for Privacy, Code of Conduct etc
• Promptly report any known or suspected loss, theft or unauthorized disclosure or use of PI to the General Counsel/Chief Compliance Officer or Chief Information Officer
• Adhere to the company's compliance program
• Safeguard the company's intellectual property, information, and assets
• Other duties as assigned

Preferred Qualifications

• Key Skills a. Data Engineering and SQL
• Key Skills b. Python
• Key Skills c. PySpark
• Key Skills d. Azure Data lake and ADF
• Key Skills e. Databricks
• Key Skills f. CICD
• Key Skills g. Strong communication