Remote Azure DataBricks Engineer

Summary

Join our European-American team as an Azure DataBricks Engineer to migrate pipeline's from SQL, Yellowbrick and Oracle for a financial services company in the USA. The project may also include real-time data integration tasks.

Requirements

• Documented solid experience as a Data Engineer
• Excellent knowledge of DataBricks and Apache Spark
• Excellent knowledge of Python
• Experience in data migration
• Experience working in the Microsoft Azure environment (e.g. Data Factory, Synapse, Logic Apps, Data Lake)
• Experience with SQL and Oracle
• Knowledge of YellowBrick or similar platforms
• Interpersonal and team skills
• Ability to take initiative and work independently
• English proficiency for fluent communication within the team

Responsibilities

• Planning and executing database migration
• Integrating databases in real-time
• Collaborating closely with data engineer, data scientist and architect teams

Preferred Qualifications

• Experience in designing and optimizing data flow using DBT, SSIS, TimeXtender or similar solutions (ETL, ELT)
• Experience with any big data or NoSQL platforms (Redshift, Hadoop, EMR, Google Data etc.)

Benefits

• Private medical care (Medicover)
• Multisport card for contractors