Data Engineer

Summary

Join Sunscrapers, a leading software development company, as a Data Engineer in Warsaw or remotely. Advance your career building a robust data platform for leading healthcare brands. You will leverage cutting-edge technologies like Apache Spark and Databricks to design and implement scalable data solutions. This role requires significant experience in data engineering, strong technical skills, and a commitment to data governance. Sunscrapers offers a competitive compensation package, flexible working hours, remote work possibilities, and a supportive work environment.

Requirements

• At least 3 years of professional experience as a data engineer
• Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar
• Excellent command of spoken and written English (at least C1)
• Experience in designing data infrastructure using Python, PySpark, Apache Spark, and Delta Spark
• Experience in managing production level Databricks clusters
• Proficiency in SQL and experience with Data Lakehouse
• Great analytical skills and attention to detail - asking questions and proactively searching for answers
• Creative problem-solving skills
• Great customer service and troubleshooting skills

Responsibilities

• Design and optimize data infrastructure using Python, PySpark, Apache Spark, and Delta Spark
• Implement strong data governance frameworks to ensure quality, security, and compliance
• Connect Delta Tables to a SQL engine (like Databricks SQL) for efficient querying and analytics
• Leverage strong DevOps expertise to deploy and maintain data systems in Azure
• Create batch and streaming pipelines for data processing

Preferred Qualifications

• Familiarity with CI/CD pipelines, containerization (Docker) and orchestration (Kubernetes)
• Experience with real-time data tools like Kafka or Azure Event Grid
• Experience with BigQuery
• Experience in managing data governance in the healthcare space

Benefits

• Flexible working hours and remote work possibility
• Comfortable office in a penthouse in central Warsaw equipped with all the necessary tools to conquer the universe (Macbook, external screen, ergonomic chairs)
• Fully equipped kitchen with fruit, hot and cold drinks
• Multisport card & Private medical care
• Culture of good feedback: evaluation meetings, mentoring