Mid Big Data Engineer

Summary

Join Capco Poland, a global technology and management consultancy, as a skilled Mid Big Data Engineer to design, implement, and maintain scalable data pipelines and solutions on-prem/cloud projects for large-scale data processing and analytics.

Requirements

• Min. 3-4 years of experience as a Data Engineer/Big Data Engineer
• University degree in computer science, mathematics, natural sciences, or similar field and relevant working experience
• Excellent SQL skills, including advanced concepts
• Very good programming skills in Scala
• Experience in Spark and Hadoop
• Experience in OOP
• Experience using agile frameworks like Scrum
• Interest in financial services and markets

Responsibilities

• Design, develop and maintain robust data pipelines using Scala, Spark, Hadoop, SQL for batch and streaming data processing
• Collaborate with cross-functional teams to understand data requirements and design efficient solutions that meet business needs
• Optimize Spark jobs and data processing workflows for performance, scalability and reliability
• Ensure data quality, integrity and security throughout the data lifecycle
• Troubleshoot and resolve data pipeline issues in a timely manner to minimize downtime and impact on business operations
• Stay updated on industry best practices, emerging technologies, and trends in big data processing and analytics
• Document, design specifications, deployment procedures and operational guidelines for data pipelines and systems
• Provide technical guidance and mentorship for new joiners

Benefits

• Employment contract and/or Business to Business - whichever you prefer
• Possibility to work remotely
• Speaking English on daily basis, mainly in contact with foreign stakeholders and peers
• Multiple employee benefits packages (MyBenefit Cafeteria, private medical care, life-insurance)
• Access to 3.000+ Business Courses Platform (Udemy)
• Access to required IT equipment
• Paid Referral Program
• Participation in charity events e.g. Szlachetna Paczka