Senior Big Data Engineer

Summary

Join Capco Poland as a Senior Big Data Engineer and contribute to transformative projects for major global banks. You will design, implement, and maintain scalable data pipelines and solutions using cutting-edge technologies. This role involves working within an agile environment, collaborating with high-performing teams, and utilizing various data and cloud technologies. You will develop end-to-end ETL processes with Spark/Scala and ensure the quality of your deliverables. Capco offers a competitive compensation package, remote work possibilities, and numerous employee benefits.

Requirements

• 5+ years of experience with Python
• 2+ years of experience with Spark and Scala
• Knowledge of good practices in writing code: clean code, software design patterns, functional style of writing code, TDD
• Agile approach for software development
• Experience and expertise across data integration and data management with high data volumes
• Experience working in agile continuous integration/DevOps paradigm and tool set (Git, GitHub, Jenkins, Sonar, Nexus, Jira)
• Experience with different database structures, including Postgres, SQL, Hive

Responsibilities

• Develop, test and deploy technical and functional specifications from Solution Designers/Business Architects/Business Analysts, guaranteeing correct operability and compliance with internal quality levels
• Develop end-to-end ETL processes with Spark/Scala, including data transfer to/from the data lake, technical validations, and business logic
• Utilize Scrum methodology and be part of a high-performance team
• Document solutions in JIRA, Confluence, and ALM
• Certify delivery and integration with other components, designing and performing tests to ensure quality

Preferred Qualifications

• CI/CD: Jenkins, GitHub Actions
• Orchestration: Control-M, Airflow
• Scripting: Bash, Python
• Software development life cycle: HP ALM
• Basics of cybersecurity and quality: Sonar, Fortify
• Basics of cloud computing: Docker, Kubernetes, OS3, Azure, AWS

Benefits

• Employment contract and/or Business to Business - whichever you prefer
• Possibility to work remotely
• Multiple employee benefits packages (MyBenefit Cafeteria, private medical care, life-insurance)
• Access to 3.000+ Business Courses Platform (Udemy)
• Access to required IT equipment
• Paid Referral Program
• Participation in charity events e.g. Szlachetna Paczka
• Ongoing learning opportunities to help you acquire new skills or deepen existing expertise