Data Engineer (with Spark, Airflow)

Summary

Join Accesa as a Data Engineer and contribute to the optimization of data management strategies for a prominent financial client. You will be responsible for creating and maintaining data transformation pipelines, working with large financial datasets, and leading process improvements. This role requires strong analytical skills, experience with Big Data pipelines and architectures, and proficiency in SQL and relational databases. You will collaborate with cross-functional teams to address data-related issues and support their infrastructure needs. The position offers a comprehensive benefits package encompassing physical, emotional, social, and work-life fusion aspects.

Requirements

• 3+ years of experience in a similar role, preferably within Agile teams
• Strong analytical skills in working with both structured and unstructured data
• Skilled in SQL and relational databases for data manipulation
• Experience in building and optimizing Big Data pipelines and architectures
• Knowledge of Apache Spark framework and object-oriented programming in Java; experience with Python is a plus
• Experience with ETL processes, including scheduling and orchestration using tools like Apache Airflow (or similar)
• Proven experience in performing data analysis and root cause analysis on diverse datasets to identify opportunities for improvement

Responsibilities

• Drive Data Efficiency: Create and maintain optimal data transformation pipelines
• Master Complex Data Handling: Work with large, complex financial data sets to generate outputs that meet functional and non-functional business requirements
• Lead Innovation and Process Optimization: Identify, design, and implement process improvements such as automating manual processes, optimizing data delivery, and re-designing infrastructure for higher scalability
• Architect Scalable Data Infrastructure: Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using open-source technologies
• Unlock Actionable Insights: Build/use analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
• Collaborate with Cross-Functional Teams: Work clients and internal stakeholders, including Senior Management, Departments Heads, Product, Data, and Design teams, to assist with data-related technical issues and support their data infrastructure needs

Preferred Qualifications

• Expertise in manipulating and processing large, disconnected datasets to extract actionable insights
• Automate CI/CD pipelines using ArgoCD, Tekton, and Helm to streamline deployment and improve efficiency across the SDLC
• Manage Kubernetes deployments on OpenShift, focusing on scalability, security, and optimized container orchestration
• Technical skills in the following areas are a plus: relational databases (e.g. Postgresql), Big Data Tools: (e.g. Databricks), and workflow management (e.g. Airflow), and backend development using Spring Boot

Benefits

• Premium medical package for both our colleagues and their children
• Dental coverage up to a yearly amount
• Eyeglasses reimbursement every two years
• Voucher for sport equipment expenses
• In-house personal trainer
• Individual therapy sessions with a certified psychotherapist
• Webinars on self-development topics
• Virtual activities
• Sports challenges
• Special occasions get-togethers
• Yearly increase in days off
• Flexible working schedule
• Birthday, holiday and loyalty gifts for major milestones