Data Engineer

Summary

Join Macropay, a fintech leader in payment orchestration, as a Data Engineer to design, build, and maintain scalable, secure, and high-performance cloud-based data pipelines. You will collaborate with various teams to translate business requirements into robust data models and ETL/ELT workflows, focusing on real-time and batch analytics within our payments platform. This role involves architecting and implementing Kafka-based streaming pipelines, processing event streams, and leveraging Snowflake as our central data warehouse. You will define best practices for data ingestion and ensure data quality and performance. The position requires a Bachelor's degree in a related field and 5+ years of professional experience with Java or Python. Macropay offers a competitive salary, comprehensive benefits, professional development opportunities, and a full remote work environment.

Requirements

• Bachelor's degree in Computer Science, Software Engineering, or a related field (equivalent experience is valued)
• 5+ years of professional experience
• Solid technology background using Java or Python as a primary language
• 3+ years of programming experience with Java / Python
• Experience in data engineer design and delivery with cloud based data Warehouse technologies, in particular Snowflake, or Redshift,BigQuery
• Development with cloud services, especially Amazon Web Services
• Demonstrable experience in designing and implementing data pipeline architectures based on Kafka in cloud environments, preferably AWS
• Deep understanding of distributed systems and high availability design principles
• Experience in building and optimizing data pipelines using technologies like Apache Kafka, Apache Flink, Apache Spark, etc., including real-time processing frameworks such as Apache Flink or Apache Spark Streaming
• Excellent communication and teamwork skills
• Ability to independently and proactively solve problems

Responsibilities

• Define and enforce best practices for data ingestion, cataloging, and lineage across our cloud infrastructure (AWS S3, Glue, EMR, Lambda, etc.)
• Develop and maintain real-time processing applications using Kafka (Producers, Consumers, Streams API) or similar technologies to aggregate, filter, and enrich streaming data from multiple sources
• Collaborate with development and analytics teams to understand and fulfill the company's data requirements
• Develop efficient and scalable data pipelines for data transformation and enrichment
• Implement monitoring and alerting mechanisms to ensure the integrity and availability of data streams
• Work with the operations team to optimize the performance and efficiency of the data infrastructure
• Automate management and maintenance tasks of the infrastructure using tools such as Terraform, Ansible, etc
• Stay updated on best practices and trends in data architectures, especially in the realm of real-time data ingestion and processing
• Monitor and troubleshoot data workflows using tools such as CloudWatch, Prometheus, or Datadog—proactively identifying bottlenecks, ensuring pipeline reliability, and handling incident response when necessary
• Ensure data quality and performance
• Define and test disaster recovery plans (multi-region backups, Kafka replication, Snowflake Time Travel) and collaborate with security/infra teams on encryption, permissions, and compliance

Preferred Qualifications

• Experience with other streaming platforms such as Apache Pulsar or RabbitMQ
• Familiarity with data lake architectures and technologies such as Amazon S3, Apache Hadoop, or Apache Druid
• Relevant certifications in cloud platforms such as AWS (optional)
• Understanding of serverless architecture and event-driven systems
• Previous professional experience in FinTech / online payment flows
• Experience with data visualization tools like Tableau, PowerBI, or Apache Superset
• Understanding of machine learning concepts and frameworks for real-time data analytics
• Previous experience in designing and implementing data governance and compliance solutions

Benefits

• Competitive salary and comprehensive benefits package
• Opportunity to shape HR strategy for a global, innovative fintech company
• Professional development opportunities and resources
• A collaborative, inclusive, and dynamic work culture
• Full Remote Work