Cloud Data Engineer

Summary

Join our team as a highly skilled Cloud Data Engineer with a strong backend development focus and experience in AWS/GCP. You will design, implement, and maintain our data-centric, Cloud-hosted backend infrastructure and services. A key role involves building and scaling data processing pipelines and APIs, ensuring high availability, security, and performance. You will collaborate with architects and security teams, troubleshoot issues, and stay current with Cloud technologies. Mentoring junior team members will also be part of your responsibilities. This position requires a Bachelor's degree and significant experience in backend development and data engineering within a cloud environment.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, or a related field
• At least 5 years of experience in backend development in a Cloud environment (AWS or GCP preferred)
• At least 3 years of experience in Data Engineering in a Cloud environment
• Strong proficiency in Python and Java; SQL proficiency is required
• Extensive experience with RESTful API design, implementation, and testing using relevant frameworks
• Experience with event-driven architectures and message queues (e.g., Kafka, RabbitMQ, Azure Event Hub, Amazon MSK)
• Experience with ELT data processing using frameworks such as Fivetran, dbt, Airflow, and/or PySpark
• Proven experience with containerization (Docker) and orchestration (Kubernetes)
• Experience with CI/CD pipelines and infrastructure-as-code (Pulumi/Terraform)
• Working knowledge of Linux, shell scripting, and Git
• Strong understanding of cloud security best practices, including IAM and data encryption
• Excellent problem-solving and analytical skills
• Excellent written and verbal communication skills
• Ability to collaborate effectively in a team environment

Responsibilities

• Build and optimize data pipelines using ELT frameworks such as Fivetran, dbt, Airflow, and PySpark, ensuring efficient data movement and transformation
• Design, develop, and maintain high-throughput, low-latency backend services for data ingestion, processing, and transformation using languages such as Python and Java
• Develop and maintain robust, scalable, and secure RESTful APIs using frameworks such as Spring Boot (Java) or Flask/FastAPI (Python)
• Advise/design OData APIs
• Implement and manage security protocols such as OAuth, SSO, and SSL
• Design and implement event-driven architectures leveraging technologies like Kafka or Pub/Sub
• Develop and implement CI/CD pipelines using tools like Jenkins, GitLab CI, or similar, to automate the deployment and scaling of backend services
• Containerize applications using Docker and orchestrate them using Kubernetes
• Write and maintain infrastructure-as-code (IAC) using tools such as Pulumi, Terraform or CloudFormation
• Collaborate with Architects and Security teams to ensure that the Cloud infrastructure and services are secure and compliant with industry standards (e.g., SOC 2, HIPAA, GDPR)
• Troubleshoot and resolve service-related issues, providing timely and efficient solutions
• Stay up to date with the latest developments in Cloud technology, backend development best practices, and security vulnerabilities
• Mentor and provide guidance to other team members as needed

Preferred Qualifications

• Experience with API gateway services (e.g., Apigee, Kong)
• Experience with managing Kubernetes clusters (e.g., GKE, EKS)
• Experience with OpenAPI/Swagger specifications