Remote Sr Big Data Engineer
Rackspace Technology
πRemote - United States of America, India
Please let Rackspace Technology know you found this job on JobsCollider. Thanks! π
Job highlights
Summary
The job is for a Senior Big Data Engineer who will develop and scale both stream and batch processing systems on GCP. The role requires expertise in technologies like Pub/Sub, Kafka, Kinesis, DataFlow, Flink, Hadoop, Pig, Hive, Spark, Oozie, Airflow, Map Reduce, Java, Python, Hbase, and BigTable. The candidate should have experience with GCP managed services, CI/CD, IaC, Terraform, and be proficient in the Apache Hadoop ecosystem. Remote work is required.
Requirements
- Experience with GCP managed services and understanding of cloud-based batch processing systems are critical
- Proficiency in Oozie, Airflow, Map Reduce, Java
- Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
- Expertise in public cloud services, particularly in GCP
- Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
- Familiarity with BigTable and Redis
- Experienced in Infrastructure and Applied DevOps principles in daily work
- Ability to tackle complex challenges and devise effective solutions
- Worked effectively in a remote setting, maintaining strong written and verbal communication skills
- Proven experience in engineering batch processing systems at scale
- Hands-on experience in public cloud platforms, particularly GCP. Additional experience with other cloud technologies is advantageous
Responsibilities
- Build a reusable, and reliable code for stream and batch processing systems at scale
- Implementing automation/DevOps best practices for CI/CD, IaC, etc
Share this job:
Disclaimer: Please check that the job is real before you apply. Applying might take you to another website that we don't own. Please be aware that any actions taken during the application process are solely your responsibility, and we bear no responsibility for any outcomes.
Similar Remote Jobs
- πUnited States
- πIndia
- πWorldwide
- πIndia
- πUnited Kingdom
- πColombia
- πIndia
- π°$100k-$130kπUnited States
Please let Rackspace Technology know you found this job on JobsCollider. Thanks! π