Rackspace Technology is hiring a
Sr Big Data Engineer

Summary

The job is for a Senior Big Data Engineer who will develop and scale both stream and batch processing systems on GCP. The role requires expertise in technologies like Pub/Sub, Kafka, Kinesis, DataFlow, Flink, Hadoop, Pig, Hive, Spark, Oozie, Airflow, Map Reduce, Java, Python, Hbase, and BigTable. The candidate should have experience with GCP managed services, CI/CD, IaC, Terraform, and be proficient in the Apache Hadoop ecosystem. Remote work is required.

Requirements

• Experience with GCP managed services and understanding of cloud-based batch processing systems are critical
• Proficiency in Oozie, Airflow, Map Reduce, Java
• Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
• Expertise in public cloud services, particularly in GCP
• Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
• Familiarity with BigTable and Redis
• Experienced in Infrastructure and Applied DevOps principles in daily work
• Ability to tackle complex challenges and devise effective solutions
• Worked effectively in a remote setting, maintaining strong written and verbal communication skills
• Proven experience in engineering batch processing systems at scale
• Hands-on experience in public cloud platforms, particularly GCP. Additional experience with other cloud technologies is advantageous

Responsibilities

• Build a reusable, and reliable code for stream and batch processing systems at scale
• Implementing automation/DevOps best practices for CI/CD, IaC, etc