Senior Streaming Engineer

Summary

Join our dynamic team as a Senior Big Data Engineer and leverage your expertise in Java, public cloud technologies, and stream/batch processing systems to design and develop scalable data solutions.

Requirements

• Driven and self-motivated to thoroughly clarify problem statements and develop well-structured plans to effectively solve challenges
• Proficiency in Java with solid experience in asynchronous programming paradigms, including a strong understanding of API futures and the ability to implement asynchronous APIs for scalable solutions
• Hands-on experience with reactive programming techniques, focusing on non-blocking data flow and real-time data handling
• Ability to design, debug, and optimize concurrent processes within distributed systems
• Adopt a proactive problem-solving approach by thoroughly analyzing problem statements, anticipating corner cases, and devising a comprehensive solution plan prior to coding. Begin coding with a clear algorithmic strategy to ensure efficient and effective solutions
• Expertise in public cloud services, particularly in GCP
• Experience in building reusable and reliable code for stream and batch processing systems at scale
• Experience with GCP managed services and understanding of cloud-based messaging/stream processing systems (e.g., Data Proc, Cloud Composer, GCS, DataFlow)
• Experience in Infrastructure and Applied DevOps principles, including CI/CD and IaC (e.g., Terraform)
• Knowledge in containerization technologies like Docker and Kubernetes to enhance scalability and efficiency
• Ability to tackle complex challenges and devise effective solutions, using critical thinking and innovative approaches
• Proven ability to work effectively in a remote setting, maintaining strong written and verbal communication skills
• Proven experience in engineering stream/batch processing systems at scale
• Strong programming abilities in Java and Python
• Hands-on experience with public cloud platforms, particularly GCP. Experience with other cloud technologies is a plus
• Must Have: Strong experience with Java-based frameworks and libraries for building scalable data processing and streaming applications
• Strong Java 11-17 experience
• Dataflow (Managed Apache Beam)
• Pub/Sub
• Google Associate Cloud Engineer Certification or other Google Cloud Professional level certification
• 4+ years of experience in customer-facing software/technology or consulting
• 4+ years of experience with “on-premises to cloud” migrations or IT transformations
• 4+ years of experience building and operating solutions on GCP (ideally) or AWS/Azure
• Technical degree in Computer Science, Software Engineering, or related field

Responsibilities

• Design and develop scalable data processing solutions, leveraging advanced Java techniques to ensure performance, reliability, and maintainability across streaming and batch workflows
• Build reusable and reliable code for stream and batch processing systems at scale
• Work with technologies such as Pub/Sub, Kafka , DataFlow, Flink, Hadoop, Pig, Oozie, Hive, and Spark
• Implement automation/DevOps best practices for CI/CD, IaC, Containerization, etc

Benefits

• Anticipated starting pay range for Colorado is: $116,100 - $170,280
• Anticipated starting pay range for the states of Hawaii and New York (not including NYC) is: $123,600 - $181,280
• Anticipated starting pay range for California, New York City and Washington is: $135,300 - $198,440