Big Data Lead

Summary

Join us at DemandMatrix as a hands-on SME in leveraging big data tech to solve complex data issues, utilizing cloud native services and expertise in software development with a focus on big data and large data pipelines.

Requirements

• Solid grounding in computer engineering, Unix, data structures and algorithms
• 7+ years of hands-on experience in Software Development with a focus on big data and large data pipelines
• Minimum 3 years of experience to build services and pipelines using Python
• Expertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce)
• Understanding of at least one NoSQL stores like MongoDB, Elasticsearch, HBase
• Experience with running scalable & high available systems with AWS or GCP

Responsibilities

• Spend almost half of time with hands-on coding
• Large scale text data processing
• Event driven data pipelines
• In-memory computations
• Optimization considering CPU core to network IO to disk IO

Preferred Qualifications

• Experience with Docker / Kubernetes
• Exposure with CI/CD
• Knowledge of Crawling/Scraping

Benefits

• Entire Work From Home
• Birthday Leave
• Remote Work