Data Engineer II

Summary

Join HackerRank's data team as a Data Engineer and contribute to democratizing data access. This remote, full-time position in India involves designing, building, and maintaining scalable data pipelines and a modern lake house platform using AWS. You will evaluate technologies, develop POCs, and solve technical challenges. The role requires collaboration with remote and cross-functional teams, including mentoring junior engineers. HackerRank offers a supportive and collaborative environment with various benefits.

Requirements

• 3+ years of experience with designing, developing and maintaining data engineering & BI solutions
• Experience with Data Modeling for Big Data Solutions
• Experience with Spark, Spark Structured Streaming (PySpark or Scala Spark)
• Experience with database technologies like Redshift or Trino
• Experience querying massive datasets using Languages like SQL, Hive, Spark, Trino
• Experience with performance tuning complex data warehouses and queries
• Able to solve problems of scale, performance, security, and reliability
• Self-driven, initiative taker with good communication skills, ability to lead and mentor junior engineers, work with cross-functional teams, drive architecture decisions

Responsibilities

• Evaluate technologies, develop POCs, solve technical challenges and propose innovative solutions for our technical and business problems
• Delight our stakeholders, customers and partners by building high-quality, well-tested, scalable and reliable business applications
• Design, build and maintain streaming and batch data pipelines that can scale
• Architect, develop and maintain our Modern lake house Platform using AWS native infrastructure
• Designing Complex Data Models to deliver insights and enable self-service
• Take ownership of scaling, performance, security, and reliability of our data infrastructure
• Hiring, guiding and mentoring junior engineers
• Work in agile development environment, participate in code reviews
• Collaborate with remote development teams and cross-functional teams

Preferred Qualifications

• Experience with BI Solutions like Looker, Power BI, Amazon Quicksight, Tableau etc
• Experience with ETL Design & Orchestration using platforms like Apache Airflow, MageAI etc
• Knowledge of Kafka, Kafka Connect and related technologies

Benefits

• One-time home office set up stipend
• Monthly Remote Work Enablement Stipend
• Professional Development Reimbursement
• Wellbeing Benefits (Headspace, Cleo, etc)
• Flexible paid time off, paid leave for new parents, and flexible work hours
• Insurance for all employees (term life, personal accident, medical) along with medical insurance for their dependents
• Employee stock options, flexible work hours, and time off