Data Engineer II

Summary

Join HackerRank's data team and build modern, scalable data systems that power real products. You will work across the full stack, from designing robust data pipelines to supporting search and analytics platforms, using tools like Airflow, dbt, and Spark. This hands-on role offers real ownership, taking projects from idea to production. You'll work with cloud platforms (AWS or GCP), open-source tools, and contribute to a flexible, future-ready data foundation. The ideal candidate enjoys solving complex problems, learning new technologies, and working collaboratively. This role involves evaluating technologies, developing POCs, and proposing innovative solutions for technical and business problems. You will also build high-quality, scalable, and reliable business applications.

Requirements

• 3+ years of experience with designing, developing and maintaining data engineering & BI solutions
• Experience with Data Modelling for Big Data Solutions
• Experience with Spark, Spark Structured Streaming (Scala Spark)
• Experience with database technologies like Redshift or Trino
• Experience querying massive datasets using Languages like SQL, Hive, Spark, and Trino
• Experience with performance tuning complex data warehouses and queries
• Able to solve problems of scale, performance, security, and reliability
• Self-driven, initiative taker with good communication skills, ability to lead and mentor junior engineers, work with cross-functional teams, and drive architecture decisions

Responsibilities

• Evaluate technologies, develop POCs, solve technical challenges and propose innovative solutions for our technical and business problems
• Delight our stakeholders, customers and partners by building high-quality, well-tested, scalable and reliable business applications
• Design, build and maintain streaming and batch data pipelines that can scale
• Architect, develop and maintain our Modern lake house Platform using AWS native infrastructure
• Designing Complex Data Models to deliver insights and enable self-service
• Take ownership of scaling, performance, security, and reliability of our data infrastructure
• Hiring, guiding and mentoring junior engineers
• Work in an agile development environment and participate in code reviews
• Collaborate with remote development teams and cross-functional teams

Preferred Qualifications

Experience with ETL Design & Orchestration using platforms like Apache Airflow, MageAI