Associate Staff Big Data Engineer

Summary

Join Nagarro, a rapidly growing Digital Product Engineering company, as a Big Data Engineer. You will leverage your 5+ years of experience to develop and maintain data pipelines, work with data lakes and warehouses, and design effective data storage solutions. Strong knowledge of Apache Kafka, Apache Spark, Python, Databricks, and Scala is essential. You will collaborate with teams, solve complex data engineering challenges, and ensure adherence to data governance and security best practices. A Bachelor's or Master's degree in a related field is required. This role offers the opportunity to work on large-scale projects and contribute to a dynamic, non-hierarchical work culture.

Requirements

• Total experience 5+years
• Excellent knowledge and experience in Big data engineer
• Strong working knowledge in Apache Kafka, Apache Spark, Python, Databricks and Scala
• A strong understanding of data modeling, data structures, data warehouses, databases, and ETL processes
• Experience with building and maintaining a cloud system
• Expertise in Spanner for high-availability, scalable database solutions
• Knowledge of data governance and security practices in cloud-based environments
• Problem-solving mindset with the ability to tackle complex data engineering challenges
• Strong communication and teamwork skills, with the ability to mentor and collaborate effectively
• Bachelor’s or master’s degree in computer science, Information Technology, or a related field

Responsibilities

• Develop and maintain data pipelines to ingest, transform, and load large datasets from various structured and unstructured sources
• Work with data lakes and data warehouses to design and implement effective data storage solutions
• Writing and reviewing great quality code
• Understanding the client’s business use cases and technical requirements and be able to convert them in to technical design which elegantly meets the requirements
• Mapping decisions with requirements and be able to translate the same to developers
• Identifying different solutions and being able to narrow down the best option that meets the client’s requirements
• Defining guidelines and benchmarks for NFR considerations during project implementation
• Writing and reviewing design document explaining overall architecture, framework, and high-level design of the application for the developers
• Reviewing architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed
• Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it
• Understanding and relating technology integration scenarios and applying these learnings in projects
• Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken
• Carrying out POCs to make sure that suggested design/technologies meet the requirements

Preferred Qualifications

Familiarity with data modeling, data warehousing, and building distributed systems