Lead Data Engineer

Summary

Join a high-performance team of Data Engineers at NBCUniversal, a leading entertainment company. This fully remote position involves designing, building, and maintaining data pipelines from various sources. You will contribute to the team's agile processes, work with stakeholders to define stories, and collaborate with Principal Engineers and Architects. The role requires developing and championing best practices, focusing on automation and GitOps. You will also develop solutions using data blending, profiling, and machine learning to deliver key insights. This position offers competitive compensation and a comprehensive benefits package.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, Data Engineering, Physics, or a related field (or foreign degree equivalent)
• Five (5) years of experience in the job offered, as a Software Engineer, or in a related occupation
• Programming skills in one or more of the following: Python, Java, Scala, R, SQL
• Writing reusable/efficient code to automate analysis and data processes
• Developing data catalogs and data cleanliness to ensure clarity and correctness of key business metrics
• Building and maintaining dimensional data warehouses in support of BI tools (four (4) years of experience)
• Near Real Time & Batch Data Pipeline development for Big Data (three (3) years of experience)
• Processing structured and unstructured data into a form suitable for analysis and reporting, including integration with a variety of data metric providers (advertising, web analytics, and consumer devices) (three (3) years of experience)
• Hands on programming of the following (or similar) technologies: Apache Beam, Scio, Apache Spark, and Snowflake (three (3) years of experience)
• Progressive data application development, working in large scale/distributed SQL, NoSQL, and/or Hadoop environment (three (3) years of experience)
• Building streaming data pipelines using Kafka, Spark or Flink (two (2) years of experience)
• Implementing scalable, distributed, and highly available systems using Google Cloud (one (1) year of experience)

Responsibilities

• Help manage a high-performance team of Data Engineers
• Contribute to and help lead team in designing, building, testing, scaling and maintaining data pipelines from a variety of source systems and streams (Internal, third party, cloud based, etc.), according to business and technical requirements
• Deliver observable, reliable and secure software, embracing “you build it you run it” mentality, and focus on automation and GitOps
• Continually work on improving the codebase and have active participation and oversight in all aspects of the team, including agile ceremonies
• Take an active role in story definition, assisting business stakeholders with acceptance criteria
• Work with Principal Engineers and Architects to share and contribute to the broader technical vision
• Develop and champion best practices, striving towards excellence and raising the bar within the department
• Develop solutions combining data blending, profiling, mining, statistical analysis, and machine learning, to better define and curate models, test hypothesis, and deliver key insights
• Operationalize data processing systems (dev ops)

Benefits

• Medical, dental and vision insurance
• 401(k)
• Paid leave
• Tuition reimbursement
• A variety of other discounts and perks