Senior Data Engineer

Summary

Join NBCUniversal as a Data Architect and leverage your expertise in data modeling, architecture, and cloud technologies to design, develop, and maintain robust data solutions. You will play a key role in shaping our data infrastructure, implementing CI/CD pipelines, and collaborating with development teams to ensure high code quality. This fully remote position requires 5+ years of experience in data modeling and distributed computing, along with proficiency in AWS technologies and CI/CD pipelines. You will also be responsible for educating business partners on best practices and contributing to the development of future enhancements. The ideal candidate possesses strong problem-solving, communication, and analytical skills, and a passion for sharing knowledge and improving team skills. A competitive salary and comprehensive benefits package are offered.

Requirements

• 5+ Years Experience in Data Modeling, Data architecture, Data Quality, Metadata, ETL and Data Warehouse methodologies and technologies
• 3+ years experience in distributed computing solutions such as Spark, MapReduce, Snowflake, Databricks, or Kubernetes
• 3+ years experience with AWS technologies, with preference for Managed Airflow, EMR, Lambda, ECS, EKS
• Experience in designing and managing CI/CD pipelines, preferably using GitHub Actions
• Experience in any combination of the following: SQL, Linux, MicroStrategy, Tableau, Python, APIs, Spark, Scala, Pandas
• Strong problem-solving skills
• Strong oral and written communication and influencing skills, with the ability to communicate new concepts and drive change in processes and behaviors and to communicate complex technical topics to management and non-technical audiences
• Strong knowledge of data security practices and privacy regulations (e.g., GDPR, CCPA) with a proven ability to implement and maintain robust data protection measures
• Bachelor’s degree in Engineering, Computer Science, Information Systems or related field with 5+ years of relevant experience

Responsibilities

• Reviews internal and external business and product requirements for data operations and activity and suggests changes and upgrades to systems and storage to accommodate ongoing needs
• Design, develop, and maintain CI/CD pipelines using GitHub Actions to automate deployment, testing, and monitoring of applications
• Implement and manage serverless solutions (e.g., AWS Lambda, EMR Serverless, Kafka, SNS, SQS, Athena etc.) as part of the application architecture
• Implement infrastructure as code (IaC) practices using tools like Terraform, AWS CloudFormation, or similar to manage cloud infrastructure
• Work with development teams to set up automated testing frameworks, ensuring high test coverage and code quality
• Must understand the basics of relational data modeling and be able to clearly articulate the reasons to use non-relational systems in our architecture
• Educate and inform business partners on architecture, capabilities, best practices and solutions to build out future enhancements
• Assist in analyzing business requirements, source systems, understand underlying data sources, transformation requirements, data mapping, data model and metadata for reporting solutions
• Writing easily understood documentation and architecture diagrams and keeping them up to date as code and frameworks change over time

Preferred Qualifications

• Proven ability to develop data applications using Spark Scala
• Troubleshoot complex data pipelines, including addressing scale-related issues such as partitioning, resolving data skews, and optimizing performance by reviewing Spark UI
• Additionally, the candidate should be able to consider data model design in scale and performance decisions before implementing solutions
• Understanding of how to manage code in the Enterprise Git repository with appropriate branching and documentation skills
• Ability to read external API documentation and write pipelines to extract data from our partners’ systems
• Strong analytical focus, results-oriented and execution driven
• Ability and desire to work within a cross-functional team environment with people from multiple business units, vendors, countries and cultures
• Flexibility to adjust to changing requirements, schedules and priorities
• Ability to work independently under minimum supervision and proactive in solving issues
• Energetic, committed and solution focused with the ability to perform under pressure and meeting targets
• Strong desire to share knowledge, teach others, and improve the overall skills of the team

Benefits

• Fully Remote: This position has been designated as fully remote, meaning that the position is expected to contribute from a non-NBCUniversal worksite, most commonly an employee’s residence
• This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks