Senior Data Engineer

Summary

Join Natera as a Senior Data Engineer and play a crucial role in designing and developing enterprise-scale data engineering solutions. You will build robust data infrastructure for processing large datasets and integrate machine learning capabilities into production data workflows. Collaborate with business analysts, data scientists, and AI engineers to translate data requirements into scalable architectures and integrate AI/ML pipelines. Responsibilities include designing and implementing ETL processes, optimizing SQL, analyzing complex data using distributed computing, and ensuring data quality. The ideal candidate possesses a Bachelor's degree in Computer Science or equivalent, 8+ years of software development experience, and 5+ years of experience with various SQL platforms. Experience with PySpark, Iceberg tables, and data pipeline orchestration tools is also required.

Requirements

• Bachelor's degree in Computer Science or equivalent experience
• 8+ years of overall software development experience, preferably building complex data solutions
• Minimum 5 years of experience working with several SQL platforms (Redshift, Athena, MySQL/Postgres, Snowflake, BigQuery)
• Minimum 3 years of experience working with PySpark and Iceberg tables / Parquet files
• Ability to lead a team of data engineers
• Big Data Processing: Extensive experience with Apache Spark and PySpark for large-scale data processing and 100+ GB scale data warehousing
• Modern Data Lakehouse: Proficiency with Apache Iceberg for managing large-scale table formats and data lake architectures
• Experienced with data warehouse applications
• Experience with data pipeline orchestration tools such as Apache Airflow
• Experienced with data visualization tools such as Power BI, Qlik Sense, Tableau, QuickSight, Looker, or similar
• Strong programming proficiency in Python and SQL
• Understanding of data quality, governance, and monitoring practices

Responsibilities

• Understand business requirements, processes, and data sources to design appropriate technical solutions
• Collaborate with business analysts to translate data requirements into logical data models and scalable architectures
• Design and implement highly scalable ETL processes and data transformations for >100 GB-scale data warehousing environments
• Develop and optimize SQL views, procedures, and functions across multiple database platforms
• Serve as subject matter expert in Structured Query Language (SQL) and database performance optimization
• Analyze and process complex data with large volumes using distributed computing frameworks
• Ensure the integrity and quality of data stored in data warehouses and data marts
• Build and maintain machine learning pipelines for model training, validation, and deployment
• Collaborate with data scientists and AI engineers to integrate AI/ML pipelines into data workflows

Preferred Qualifications

• Familiarity with machine learning frameworks (e.g., TensorFlow, PyTorch) and libraries (e.g., scikit-learn)
• Experience implementing AI/ML models for data processing, predictive analytics, and anomaly detection
• Experienced in leveraging large language models (LLMs) during development (e.g. Cursor) and used within applications
• Ability to collaborate with data scientists and AI engineers to integrate AI/ML pipelines into data workflows
• Understanding of bioinformatics / genomics (e.g. secondary and tertiary analysis tooling) and file format standards (e.g. VCF, BAM, FASTQ)
• Experience with distributed computing and cluster management

Benefits

• Competitive Benefits - Employee benefits include comprehensive medical, dental, vision, life and disability plans for eligible employees and their dependents
• Additionally, Natera employees and their immediate families receive free testing in addition to fertility care benefits
• Other benefits include pregnancy and baby bonding leave, 401k benefits, commuter benefits and much more
• We also offer a generous employee referral program!