Senior Platform and Data Lake Engineer

Summary

Join TetraScience as a Senior Platform and Data Lake Engineer and play a critical role in building and maintaining our data infrastructure. You will work with cross-functional teams to ensure seamless ingestion, processing, and storage of large scientific datasets. This position requires extensive experience in data pipeline infrastructure and hands-on Databricks experience. You will design, develop, and optimize data lake solutions and pipelines, architect services for customer data processing, and implement data quality frameworks. The ideal candidate possesses expert-level skills in Python, Java, Typescript, and Lake House architecture, along with extensive cloud-based data storage and processing experience. TetraScience offers a comprehensive benefits package including 100% employer-paid benefits, unlimited PTO, 401K, flexible working arrangements, and company-paid life insurance.

Requirements

• 8+ years of experience in the software development industry, preferably in data engineering, data warehousing or data analytics companies and teams
• 3+ year of experience with the DataBricks ecosystem
• Expert level of Python, Java, and Typescript
• Expert level of understanding and hands-on experience with Lake House architecture
• Expert level of experience with Spark/Glue and Delta tables/iseberg
• Experienced in designing and implementing complex, scalable data pipelines/ETL services
• Extensive in cloud-based data storage and processing technologies, particularly AWS services such as S3, Step Functions, Lambda, and Airflow
• Ability to articulate ideas clearly, present findings persuasively, and build rapport with clients and team members

Responsibilities

• Design, develop, and optimize data lake solutions to support our scientific data pipelines and analytics capabilities
• Design, develop, and optimize data pipelines and workflows within the Databricks platform
• Design and architect services to meet customer data processing needs
• Implement data quality and governance frameworks to ensure data integrity and compliance

Preferred Qualifications

• Knowledge of basic DevOps and MLOps principles
• Working knowledge of Snowflake
• Experience in working with Data Scientists and ML Developers
• Experience in management and lead developer roles from technology services companies
• Hands-on experience with data warehousing solutions and ETL tools

Benefits

• 100% employer-paid benefits for all eligible employees and immediate family members
• Unlimited paid time off (PTO)
• 401K
• Flexible working arrangements - Remote work
• Company paid Life Insurance, LTD/STD
• A culture of continuous improvement where you can grow your career and get coaching