Data Engineer - Data Engineering

Summary

Join Arine, a rapidly growing healthcare technology company, as a key technical leader in data engineering. You will drive the design, development, and optimization of scalable data ingestion pipelines within the Arine platform, leveraging your expertise in Python and AWS. This role involves architecting solutions for diverse file types and large-scale healthcare datasets, building reusable tools, and collaborating with cross-functional teams. You will also mentor junior engineers and contribute to high-quality technical documentation. Arine offers a dynamic role with unparalleled learning and growth prospects, contributing to a company revolutionizing the healthcare industry. The base salary range is $165,000-180,000/year.

Requirements

• 10+ years of professional experience in data engineering with focus on large-scale data ingestion and infrastructure
• Deep expertise in Python programming and modern data engineering tools
• Experience creating an automated production grade ETL process using Python and SQL
• Strong understanding of ETL/ELT frameworks and distributed data processing
• Experience with data processing, validation, cleaning and debugging data sets
• Experience with API integration for seamless data exchange between systems
• Proven experience handling and processing various file types and formats, including specialized healthcare standards such as HL7, 834, 837, and NCPDP
• Experience integrating and consolidating data from diverse source systems into a unified repository, including data from EHR and claim systems, as well as from file-based and API integrations
• Experience with processing large data sets (over 10GB)
• Experience with incremental data processing and change data capture (CDC) methodologies
• Strong experience designing scalable data architectures in AWS environment
• Deep understanding of software engineering principles including test-driven development, loose coupling, single responsibility, and modular design
• Experience with containerization technologies (Docker, Kubernetes) and building configuration-driven, maintainable systems
• Proven ability to build tools and systems that can be operated by diverse engineering profiles through configuration rather than code changes
• Passion for building new and improving existing data infrastructure with robust, maintainable, and operationally excellent data systems
• Strong collaboration and communication skills; comfortable working with diverse technical and non-technical stakeholders
• Excellent verbal and written communication skills with ability to explain technical infrastructure concepts to diverse audiences
• Ability to pass a background check
• Must live in and be eligible to work in the United States

Responsibilities

• Act as the team architect by leading system design reviews, offering recommendations, conducting comprehensive peer reviews, and demonstrating expert-level proficiency in Python and AWS services
• Architecting and implementing scalable data ingestion pipelines handling different file types into Arine platform
• Develop reusable components that can be integrated into data pipelines to enhance efficiency and minimize future implementation time
• Creating configuration-driven, containerized toolsets that can be easily used and maintained by diverse engineering profiles
• Work collaboratively with cross-functional teams to ensure their data requirements are met through ETL components
• Implementing incremental data ingestion strategies for large-scale healthcare datasets
• Building monitoring and alerting systems for data ingestion processes and pipeline health
• Applying software engineering best practices including test-driven development and modular design to data infrastructure
• Refactoring and rebuilding existing data ingestion processes to improve scalability and operational efficiency
• Working with containerization technologies (Docker, Kubernetes) to create portable and maintainable data solutions
• Identify and escalate inefficiencies within and across teams
• Provide technical guidance, mentorship to junior engineers, and promote best practices and coding standards
• Author and support high-quality technical documentation, assisting junior engineers in doing the same

Preferred Qualifications

Familiarity with healthcare data and regulatory environments (HIPAA compliance)

Benefits

The base salary range for this position is: $165,000-180,000/year