Softrams is hiring a
Senior Data Engineer, Remote - United States

Summary

Softrams is seeking a Data Engineer for federal health IT solutions. The role involves developing scalable data pipelines on AWS, collaborating with cross-functional teams, and ensuring data quality, integrity, and security. The ideal candidate should have a Master's degree in computer science or related field with 4+ years of experience in data engineering, proficiency in Python, Apache Spark, SQL, and AWS services, and knowledge of handling various data formats.

Requirements

• Master's degree in computer science, Data Engineering, or a related field with a minimum of 4 years of experience in data engineering (PhD is a plus)
• At least 5 years of experience. in programming with Python, focusing on data engineering tasks and scripting
• A minimum of 3 years of hands-on experience. with Apache Spark for large-scale data processing, including building data visualizations using PySpark and Jupyter Notebooks
• Proficiency in data manipulation and analysis using Python libraries such as NumPy and Pandas
• Proven expertise in designing and managing data pipelines using AWS services, including AWS EMR and AWS S3
• At least 2 years of experience. utilizing Jupyter Notebooks for data exploration, analysis, visualization, and collaboration
• Strong knowledge of handling various data formats, including CSVs and Parquet files
• Extensive experience with cloud infrastructure, specifically AWS, and a thorough understanding of its services and capabilities

Responsibilities

• Develop, implement, and optimize scalable data pipelines on AWS to ensure efficient processing and storage of large datasets
• Collaborate with cross-functional teams to define data requirements and establish effective data governance frameworks
• Design and maintain robust data infrastructure to support diverse data workloads, including batch processing using AWS EMR
• Manage large volumes of structured and unstructured data, utilizing AWS S3 and AWS Redshift for efficient storage and querying
• Create, maintain, and enhance data ingestion processes using Apache Spark to ensure data quality, integrity, and consistency
• Utilize PySpark and Jupyter Notebooks to build data visualizations that support data-driven decision-making and provide insights
• Work closely with stakeholders to understand data needs and develop innovative solutions that drive data-driven decision-making
• Utilize Jupyter Notebooks for data exploration, visualization, and sharing of insights to support data science and analytics efforts
• Implement and enforce best practices for data security and compliance, particularly when handling sensitive healthcare data

Preferred Qualifications

• 2+ years of experience. in Scala programming
• Familiarity with EMR Studios and Anaconda for data engineering and analytics
• Experience working with AWS Bedrock and other LLM SaaS platforms (such as OpenAI or similar) for AI/ML projects
• Proven experience in curating datasets for use in DAGs or model training processes
• Background in data engineering within the healthcare domain, particularly with administrative claims data or health insurance claims data
• Knowledge of CMS (Center for Medicare and Medicaid Services) protocols and experience with CMS’s Integrated Data Repository (IDR)
• Understanding of the Hadoop ecosystem and distributed computing concepts

Benefits

• 65%-75% company-sponsored (including dependents) premiums towards medical, dental and vision insurance. For eligible plans and tiers, we provide 100% company-paid. medical insurance
• Retirement 401(k) plan with employer matching. Immediate vesting
• Vacation and sick leave
• Maternity and parental leave
• Discretionary bonuses, spot awards, gifts, and tenure-based rewards
• Company-sponsored role-based training and certifications
• Monthly DoordashDashPass subscription
• Group discounts via LifeMart ADP