Remote Data Engineer

Summary

Join the CDC Foundation as a Data Engineer to play a crucial role in advancing data infrastructure and collaborating with public health agencies. This fully remote opportunity is part of the Workforce Acceleration Initiative, a federally funded program aimed at accelerating information system improvements for the nation's public health agencies.

Requirements

• Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field
• 5 years of experience with project oversight, including communications to end-users and technical staff
• Proficiency in programming languages commonly used in data engineering, such as Python, Java, Scala, or SQL
• Experience with Microsoft Azure cloud technologies and frameworks or similar (AWS)
• Strong understanding of database systems, including relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra)
• Experience regarding engineering best practices such as source control, automated testing, continuous integration and deployment, and peer review
• Knowledge of data warehousing concepts and tools
• Experience with cloud computing platforms
• Expertise in data modeling, ETL (Extract, Transform, Load) processes, and data integration techniques
• Solid understanding of FHIR and API-based architectures
• Familiarity with agile development methodologies, software design patterns, and best practices
• Strong analytical thinking and problem-solving abilities
• Excellent verbal and written communication skills, including the ability to convey technical concepts to non-technical partners effectively
• Experience working with project management and tracking software (e.g., JIRA and DevOps)
• Flexibility to adapt to evolving project requirements and priorities
• Outstanding interpersonal and teamwork skills; and the ability to develop productive working relationships with colleagues and partners
• Experience working in a virtual environment with remote partners and teams
• Proficiency in Microsoft Office

Responsibilities

• Document the existing architecture of the case-based disease surveillance system, including system interactions and data generation, storage, processing, and analysis
• Maintain current and develop the future-state case-based disease surveillance system architectural diagrams to ensure the system continues to be robust, scalable, and aligned with the SD-DOH’s public health goals
• Facilitate and collaborate with end-users and technical staff to understand requirements and deliver effective surveillance system solutions
• Evaluate, design, and implement both the existing and proposed future-state disease surveillance system solutions. Ensure that the solutions meet the needs of all stakeholders within SD-DOH, including being adaptable to evolving challenges
• Document all aspects of the system architecture, including design decisions, data flow processes, and system requirements. Maintain clear and comprehensive records to support ongoing system maintenance and future upgrades
• Create a final requirements document with all necessary technical specifications for a future-state case-based surveillance system listed clearly and concisely
• Create and manage the systems and pipelines that enable efficient and reliable flow of data, including ingestion, processing, and storage
• Collect data from various sources, transforming and cleaning it to ensure accuracy and consistency. Load data into storage systems or data warehouses
• Optimize data pipelines, infrastructure, and workflows for performance and scalability
• Monitor data pipelines and systems for performance issues, errors, and anomalies, and implement solutions to address them
• Implement security measures to protect sensitive information
• Collaborate with data scientists, analysts, and other partners to understand their data needs and requirements, and to ensure that the data infrastructure supports the organization's goals and objectives
• Collaborate with cross-functional teams to understand data requirements and design scalable solutions that meet business needs
• Implement and maintain ETL processes to ensure the accuracy, completeness, and consistency of data
• Design and manage data storage systems, including relational databases, NoSQL databases, and data warehouses
• Provide technical guidance to other staff
• Work closely with a multidisciplinary team, including data content experts, analysts, data scientists, epidemiologists, IT staff, and other organizational personnel
• Communicate effectively with partners at all levels of the organization to gather requirements, provide updates, and present findings