Data Engineer

Summary

Join the CDC Foundation as a Data Engineer to design, build, and maintain data infrastructure for a public health organization. This role will play a crucial part in advancing the CDC Foundation's mission by providing technical expertise to support the development of the South Carolina DPH Data Pipeline.

Responsibilities

• Assess existing data pipeline and work with programs to envision an ideal future state
• Recommend and participate in the development of the systems and pipelines that enable efficient and reliable flow of data, including ingestion, processing, and storage
• Collect data from various sources and transforming it to conform with defined rules to ensure accuracy and consistency. Load data into storage systems or data warehouses
• Optimize data pipelines, infrastructure, and workflows for performance and scalability
• Monitor data pipelines and systems for performance issues, errors, and anomalies, and implement solutions to address them
• Implement security measures to protect sensitive and highly confidential information
• Collaborate with data scientists, analysts, and other partners to understand their data needs and requirements, and to ensure that the data infrastructure supports the organization's goals and objectives
• Collaborate with cross-functional teams to understand data requirements and design scalable solutions that meet business needs
• Implement and maintain ETL processes to ensure the accuracy, completeness, and consistency of data
• Design and manage data storage systems, including relational databases, NoSQL databases, and data warehouses
• Knowledgeable about industry trends, best practices, and emerging technologies in data engineering, and incorporating the trends into the organization's data infrastructure
• Provide technical guidance to other staff
• Communicate effectively with partners at all levels of the organization to gather requirements, provide updates, and present findings