SprintFWD is hiring a
Lead Data Engineer in United States

Summary

Comma8 is seeking a Data Engineer with 3-5 years of experience to lead the design and growth of data solutions. The role involves managing large data volumes, constructing streaming data pipelines, automating complex workflows, mentoring a team, and ensuring high operational efficiency. Benefits include health insurance, dental insurance, vision insurance, life insurance, 401K plan, fitness and wellness benefits, continued education courses, conference attendance, and flexible vacation, sick, and mental-health days.

Requirements

• 3-5 years of data engineering experience developing large data pipelines
• Strong SQL skills and ability to create queries to extract data and build performant datasets
• Hands-on experience with data integration tools (e.g. Apache Spark, Apache Kafka)
• Hands-on experience with cloud-based data services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
• Experience with version control systems (e.g., Git) and collaborative development practices
• Strong programming skills in Python
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query)
• Solid experience with data integration toolsets (i.e Airflow) and writing and maintaining Data Pipelines
• Strong in Data Modeling techniques and Data Warehousing standard methodologies and practices
• Familiar with Scrum and Agile methodologies

Responsibilities

• Lead the design and growth of our Products and Data Warehouses around our clients Analytics
• Design and develop scalable data warehousing solutions, building ETL pipelines in Big Data environments (cloud, on-prem, hybrid)
• Manage the transformation of large daily batch data volumes in the cloud using Apache Spark, EMR, and Glue, ensuring streamlined processing and cost savings
• Construct and maintain high-throughput streaming data pipelines using technologies like Kinesis, Spark Streaming, and Elasticsearch, while minimizing response lag
• Automate and orchestrate complex data workflows using Python, Apache Airflow, and Step Functions to eliminate bottlenecks in data pipelines
• Mentor and guide a team, providing technical expertise in SQL query execution, data manipulation, data visualization, and performance optimization
• Develop, test, and deploy scalable reverse ETL solutions using API Gateway, Python (Flask), and Lambda, achieving near-zero latency and high scalability
• Help architect data solutions/frameworks and define data models for the underlying data warehouse and data lakes
• Collaborate with key stakeholders to map, implement, and deliver successful data solutions
• Maintain detailed documentation of your work and changes to support data quality and data governance
• Ensure high operational efficiency and quality of your solutions to meet SLAs and support commitment to our clients
• Be an active participant and advocate of agile/scrum practice to ensure health and process improvements for your team

Preferred Qualifications

Nice to have experience with Cloud technologies like AWS (S3, EMR, EC2)

Benefits

• Health insurance
• Dental insurance
• Vision insurance
• Life insurance
• 401K plan
• Various Fitness and Wellness Benefits (Group Classes, Free Training, Products, etc.)
• Continued Education Courses
• Conference Attendance
• Flexible vacation, sick, and mental-health days