Remote Data Engineer

Summary

Join our team as a Data Engineer and contribute to designing and creating impactful software powering enterprise data platform solutions. You will work within a team of globally dispersed cloud Architects, Engineers, and Consultants to design and create end-to-end Cloud-based solutions with a focus on application and data.

Responsibilities

• Design and development of end-to-end Cloud-based solutions with heavy focus on application and data and good understanding of infrastructure
• Translate complex functional and technical requirements into detailed designs
• Write high-performance, reliable, and maintainable code
• Develop test automation and associated tooling needed for the project
• Work on complex and varied Cloud-based projects including tasks such as collecting, parsing, managing, analyzing, and visualizing very large datasets etc
• Maintain and execute DataOps tasks such as performance optimization of ETL/ELT pipeline, diagnosis and troubleshooting of pipeline issues, interpreting Data Observability Dashboards, Enhancements etc
• Perform Data Pipeline specific DevOps activities such as Infrastructure provisioning, writing IaC code, implementing data security etc
• Analyze potential issues and complete root cause analysis and assign issues to be resolved
• Follow up with Data Engineering team members to see fixes through completion
• Review bug descriptions, functional requirements, and design documents, incorporating this information into test plans and test cases
• Performance tuning for batch and real-time data processing
• Secure components of clients' Cloud Data platforms
• Health-checks and configuration reviews
• Data pipelines development – ingestion, transformation, cleansing
• Data flow integration with external systems
• Integration with data access tools and products
• Foundational CI/CD for all infrastructure components, data pipelines, and custom data apps
• Common operational visibility of the data platform from data platform infrastructure to data pipelines, machine learning apps
• Assist client application developers and advise on efficient data access and manipulations
• Define and implement efficient operational processes