Remote Data Engineer

Summary

Join our data team as a talented Data Engineer with expertise in AWS to design, build, and maintain robust data pipelines and architectures.

Requirements

• Experience working in AWS/Public Cloud-based infrastructure
• Basic understanding and hands-on skills in Terraform/IaC
• Strong knowledge and experience in Data pipelines through modern cloud computing
• Databricks, Glue, Lambda etc
• Hands on experience in Data storage and warehousing through cloud storage
• S3, managed warehousing, Postgres, RDS, Redshift, etc
• Hands-on experience in Data analysis, coding, and testing
• Python, Spark, Scala, Shell
• Experience creating data architecture diagrams, dictionaries, documentation, and test plans
• Experience with basic data monitoring and alerting patterns and tools

Responsibilities

• Understands data architecture and overall business requirements around the data platform
• Collects key data requirements from product/business groups on a regular basis
• Accountable for data strategy on how the organization collects, processes, uses, governs, and stores data
• Owns data engineering life cycle including research, proof of concept, architecture, development, testing, deployment, and maintenance of data platform
• Data collection through pipelines from various sources, including databases, APIs, partners, and external providers
• Standardize data by converting different data formats from sources to a standard format and cleanse it by removing inconsistencies
• Design and develop ETL or ELT, cleansing, aggregation, enrichment, and making data usable for analytics
• Make consumable data available on high-throughput data stores and build standard data consumption patterns (APIs, flat file, Database tables etc.) for users and systems to fetch data at scale
• Add instrumentation across data pipelines for monitoring and alerting on internal problems before they result in user-visible outages
• Build processes and diagnostic tools to troubleshoot, maintain, and optimize data processes and respond to customer and production incidents
• Implements end-to-end automated CICD practices for building, scanning, packaging, testing, and deploying on a data platform, with the ability to continuously deliver secure solutions at scale
• Adopt continuous learning of modern data engineering practices. Maintain industry standards through incremental adoption of new technology and best practices
• Create and continuously maintain high-quality, up-to-date documentation of data processes, data dictionaries, and transformations to meet audit and compliance standards

Benefits

• Compensation commensurate with experience
• Comprehensive benefits package including medical, dental and vision insurance
• Health Savings Account
• Generous PTO and Holiday Pay
• 401(k) retirement plan
• Remote work-from-home option consideration