Experience.com is hiring a
Senior Data Engineer, Remote - India

Summary

The Senior Data Engineer (GCP) is a professional responsible for developing and maintaining scalable data pipelines, collaborating with cross-functional teams, and improving data models. The position requires 8+ years of experience in ETL and ELT data from various sources, Python, Pandas, Numpy, SciPy, GCP, BigQuery, PySpark, Pub/Sub, and advanced proficiency in Google Cloud services. The role also involves mentoring a team of Data Engineers and demonstrating a drive to learn new technologies.

Requirements

• 8+ years of Strong experience in ETL and ELT data from various sources in Data Warehouses
• 8+ years of experience in Python, Pandas, Numpy, and SciPy
• 5+ years of Experience in GCP
• 5+ years of Experience in BigQuery, PySpark, and Pub/Sub
• 5+ years of Experience working with and creating data architectures
• Advanced proficiency in Google Cloud services such as Dataflow, Dataproc, Dataprep, Data Studio, and Cloud Composer
• Proficient in writing complex Spark (PySpark) User Defined Functions (UDFs), Spark SQL, and HiveQL
• Good understanding of Elastic search
• Experience in assessing and ensuring data quality, data testing, and addressing data quality issues
• Excellent understanding of Spark architecture and underlying frameworks including storage management
• Solid background in database design and development, database administration, and software engineering across full life cycles
• Experience with NoSQL data stores like MongoDB, DocumentDB, and DynamoDB
• Knowledge of data governance principles and practices, including data lineage, metadata management, and access control mechanisms
• Experience in implementing and optimizing data security controls, encryption, and compliance measures in GCP environments
• Ability to troubleshoot complex issues, perform root cause analysis, and implement effective solutions in a timely manner
• Proficiency in data visualization tools such as Tableau, Looker, or Data Studio to create insightful dashboards and reports for business users
• Strong communication and interpersonal skills to effectively collaborate with technical and non-technical stakeholders, articulate complex concepts, and drive consensus
• Experience with agile methodologies and project management tools like Jira or Asana for sprint planning, backlog grooming, and task tracking

Responsibilities

• Collaborate with cross-functional teams to define, prioritize, and execute data engineering initiatives aligned with business objectives
• Design and implement scalable, reliable, and secure data solutions by industry best practices and compliance requirements
• Drive the adoption of cloud-native technologies and architectural patterns to optimize the performance, cost, and reliability of data pipelines and analytics solutions
• Mentor and lead a team of Data Engineers
• Demonstrate a drive to learn and master new technologies and techniques
• Apply strong problem-solving skills with an emphasis on building data-driven or AI-enhanced products
• Coordinate with ML/AI and engineering teams to understand data requirements