Remote Senior GCP Data Engineer

Summary

Join Experience.com as a Senior GCP Data Engineer in Chennai (with remote options) and contribute to building and maintaining scalable data pipelines. You will collaborate with data scientists and business teams, improving data models and fostering data-driven decision-making. This role requires expertise in GCP, big data technologies, and data engineering best practices. You will mentor a team of data engineers and drive the adoption of cloud-native technologies. The ideal candidate possesses strong problem-solving skills and excellent communication abilities. Experience.com offers a dynamic work environment focused on impactful experience management.

Requirements

• 8+ years of Strong experience in ETL and ELT data from various sources in Data Warehouses
• 8+ years of experience in Python, Pandas, Numpy, and SciPy
• 5+ years of Experience in GCP
• 5+ years of Experience in BigQuery, PySpark, and Pub/Sub
• 5+ years of Experience working with and creating data architectures
• Certified in Google Cloud Professional Data Engineer
• Advanced proficiency in Google Cloud services such as Dataflow, Dataproc, Dataprep, Data Studio, and Cloud Composer
• Proficient in writing complex Spark (PySpark) User Defined Functions (UDFs), Spark SQL, and HiveQL
• Good understanding of Elastic search
• Experience in assessing and ensuring data quality, data testing, and addressing data quality issues
• Excellent understanding of Spark architecture and underlying frameworks including storage management
• Solid background in database design and development, database administration, and software engineering across full life cycles
• Experience with NoSQL data stores like MongoDB, DocumentDB, and DynamoDB
• Knowledge of data governance principles and practices, including data lineage, metadata management, and access control mechanisms
• Experience in implementing and optimizing data security controls, encryption, and compliance measures in GCP environments
• Ability to troubleshoot complex issues, perform root cause analysis, and implement effective solutions in a timely manner
• Proficiency in data visualization tools such as Tableau, Looker, or Data Studio to create insightful dashboards and reports for business users
• Strong communication and interpersonal skills to effectively collaborate with technical and non-technical stakeholders, articulate complex concepts, and drive consensus
• Experience with agile methodologies and project management tools like Jira or Asana for sprint planning, backlog grooming, and task tracking

Responsibilities

• Collaborate with cross-functional teams to define, prioritize, and execute data engineering initiatives aligned with business objectives
• Design and implement scalable, reliable, and secure data solutions by industry best practices and compliance requirements
• Drive the adoption of cloud-native technologies and architectural patterns to optimize the performance, cost, and reliability of data pipelines and analytics solutions
• Mentor and lead a team of Data Engineers
• Demonstrate a drive to learn and master new technologies and techniques
• Apply strong problem-solving skills with an emphasis on building data-driven or AI-enhanced products
• Coordinate with ML/AI and engineering teams to understand data requirements

Benefits

Remote work, flexible hours