Data Engineer

Summary

Join Mercari as a Data Engineer and design, build, and operate ETL pipelines at scale. You will automate data and machine learning processes, design data structures, and build knowledge graphs. Develop and operate APIs/tools, provide technical solutions using Big Data technologies, and create technical design documents. Design and develop data platforms using Python, Spark, and BigQuery. Build DevOps platforms for CI/CD. Remote work is permitted within the U.S. The position offers a competitive salary and a comprehensive benefits package.

Requirements

• Bachelor of Science degree in Computer Science or closely related field of study and five (5) years of experience as a Data Engineer, Data Specialist, or related occupation where required experience gained
• Five (5) years of experience in Confluence
• Five (5) years of experience in JIRA
• Five (5) years of experience in GIT
• Five (5) years of experience in CI/CD pipelines
• Five (5) years of experience in Agile methodologies
• Five (5) years of experience in Java
• Five (5) years of experience in Python
• Five (5) years of experience in Data Modeling or Data Warehouse
• Five (5) years of experience in ETL: Apache Airflow
• Five (5) years of experience in Container: Docker or Kubernetes
• Five (5) years of experience in API: gRPC, Tensorflow Serving, or Flask (REST)
• Five (5) years of experience in Database: MySQL, Postgres, Oracle, SqlServer, or Google Spanner
• Five (5) years of experience in Distributed Processing: Apache Beam or Apache Spark
• Five (5) years of experience in Machine Learning: Tensorflow, Keras, or Scikit-Learn, etc
• Five (5) years of experience in Cloud: Google Cloud (BigQuery, Google Dataflow, or Google Dataproc, etc.)

Responsibilities

• Design, build, and operate ETL pipelines at scale
• Automate processes related to data products and machine learning products
• Design data structure for data products
• Build knowledge graphs, flow charts, and system diagrams for problem analysis
• Develop and operate API/tools related to data products and machine learning products
• Provide technical solutions using Big Data technologies and create technical design documents
• Design and develop Data Platform using Python, Spark, and BigQuery
• Build Devops platform for Continuous Integration/Continuous deployment stack for ETL applications teams
• Profile, debug, and optimize apps

Benefits

• Work remote from anywhere in the US and receive flexible time off
• Top-notch insurance plans, best-in-class new parent offerings and access to mind and body wellness apps
• As we grow, your career opportunities grow, we provide access to new tools, technologies and learning opportunities
• We provide a strong financial foundation and rewards that go beyond the paycheck
• We are each other's biggest fans- we celebrate from virtual coffee breaks to recognition programs