Data Engineer

Summary

Join PayPay's Data Engineering team and help scale our platform using cutting-edge technologies. We are looking for talented Data Engineers to design, develop, and maintain scalable data ingestion pipelines. You will optimize and manage large-scale data pipelines, implement data processing workflows, and maintain and enhance the Data Lakehouse. Collaboration with cross-functional teams is essential to ensure seamless data flow. This is a full-time, work-from-anywhere position in Japan with flexible hours and generous benefits.

Requirements

• 5+ years experience as a Data Engineer or in a similar role
• Hands-on experience with Databricks, Delta Lake, Spark, and Scala
• Experience designing, building, and operating a DataLake or Data Warehouse
• Knowledge of Data Orchestration tools such as Airflow, Dagster, Prefect
• Familiarity with change data capture tools like Canal, Debezium, and Maxwell
• Experience in at least one primary language (e.g. Scala, Python) and SQL (any variant)
• Experience with data cataloging and metadata management using AWS Glue Data Catalog, Lakeformation, or Unity Catalog
• Proficiency in Terraform for infrastructure as code (IaC)
• Strong problem-solving and debugging skills to troubleshoot complex data issues
• Excellent communication and collaboration skills
• Able to make sound decisions and learn quickly while factoring in complex technical contexts

Responsibilities

• Design, develop, and maintain scalable data ingestion pipelines using Databricks, Airflow, Lambda, and Terraform
• Optimize and manage large scale data pipelines to ensure high performance, reliability, and efficiency
• Implement data processing workflows using Spark, Delta Lake, PySpark, and Scala
• Maintain and enhance Data Lakehouse with Unity Catalog for effective data management and discovery
• Build automation, framework, and improved tooling to support end to end data engineering workflows
• Collaborate with cross-functional teams to ensure seamless data flow and integration across the organization
• Implement best practices for observability, data governance, security, and compliance

Benefits

• Social Insurance (health insurance, employee pension, employment insurance and compensation insurance)
• 401K
• Translation/Interpretation support
• VISA sponsor + Relocation support
• Annual leave (up to 14 days in the first year, granted proportionally according to the month of employment. Can be used from the date of hire)
• Personal leave (5 days each year, granted proportionally according to the month of employment）
• PayPay's own special paid leave system, which can be used to attend to illnesses, injuries, hospital visits, etc., of the employee, family members, pets, etc
• Annual salary paid in 12 installments (monthly)
• Special Incentive once a year *Based on company performance and individual contribution and evaluation
• Late overtime allowance
• Payroll payment can be changed to digital salary payment “PayPay Paycheck” for an amount set by you
• Work From Anywhere at Anytime
• Super Flex Time (No Core Time)
• Every Sat/Sun/National holidays (In Japan)/New Year's break/Company-designated Special days