Platform Engineer

Summary

Join PayPay's Streaming Platform team as a senior engineer and play a key role in enhancing our Kafka-based architecture. You will lead the development, operation, and modernization of our streaming infrastructure, ensuring high performance and reliability for over 69 million users. Responsibilities include managing self-hosted and AWS MSK Kafka clusters, optimizing performance, and collaborating with application teams. This hands-on role requires extensive Kafka experience and expertise in AWS, Terraform, and Ansible. You will contribute to building scalable, secure, and observable data pipelines, automating infrastructure workflows, and guiding best practices. The position offers a hybrid workstyle and a comprehensive benefits package.

Requirements

• Minimum of 3 years of engineering experience with Apache Kafka in production environments
• Minimum of 3 years of experience in AWS, Terraform, Ansible and Linux Administration
• Strong hands-on experience with Kafka cluster operations, including setup, tuning, and maintenance
• Experience with Kafka authentication and authorization operations
• Familiarity with AWS cloud platform, especially Amazon MSK (Managed Streaming for Kafka)
• Proficiency in one or more general-purpose programming languages (e.g., Python, Java, Go)
• Experience with infrastructure automation and configuration management tools, such as Terraform or Ansible
• Understanding of modern system design using microservice architecture
• Working knowledge of Git and CI/CD tools

Responsibilities

• Tech leadership, architecture, development, and operations for Apache Kafka, including both self-hosted clusters and AWS MSK
• Ensure deployment, management, and smooth operation of highly available Kafka infrastructure
• Monitoring and alerting of Kafka for reliability, throughput, and latency
• Performance tuning and instrumentation of Kafka clusters and pipelines
• Partner with application teams to guide Kafka topic design, schema management, and best practices
• Contribute to the development of scalable, secure, and observable data pipelines
• Automate infrastructure and operations workflows using infrastructure-as-code tools

Preferred Qualifications

• Deep expertise in running and scaling Apache Kafka both self-hosted and in managed cloud environments (e.g., AWS MSK)
• Experience migrating or operating Kafka in KRaft mode (no ZooKeeper)
• Experience in data replicate between Kafka clusters, enabling seamless data migration, disaster recovery, and cross-region data synchronization
• Exposure to Kafka security, including ACLs, TLS, SASL, and IAM-based auth on MSK
• Deep expertise in Kafka internals, including broker tuning, partitioning, replication, and fault tolerance
• Contributions to Kafka-related open source projects or community involvement
• Experience with Kafka Connect, Kafka Streams, or other stream processing frameworks
• Experience guiding cross-team adoption of Kafka in microservice architectures
• Experience in operating distributed systems
• Working experience in a full remote environment
• Bachelor’s or Master’s Degree in Computer Science or a related field

Benefits

• Social Insurance (health insurance, employee pension, employment insurance and compensation insurance)
• 401K
• Translation/Interpretation support
• VISA sponsor + Relocation support
• Super Flex Time (No Core Time)
• In principle, 9:00am-5:45pm + 1h break (actual working hours: 7h45m + 1h break)
• Every Sat/Sun/National holidays (In Japan)/New Year's break/Company-designated Special days
• Annual leave (up to 14 days in the first year, granted proportionally according to the month of employment. Can be used from the date of hire)
• Personal leave (5 days each year, granted proportionally according to the month of employment）
• ���PayPay's own special paid leave system, which can be used to attend to illnesses, injuries, hospital visits, etc., of the employee, family members, pets, etc
• Annual salary paid in 12 installments (monthly)
• Based on skills, experience, and abilities
• Reviewed once a year
• Special Incentive once a year *Based on company performance and individual contribution and evaluation
• Late overtime allowance
• ���Payroll payment can be changed to digital salary payment “PayPay Paycheck” for an amount set by you