Principal Data Engineer

Summary

Join the Segment team at Twilio as a Principal Data Engineer, leading the technical direction of the core data processing infrastructure. Build and scale large-scale distributed systems supporting Segment’s customer data platform (CDP). This role requires deep data engineering expertise, a strong product sense, and the ability to make high-leverage architectural decisions. You will collaborate with cross-functional teams to design and implement scalable data pipelines, high-throughput compute engines, and durable storage solutions. Lead end-to-end delivery of major platform features and improvements with measurable business impact. Mentor and guide a team of data engineers.

Requirements

• 8+ years of software engineering experience, with at least 5+ years focused on large-scale data engineering
• Proficiency in Scala or Java, with hands-on experience building systems using Apache Spark or similar distributed compute engines
• Deep understanding of distributed systems, stream processing, and data pipeline design patterns
• Experience processing large volumes of data using cloud-native and open-source big data tools (e.g., Spark, Kafka, Parquet, Delta Lake)
• Track record of leading technical initiatives and collaborating across engineering and product teams
• Strong problem-solving skills and a bias toward ownership and execution
• Experience with AWS and infrastructure-as-code tools
• Excellent written and verbal communication skills

Responsibilities

• Architect and implement large-scale data processing systems that power core Segment capabilities
• Design and optimize distributed compute frameworks leveraging Apache Spark, Scala, and cloud-native services
• Drive scalability and performance improvements across our data pipelines, enabling real-time and batch data processing
• Partner with product, infrastructure, and platform teams to build reusable systems that accelerate development across Segment
• Mentor and guide a team of data engineers, setting technical standards and fostering a culture of operational excellence
• Lead end-to-end delivery of major platform features and improvements with measurable business impact
• Ensure systems are designed for observability, maintainability, security, and compliance

Preferred Qualifications

• Experience with other data technologies such as Apache Flink, Trino, Snowflake, or Databricks
• Familiarity with data governance, quality frameworks, and GDPR/CCPA compliance
• Exposure to real-time personalization or ML-driven analytics use cases
• Experience working in globally distributed teams

Benefits

• Health care insurance
• 401(k) retirement account
• Paid sick time
• Paid personal time off
• Paid parental leave