Lead Data Engineer

Summary

Join Mysten Labs' Data team as a Staff Data Engineer to design and execute the next iteration of our data systems. You will design web3 analytics systems at web2 scale and help define new standards for blockchain analytics. This role combines hands-on work with data architecture planning, partnering with Data and Engineering heads to process high-volume data reliably and robustly. You will work across Mysten Labs, becoming an expert on Sui blockchain data structures. The position requires a combination of individual contribution and architectural planning. You will be instrumental in building the future of web3.

Requirements

• 5+ years experience in data engineering
• Strong SQL and Python
• Strong and informed opinions on data orchestrators, catalogs, governance, and testing frameworks
• Experience combining in-house and external data

Responsibilities

• Design and implement scalable ingestion pipelines with a reusable and modular framework
• Build robust, reusable frameworks to ingest data from internal sources (e.g., Prod DBs, cloud buckets, etc) and external APIs or files (e.g., CSVs, webhooks)
• Ensure idempotency, backfill support, and error handling in pipeline design
• Optimize and own data warehousing, with clear table definitions, schemas, cost efficiency
• Architect a centralized data lake/warehouse with clear schemas and partitioning strategies
• Support both batch and streaming workloads, and optimize for cost and performance
• Enable data discoverability, usability, and governance
• Implement or integrate data cataloging and lineage tools
• Define naming conventions, documentation standards, and ownership metadata to make data self-serve and intuitive for data scientists, analysts, and product / GTM teams
• Set up a mechanism for scalable access controls (with RBAC or ABAC), PII tagging, and data obfuscation
• Enable approaches for data quality checks, validation pipelines, and alerting for broken or stale data
• Develop a strong understanding of how to use on-chain and off-chain data together

Preferred Qualifications

• Experience with, or interest in learning, Rust
• Prior blockchain and cryptocurrency experience
• Experience designing and implementing streaming data solutions
• Experience or interest in security analytics and data for security teams