Senior Data Engineer

Summary

Join PrizePicks, a rapidly growing sports company, as a Senior Data Engineer and contribute to building and maintaining analytics tools and workflows. You will enhance core data platforms, develop new integrations, and manage Kubernetes-based infrastructure on GCP. Collaborate with various teams to improve data transformation processes and implement Infrastructure as Code solutions. Develop and manage CI/CD pipelines, ensuring data workflows are thoroughly tested and deployed efficiently. Serve as a Data Engineering thought leader, staying current with emerging technologies and mentoring junior team members. This role requires on-call support and a commitment to maintaining system reliability.

Requirements

• Proven experience in designing and implementing both real-time and batch data ingestion frameworks, with a strong focus on data modeling, ETL/ELT best practices, and maintaining high data quality and reliability
• Skilled in managing Kubernetes-based infrastructure on Google Cloud Platform (GCP), leveraging GitOps principles with Argo CD for automated and consistent deployment workflows
• Experience in designing and automating data governance processes and tool integrations across complex technical environments, ensuring data protection and integrity throughout the product lifecycle and data flows
• 5+ years of experience in a data Engineering, or data-oriented software engineering role creating and pushing end-to-end data engineering pipelines
• 2+ years of experience acting as technical lead and providing mentorship and feedback to junior engineers
• Extensive experience building and optimizing cloud-based data streaming pipelines and infrastructure
• Extensive experience exposing real-time predictive model outputs to production-grade systems leveraging large-scale distributed data processing and model training
• Experience in most of the following: SQL/NoSQL databases/warehouses: Postgres, BigQuery, BigTable, Materialize, AlloyDB, etc
• Replication/ELT services: Data Stream, Hevo, etc
• Data Transformation services: Spark, Dataproc, etc
• Scripting languages: SQL, Python, Go
• Cloud platform services in GCP and analogous systems: Cloud Storage, Cloud Compute Engine, Cloud Functions, Kubernetes Engine etc
• Data Processing and Messaging Systems: Kafka, Pulsar, Flink
• Code version control: Git
• Data pipeline and workflow tools: Argo, Airflow, Cloud Composer
• Monitoring and Observability platforms: Prometheus, Grafana, ELK stack, Datadog
• Infrastructure as Code platforms: Terraform, Google Cloud Deployment Manager
• Other platform tools such as Redis, FastAPI, and Streamlit
• Excellent organizational, communication, presentation, and collaboration experience with organizational technical and non-technical teams
• Graduate degree in Computer Science, Mathematics, Informatics, Information Systems or other quantitative field

Responsibilities

• Enhance the capabilities of our existing Core Data platforms and develop new integrations with both internal and external APIs within the Data organization
• Work closely with DevOps, architects, and engineers to ensure the success of the Core Data platform
• Collaborate with Analytics Engineers to enhance data transformation processes, streamline CI/CD pipelines, and optimize team collaboration workflows
• Architect and implement Infrastructure as Code (IaC) solutions to automate and streamline the deployment and management of data infrastructure. Ensure scalable, consistent configurations aligned with data engineering standards and best practices
• Develop and manage CI/CD pipelines to automate and streamline the deployment of data solutions. Ensure that data workflows are thoroughly tested, integrated, and deployed efficiently, following best practices for version control, automation, and quality assurance
• Ensure code is thoroughly tested, effectively integrated, and efficiently deployed, in alignment with industry best practices for version control, automation, and quality assurance
• Serve as a Data Engineering thought leader within the broader PrizePicks technology organization by staying current with emerging technologies, implementing innovative solutions, and sharing knowledge and best practices with junior team members and collaborators
• Provide on-call support as part of a shared rotation between the Data and Analytics Engineering teams to maintain system reliability and respond to critical issues

Preferred Qualifications

Experience building real-time production data engineering pipelines in a daily fantasy sports platform

Benefits

• Company-subsidized medical, dental, & vision plans
• 401(k) plan with company match
• Annual bonus
• Flexible PTO to encourage a healthy work/life balance (2 weeks STRONGLY encouraged!)
• Generous paid leave programs, including 16-week paid parental leave and disability benefits
• Workplace flexibility and modern work schedules focused on getting the job done, not hours clocked
• Company-wide in-person events and team outings
• Lifestyle enhancement program
• Company equipment provided (Windows & Mac options)
• Annual performance reviews with opportunities for growth and career development