Senior Data Engineer

Summary

Join BeyondTrust as a Senior Data Engineer and contribute to building and enhancing a state-of-the-art data lakehouse processing billions of events daily. You will create valuable insights from vast data quantities, focusing on security and computational efficiency. The role involves optimizing data workloads, developing data processing routes, performing complex data transformations, and building new data development frameworks. You will monitor data workflows, leverage CI/CD best practices, and create ML/engineering solutions for cybersecurity challenges. BeyondTrust offers a flexible and supportive work environment.

Requirements

• Strong programming and technology knowledge in cloud data processing
• Previous experience working in matured data lakes
• Strong data modelling skills for analytical workloads
• Spark (or equivalent parallel processing framework) experience is needed, existing Databricks knowledge is a plus
• Interest and aptitude for cybersecurity; interest in identity security is highly preferred
• Technical understanding of underlying systems and computation minutiae
• Experience working with distributed systems and data processing on object stores
• Ability to work autonomously is a MUST

Responsibilities

• Optimize data workloads at a software level by improving processing efficiency
• Develop new data processing routes to remove redundancy or reduce transformation overhead
• Perform complex data transformation in order to accomplish niche business use cases
• Develop new frameworks for data development to supercharge our delivery velocity
• Monitor and maintain existing data workflows
• Use observability best practices to ensure pipeline performance
• Perform complex transformations on both real time and batch data assets
• Create new ML/Engineering solutions to tackle existing issues in the cybersecurity space
• Leverage CI/CD best practices to effectively develop and release source code

Preferred Qualifications

• Data warehousing for analytics use cases
• Highly proficient in Python (or another object orientated language)
• Distributed processing framework experience
• Spark
• Realtime processing experience
• Some experience with graph data stores