Principal Data Engineer

Summary

Join Saviynt as a Principal Data Engineer and lead the development, administration, and optimization of our ElasticSearch infrastructure. You will be responsible for upgrading ElasticSearch versions, designing scalable solutions, and ensuring optimal performance. This critical role involves collaborating with cross-functional teams, implementing multi-tenant architecture, and defining best practices for data modeling and security. You will also mentor junior engineers and troubleshoot complex issues. The ideal candidate possesses extensive experience in ElasticSearch administration and optimization, along with strong expertise in related technologies and cloud platforms.

Requirements

• 8+ years of experience in Data Engineering, DevOps, or Software Engineering, with a focus on ElasticSearch administration and optimization
• Proven experience upgrading ElasticSearch (6.x to 7.x, 7.x to 8.x) in a production environment
• Strong expertise in ElasticSearch performance tuning, cluster scaling, and optimization techniques
• Experience designing and implementing multi-tenant solutions in ElasticSearch
• Hands-on experience with indexing strategies, data partitioning, and sharding techniques
• Solid understanding of Elasticsearch security best practices, including TLS encryption, role-based access control (RBAC), and secure data storage
• Proficiency in SQL, NoSQL databases, and data storage technologies (e.g., MySQL, Redis, S3)
• Experience integrating ElasticSearch with Java/SpringBoot applications and working with RESTful APIs
• Strong knowledge of cloud platforms (AWS, Azure) and experience deploying ElasticSearch in containerized environments (Docker, Kubernetes)
• Experience with infrastructure automation tools (Terraform, Ansible, Helm, or CloudFormation) for managing ElasticSearch clusters
• Strong problem-solving and analytical skills with the ability to troubleshoot complex distributed systems
• Excellent communication and collaboration skills to work cross-functionally with engineers, architects, and business stakeholders

Responsibilities

• Lead and execute ElasticSearch upgrades from version 6.x to 7.x and from version 7.x to 8.x, ensuring minimal downtime and smooth migrations
• Design and implement best practices for scalability, performance tuning, and security of ElasticSearch clusters
• Optimize search indexing and query performance for large-scale datasets, improving search relevance and efficiency
• Develop and maintain monitoring and alerting mechanisms for ElasticSearch clusters, ensuring high availability and resilience
• Collaborate with cross-functional teams (Engineering, DevOps, Security) to integrate ElasticSearch with existing applications and infrastructure
• Evaluate and implement multi-tenant architecture within ElasticSearch to support large-scale deployments across multiple customers
• Define and enforce best practices for data modeling, indexing strategies, and cluster sharding to ensure optimal system performance
• Troubleshoot and resolve ElasticSearch issues related to query performance, data consistency, and infrastructure scaling
• Maintain and enhance security configurations, including encryption, access control, and compliance with industry standards
• Automate operational tasks using scripting and infrastructure-as-code (IaC) tools for ElasticSearch management
• Mentor and guide engineers on ElasticSearch administration, optimization, and best practices