GCP HPC DevOps Engineer

Summary

Join Xebia, a global leader in digital solutions, and lead the migration of on-premises SLURM-based HPC clusters to Google Cloud Platform (GCP). You will design, implement, and manage scalable and secure HPC infrastructure solutions on GCP, optimizing SLURM configurations and workflows for efficient cloud resource use. Responsibilities include automating cluster deployment and maintenance using scripting languages and automation tools, integrating HPC software stacks, deploying and troubleshooting applications using parallel computing frameworks, and collaborating with teams to ensure smooth migration and operation. You will provide expert support for performance tuning and cluster resource optimization, staying current with emerging HPC technologies and GCP services. This role requires extensive experience with HPC environments, Linux-based systems, and cloud migration, along with proficiency in automation tools and scripting languages. The ideal candidate will possess strong problem-solving skills and a focus on optimizing HPC workloads and resource utilization.

Requirements

• 5+ years of experience with HPC (High-Performance Computing) environments, including SLURM workload manager, MPI, and other HPC-related software
• Extensive hands-on experience managing Linux-based systems, including performance tuning and troubleshooting in an HPC context
• Proven experience migrating and managing SLURM clusters in cloud environments, preferably GCP
• Proficiency with automation tools such as Ansible and Terraform for cluster deployment and management
• Experience with Spack for managing and deploying HPC software stacks
• Strong scripting skills in Python, Bash, or similar languages for automating cluster operations
• In-depth knowledge of GCP services relevant to HPC, such as Compute Engine (GCE), Cloud Storage, and VPC networking
• Strong problem-solving skills with a focus on optimizing HPC workloads and resource utilization
• Work from EU and a work permit to work from EU are required

Responsibilities

• Leading the migration of on-premises SLURM-based HPC (High-Performance Computing) clusters to Google Cloud Platform
• Designing, implementing, and managing scalable and secure HPC infrastructure solutions on GCP
• Optimizing SLURM configurations and workflows to ensure efficient use of cloud resources
• Managing and optimizing HPC environments, focusing on workload scheduling, job efficiency, and scaling SLURM clusters
• Automating cluster deployment, configuration, and maintenance tasks using scripting languages (Python, Bash) and automation tools (Ansible, Terraform)
• Integrating HPC software stacks using tools like Spack for dependency management and easy installation of HPC libraries and applications
• Deploying, managing, and troubleshooting applications using MPI, OpenMP, and other parallel computing frameworks on GCP instances
• Collaborating with engineering, support teams, and stakeholders to ensure smooth migration and ongoing operation of HPC workloads
• Providing expert-level support for performance tuning, job scheduling, and cluster resource optimization
• Staying current with emerging HPC technologies and GCP services to continually improve HPC cluster performance and cost efficiency

Preferred Qualifications

• Google Cloud Professional DevOps Engineer or similar GCP certifications
• Familiarity with GCP’s HPC-specific offerings, such as Preemptible VMs, HPC VM images, and other cost-optimization strategies
• Experience with performance profiling and debugging tools for HPC applications
• Advanced knowledge of HPC data management strategies, including parallel file systems and data transfer tools
• Understanding of container technologies (e.g., Singularity, Docker) specifically within HPC contexts
• Experience with Spark or other big data tools in an HPC environment