HPC Engineer

Summary

Join Qarik Group, LLC as a Cloud Storage Technical Lead engineer to design, build, and maintain scalable file storage environments on AWS, collaborating with cross-functional teams to understand computational needs of scientists.

Requirements

• AWS: Deep understanding of AWS services and best practices for building scalable, secure, and cost-effective cloud environments
• DevOps: Proven experience with DevOps practices, including infrastructure as code (Terraform, Ansible), continuous integration, and continuous deployment (GitLab CI/CD)
• IAM: Prior experience integrating storage with common identity and access management solutions such as Active Directory and IAM Identity Center
• Version Control: Proficiency with Git and experience managing code repositories
• Expert level proficiency with POSIX file system semantics
• Proficiency with POSIX I/O profiling for high performance / high throughput workloads
• Expert level proficiency in at least one high performance / parallel filesystem technology such as Weka, Lustre, GPFS, CEPH or JuiceFS
• High proficiency with Amazon S3 object storage
• High proficiency with Network File System (NFS) semantics and solutions
• Knowledge of security best practices in cloud environments and experience implementing them
• Excellent communicator, ability to clearly share architecture plans, designs, risks, and implementation with a variety of stakeholders

Responsibilities

• Design, implement, and maintain scalable and high performance file storage environments on AWS
• Develop and manage infrastructure as code using tools such as Terraform and Ansible
• Automate deployment pipelines and improve CI/CD processes using GitLab CI/CD
• Collaborate with cross-functional teams to understand the computational needs of scientists and translate them into effective platform solutions
• Monitor and optimize platform performance, ensuring reliability and scalability
• Troubleshoot and resolve issues related to infrastructure, deployment, and application performance
• Provide technical guidance and mentorship to junior team members
• Identify and advance collaboration opportunities with other product teams, such as integration with existing data movement and data catalog solutions

Preferred Qualifications

• Prior experience with AWS managed services for file storage, such as EFS, FSx for Lustre, or FSx for OpenZFS
• Prior experience with at least one POSIX interface solution for S3 object storage, such as S3 Mountpoint, CunoFS, or goofys
• Prior experience with cloud data caching solutions such as Amazon ElastiCache or Amazon File Cache