Staff Data Systems Engineer

Summary

Join Index Exchange as a Staff Engineer, Data Systems and help build and maintain the next generation of big and small data solutions. You will work with exciting technologies, experiment with new tools, and engineer innovative approaches to solve challenging problems. The role requires experience in an Agile environment and expertise across all layers of the data stack. Index Exchange offers a supportive and collaborative culture, with opportunities for professional growth and development. The company handles billions of auctions daily, generating terabytes of data, and you will play a key role in managing and utilizing this data for internal and external stakeholders. This is a high-impact role in a fast-paced, growing company.

Requirements

• Bachelor/ Master’s Degree in Computer Science or Engineering related fields
• 8+ years of experience as a Senior Software Engineer in enterprise grade, large scale distributed software product development
• 5+ years of work experience designing and building high performance data pipelines and applications using Hadoop, Spark, Hive, Presto, Kafka and Flink streaming, Druid, or other similar technologies
• General Working and Operational knowledge in traditional RDBS (mysql, postgres, etc)
• Proficiency in some of the following languages: Kotlin, Scala, Java, Python, Golang, Rust, Bash
• Deep understanding of design principles of distributed systems and familiar with mainstream big data related technologies and distributed frameworks
• Strong technical leadership, mentorship, collaboration, and communication skills
• Exposure to working with a globally distributed, culturally diverse team
• Knowledge of data modeling, index patterns, query optimization
• Some Experience NoSQL systems, usage, operations, installations (cassandra, redis, scylladb, keyDB, etc)
• Experience with automated testing systems
• Experience working in Agile methodologies with continuous integration and delivery as CI/CD
• Experience working with containerization, and virtualization tools such as Kubernetes, Docker
• Some experience with scheduling tools such as airflow, oozie

Responsibilities

• Evaluate new technologies, design, implement, and maintain big and small data systems and their associated pipelines (governance, audit, installations, operations, best practices, code libraries)
• Maintain, operate, expand and govern existing big and small data systems (from small caches and RDBs in Kubernetes to large Trino/Spark/Hadoop/OLAP clusters)
• Automate and manual hardships, deployments, create CI/CD pipelines for everything, best practices, and internal methodologies for making everyone's life “just that much easier.”
• Lead and mentor team members
• Identify and assess potential solutions for technical and business suitability
• Have a deep understanding about Index and its products. Drive initiatives that produce positive outcomes across divisions

Preferred Qualifications

• An established engineer with exposure execution of projects at a lead/staff level or a functional area. A developer first and for most, that understands how to operate, scale and join various data systems
• You don’t stop until the problem gets solved and you find more than one way to solve it. You love working with other people, presenting your viewpoint but ultimately working towards the best solution, regardless of where it comes from
• Learning new frameworks and languages is exciting to you – you’re not satisfied with the status quo. We use a variety of languages and tools to solve problems and we're interested in what you're looking to learn
• You have a passion for Small and Big Data and an interest in the latest trends and developments constantly researching new tools and data technologies

Benefits

• Company paid comprehensive health and life insurance plans
• Paid Time off and flexible work schedules
• Company contribution to Provident Fund
• Participation in our company Stock options plan
• Company paid Parental Leave
• Monthly internet stipend
• Quarterly Wellness allowance
• Community engagement opportunities and donation-matching program
• Volunteer paid day off
• Annual virtual company retreats and regular community-led team events
• A workplace that supports a diverse, equitable, and inclusive environment