Remote Senior Crypto Data Engineer

Summary

Astra DAO is seeking a Senior Big Data Engineer to facilitate the operations of their Data Scientists and Engineering team. The role involves constructing frameworks, employing machine learning techniques, collaborating with coworkers, and optimizing existing structures. The Senior Big Data Engineer will be responsible for various tasks such as liaising with coworkers and clients, conceptualizing and generating infrastructure, reformulating existing frameworks, testing structures, building a data pipeline, preparing raw data, implementing proper data validation and reconciliation methodologies, and ensuring work remains backed up and accessible.

Requirements

• Bachelor's degree in Data Engineering, Big Data Analytics, Computer Engineering, or related field
• 3+ years of Python, Java or any programming language development experience
• 3+ years of SQL & No-SQL experience (Snowflake Cloud DW & MongoDB experience is a plus)
• 3+ years of experience with schema design and dimensional data modeling
• Expert proficiency in SQL, NoSQL, Python, C++, Java, R
• Expert with building Data Lake, Data Warehouse or suitable equivalent
• Expert in AWS Cloud
• Excellent analytical and problem-solving skills
• A knack for independence and group work
• Capacity to successfully manage a pipeline of duties with minimal supervision

Responsibilities

• Liaising with coworkers and clients to elucidate the requirements for each task
• Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed
• Reformulating existing frameworks to optimize their functioning
• Testing such structures to ensure that they are fit for use
• Building a data pipeline from different data sources using different data types like API, CSV, JSON, etc
• Preparing raw data for manipulation by Data Scientists
• Implementing proper data validation and data reconciliation methodologies
• Ensuring that your work remains backed up and readily accessible to relevant coworkers