Intermediate Data Engineer

Summary

Join Dev.Pro, a US-based software development company, as a Data Engineer. You will play a key role in a project connecting the art market with a digital financial ecosystem, working with blockchain technology. This position involves managing data warehouses and pipelines, migrating data to GCP, designing ETL/ELT pipelines, and collaborating on data mastering tools. You will also be responsible for evaluating the technology stack, developing data scrapers, and ensuring data quality. The ideal candidate has 4+ years of data engineering experience, proficiency in MariaDB or MySQL, and experience with cloud platforms. Dev.Pro offers a comprehensive benefits package including paid time off, health insurance, and professional development opportunities.

Requirements

• 4+ years in data engineering (extraction, transformation, migration)
• Proficiency in MariaDB or MySQL
• Experience with cloud platforms (AWS, GCP, Azure) and cloud-native solutions
• Strong skills for data manipulation and automation (any language, but Python is preferable)
• Practical skills in ETL/ELT pipeline and orchestration tools (Apache Airflow, Dataflow)
• Experience with batch and real-time data processing frameworks
• Skills in data pipeline development, including batch processing implementation
• Solid grasp of data modeling, warehousing, and scalable architecture design
• RDBMS expertise (stored procedures, triggers, indexes)
• Knowledge of database management, performance optimization, and security
• Upper-Intermediate+ English level

Responsibilities

• Oversee the data warehouse and pipeline, from planning to production delivery
• Migrate data from file storage systems to GCP, including Google Cloud Storage and BigQuery
• Design, develop, and maintain ETL/ELT pipelines for migration and integration
• Collaborate on re-implementing custom data mastering tools to enhance data cleaning and standardization
• Evaluate existing technology stack and recommend improvements for databases and orchestration tools
• Develop a scraper to extract and aggregate data from external sources, ensuring smooth integration
• Ensure data integrity, consistency, and quality through optimized processes and validation
• Work with a small, dynamic team to meet project milestones, focusing on scalability and reliability

Preferred Qualifications

• Familiarity with JavaScript for legacy systems
• Elasticsearch experience
• Knowledge of Tableau, Power BI, Looker, or similar
• Experience with PostgreSQL in data engineering
• Knowledge of data engineering services and metrics
• Experience with MLOps tools, ML models/algorithms

Benefits

• Get 30 paid rest days per year to use as holidays/vacation/other on the desired and requested dates
• 5 sick leave days, up to 60 days of medical leave, and up to 6 days of leave per year due to family reasons (i.e., wedding/funeral/baby birth)
• Get a health insurance package fully compensated by Dev.Pro
• Join fun online activities and team-building events
• Get continuous remote HR, payroll support, and overtime coverage
• Join English/Polish lessons
• Grow your expertise with mentorship support and DP University