Senior Data Engineer

Summary

Join OREDATA, a Digital Transformation & IT Consulting firm, and work as a Senior Data Engineer for a leading aviation company. You will be responsible for developing and maintaining a customer profile management system, ensuring data quality, and defining data management policies. The role involves designing and implementing big data analytics capabilities, processing unstructured data, and collaborating with stakeholders. You will also be involved in data-driven decision-making to improve business processes. Remote work and flexible time off are offered, along with professional development opportunities and various benefits.

Requirements

• Minimum 5 years of experience in a data-related role is expected for a data engineer role
• Strong programming skills and proficiency in programming languages such as [Python, Java, Scala] are essential
• Knowledge of data engineer skills such as data management, data visualization, and familiarity with data architecture is important
• Data warehousing knowledge, data modeling, cloud computing familiarity such as [AWS, Azure], and data security and governance understanding are all essential for the role

Responsibilities

• Responsible for developing and maintaining customer profile management system, including data integration, data modeling and data migration
• Responsible for monitoring, measuring and improving the quality of customer profile data metrics and reducing any redundancy in the system
• Responsible for defining and enforcing policies and procedures for managing data assets, including product data quality, data stewardship, and reference data security
• Responsible for ensuring the accuracy, completeness and consistency of customer profile data across the organization
• Responsible for analyzing customer profile data and data sources and identifying trends and patterns that can be used to improve business processes and operational efficiency, including data-driven decision-making
• Design, develop, implement, and support our emerging big data analytics capabilities through the development and maintenance of advanced data ingestion, processing, modeling, and reporting capabilities
• Design and implement new or enhanced data processes, tools or models
• Working with stakeholders, provide operational and process execution and ongoing user support
• Write a query to do almost anything
• Process unstructured data into a form suitable for analysis – and then do the analysis if needed

Preferred Qualifications

• A bachelor’s degree or master’s degree in computer science, information technology, data engineering, software engineering, or a related field is preferred
• Experience in data processing frameworks like [Apache Spark, Apache Hadoop, or Apache Flink] is beneficial for handling and processing large-scale data sets efficiently
• Database and query language expertise is preferred in database systems like [SQL, NoSQL]
• Experience working with large volumes of metadata and schemas
• Hands-on experience in designing, developing, and optimizing ETL processes using IBM DataStage or similar ETL tools such as informatica PowerCenter, Talend, or Apache NiFi
• Understanding how the role complements others working with [machine learning, data science, algorithms, business intelligence] is beneficial for collaboration
• Experience with Message Broker Systems : Kafka, etc
• Experience with data pipeline and workflow management tools: Airflow, Azkaban, Luigi, etc
• Experience with stream-processing systems: Storm, Spark-Streaming, etc
• While not mandatory, data engineer certifications such as [Certified Data Engineer (CDE) or AWC Certified Big Data] are beneficial

Benefits

• Remote working and flexible time off
• Opportunity to get company paid Professional Certificates (Google Cloud Platform, Confluent Kafka, etc)
• Access to Online Training Platforms (Udemy, Pluralsight, A Cloud Guru, Coursera, etc.)
• Opportunity to work on international projects
• Private Health Insurance
• Birthday Leave Policy
• Dynamic work ecosystem where you can take initiative and responsibility
• Open communication, flexibility and start-up spirit
• Learning & Development opportunities for both personal and professional growth