Data Engineer

Summary

Join Encora as a Data Engineer in Brazil and work from home! You will collaborate with stakeholders to define and communicate data platform requirements and architecture. Leverage your extensive experience in data lake, data warehouse, and other data storage systems. Extract business and technical requirements from customers and stakeholders. Demonstrate strong knowledge of data frameworks like Hadoop, Hive, Spark, and AWS data tools. Experience with SQL, NoSQL, and unstructured data is essential. Master ETL frameworks and workflow processes.

Requirements

• Have strong knowledge of Data frameworks: Hadoop (YARN, HDFS), Hive, Spark, Kafka, Pentaho, Airflow, AWS data tools
• Manipulate data using SQL, NoSQL and unstructured data sources, including metadata
• Have strong knowledge of ETL frameworks and workflow processes (data ingestion, clean up and preparation)

Responsibilities

• Devise, understand and communicate clearly requirements and architecture design for data platforms in an efficient manner
• Work with stakeholders at a high level
• Have large experience with architecture, governance, security, design, business mapping and understanding, performance and tuning for data lake, data warehouse and other data storage systems and transformation
• Extract business and technical requirements in high and low level by talking with customers and stakeholders

Preferred Qualifications

• Have experience with Python and Java for system administration
• Have large experience with data modeling and data design
• Have system administration experience on Linux platforms
• Have financial and/or banking marketing knowledge
• Have experience with AWS platform
• Know machine learning algorithms and workflow
• Know pipeline and workflow orchestration (Oozie, Luigi)
• Have experience with Kubernetes