Senior Data Engineer

Summary

Join ELEKS Software Engineering & Development Office as a Senior Data Engineer (DataFactory/DataBricks) in Poland, Croatia, Eastern Europe, or Argentina. This role involves working with a multinational firm delivering risk mitigation, data and analytics, and other services. You will develop and maintain data solutions using Azure Data Factory and Databricks. The position requires extensive experience with data engineering principles and technologies. You will collaborate with a team of professionals on challenging tasks. ELEKS offers opportunities for competence development and a dynamic work environment.

Requirements

• 4+ years of working experience as Data Engineer
• Experience with Databricks (building pipelines, notebooks, Spark)
• Strong knowledge of Azure Data Factory (creating and configuring data flows, pipelines)
• Experience with MS SQL Databases
• Hands-on experience with Cosmos DB
• Python programming

Responsibilities

• Develop solutions and algorithms according to technical specifications or other requirements documentation
• Analyze software code compliance with readability and performance standards
• Use version control systems to track code optimization progress, as well as to merge or split code entities. Commit changes according to version control rules
• Perform analysis, verification, and debugging of the software code at the level of application units
• Develop procedures to test code availability, collect diagnostic data, generate test data sets with desired characteristics; examine the required software characteristics, etc
• Reproduce defects logged in the defect database, establish their causes, and modify code to eliminate defects
• Determine software product compliance with the technical requirements
• Collect and analyze results of software tests. Deliver correct interpretation of diagnostic data; analyze the significance of the obtained software characteristics; can document test results. Assess software product availability
• Develop, document, and edit programming interfaces, software modules and components assembling procedures, software deployment and update procedures, as well as data migration and transformation (conversion) procedures
• Assemble software modules and components into a software product; connect the software product to the external environment. Test the availability of software product versions
• Analyze variants of software requirements implementation; evaluate their time-consumption and workload. Conduct assessment and justification of the solutions recommended for software requirements implementation
• Approve technical requirements for software with stakeholders. Evaluate and justify terms of task completion. Assign tasks to developers, monitor task implementation

Preferred Qualifications

• Knowledge of Data Lakehouse architecture, experience with Microsoft Fabric
• Experience with Entity Framework (EF6)
• Experience in creating project templates and reusable components (e.g., NuGet packages)
• Experience with the Microsoft Azure platform for building and hosting microservices and data-driven solutions (App Services, Key Vault, AKS, Storage Queues)
• Familiarity with security protocols such as OAuth 2.0 (OKTA identity, OAuth, Identity Server, Azure AD, Managed Identity, Key Vaults)
• Hands-on experience in building RESTful APIs with API Management
• Good knowledge of CI/CD systems setup and configuration
• Understanding of Open API standards
• Familiarity with ORMs, especially Entity Framework 6
• Familiarity with DDD (Domain-Driven Design)
• Understanding of canonical API approaches
• Experience with various databases (Cosmos DB, MS SQL, etc.)
• Familiarity with APMs (Application Performance Management) tools
• Experience with H3 Mapping, Power BI
• Familiarity with file storage systems (Blob Storage, File Share, CDN)

Benefits

• Close cooperation with a customer
• Challenging tasks
• Competence development
• Team of professionals
• Dynamic environment with low level of bureaucracy