Senior Azure Data Engineer

Summary

Join Techstra Solutions as an Azure Data Engineer and play a pivotal role in designing, implementing, and managing data architecture on the Azure cloud platform. Collaborate with cross-functional teams to ensure data infrastructure meets business needs and adheres to industry best practices. This role requires strong technical expertise and strategic thinking to shape the data landscape. Responsibilities include developing and maintaining data pipelines, optimizing ETL processes, extracting and transforming data, and designing ETL solutions. You will lead the development and deployment of data solutions on the Azure cloud platform and manage Azure-based data stores. The position offers an opportunity to work in Chicago, IL, Pittsburgh, PA, or remotely.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or related field
• Must have at least 2 years’ experience in Azure Data Factory
• Must have at least 7 years experience with Microsoft Azure platform and services
• Strong expertise in data modeling, ETL processes, and database management
• Proficiency in SQL, Azure SQL, and Azure Data Lake Storage
• Familiarity with data warehousing concepts and tools
• Excellent problem-solving and communication skills
• Strong project management and leadership abilities
• Ability to work collaboratively in cross-functional teams
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential

Responsibilities

• Develop and maintain data pipelines using Azure Data Factory and other relevant Azure technologies
• Optimize ETL processes for performance and scalability, leveraging Azure services such as Azure Data Factory Mapping Data Flows, Azure Data Lake Analytics, or Azure Synapse Pipelines
• Extract data from various sources, transform it into a usable format, and load it into Azure data storage solutions such as Azure SQL Database, or Azure Synapse Analytics
• Collaborate with cross-functional teams to understand data requirements and design scalable and efficient ETL processes using Azure services
• Identify, design and implement ETL solutions for extraction and integration of data to and from data warehouses and data marts for the purposes of reporting, decision support and analysis
• Lead the design, development, and deployment of data solutions on the Azure cloud platform
• Manage Azure SQL Databases, Azure Data Factory, Azure Data Warehouses, and other Azure-based data stores
• Designing and building data pipelines using API ingestion and Streaming ingestion methods

Preferred Qualifications

• Azure certifications (e.g., Azure Data Engineer, Azure Solutions Architect) preferred
• Knowledge in Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, Power BI is desirable
• Knowledge of C# and Hands-on with PowerShell scripting is desirable
• Working knowledge of Python is desirable