Data Engineer

Summary

Join a global Microsoft solutions integrator as a Data Engineer in our Data & Analytics organization. This is a full-time, well-benefited career opportunity for a highly experienced Data Engineer with hands-on knowledge in data architecture, especially Spark and Delta/Data Lake technologies.

Requirements

• 5+ years of Data Engineering experience including 2+ years designing and building Databricks data pipelines is REQUIRED
• Azure cloud is highly preferred, however will consider AWS, GCP or other cloud platform experience in lieu of Azure
• Experience with conceptual, logical and/or physical database designs is a plus
• 2+ years of hands-on Python/Pyspark/SparkSQL and/or Scala experience is REQUIRED
• 2+ years of experience with Big Data pipelines or DAG Tools (Data Factory, Airflow, dbt, or similar) is REQUIRED
• 2+ years of Spark experience (especially Databricks Spark and Delta Lake) is REQUIRED
• 2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem, including Data/Delta Lakes, is REQUIRED
• Experience with source control (git) on the command line is REQUIRED
• 2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data is HIGHLY DESIRED
• Data modeling / data profiling capabilities with Kimball/star schema methodology is a plus
• Professional experience with Kafka, or other live data streaming technology, is HIGHLY DESIRED
• Professional experience with database deployment pipelines (i.e., dacpac’s or similar technology) is HIGHLY DESIRED
• Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED

Responsibilities

• Scope and execute together with team leadership
• Work with the team to understand platform capabilities and how to best improve and expand those capabilities
• Strong independence and autonomy
• Design, development, enhancement, and maintenance of complex data pipeline products which manage business-critical operations and large-scale analytics applications
• Experience leading mid- and senior-level data engineers
• Support analytics, data science and/or engineering teams and understand their unique needs and challenges
• Instill excellence into the processes, methodologies, standards, and technology choices embraced by the team
• Embrace new concepts quickly to keep up with fast-moving data engineering technology
• Dedicate time to continuous learning to keep the team appraised of the latest developments in the space
• Commitment to developing technical maturity across the company