Senior Data Engineer

Summary

The job is for a Data Engineer at Inovalon, where the candidate will work on creating a configurable and robust data ingestion platform using advanced cloud-based data analytics and data-driven intervention platforms. The role requires extensive experience in data processing/pipeline, ETL, Query Plans & Optimization, and working with tools like ADF (Azure Data Factory), Databricks, MS-SQL.

Requirements

• Minimum of 10 years industry experience working in data & reporting area, knowledge of healthcare data will be a plus
• 10+ years working on data process/pipeline, ETL, Data Workflow, Query Plans & Optimization
• 10+ experience in MS SQL, T-SQL, ETL Jobs
• 5+ years working on databricks, PySpark/SparkSQL, data frames, unity catalog
• 5+ years’ experience in writing query plans, indexes etc
• 5+ years’ experience working on Azure Cloud is preferred utilizing ADF (Azure Data Factory), Azure Delta Lake, Azure SQL Server, Data Sync, Log Insights and Analytics
• Experience working with Role based security at database, ETL jobs, data exports level
• Experience with HIPPA and PHI will be a plus
• Ability to effectively communicate with internal and external customers
• Excellent verbal and written communication skills
• Excellent computer proficiency (MS Office – Word, Excel and Outlook)
• Must be able to work under pressure and meet deadlines
• Ability to work independently and to carry out tasks to completion following standard accepted practices

Responsibilities

• Work with the agile team to participate in agile ceremonies like grooming, planning, standup, retrospective, demos
• Actively contribute to grooming, and standup, create & update tasks estimate and status
• Design and develop modern data ingestion pipeline utilizing tools like ADF (Azure Data Factory), Databricks, MS-SQL
• Extensively work on Azure Cloud Platform
• Write complex queries, stored procedures, functions, for various job execution, manipulating data
• Create highly configurable design to apply various business rules, filters, mapping based on various clients
• Work with data architects and business analysts to create a logical data model and create DDL scripts for physical model creation
• Read data files (flat, excel, csv, json etc) from sftp servers, validate the data utilizing various business rules and configurations
• Design and code various data architecture component like data validation, cleansing, de-duping, Symantec layer
• Work on large data to ensure configurable ingestion of data, dynamic rule & validation of data, cleansing, transforming and loading into the data warehouse
• Design and implement data validation and quality checks to ensure the accuracy and completeness of the data in the data warehouse
• Perform performance of queries and data processing, identify and resolve any issues looking at query plans, create appropriate indexes, resolve dead locks and create table hints
• Understand data partition, performance, multi-threading/processing of high volume data
• Participate in design discussions, data analysis, data model creation etc
• Product quality design diagrams (using MS-Visio, Draw.io etc) and documentations (MS-Work, Excel etc)
• Maintain compliance with Inovalon’s policies, procedures and mission statement
• Adhere to all confidentiality and HIPAA requirements as outlined within Inovalon’s Operating Policies and Procedures in all ways and at all times with respect to any aspect of the data handled or services rendered in the undertaking of the position
• Fulfill those responsibilities and/or duties that may be reasonably provided by Inovalon for the purpose of achieving operational and financial success of the Company

Benefits

Some travel (less than 10%) may be required for this position, primarily for training and collaboration purposes