Data Engineer and Integration Developer

Summary

Join Nimble Gravity, a team of passionate data experts, as a Data Engineer & Integration Developer - Databricks. This remote role focuses on building custom connectors for legacy systems, automating data extraction processes, and integrating data from diverse sources. You will leverage your Databricks certification, Python expertise, and experience with complex data extraction scenarios to design and develop data workflows, collaborate with stakeholders, and ensure efficient integration and deployment of data solutions.

Requirements

• Databricks certification is required
• 5+ years in a Data Scientist or related role
• Proven expertise in data integration and automation
• Familiarity with handling diverse and complex datasets
• Proficiency in Python development
• Experience with Airflow, dbt, and similar tools and frameworks
• Experience with cloud providers
• Expertise in web scraping and automation
• Strong understanding of data extraction methods from various sources: legacy systems, APIs, Excel files, SharePoint, and PDFs
• Demonstrated experience in designing and implementing data integration solutions
• Fluent in English (spoken and written)
• Excellent communication and problem-solving skills
• Ability to work effectively in a collaborative, remote environment

Responsibilities

• Custom Connector Development: Design and develop Python-based custom connectors for legacy systems to facilitate seamless data integration
• Automation and Data Extraction: Create automation scripts for data extraction from a variety of sources, including legacy systems, APIs, Excel files, SharePoint, and PDFs
• Web Scraping: Utilize web scraping techniques to gather data from online sources, ensuring accuracy and reliability
• Collaboration and Stakeholder Engagement: Work closely with Data Stewards and stakeholders to understand requirements, provide updates, and ensure that solutions align with business objectives
• Integration and Deployment: Employ tools such as Airflow and DBT to ensure efficient integration and deployment of data workflows
• Process Documentation: Create detailed technical documentation to support process replication, maintenance, and knowledge sharing
• Continuous Improvement: Stay updated with emerging tools and techniques to enhance data integration and automation practices
• Databricks Integration and Development: Design, develop, and optimize data workflows and pipelines within the Databricks platform, ensuring high performance, scalability, and alignment with best practices

Preferred Qualifications

US Visa (preferred)