Senior Data Engineer

Summary

Join NearForm, a digital and AI engineering consultancy, as a remote Senior Data Engineer based in India. You will build server-side components of interactive applications for enterprise clients, utilizing Python, GCP, Pyspark, SQL, and BigQuery. Responsibilities include designing data pipelines, implementing security, executing tests, and optimizing applications. This role requires significant experience in server-side development, agile environments, and specific technologies like Snowflake or Databricks. NearForm offers flexible work arrangements, a wellness hub, and a supportive team environment. The recruitment process involves a talent call, an assessment, and a video call with a hiring manager.

Requirements

• Experience delivering at a Senior Developer level in an enterprise environment
• Practical experience of delivering in an agile environment
• Practical experience of knowledge of developing real-world solutions and platforms
• Deep understanding of server side code, with experience developing in Python and utilising SQL
• Production experience of Snowflake or Databrick platforms
• Deep understanding of ELT/ETL pipelines and toolchains
• Deep understanding of Relational and Document data stores
• Deep understanding of how modern web services are built
• Deep understanding of modern testing methodologies, tools and practices
• Deep understanding of versioning control tools including Git
• Understanding of tools such as AWS serverless compute, Apigee, CloudRun and containerisation
• Understanding of security and performance considerations
• Understanding of architectural and design patterns in one or more of AWS/GCP/Azure
• Understanding of deployment tools and systems integration
• Understanding with a wide variety of open-source technologies and tools
• Excellent analytical and multitasking skills
• Excellent communication and collaboration skills
• Empathy and people skills

Responsibilities

• Designing & building data pipelines and solutions using Python, GCP, Pyspark, SQL, and BigQuery
• Supporting system design, development and maintenance and take responsibility for personal technical quality standards within the project team
• Assisting with defining structured practices, especially in source code management, building and deployment
• Designing and implementing data storage solutions
• Implementing security and data protection
• Executing meticulous automated tests in line with best practice
• Troubleshooting and debugging applications
• Optimising applications for maximum speed and scalability
• Using appropriate tools to maintain version control and build processes
• Getting feedback from, and building solutions for, users and clients
• Working with and supporting the Delivery Architect (team lead) in project execution and timely delivery
• Collaborating with client teams
• Writing documentation and guides
• Following new and emerging technologies
• Contributing to NearForm tooling and open-source projects
• Understand and embrace the NearForm culture

Preferred Qualifications

• Experience with tools such as FastAPI, Pydantic, Flask and Django
• Experience with other programming languages e.g. Javascript and Java
• Experience with multiple cloud technologies
• Experience debugging infrastructure
• Infrastructure as code technology such as Terraform and Cloud Formation
• Previous consultancy experience

Benefits

• Work remotely; we have a genuine dedication to work/life balance
• Work flexibly; we appreciate there are more important things in life than work so our flexible working culture allows you to work around what matters - school run, no problem!
• The Wellness Hub: We have a genuine commitment to fostering/improving NearFormers’ wellbeing; we offer resources and support, including a NearForm advice line which offers confidential support for anything from relationship issues to staying healthy