Senior Data Consultant

Summary

Join Cyberlogic's growing data consultancy team as a Senior Data Consultant! Based remotely in Johannesburg, Stellenbosch, or Cape Town, you will play a critical role in designing, developing, and maintaining data pipelines. Collaborate with a skilled team to deliver high-impact solutions using Azure technologies and PySpark. Leverage your expertise in data engineering and cloud platforms to build scalable solutions and mentor junior team members. This role requires strong technical skills and a collaborative mindset. If you're passionate about data and driving innovation, apply now!

Requirements

• 8-10 years of experience in data engineering or a related role with a strong background in building and optimizing data pipelines
• Expertise in PySpark for large-scale data processing and distributed computing
• Strong experience with Azure data technologies (Azure Data Factory, Azure Databricks, Azure SQL Database, etc.)
• Proficiency in SQL and experience working with relational and NoSQL databases
• Experience with cloud-based data platforms and services, especially in Azure
• Solid understanding of ETL processes and data modeling techniques
• Strong problem-solving and troubleshooting skills
• Experience working with version control systems (e.g., Git) and continuous integration/continuous deployment (CI/CD) practices
• Strong communication skills and the ability to work effectively within a collaborative, small-team environment
• A proactive, self-starter attitude with a passion for data engineering and technology

Responsibilities

• Design, build, and maintain scalable data pipelines for the collection, transformation, and storage of data
• Work with large, complex datasets to ensure efficient processing and integration
• Develop data engineering solutions using Azure technologies and PySpark for distributed data processing
• Implement ETL processes and automate data workflows to support analytics, reporting, and business intelligence initiatives
• Collaborate with data scientists, business analysts, and stakeholders to understand data requirements and deliver actionable insights
• Ensure data quality, consistency, and reliability across all pipelines and datasets
• Optimize data models, storage, and processing workflows for performance and scalability
• Contribute to the architecture and design decisions for cloud-based data platforms
• Mentor junior team members and foster a collaborative and innovative team culture
• Troubleshoot and resolve issues related to data pipelines, ensuring high availability and performance

Preferred Qualifications

• Experience with data warehousing solutions (e.g., Azure Synapse Analytics, Snowflake)
• Familiarity with other big data technologies (e.g., Hadoop, Kafka, Spark Streaming)
• Experience with infrastructure as code (IaC) tools such as Terraform or Azure Resource Manager (ARM)
• Knowledge of containerization technologies (e.g., Docker, Kubernetes)
• Background in Agile development methodologies

Benefits

Remote work (Johannesburg / Stellenbosch / Cape Town)