Rackspace Technology is hiring a
Senior Data Engineer, Remote - Vietnam
![Logo of Rackspace Technology](https://cdn.jobscollider.com/logo/rackspace-technology-d583.webp)
Summary
The job is for a Senior Data Engineer or Data Architect with extensive experience in Spark Streaming, Azure Functions, Synapse, and other Azure data services. The role involves designing, developing, and maintaining real-time data streaming pipelines, implementing CI/CD pipelines, and collaborating with cross-functional teams to deliver high-quality data solutions.
Requirements
- Proven experience in real-time data streaming using Spark Streaming or Azure Functions
- Strong proficiency in Python for data processing and automation tasks
- Hands-on experience with Kafka for data ingestion and message queuing
- Familiarity with Redis for caching and fast data retrieval
- Knowledge of data lake architectures and best practices for data storage and retrieval
- Solid understanding of software engineering principles, including design patterns, testing, and documentation
- Strong problem-solving skills and ability to work in a fast-paced, collaborative environment
Responsibilities
- Design, develop, and maintain data architectures using Spark and Lakehouse methodologies
- Utilize Synapse services including Spark pools, dedicated SQL pools, and pipelines to build robust data solutions
- Leverage other Azure data services to enhance and optimize data workflows
- Implement and manage CI/CD pipelines to ensure smooth deployment and integration processes
- Load, merge, and process machine logs from Kafka, ensuring efficient data flow and transformation
- Integrate processed data into Redis cache and send it to the data lake for long-term storage and analysis
- Implement and optimize data processing solutions using Python
- Apply software engineering best practices, including code reviews, version control, and continuous integration/deployment (CI/CD)
Preferred Qualifications
- Experience with cloud platforms such as Azure
- Knowledge of other programming languages and frameworks related to data engineering
- Familiarity with data modeling, ETL processes, and data lake concepts
This job is filled or no longer available
Similar Jobs
- 💰~$150k-$190k📍United States
- 💰~$144k-$185k📍Canada
- 💰$40k-$80k📍Worldwide
- 💰~$48k-$59k📍Worldwide
- 💰~$48k-$59k📍Worldwide
- 💰~$48k-$59k📍Worldwide
- 💰~$48k-$59k📍Worldwide
- 💰~$48k-$59k📍Spain
- 💰~$150k-$222k📍United States
- 💰~$48k-$59k📍United Kingdom