Senior LLM/RAG Engineer

Summary

Join Data Meaning, a leading Business Intelligence and Data Analytics consulting firm, as a Senior LLM/RAG Engineer. This role focuses on developing and implementing Retrieval-Augmented Generation (RAG) techniques to improve metadata quality using Azure OpenAI. You will design and implement API connections, build scalable applications integrating RAG and AI, and collaborate with various teams. Strong Python skills, API integration experience, and knowledge of RAG systems are essential. The position requires a Bachelor's degree in a related field and experience with cloud platforms, preferably Microsoft Azure. Data Meaning offers a remote work environment within a collaborative and innovative culture.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, or related field
• Strong proficiency in Python and experience with API integrations
• Knowledgeable in implementing and managing RAG or similar AI-driven systems
• Understanding of data governance, metadata management, and related best practices
• Experience with cloud platforms, preferably Microsoft Azure

Responsibilities

• Design and implement API connections to fetch metadata from various sources, including Collibra
• Maintain and optimize API performance to ensure efficient data retrieval
• Develop and maintain a RAG system to augment metadata with accurate and contextually relevant information
• Utilize Azure OpenAI to generate and enhance metadata descriptions
• Build and deploy scalable applications that integrate RAG and AI technologies
• Ensure application compliance with security, privacy, and data management standards
• Collaborate with data governance and IT teams to align RAG outputs with organizational needs
• Support project management with updates, documentation, and insights from the RAG system
• Document all processes and maintain records for internal standards and regulatory compliance
• Develop reports to monitor the effectiveness and accuracy of the RAG system

Preferred Qualifications

• Experience with Collibra APIs is highly advantageous
• Master’s degree or advanced certifications in data science, AI, or cloud technologies
• Proven ability to document processes and create detailed reports