Senior Data Engineer

Summary

Join Quorum's innovative team as a Senior Data Engineer and lead the development and scaling of the data infrastructure for Quorum Copilot, our groundbreaking Generative AI tool for public affairs data analysis. You will architect and implement advanced data pipelines, ensuring scalability and efficiency for processing vast datasets. Collaborate with a team of engineers, product managers, and designers to build robust, secure, and cost-effective cloud-based systems. Leverage your expertise in big data tools, machine learning frameworks, and database technologies to power our AI product. Contribute to innovative data architecture and drive strategic decisions on technology selection. This role requires a Bachelor's or Master's degree in a related field, 5+ years of data engineering experience, and proficiency in various technologies.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or related field
• 5+ years in data engineering with a track record of developing and scaling data-driven products
• Expertise in building data pipelines, architectures, and datasets, with experience in AWS cloud services (EC2, EMR, RDS, Redshift)
• Proficient in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch)
• 3+ years experience with Python
• Deep knowledge of SQL and NoSQL databases, workflow management tools (Azkaban, Luigi, Airflow), and an understanding of the machine learning model deployment cycle

Responsibilities

• Architect and implement highly scalable advanced Retrieval-Augmented Generation (RAG) data pipelines to support the aggregation and analysis of legislative bills, social media posts, documents, and testimonial content
• Design robust data pipelines that enable real-time processing and analysis of vast datasets to inform public affairs strategies and decision-making
• Design and implement data cleansing and transformation pipelines to power the AI product
• Lead cloud-based deployments, primarily in AWS, ensuring optimal performance, security, and cost-efficiency
• Innovate and iterate on data architecture to support Quorum Copilot's dynamic needs, ensuring the product remains at the cutting edge of AI-driven public affairs tools
• Drive build vs buy, tool selection, and solution trade-off analysis using standard engineering principles

Preferred Qualifications

• Experience or working knowledge of vector databases and advanced Retrieval-Augmented Generation (RAG) systems (Langchain, Pinecone, OpenAI/ChatGPT)
• You care deeply about the code you write and either have significant technical skills solving real-world problems or the drive to quickly develop them
• You have an interest in learning new development languages, tools, and platforms used by the wider team
• You're a highly motivated self-starter who wants to make valuable contributions to a team from day 1
• You're an engaged learner—you want to expand your skills, get your hands dirty, and find solutions to challenging problems that require tradeoffs
• You want to work in a fun environment where you can form real friendships with other professionals and feel a strong sense of belonging at a growing tech company
• You're excited to pitch in wherever needed, from helping in recruiting to supporting customers
• You are looking to join a dynamic team immediately

Benefits

• Standard Brazil Holidays
• Monthly Work from Home Stipend
• Mental and Dental Care by SulAmérica
• Monthly Professional Development/Mental Health stipend
• Alelo Food Voucher
• Incompany English classes
• Free Subscription to the Calm app
• Flexible Paid Time Off
• Inclusion & Diversity Affinity Groups to support belonging
• 12 weeks of paid parental leave
• This is a CLT opportunity
• We are a remote-first team with flexible work options: work remotely, or you may choose to come into our office in Belo Horizonte