Senior Data Engineer

Summary

Join Quorum as a Senior Data Engineer and lead the development and scaling of the data infrastructure for Quorum Copilot, our groundbreaking Generative AI tool for public affairs data analysis. You will architect and implement advanced data pipelines, ensuring our systems are scalable, efficient, and secure. Collaborate with a team of engineers, product managers, and designers to support advanced AI and machine learning models. Leverage your expertise in big data tools, cloud technologies (AWS), and machine learning frameworks. Contribute to a dynamic, remote-first team with flexible work options and a commitment to professional development.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or related field
• 5+ years in data engineering with a track record of developing and scaling data-driven products
• Expertise in building data pipelines, architectures, and datasets, with experience in AWS cloud services (EC2, EMR, RDS, Redshift)
• Proficient in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch)
• 3+ years experience with Python
• Deep knowledge of SQL and NoSQL databases, workflow management tools (Azkaban, Luigi, Airflow), and an understanding of the machine learning model deployment cycle

Responsibilities

• Architect and implement highly scalable advanced Retrieval-Augmented Generation (RAG) data pipelines to support the aggregation and analysis of legislative bills, social media posts, documents, and testimonial content
• Design robust data pipelines that enable real-time processing and analysis of vast datasets to inform public affairs strategies and decision-making
• Design and implement data cleansing and transformation pipelines to power the AI product
• Lead cloud-based deployments, primarily in AWS, ensuring optimal performance, security, and cost-efficiency
• Innovate and iterate on data architecture to support Quorum Copilot's dynamic needs, ensuring the product remains at the cutting edge of AI-driven public affairs tools
• Drive build vs buy, tool selection, and solution trade-off analysis using standard engineering principles

Preferred Qualifications

• Top candidates will have experience or working knowledge of vector databases and advanced Retrieval-Augmented Generation (RAG) systems (Langchain, Pinecone, OpenAI/ChatGPT)
• You care deeply about the code you write and either have significant technical skills solving real-world problems or the drive to quickly develop them
• You have an interest in learning new development languages, tools, and platforms used by the wider team
• You're a highly motivated self-starter who wants to make valuable contributions to a team from day 1
• You're an engaged learner—you want to expand your skills, get your hands dirty, and find solutions to challenging problems that require tradeoffs
• You want to work in a fun environment where you can form real friendships with other professionals and feel a strong sense of belonging at a growing tech company
• You're excited to pitch in wherever needed, from helping in recruiting to supporting customers
• You are looking to join a dynamic team immediately

Benefits

• Flexible Paid Time Off
• Standard Brazil holidays plus additional company-wide days off for team members to rest and recharge
• Free Subscription to the Calm App
• Free Subscription to LinkedIn Learning to support professional development
• Invest in Yourself Days - one designated day per quarter is dedicated to your professional development!
• Monthly Work from Home Stipend
• Mental and Dental Care by SulAmérica
• Virtual and in-person team events
• Mental Health stipend
• Alelo Food Voucher
• In company English classes
• Inclusion & Diversity Affinity Groups to support belonging
• 12 weeks of paid parental leave
• This is a CLT opportunity