Software Engineer AI Workflows

Summary

Join Flume Health, a software company focused on streamlining healthcare data integration, as a Software Engineer specializing in AI workflows. You will be responsible for building and orchestrating pipelines that utilize foundation models like OpenAI's or Google's large language models (LLMs) to generate and refine data mappings. This role involves architecting AI workflows, implementing intelligent retry and refinement loops, developing prompt engineering strategies, orchestrating pipelines with tools like Python and Airflow, collaborating with other teams, staying up-to-date on the LLM ecosystem, and ensuring performance, traceability, and compliance. If you enjoy working with AI processes, designing data-driven feedback loops, and operating in a fast-paced, compliance-aware environment, this opportunity is for you.

Requirements

• 3+ years of software engineering experience, ideally in data, ML, or AI-driven product development
• Demonstrated experience building or orchestrating workflows that integrate with LLM APIs (e.g., OpenAI, Anthropic, Google)
• Proficiency with Python and orchestration tools (e.g., Airflow, Dagster, Prefect)
• Familiarity with prompt engineering concepts and iterative refinement techniques (e.g., looped retries, RAG, validation-based reruns)
• Comfortable working in cloud-native environments (GCP or AWS) using Docker and Kubernetes
• Strong communication skills able to translate business needs into repeatable, explainable, and traceable AI workflows

Responsibilities

• Architect AI workflows that interpret user-provided specifications and generate data mappings using LLM APIs (e.g., OpenAI, Vertex AI), refining outputs based on validation feedback
• Implement intelligent retry and refinement loops to improve mapping accuracy using techniques like prompt chaining, structured validation, and contextual injection
• Develop prompt engineering strategies with support for domain-specific augmentation (e.g., via vector search or RAG)
• Orchestrate pipelines with tools like Python and Airflow, integrating structured QA steps to monitor output quality and flag failures
• Collaborate across Data, Platform, and Product teams to embed these workflows into Flume’s core systems, ensuring performance, traceability, and compliance
• Stay up to date on the evolving LLM ecosystem understanding tradeoffs in cost, latency, and capability across vendors

Preferred Qualifications

• Experience with vector databases or semantic search tools (e.g., Pinecone, Weaviate, Qdrant)
• Exposure to regulated data environments, such as healthcare or finance
• Understanding of LLM orchestration frameworks (LangChain, DSPy, HuggingFace Accelerate)
• Interest in evaluating the effectiveness of LLM outputs using custom metrics or domain-specific rules
• Contributions to open-source AI tooling or AI workflow libraries

Benefits

• Flexible PTO - you’re going to be working hard so enjoy time off
• A robust stock option plan to give our employees a direct stake in Flume’s success
• WFH stipend - we’ve always been remote first
• Competitive compensation and 401k with a 4% match
• Comprehensive health coverage (medical, dental, vision)