AI Software Engineer

Summary

Join Caregentic as a Senior or Staff AI Software Engineer and take end-to-end ownership of key backend services and AI-driven applications. You will build scalable, resilient systems using Python, FastAPI, Apache Kafka, and GraphQL within a modern microservice architecture. Collaborate with cross-functional teams, mentor other engineers, and engage with clients to showcase Caregentic's AI capabilities. The role involves designing and building LLM services, including RAG systems, agentic workflows, and evaluation pipelines. You will also lead conversational AI development and own vector search and embedding pipelines. Caregentic offers a fully remote work environment and a competitive compensation package. This is an opportunity to make a meaningful impact on healthcare delivery using cutting-edge technology.

Requirements

• 7+ years of backend development experience in production environments, specifically strong Python skills, including async programming and type hints
• Experience building and monitoring production-quality ML and AI systems
• Hands-on expertise with RAG frameworks and agentic workflows
• Solid understanding of PostgreSQL database design and optimization
• Familiarity with Docker and containerization
• Strong testing practices using pytest
• Shipped production LLM systems: ideally RAG architectures, agent/tool use, and prompt/system design, with LangChain (tracing/evals via LangSmith), embeddings, and vector databases (Databricks Vector Search preferred). Deep expertise in retrieval quality, including chunking, metadata, hybrid search, reranking, and grounding

Responsibilities

• Design and develop robust, scalable, event-driven services using Python, FastAPI, Apache Kafka and GraphQL
• Build fundamental LLM agents and integrate them into our product
• Work with DevOps on deployments, monitoring, and reliability improvements
• Maintain and optimize PostgreSQL databases and data models
• Collaborate across product and engineering teams to define requirements and architect features
• Drive engineering best practices through code reviews and mentorship
• Engage with current and prospective clients to drive understanding of the Caregentic AI architecture and capabilities
• Design, build, and operate LLM services, including RAG systems (LangChain), agentic workflows, and evaluation pipelines (LangSmith, deepeval, A/B testing)
• Own vector search & embeddings pipelines from schema and metadata design to model benchmarking, cost/latency optimization, and Databricks Vector Search integration
• Lead conversational AI development enhancing NLU policies, safety guardrails, and custom action servers, plus integrating assistants with microservices

Preferred Qualifications

• Experience with microservice architectures is preferred
• Experience with GraphQL APIs
• Experience with event-driven systems and message queues
• Experience with major cloud providers (e.g., AWS)
• Experience with Strawberry GraphQL
• Experience with Kafka, Procrastinate, or similar background job systems
• Experience Apollo Federation or multi-service GraphQL experience
• Healthcare or regulated industry experience (HIPAA, HL7, FHIR)
• Experience with Terraform or infrastructure-as-code
• Experience with Kubernetes
• Experience mentoring engineers or leading technical projects
• Experience working with LangSmith tracing, AWS Bedrock, LangChain runnables
• Experience maintaining forked open source repositories
• Experience with managing conversation states in conversational agents - experience with LangGraph or similar

Benefits

• Comprehensive healthcare coverage (medical, dental, vision)
• Time off: Flexible PTO policy with 20 vacation days and 5 sick days per year
• Remote work environment
• Top-tier development equipment
• Opportunity to make a meaningful impact on healthcare delivery