Lead Engineer, Back-end

Summary

Join our Engineering team as a Back-end Engineer to partner with cross-functional teams in identifying and implementing innovative technologies that ensure server-side applications are robust, secure, efficient, and drive business growth.

Requirements

• 8 years minimum experience in software programming, specializing in back-end software development
• Programming Languages: Proven experience with Python, and experience with Go or Java is a plus
• Workflow Orchestration: Deep understanding of designing and implementing complex workflows. Experience with LangChain is a plus
• AI Integration: Familiarity with AI frameworks (e.g., TensorFlow, PyTorch) and working knowledge of LLMs
• APIs and Communication Protocols: Experience with RESTful APIs, gRPC, and asynchronous communication
• Databases: Proficiency with both SQL and NoSQL databases; experience with state management
• Messaging Systems: Experience with message brokers like RabbitMQ or Apache Kafka
• Containerization and Orchestration: Experience with Docker and Kubernetes for deploying scalable applications
• Security: Understanding of authentication, authorization, encryption, and secure coding practices
• AI and ML Expertise: Experience with Retrieval-Augmented Generation (RAG) systems. Knowledge of prompt engineering and persona-based AI model interactions. Familiarity with Hugging Face Transformers or similar libraries
• DevOps Skills: Experience with CI/CD pipelines and infrastructure as code tools (e.g., Terraform, Ansible). Familiarity with cloud platforms such as AWS, Google Cloud Platform, or Azure
• Monitoring Tools: Experience with monitoring and logging tools like Prometheus, Grafana, ELK Stack, or OpenTelemetry
• Strong experience with workflow orchestration tools (e.g., Prefect, Temporal.io, Apache Airflow)
• Experience integrating with AI/ML models, particularly Large Language Models (LLMs)
• Proven experience in building distributed systems and microservices architecture
• Act as product evangelist with deep curiosity on technology trends and being able to go deep into technology and relate technical enhancements to build a product vision
• Must be a compelling and clear communicator; able to explain the full possibilities of the tech stack/architecture and represent your team to internal and external audiences with differing levels of technical fluency
• Self-motivated and focused on achieving excellence as a team

Responsibilities

• Be a hands-on leader to engineering teams in successfully delivering scalable, maintainable, and secure features to our customers; advocating for clean code, process enhancements, and continuous improvements
• Integrate Foundation Model LLMs and internal RAG systems into backend services
• Implement workflow orchestration logic to manage task dependencies and execution order
• Collaborate with AI specialists to effectively integrate LLMs and RAG systems
• Work closely with frontend developers and other team members to ensure seamless integration of backend components
• Ensure the system is scalable and efficient, capable of handling high loads and complex workflows
• Implement asynchronous processing, caching mechanisms, and optimize communication protocols
• Implement logging, monitoring, and alerting mechanisms to track system performance and agent activities
• Adhere to best practices for data privacy and security, including encryption and compliance with relevant regulations
• Write clear, comprehensive technical documentation for developed components
• Develop unit tests, integration tests, and conduct thorough testing to ensure reliability and robustness
• Complete assigned tasks in a timely manner within project constraints
• Cultivate a collaborative working environment and a culture of technical ownership