Design, ship, and scale production-grade AI systems, integrating LLMs with focus on reliability, safety, and performance. Collaborate across teams and mentor engineers.
Company Overview
Role Overview
Impact You'll Drive
Key Responsibilities
Must-Have Qualifications
Nice-to-Have Qualifications
Example Tech You'll Touch
How We Build
Success in 90 Days
Function was founded with a singular focus: empower you to live 100 healthy years. We’re doing that by using the best available technology to make sure people don't suffer or die a preventable death. Function has been recognized as one of Fast Company’s Most Innovative Companies of 2024, and is venture-backed by Andreessen Horowitz (a16z). Hundreds of thousands of members have joined Function to take control of their health. We are growing our team and seeking out world-class talent that deeply believes in our mission to positively impact global health, has a relentless bias toward action and a growth mindset. Function fosters a collaborative and dynamic environment, where every day we are building the future.
You will design, ship, and scale production-grade, stateful multi-agent systems end to end—spanning orchestration graphs, model serving, real-time inference, and observability.
You’ll partner with product, infra, and research to integrate LLMs and multimodal models (voice, vision, structured data) into consumer and internal workflows with strong safety, reliability, and cost controls.
This is a hands-on role for a high-ownership engineer with deep systems expertise and a track record of delivering AI agents at scale.
- Ship agentic features that move core product KPIs with measurable quality and latency targets.
- Establish evaluation gates and on-call reliability for AI systems that handle real users and revenue.
- Reduce cost-to-serve via model routing, KV cache reuse, and retrieval quality improvements.
- Architect and build stateful, graph-based agent workflows with tool use, planning, and memory.
- Integrate LLMs and multimodal models via structured I/O (JSON Schema, Pydantic validators) and function/tool calling.
- Build high-reliability APIs and streaming services for real-time inference, speech, and vision.
- Own production readiness: tracing, logging, metrics, rate limiting, circuit breakers, and SLOs.
- Stand up eval pipelines: offline golden sets, LLM-as-judge with human rubrics, online A/B, and regression tests in CI.
- Implement retrieval and memory: hybrid search, vector and graph retrieval, semantic caches, and long-horizon context.
- Optimize cost/latency: model routing, prompt and tool selection, quantization, and KV cache/prefill strategies.
- Lead cloud-native deployments on Kubernetes with GPU autoscaling, canary/shadow releases, and feature flags.
- Partner cross-functionally to translate research into robust production systems and iterate quickly behind evaluation gates.
- Mentor engineers through code reviews, design docs, and architecture decisions.
Must-Have Qualifications
- 2.5+ years building agentic AI systems; 6+ years as a full-stack or ML engineer, building production backends or ML systems in Python, Go, or similar.
- Fluency with agentic orchestration (e.g., LangGraph, PydanticAI, DSPy, LlamaIndex) and tool/function calling.
- Experience integrating frontier LLMs and multimodal models via managed APIs or self-hosted serving.
- Deep understanding of model serving and inference optimization (vLLM/Triton/TGI/SGLang, batching, KV cache reuse).
- Strong with API design and backend frameworks (FastAPI, Flask) and event-driven architectures.
- Data systems expertise with PostgreSQL and Redis, including caching, token streaming, and throughput tuning.
- Retrieval and memory: vector databases (pgvector, Pinecone, Weaviate, Milvus), hybrid search, and graph/knowledge storage.
- Production evals: LLM-as-judge, human-in-the-loop, rubric design, and CI-integrated regression tests.
- Observability and SRE: OpenTelemetry traces, metrics, structured logs, SLOs, dashboards, and on-call triage.
- Cloud-native delivery: Kubernetes, Terraform, Docker, GPU scheduling/autoscaling on AWS or GCP.
- CI/CD proficiency with GitHub Actions and test automation for prompts, tools, and agents.
- Clear, concise communication and high ownership in fast-paced environments.
Nice-to-Have Qualifications
- Real-time multimodal systems: streaming ASR, low-latency TTS, WebRTC, and vision pipelines.
- Post-training/fine-tuning: DPO/ORPO, RLHF, preference data generation, and safety alignment.
- RAG expertise beyond basics: Graph RAG, multi-hop retrieval, rerankers, query planning, and freshness policies.
- Safety and governance: policy-as-code, red-teaming, PII handling, audit logs, and role-based tool authorization.
- Regulated data experience (HIPAA, SOC 2, GDPR) and data residency controls.
- Personalization at inference time, long-term memory agents, session state, and episodic memory stores.
- Experience with consumer-scale AI apps, high-traffic systems, or on-device/edge acceleration (WebGPU).
- Orchestration: LangGraph, PydanticAI, DSPy, LlamaIndex
- Serving: vLLM, Triton, TGI, SGLang; OpenAI/Anthropic-compatible APIs
- Backend: Python, Go, FastAPI, gRPC, Kafka/PubSub
- Data: PostgreSQL, Redis, pgvector, Pinecone/Milvus/Weaviate
- Observability: OpenTelemetry, Prometheus, Grafana, Sentry
- Infra: Kubernetes, Terraform, Docker, GPU operators, Karpenter/Cluster Autoscaler
- Evals & QA: RAGAS/DeepEval-style frameworks, golden sets, canary/shadow testing
How We Build
- Evaluation-driven development: every change to prompts, tools, routing, or retrieval passes automated eval gates.
- Structured outputs by default: JSON Schema/Pydantic validation, strict tool contracts, and idempotent handlers.
- Safety-first tooling: guardrails, content and data policies, tool sandboxing with timeouts and scopes.
- Pragmatic iteration: short cycles, feature flags, shadow traffic, and fast rollback.
- Launch a production agentic workflow with clear SLOs, tracing, and dashboards.
- Stand up an eval harness with golden sets and CI gates for the top use case.
- Improve latency and cost with routing and KV cache strategies while maintaining quality.
At Function, we celebrate diversity and are committed to building a diverse and inclusive workforce. As an equal opportunity employer, we do not discriminate on the basis of race, color, gender identity, ancestry, religion, age, sexual orientation, national origin, disability, marital status, Veteran status, or any other occupationally irrelevant criteria.
Join the Function Health team and become a part of our mission to build a healthier future for all. Discover more about us and how we're changing the face of healthcare at Function Health.
Important Notice: Legitimate communication from the Function Health team will always come from an email address ending in @functionhealth.com. Function Health will never request personal information such as banking details or payment during the hiring process. Please be cautious of communications or job offers that come from other email domains, instant messaging platforms, or unsolicited calls. If you ever have doubts about the legitimacy of a communication, please reach out to us directly at [email protected].
Top Skills
Docker
Dspy
Fastapi
Flask
Go
Grafana
Kubernetes
Langgraph
Llamaindex
Milvus
Opentelemetry
Pinecone
Postgres
Prometheus
Pydanticai
Python
Redis
Sglang
Terraform
Tgi
Triton
Vllm
Weaviate
Similar Jobs
Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
The Payroll & Benefits Coordinator will support payroll processing, benefits administration, and contractor lifecycle operations, ensuring accuracy and compliance.
Top Skills:
Remote.Com
Fintech • Financial Services
As a Security Engineer, you'll architect and maintain security systems, enhance security posture, manage incidents, and integrate AI tools for proactive security solutions.
Top Skills:
CrowdstrikeGoNode.jsPythonSlackSoarTerraformTerragruntTypescript
Fintech • Financial Services
As an Information Security Engineer, you will architect and maintain security systems, automate workflows, respond to incidents, and measure security operations effectiveness.
Top Skills:
CrowdstrikeGoNode.jsPythonSlackSoarTerraformTerragruntTypescript
What you need to know about the Montreal Tech Scene
With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.
Key Facts About Montreal Tech
- Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
- Major Tech Employers: SAP, Google, Microsoft, Cisco
- Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
- Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
- Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
- Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal
