Staff Software Engineer - Backend Core AI

About the Role

We’re hiring a Staff Engineer – AI Frameworks to architect and lead the development of the foundational multi-agent infrastructure powering the next generation of intelligent systems for financial institutions.
This role is not about plugging in pre-built models—it’s about designing and scaling custom AI orchestration frameworks that bind language models, memory, judgment modules, and tool use into autonomous systems that are trustworthy, composable, and policy-aligned.

You’ll work at the intersection of machine learning, distributed systems, and agentic reasoning, partnering with researchers, backend engineers, and product leaders to bring real-time, LLM-driven intelligence into production at scale.
This is a rare opportunity to define and build the AI runtime and execution architecture for mission-critical agents in a high-regulation, high-trust industry.

What You’ll Own

As a Staff Software Engineer specializing in Agentic AI Orchestration, you will be at the forefront of building the intelligent backbone of our platform. This pivotal role involves taking complete ownership of the architectural design and implementation that ensures our autonomous agents operate with unparalleled efficiency, security, and responsiveness. You will be instrumental in maintaining strict sub-second latency targets (under 1 second) across all critical operations, including sophisticated agent routing, the application of banking-grade security guardrails, and the secure, PII-safe execution and guardrails of tool calls.

Beyond architecture, you will serve as the primary technical lead, guiding the strategic direction and hands-on development across several critical areas. This includes optimizing agent routing mechanisms, enhancing dynamic planning capabilities, evolving memory management systems for optimal token usage, establishing robust evaluation frameworks, and pioneering RL-based prompt tuning techniques. Your contributions will directly impact the reliability, safety, and performance of our next-generation AI agents.

What You’ll Do

• This role demands a multifaceted technical leader capable of both high-level design and hands-on implementation. Your responsibilities will include:
• Lead Design and Implementation of Advanced Routing: Spearhead the architectural design and hands-on implementation of hierarchical and plan-based routing systems for our autonomous agents. A critical aspect of this will be enforcing tight latency budgets, specifically targeting performance within an 800–1500 ms window to ensure seamless user interactions and agent responsiveness.
• Build Secure PII Pipelines: Develop and implement robust PII (Personally Identifiable Information) masking and transactional guardrails. This involves designing secure data flows and also rigorously enforcing idempotency and transaction safety across all external tool calls made by our agents, ensuring data integrity and compliance.
• Own Model Selection and Inference Strategy: Drive the strategic selection and optimization of inference models. This encompasses evaluating and integrating a diverse range of models, including large language models, as well as fine-tuned or custom small models. You will also be responsible for leveraging high-performance inference engines to maximize throughput and minimize latency.
• Design Efficient Memory Layers: Architect sophisticated memory layers for our agents. This includes designing short-term rolling windows for immediate context, retrieval memory for efficient information access, and audited long-term state management, all engineered to minimize token usage and optimize computational resources.
• Ship Comprehensive Observability: Implement comprehensive observability solutions crucial for understanding and debugging agent behavior. This involves shipping systems for per-turn token and latency budgeting, detailed attribution (identifying which prompt or tool consumed what resources), and proactive red-flag alerting for anomalous behavior or performance degradation.
• Stand Up Evals & Reinforcement learning: Establish and maintain robust evaluation frameworks and auto-evolution loops. This includes building comprehensive regression suites to track performance over time, developing sophisticated reward models for agent behavior optimization, and implementing bandit algorithms for dynamic, data-driven selection of prompts, models, and tools.
• Mentor and Elevate Standards: Act as a technical mentor for senior engineers within the team, fostering their growth and development. You will also be responsible for setting and upholding high standards for code quality, defining best practices for incident response, and establishing effective change management processes to ensure smooth and reliable system evolution.

What We’re Looking For

Required Qualifications

• To succeed in this role, you should possess a strong foundation in distributed systems and practical experience with AI agent technologies:
  Extensive Distributed Systems Experience: 8+ years of professional experience in designing, building, and maintaining high-throughput, low-latency distributed systems. Proficiency in at least one of the following programming languages is essential: Go, Rust, Java, C++, or Python.
• Proven LLM Agent Deployment: Demonstrated hands-on experience deploying, tuning and working with multi-agent frameworks and LLM (Large Language Model) agents, implementing function-calling mechanisms, or operating production-grade RAG (Retrieval-Augmented Generation) systems at scale.
• Deep Streaming and Orchestration Knowledge: In-depth knowledge of streaming technologies (e.g., WebRTC/LiveKit, gRPC), asynchronous orchestration patterns, the use of idempotency keys, and ensuring exact-once semantics in distributed environments.
• Practical Security and Compliance Chops: Strong practical understanding and experience with security and compliance requirements, including secure PII handling, robust key management strategies, implementation of comprehensive audit trails, and policy enforcement within a highly regulated environment.
• Strong Optimization Instincts: Possess strong instincts and hands-on profiling experience in prompt engineering and token optimization for large language models, demonstrating an ability to maximize efficiency and minimize costs.
• Reinforcement Learning Expertise: Experience with RLHF (Reinforcement Learning from Human Feedback) / RLAIF (Reinforcement Learning from AI Feedback), reward modeling, online learning techniques (e.g., bandit algorithms), and building sophisticated evaluation harnesses for AI models

Preferred Experience

• Candidates with the following additional qualifications will be particularly well-suited for this role:
• Voice Systems Experience: Familiarity with voice systems, including managing STT (Speech-to-Text) and TTS (Text-to-Speech) latency, and practical experience with telephony or voice quality assurance.
• Banking/FinServ Background: Prior experience in the banking or financial services industry, including an understanding of relevant regulations and compliance standards.

What Makes This Role Special

• You’ll define the core AI infrastructure powering autonomous financial workflows across millions of users
• You’ll lead the engineering strategy behind multi-agent AI systems—designing how autonomous AI think

Compensation

•  Compensation is expected to be between $210,000 - $240,000. Exact compensation may vary based on skills and location.

What We Offer

• Health: medical, dental, and vision insurance and wellbeing resources and programs
• Time away: Public holidays and discretionary PTO package for flexible days off with manager approval
• Financial: 401K, ESPP, Basic life and AD&D insurance, long-term and short-term disability
• Family: parental leave
• Development: Access to internal professional development resources.