Design, build, and operate end-to-end training and inference infrastructure for large language and multimodal models. Improve efficiency (memory, parallelism, kernel optimizations), ensure robust scalable training and RL pipelines, optimize low-latency/high-throughput serving (quantization, caching, speculative decoding), manage multi-GPU and multi-cloud orchestration, and productionize new algorithms with strong observability and reproducibility.
About Goaly
About the Role
Key Responsibilities
Requirements
Bonus Points
At Goaly, our mission is to make custom AI affordable for every business. Our founding team comes from the front lines of top AI labs and tech giants (Meta MSL, TikTok AI, Google DeepMind, xAI, Microsoft Research, etc.), where we built large-scale training infrastructure powering trillion-parameter models and scaled GenAI models to a global user base. Now, we are building something we wish we had before: a platform that makes training and adapting custom AI affordable for all modern companies, not just Big Tech. Our north star is ambitious: for a domain-specific task, reach 90% of SOTA performance at less than 10% of the cost. To get a taste of what we are doing, see our first tech blog.
You will sit at the intersection of systems engineering and applied ML, building specialized infrastructure that keeps large language and multimodal models fast, reliable, and cost-effective. You will partner with research, product, and infra teams to ship production-ready platforms for training and serving AI at scale.
- Efficiency & performance: Improve LLM training and inference efficiency through better memory utilization, optimized parallelism, and kernel-level innovations (e.g. FlashAttention, CUDA/Triton).
- Training & RL robustness: Build scalable, stable training and RL pipelines with strong reproducibility, observability, and debuggability.
- Serving & inference optimization: Design and tune high-throughput, low-latency model serving systems, including quantization, caching, and speculative decoding.
- Scalability & infrastructure: Own end-to-end training and inference infrastructure — from data ingestion and checkpointing to multi-GPU and multi-cloud orchestration.
- Production enablement: Work closely with researchers and product engineers to turn new algorithms into reliable, production-ready systems.
- 5+ years building or operating ML infrastructure at scale, ideally supporting large language or multimodal models.
- Deep understanding of GPU architecture, distributed training frameworks (PyTorch, DeepSpeed, Megatron, Ray), and parallelism strategies.
- Hands-on experience running inference stacks (vLLM / SGLang, TGI, Triton) and optimizing them via low-level profiling.
- Strong software engineering fundamentals in Python and one of C++/Rust/Go, with clean, reliable code shipped to production.
- Working knowledge of modern data pipelines, feature stores, and vector databases used in production AI systems.
- Comfort automating infrastructure with Kubernetes, Terraform/Pulumi, and observability stacks (Prometheus, Grafana, OpenTelemetry).
- Experience deploying open-source LLMs (Llama 3, Qwen, DeepSeek) or training custom foundation models.
- Contributions to ML systems tooling (compilers, kernels, inference runtimes) or open-source infrastructure projects.
- Background in reinforcement learning, evaluation harnesses, or alignment tooling that hardens production AI systems.
Similar Jobs
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Lead global M&A, investments, and post-closing tax integration/compliance. Partner with internal and external stakeholders on structuring, perform tax modeling (e.g., Sections 382/383), research complex tax issues, manage income tax audits, and support Treasury, state planning, and special tax projects.
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Lead and own a portfolio of issuer and processor partner relationships end-to-end. Drive partner onboarding, technical integration, BIN setup, approvals, pilot and GA readiness. Coordinate cross-functional stakeholders, define milestones, mitigate risks, build governance/SLAs/incident protocols, and advise leadership on partner strategy and regulatory changes. Engage executive partners and use AI tooling to streamline reporting and diligence.
Top Skills:
AIBinLlmMastercardVisa
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Outbound senior account executive focused on restaurant SMBs. Build pipeline via cold outreach, prospecting, discovery, demos, and field visits. Close new-logo deals selling the Square ecosystem, partner with Business Development, Product and Marketing, use Salesforce to track activity, and exceed monthly sales targets and KPIs.
Top Skills:
SalesforceSquare
What you need to know about the Montreal Tech Scene
With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.
Key Facts About Montreal Tech
- Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
- Major Tech Employers: SAP, Google, Microsoft, Cisco
- Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
- Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
- Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
- Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal
