Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Gaming
Designs and implements live-ops events and content for a mobile game, including balancing systems, analyzing player feedback and metrics, improving design tools, and collaborating across disciplines. Creates new event types, prototypes features, writes design documentation and user stories, leads brainstorming, and advocates for player-focused design. The role requires strong game systems, content, narrative, math, balancing, pacing, and mobile gaming expertise.
Top Skills:
Unity
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Build and own foundational Android platform infrastructure, including architecture, shared libraries, tooling, observability, performance systems, and developer experience. Lead cross-organizational technical initiatives, establish Android engineering standards, evaluate AI-assisted development tools, and build AI-powered platform capabilities. Collaborate across mobile, web, backend, product, and design teams while mentoring engineers and driving large-scale organizational impact.
Top Skills:
AndroidArtifactoryBuildkiteCi/CdClaude CodeCodexGooseGradleJetpack ComposeKochikuKotlinKotlin MultiplatformMoleculePaparazziSqldelightStructured ConcurrencyWireWorkmanager
Blockchain • Fintech • Mobile • Payments • Software • Financial Services
Staff Android Software Engineer responsible for owning Cash App’s foundational Android platform, including architecture, shared libraries, infrastructure, developer tooling, observability, performance, and experimentation systems. The role drives AI-native engineering practices, leads cross-organizational technical initiatives, establishes standards, mentors engineers, and partners across mobile, web, backend, product, and design teams. Requires deep Android expertise and organizational-level impact.
Top Skills:
A/B TestingAndroidArtifactoryBuildkiteCi/CdClaude CodeCodexCustom Gradle PluginsDeep LinkingDependency InjectionFeature FlaggingGooseGradleJetpack ComposeKochikuKotlinKotlin MultiplatformMobile Application Performance ProfilingMoleculePaparazziSqldelightStructured ConcurrencyWireWorkmanager
What you need to know about the Montreal Tech Scene
With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.
Key Facts About Montreal Tech
- Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
- Major Tech Employers: SAP, Google, Microsoft, Cisco
- Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
- Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
- Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
- Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal
