LanceDB Logo

LanceDB

Senior Software Engineer

Posted 18 Days Ago
Remote
Hiring Remotely in Canada
Senior level
Remote
Hiring Remotely in Canada
Senior level
Design and maintain distributed Lance dataset operations, develop high-performance indices and data formats, and improve query performance across Spark, Ray, and Trino. Contribute Rust-based table formats and encodings, integrate Lance with major open-source data infrastructure systems, operate internal data processing infrastructure, and promote the project through open-source communities and conferences.
The summary above was generated by AI

About LanceDB

AI advances at the speed of its research, and research moves at the speed of its data. LanceDB is the AI-native Multimodal Lakehouse: one system where a researcher curates petabytes of video, audio, and every signal derived from them with a few lines of Python, and the next training run starts as fast as the next idea. Customers like Runway, Midjourney, and Netflix build the future of AI on LanceDB, from frontier and world models to robots and autonomous vehicles.

About the Role

We're looking for a Senior Software Engineer to help expand the reach of Lance and LanceDB within the broader data infrastructure ecosystem. You'll work at the intersection of high-performance computing, big data, and open-source systems. You will contribute scale and performance improvements, integrations with the wider data and AI ecosystem, simplifying distributed operations, and usability and maintainability enhancements.

What You'll Do

  • Designing and maintaining efficient distributed Lance dataset operations

  • Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino

  • Working on table formats, data encodings, and various aspects of the Lance format in Rust

  • Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems

  • Operating and improving internal data processing infrastructure

  • Promoting the Lance format in open-source communities and at Big Data conferences

What We're Looking For

  • 10+ years of experience building high-performance databases, big data systems, or large-scale data services

  • Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)

  • Strong experience with high-performance computing in C++, Java, and/or Scala

  • Experience with Rust (or willingness to learn it)

  • Proven ability to move fast, work independently, and collaborate with a high-caliber team

Nice to Have

  • Contributor, committer, or PMC member in Apache or other large open-source projects

  • Experience with Apache Arrow, DataFusion, Parquet, Iceberg, or Delta Lake

  • Track record of driving large features or integrations in distributed systems

  • Strong community presence and passion for open-source collaboration

Similar Jobs

Yesterday
Remote
Senior level
Senior level
Fintech • Information Technology
Design and ship autonomous agent execution for brokerage accounts. Build account-level policies, guardrails, rule-evaluation services, and audit mechanisms protecting trading systems. Partner with Product, Compliance, Risk, identity, and tools teams to define safe unattended actions. Lead architecture, production reliability, design reviews, implementation, and mentorship while working primarily in Go and using Python or TypeScript.
Top Skills: GoLangchainMcpOauthPythonTypescript
Yesterday
Remote
Senior level
Senior level
Fintech • Information Technology
Build and own Alpaca’s agent-facing developer platform, including MCP servers, tools, skills, and trading and account APIs. Define architecture, evolve integrations as standards change, ensure production reliability, and partner with Product, Compliance, Risk, and Platform Engineering. Lead through hands-on development, design reviews, and mentorship while shipping platform capabilities used by developers and financial partners.
Top Skills: APIsGoLangchainMcpPythonSdksTypescript
3 Days Ago
Remote
Senior level
Senior level
Payments
Build and operate security platform services in Go and Python on AWS and Kubernetes. Own the NATS event bus, security scanning gates, CI/CD integrations, finding workflows, triage automation, and remediation routing. Develop AI-assisted security workflows, improve scanner accuracy and reliability, participate in threat modeling and secure design reviews, and support on-call operations for owned services.
Top Skills: AWSClaude CodeCodeqlCodexConftestCursorElasticsearchGitGitlab CiGitleaksGoGrypeKubernetesKyvernoMySQLNatsOpaOwasp Top 10PostgresPythonReactReact NativeSastSbomScaSemgrepSigstoreSlsaSnykTrivyTypescript

What you need to know about the Montreal Tech Scene

With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.

Key Facts About Montreal Tech

  • Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
  • Major Tech Employers: SAP, Google, Microsoft, Cisco
  • Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
  • Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
  • Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
  • Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account