Staff/Principal ML Engineer

We are looking for an exceptional

Staff/Principal ML Engineer to join our client, a cutting-edge real-time AI research lab and inference provider backed by top-tier venture funds.

Powering hundreds of millions of daily active users and serving over

10 trillion LLM tokens monthly , the platform operates at massive scale behind low-latency voice and conversational

AI APIs .

In this role, you will take true end-to-end technical ownership of inference optimization, containerizing foundational models and scaling sub-second, high-concurrency workloads on high-performance GPU infrastructure.

What You’ll Be Doing

• Taking complete end-to-end technical ownership of the model serving and inference optimization lifecycle.

Taking cutting-edge speech, voice, and LLM architectures directly from the AI research team and containerizing them for production deployment.

• Optimizing serving runtimes and kernels for sub-second latency and ultra-high-concurrency workloads on distributed

NVIDIA GPU clusters .

• Architecting and scaling distributed inference platforms using

Ray, Kubernetes, and Docker to ensure 99.99% operational reliability under heavy global traffic.

• Diagnosing low-level system bottlenecks, memory allocations, and

GPU compute performance across custom model serving pipelines.

• Driving architectural standards, mentoring senior team members, and collaborating cross-functionally between research and core platform engineering.

What We’re Looking For

Staff/Principal-level software engineering and ML systems background (equivalent to Google L5/L6 or Meta E5/E6).

• Deep systems-level experience profiling, benchmarking, and optimizing high-throughput, low-latency machine learning pipelines in production.
• Consultant mindset: exceptional technical leadership, high ownership, ability to make sound architectural trade-offs, and clear communication across engineering squads.
• Strong problem-solving abilities within low-level hardware-software interfaces and GPU architectures.

Must-Haves

• Strong programming mastery in

C++, Python, and/or Rust.

• Hands-on expertise with high-performance model serving frameworks

v LLM and Tensor RT-LLM (TRT-LLM).

• Deep knowledge of CUDA programming, GPU kernel optimization, and NVIDIA GPU architecture.
• Proven production experience with containerization and distributed orchestration using Kubernetes, Ray, and Docker.
• Demonstrable track record deploying and running large-scale LLM/Gen AI inference systems in production.
• Fluent English for daily international technical collaboration.
• #J-18808-Ljbffr