Senior ML Engineer – VLM

Responsibilities

• Own the offline dataset pipeline — design, implement, test, and deploy Cloud-based pipelines that convert logged multi-sensor data into VLM/VLA training datasets, spanning geometric labels (3D/2D detection, tracking, segmentation, depth) through semantic, scenario-level, and action/trajectory-grounded annotations.
• Build VLM-assisted auto-labeling — develop open-vocabulary detection, dense captioning, semantic enrichment, and scene/scenario description generation that move beyond closed-set bounding boxes, using foundation models to scale annotation and cut manual labeling cost.
• Generate reasoning-grounded labels — produce language-grounded reasoning and chain-of-causation style annotations, temporally aligned to ego-motion and trajectories, to support VLA training and explainable driving behavior.
• Mine and curate the long tail — surface rare, difficult, and high-uncertainty scenarios, and build curated datasets that measurably improve downstream VLM/VLA model metrics rather than simply adding volume.
• Close the data flywheel — define dataset schemas, quality metrics, and validation; track auto-labeling quality against model requirements; route model failures back into re-labeling and retraining loops.
• Partner with the end-to-end model team — co-define dataset specifications with VLM/VLA model developers, own the quality bar and delivery cadence, and operationalize a continuous dataset delivery loop into their training pipelines.
• Scale on cloud infrastructure — build distributed, reproducible pipelines using columnar data formats and distributed compute, with disciplined software practices, version control, and documentation.
• Lead and mentor — serve as project lead, guide less-experienced engineers, run design reviews, set coding and annotation standards, and drive alignment across team interfaces to the rest of the organization.
• Stay current — track the latest advances in multimodal models, auto-labeling, and end-to-end autonomous driving, and translate relevant research into production data systems.

Requirements

• Considered highly skilled and proficient in discipline; conducts complex, important work under minimal supervision and with wide latitude for independent judgment.
• Scope of

Influence: Expected to drive alignment across team interfaces to the rest of the organization.

Designs, maintains, and owns team technical solutions and drives consensus.

Mentors and guides engineers within the group.

• Bachelor’s Degree in Computer Science, Robotics, Electrical Engineering, or related technical field plus competences typically acquired through 6+ years of experience; OR Master’s Degree in a related technical field plus competences typically acquired through 3+ years of experience.
• Computer Vision & Deep Learning — model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation.
• Multimodal / VLM experience — hands-on work with vision-language models, open-vocabulary or zero-shot recognition, dense captioning, or semantic embeddings / search applied to perception data.
• Model Data Curation — building targeted datasets that measurably improve downstream model performance; large-scale Parquet data processing (Databricks, Daft, Pandas, etc.).
• Distributed ML & data frameworks — Py Torch, Lightning, Ray, Spark, or equivalent for training and large-scale data processing.
• Scaled MLOps & Tooling — experiment tracking, model registry, MLflow / Weights & Biases, and ML metrics, evaluation, and quality.
• Development Tools & Eco-System (at scale) — strong Python software development, VDI and cloud-based development environments, CI systems (Git Hub Actions), and Docker.
• #J-18808-Ljbffr