Applied Scientist - LLM, Alexa Conversational Modelling Intelligence

Job Description

As an Applied Scientist II in the Alexa Conversational Modelling Intelligence team within Alexa AI, you will drive model post‑training for Large Language Models that power Alexa+.

You will adopt state‑of‑the‑art techniques—supervised fine‑tuning, RLHF, preference optimisation—and conduct rigorous experiments to translate findings into production‑ready solutions that improve the experience for millions of users worldwide.

Key Responsibilities

• Own full model development lifecycle: data curation, model training, evaluation, and deployment of LLMs that power Alexa+.
• Design and execute training recipes (SFT, RLHF, preference optimisation) and iterate rapidly on data, hyper‑parameters, and architectures to meet quality and efficiency goals.
• Build robust evaluation frameworks, develop benchmarks, implement LLM‑as‑a‑judge pipelines, and conduct defect analysis to identify gaps in conversational quality, safety, and fluency.
• Translate evaluation insights into targeted improvements that close identified gaps.
• Collaborate closely with research scientists and engineering partners to scale models from experimentation to production.
• Document methods and contribute to shared tooling and infrastructure so the team can build on your work.
• Publish findings at top‑tier NLP/ML conferences to advance the state of the art while grounding work in measurable outcomes.
• About The Team

The Alexa Conversational Modelling Intelligence team builds LLM‑based conversational technologies that customers love.

Our mission is to push the envelope in LLMs for Alexa to deliver the best‑possible customer experience.

A Day in the Life

Your day will involve launching and monitoring training runs, analysing experiment results, and iterating on model recipes based on evaluation data.

You will also participate in science reviews, sync with engineering partners on deployment readiness, and deep‑dive into model outputs to understand behavioural patterns, balancing hands‑on experimentation with collaborative problem‑solving across the Alexa AI organisation.

• Basic Qualifications
• Ph D in computer science, machine learning, engineering or related fields.
• Knowledge of at least one programming language such as Java, C#, Java Script, Python, Ruby or Perl.
• Experience designing experiments and performing statistical analysis of results.
• Hands‑on experience building, training and evaluating LLMs.
• Preferred Qualifications
• Publications at top‑tier conferences such as CVPR, ICCV, ECCV or Neur IPS.
• Experience working with large, complex data sets.
• Experience collaborating with science, data‑processing and software engineering teams.
• Strong written and verbal communication skills to convey technical information to non‑technical audiences.
• Experience building and deploying LLM solutions in production or at scale.
• Hands‑on experience with LLM training and fine‑tuning via pre‑training, SFT or RLHF/preference optimisation.
• Experience in LLM evaluation, building benchmarks, LLM‑as‑a‑judge, or defect/quality analysis.
• Familiarity with modern training/inference infrastructure such as distributed training, RL frameworks and model serving.
• Equal Opportunity Employment

Amazon is an equal opportunities employer.

We make recruiting decisions based on your experience and skills.

Our inclusive culture empowers Amazonians to deliver the best results for our customers.

If you have a disability and need a workplace accommodation, please visit https://amazon. jobs/content/en/how-we-hire/accommodations for more information.

• Gender & Diversity m/w/d
• #J-18808-Ljbffr