Dream Team

🏛️ Before LLMs

Before LLM APIs, AI teams focused on specialists for model development from scratch.

AI Product Manager

• Defines product vision and metrics

• Aligns AI with business goals

Data Scientist

• Designs AI/ML models

• Runs experiments and analyses data

• Builds and tunes models

• Maintains model performance

ML Engineer

• Bridges data science and production

• Scales models in production

• Integrates models into codebases

Data Engineer

• Sets up data pipelines

• Ensures clean data for training

• Provides scalable infrastructure

• Foundation for data-driven AI

Full Stack Developer (they normally sit outside the model team)

• Integrates AI with frontend and backend

• Connects data science to user experience

• Delivers speed and functionality

🚀 With LLMs

With LLM APIs, teams evolve toward integration-focused workflows while maintaining core expertise.

AI Product Manager

• Defines vision and metrics

• Aligns capabilities with business goals

Full Stack Developer

• Builds UI and backend logic

• Integrates AI features

• Handles data flow between UI, backend, APIs

• Maintains product performance

ML Engineer

• Evaluates quality of LLM outputs

• Ensures alignment with goals

• Identifies bugs and performance issues

• Tests for accuracy and bias

AI Engineer

• Builds full-stack AI applications

• Uses LLMs for real problems

• Applies LLMs to products

• Measures AI performance

• Prompt engineering

Data Scientist (evolved role)

• Adapts expertise to LLM evaluation and optimisation

• Analyses LLM performance patterns

• Designs experiments for prompt effectiveness

• Maintains statistical rigour in AI assessments

⚡ Advanced Teams

For fine-tuning and advanced features, specialist roles expand their responsibilities.

Data Scientist (expanded scope)

• Fine-tunes models with company data

• Evaluates LLM performance

• Runs experiments with different models

• Identifies and fixes biases

• Bridges traditional ML with LLM capabilities

Data Engineer (enhanced role)

• Manages pipelines for fine-tuning

• Ensures data quality

• Maintains data lakes and warehouses

• Supports real-time data via prompts

• Adapts infrastructure for LLM workflows

ML Engineer (evolved responsibilities)

• Integrates LLM API calls into production

• Ensures scalable API integration

• Sets up MLOps for LLM integration

• Monitors API performance

• Combines traditional MLOps with LLM operations

AI Engineer Current job market (2025) shows AI engineer positions require hybrid skill sets combining traditional ML expertise with LLM integration capabilities. Organisations value both foundational AI/ML engineering skills and modern LLM application abilities.

🎯 Scaling Infra

Essential capabilities for scaling AI.

LLM Accessibility

• Provides easy LLM access

• Standardises integration

• Reduces adoption barriers

Observability Platform

• Monitors AI performance

• Tracks usage and costs

• Identifies optimisation opportunities

Evaluation Support

• Measures AI effectiveness

• Provides testing frameworks

• Ensures quality standards

Feedback Loop

• Captures user interactions

• Enables continuous improvement

• Informs feature development

Experimentation Sandbox

• Safe testing environment

• Rapid prototyping

• Risk-free exploration

Recipes and Demos

• Implementation examples

• Accelerates development

• Models best practices