> For the complete documentation index, see [llms.txt](https://jamiewen00.gitbook.io/ai-engineering-handbook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://jamiewen00.gitbook.io/ai-engineering-handbook/roles-and-teams/ai-team.md).
# AI Team
## Before LLMs
Before LLM APIs, AI teams focused on specialists for model development from scratch.
**AI Product Manager**: Defines product vision and metrics. Aligns AI with business goals
**Data Scientist**: Designs AI/ML models. Runs experiments and analyses data. Builds and tunes models. Maintains model performance
**ML Engineer**: Bridges data science and production. Scales models in production. Integrates models into codebases
**Data Engineer**: Sets up data pipelines. Ensures clean data for training. Provides scalable infrastructure.Lays the foundation for data-driven AI.
**Software Engineer** (normally sits outside the model team): Integrates AI with frontend and backend. Connects data science to user experience. Delivers speed and functionality.
## With LLMs
With LLM APIs, teams evolve towards integration-focused workflows while maintaining core expertise.
**AI Product Manager**: Defines vision and metrics. Aligns capabilities with business goals
**Software Engineer**: Builds UI and backend logic. Integrates AI features. Handles data flow between UI, backend, and APIs. Maintains product performance.
**ML Engineer**: Evaluates quality of LLM outputs. Ensures alignment with goals. Identifies bugs and performance issues. Tests for accuracy and bias
**AI Engineer**: Builds full-stack AI applications. Uses LLMs to solve real problems. Applies LLMs to products. Measures AI performance. Performs prompt engineering.
**Data Scientist** (evolved role): Adapts expertise to LLM evaluation and optimisation. Analyses LLM performance patterns. Designs experiments for prompt effectiveness. Maintains statistical rigour in AI assessments.
## Advanced Teams
For fine-tuning and advanced features, specialist roles expand their responsibilities.
**Data Scientist** (expanded scope)
* Fine-tunes models with company data
* Evaluates LLM performance
* Runs experiments with different models
* Identifies and fixes biases
* Bridges traditional ML with LLM capabilities
**Data Engineer** (enhanced role)
* Manages pipelines for fine-tuning
* Ensures data quality
* Maintains data lakes and warehouses
* Supports real-time data via prompts
* Adapts infrastructure for LLM workflows
**ML Engineer** (evolved responsibilities)
* Integrates LLM API calls into production
* Ensures scalable API integration
* Sets up MLOps for LLM integration
* Monitors API performance
* Combines traditional MLOps with LLM operations
**AI Engineer**
The current job market (2025) shows AI engineer positions require hybrid skill sets, combining traditional ML expertise with LLM integration capabilities. Organisations value both foundational AI/ML engineering skills and modern LLM application abilities.
***
## Scaling Infra
Essential capabilities for scaling AI.
**LLM Accessibility**
* Provides easy LLM access
* Standardises integration
* Reduces adoption barriers
**Observability Platform**
* Monitors AI performance
* Tracks usage and costs
* Identifies optimisation opportunities
**Evaluation Support**
* Measures AI effectiveness
* Provides testing frameworks
* Ensures quality standards
**Feedback Loop**
* Captures user interactions
* Enables continuous improvement
* Informs feature development
**Experimentation Sandbox**
* Provides a safe testing environment
* Enables rapid prototyping
* Allows risk-free exploration
**Recipes and Demos**
* Offers implementation examples
* Accelerates development
* Models best practices
## Transitioning
> 💡 What types of leaders do you need to transition from a traditional team to an AI team?
Operator — focused on maintenance and functioning, through stability and security.
Guide — focused on humanistic growth and supporting others, through affiliation and bonding.
Achiever — focused on victory and success, through sheer drive and achievement.
Catalyst — focused on transformation and innovation, through inspiration and engagement.
***
> 💡 Leaders need to cover all four areas, but which ones should they focus on?
1. Tech Lead EM: tech-led, people + delivery
2. Team Lead EM: people-led, tech + delivery
3. Delivery EM: delivery-led, tech + people
4. Product EM: product-led, tech + delivery. (I think this can be a technical product manager)
5. Lead of Leads EM: although they are team and delivery focused, you need to be one of 1-4 before you can be a lead of leads.