AI Engineer Certificate

Part 1 · What Is an AI Engineer? (20 min)

• AI engineer vs ML engineer vs data scientist: where the lines are and why they matter
• The production gap: why most demos fail when they hit real users
• The coding-agent-first workflow: how to direct an agent, not just prompt it
• Tour of the stack you will be using across this programme

Part 2 · How LLMs Work (30 min)

• Tokens, not words: why this matters for cost and behaviour
• The context window: what goes in, what gets forgotten, what costs money
• Temperature, top-p, and sampling: when to turn the dial and when to leave it
• System prompts, user prompts, assistant turns: the message structure
• Model families and their tradeoffs: Claude, GPT-4o, Gemini, Mistral, open-source
• Cut-off dates and knowledge limitations: how to reason about what a model knows

Part 3 · APIs, Structured Outputs, and Validation (40 min)

• Multi-provider API patterns: how to write provider-agnostic code
• Managing tokens: counting, budgeting, pricing considerations
• Structured outputs and data contracts: getting reliable JSON from a model
• Output validation: what to do when the model ignores your schema
• Multimodal inputs: sending documents, images, and PDFs to the API
• Error handling, retries, and fallbacks

Part 4 · Prompt Design for Production (30 min)

• Why prompts that work once often fail at scale
• System prompt architecture: separating instructions, context, and persona
• Few-shot examples: when to include them and how many
• Chain-of-thought and when it actually helps
• Negative examples: telling the model what NOT to do
• Prompt versioning: treating prompts as code

Part 1 · Embeddings and Semantic Search (25 min)

• What embeddings are: text as points in vector space
• How similarity search works: cosine, dot product, L2 distance
• Embedding use cases: semantic search, recommendations, classification, anomaly detection
• Choosing an embedding model in 2026: OpenAI text-embedding-3-large, Cohere, BGE-M3 for self-hosting
• Matryoshka embeddings: flexible dimensions for storage-cost tradeoffs

Part 2 · Vector Databases and RAG Architecture (35 min)

• What vector databases do and why you need one
• Comparing options: Qdrant, Pinecone, Chroma, pgvector, Weaviate
• Chunking strategies: fixed-size vs semantic vs late chunking
• Dense retrieval, sparse retrieval (BM25), and hybrid approaches
• Reranking: why first-pass retrieval is often not enough
• The full RAG pipeline: ingest, embed, store, retrieve, generate
• GraphRAG: when knowledge graphs improve multi-hop reasoning

Part 3 · Context Engineering (30 min)

• The context lifecycle: what goes in, when, and in what order
• Prompt caching: how to reduce cost on repeated prefixes
• Context compression: summarisation, distillation, selective retrieval
• Window budgeting: allocating tokens across system, history, retrieved context, and query
• The difference between context engineering and prompt engineering

Part 4 · Function Calling, MCP, and Open-Source (30 min)

• Function calling: how tools are registered and invoked
• Model Context Protocol (MCP): the universal standard for agent-tool integration
• A2A (Agent-to-Agent): how agents discover and delegate to each other
• Open-source models with Ollama: running Llama, Mistral, and others locally
• Open vs closed source tradeoffs: latency, cost, control, capability

Part 1 · Agent Fundamentals and ReAct (25 min)

• What makes a system an agent: the perceive-reason-act loop
• ReAct prompting: Reasoning + Acting as the core pattern
• Single vs multi-agent architectures: when to add an agent vs extend an existing one
• The supervisor pattern: one orchestrator, multiple specialists
• Agent cards and capability discovery with A2A

Part 2 · Orchestration with LangGraph (35 min)

• State graphs: modelling agent behaviour as a graph
• Nodes, edges, and conditional routing
• Checkpointing: saving and resuming agent state
• Human-in-the-loop: when to pause and ask
• MCP in agentic workflows: connecting tools at the orchestration layer
• SDK-native agents: building without a framework

Part 3 · Memory, Failure Modes, Security, and n8n (30 min)

• In-context memory: what is in the window right now
• External memory: vector stores and knowledge bases as agent memory
• Episodic memory: remembering across sessions
• Agentic failure modes: tool call loops, context pollution, cascading errors, hallucinated tool outputs
• Cost engineering in agents: model routing, token budgeting across multi-step workflows
• Prompt injection attacks: direct and indirect (via retrieved content)
• Guardrails: validating inputs, outputs, and tool call parameters
• n8n and workflow automation: connecting your agent to real business workflows

Evals Session · Evaluation-Driven Development (40 min)

• Why agents that seem to work often do not: the silent failure problem
• Evaluation-driven development: writing your eval before you write your agent
• Building a golden dataset: what good looks like, documented
• Rule-based evals: fast, cheap, brittle
• LLM-as-judge: using a model to evaluate a model
• Setting up Langfuse: tracing, logging, and dashboards
• Regression testing: catching regressions before they reach users
• Failure mode analysis: turning production failures into eval cases

Part 1 · Deployment Architecture (25 min)

• Serverless vs container: latency, cold starts, cost tradeoffs
• Streaming responses: why and how
• Async queues for long-running agent tasks
• API gateway patterns for LLM systems
• Managing secrets and credentials in production

Part 2 · LLM Ops (30 min)

• Prompt versioning: treating prompts as code artifacts with history
• Model versioning: handling model updates without breaking production
• CI/CD for LLM systems: what a pipeline looks like
• Running evals automatically on every pull request
• Rollback strategies: how to safely revert a bad prompt or model change

Part 3 · Observability, Cost, and Safety (35 min)

• Langfuse deep dive: traces, sessions, users, scores
• Silent degradation: the failure mode no one sees coming
• Cost optimisation at scale: prompt caching, batching, model routing
• Token cost monitoring: per-user, per-session, per-model breakdowns
• Safety and guardrails in production: input validation, output filtering, kill switches
• Fallbacks and safe defaults: what the system does when it cannot answer

Part 4 · Fine-Tuning, System Cards, and Shipping (30 min)

• Fine-tuning vs prompting vs RAG: the decision framework
• When fine-tuning is worth it and when it is not
• The system card: documenting what your system does, how it fails, and what to watch
• Documentation standards for production AI systems
• What good AI engineering looks like in 2026: the skills that matter
• Capstone showcase: presenting your system to the cohort