LLMOps: Large Language Models in Production Specialization

Run large language model applications in production: evaluation, serving on Kubernetes, efficient inference and security.

Evaluate and monitor LLM applications so you know when they get better or worse, deploy and scale models on Kubernetes, make inference faster and cheaper with batching, caching and quantisation, and defend your applications against prompt injection and data leaks. Every course is hands-on, with labs and a capstone project your instructors review.

Prepares you for

LLMOps engineer, AI infrastructure engineer, MLOps engineer

4 courses in this Specialization

• LLMOps: Evaluating, Tracing and Monitoring LLM Applications: Run LLM features like production software: golden sets and judges, regression gates with replayed recordings, OpenTelemetry traces, cost and guardrail metrics, and A/B tests. (Advanced, about 17 hours)
• Operationalising LLMs on Kubernetes: Deploy, scale, observe and upgrade open large language models on Kubernetes: inference servers, model storage, GPU scheduling, queue-based autoscaling and safe rollouts. (Advanced, about 24 hours)
• Efficient Inference: Quantisation, Distillation and Batching: Make language models faster, smaller and cheaper to serve: find where time and memory go, batch continuously, quantise weights and activations with the maths behind it, and distil, prune and speculate. (Advanced, about 18 hours)
• Securing LLM Applications: Protect your own LLM features: threat-model them with the OWASP Top 10, contain prompt injection in layers, handle output and tools safely, keep data private, and prove it with an automated test suite. (Intermediate, about 22 hours)