# Library

Curated links for AI engineers: interactive explanations, playgrounds, roadmaps, agents, evals, tracking.

## LLM internals

See what happens inside a transformer, from tokens to the next-token probability.

- [LLM Visualization](https://bbycroft.net/llm): Fly through nano-GPT, GPT-2 and GPT-3 in 3D and follow one token through every layer and matrix multiply.
- [Transformer Explainer](https://poloclub.github.io/transformer-explainer/): Type a prompt into a live GPT-2 running in the browser and watch embeddings, attention heads and the temperature-adjusted output distribution change.
- [Attention in transformers, step by step (3Blue1Brown)](https://www.3blue1brown.com/lessons/attention/): An animated lesson that shows queries, keys and values as geometry, so the attention formula stops being symbols.
- [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/): Diagram-by-diagram walk through the original encoder-decoder transformer, one tensor shape at a time.
- [The Tokenizer Playground](https://huggingface.co/spaces/Xenova/the-tokenizer-playground): Paste text and compare how the tokenizers of GPT, Llama and other models split it, with token counts side by side.
- [The Annotated Transformer](https://nlp.seas.harvard.edu/annotated-transformer/): The 'Attention Is All You Need' paper interleaved line by line with runnable PyTorch code.
- [How To Scale Your Model](https://jax-ml.github.io/scaling-book/): Worked arithmetic for how transformers map onto accelerators: memory, FLOPs, and which parallelism to use when.
- [On the Biology of a Large Language Model](https://transformer-circuits.pub/2025/attribution-graphs/biology.html): Attribution graphs that trace which internal features a production model uses for multi-step reasoning, planning rhymes and arithmetic.

## Embeddings and retrieval

How text becomes vectors, how to look at those vectors, and how retrieval is built on them.

- [Embedding Projector](https://projector.tensorflow.org/): Load word2vec or MNIST embeddings and rotate them in PCA, t-SNE or UMAP to see which points end up as neighbours.
- [Understanding UMAP](https://pair-code.github.io/understanding-umap/): Drag n_neighbors and min_dist on toy datasets and a 3D mammoth to see what UMAP keeps and what it distorts.
- [Sentence Transformers documentation](https://www.sbert.net/): The library docs for computing, fine-tuning and evaluating text embeddings and rerankers, with semantic search examples.
- [MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard): Compare embedding models by task type, language, size and memory before picking one for retrieval.
- [Contextual Retrieval](https://www.anthropic.com/engineering/contextual-retrieval): Measured retrieval failure rates for plain chunking versus chunk context, BM25 hybrid search and reranking.

## Agents and tools

Patterns, protocols and failure modes of LLM systems that call tools and act in loops.

- [Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents): Workflows versus agents, five composable patterns, and when to skip frameworks and keep the loop simple.
- [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/): A structured survey of planning, memory and tool use in LLM agents, with the papers behind each idea.
- [Model Context Protocol specification](https://modelcontextprotocol.io/specification/2026-07-28): The current MCP spec: how a client and server negotiate tools, resources, prompts, auth and transport.
- [Writing effective tools for agents](https://www.anthropic.com/engineering/writing-tools-for-agents): How to design, prototype and evaluate tool definitions so a model picks and calls them correctly.
- [Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents): What to put in a finite context window over a long run: compaction, note-taking, sub-agents and just-in-time retrieval.
- [How we built our multi-agent research system](https://www.anthropic.com/engineering/multi-agent-research-system): A production write-up of an orchestrator with parallel sub-agents, including token cost, prompting lessons and how it was evaluated.
- [12-Factor Agents](https://github.com/humanlayer/12-factor-agents): Twelve engineering principles for LLM software that has to run reliably in production: own your prompts, own your control flow, and so on.

## Evals and benchmarks

How to measure whether an LLM product works, and what public benchmarks show.

- [Your AI Product Needs Evals](https://hamel.dev/blog/posts/evals/): A case study of building unit tests, human review and LLM-as-judge for a real product, and why that loop drives every improvement.
- [AI Evals FAQ](https://hamel.dev/blog/posts/evals-faq/): Direct answers to the common questions: error analysis first, binary judgments over scales, how many examples, when to trust a judge.
- [Evaluating the Effectiveness of LLM-Evaluators](https://eugeneyan.com/writing/llm-evaluators/): A review of the research on LLM-as-judge: which setups agree with humans, known biases and how to measure a judge.
- [Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents): How to grade multi-step agent runs: tasks, trials, graders and transcripts, and when to use code, model or human grading.
- [Inspect](https://inspect.aisi.org.uk/): The UK AI Security Institute's open-source framework for writing evals with datasets, solvers, scorers, tools and sandboxes.
- [Berkeley Function Calling Leaderboard](https://gorilla.cs.berkeley.edu/leaderboard.html): See how models compare on tool calling across single, parallel, multi-turn and agentic categories.

## ML intuition

Small models in the browser for building a feel for training, convolution and generation.

- [A Neural Network Playground](https://playground.tensorflow.org/): Train a tiny network in the browser and watch the decision boundary form as you change layers, features and regularisation.
- [CNN Explainer](https://poloclub.github.io/cnn-explainer/): Click any neuron of a small image classifier to see the kernel, the dot products and the activation map it produces.
- [Diffusion Explainer](https://poloclub.github.io/diffusion-explainer/): Follow Stable Diffusion from prompt to image and see how guidance scale and seed change each denoising step.
- [MLU-Explain](https://mlu-explain.github.io/): Visual essays from Amazon on precision and recall, ROC curves, cross-validation, bias-variance and double descent.
- [Why Momentum Really Works](https://distill.pub/2017/momentum/): Move the step size and momentum sliders and see gradient descent converge, oscillate or diverge on a quadratic.
- [Understanding Deep Learning](https://udlbook.github.io/udlbook/): A free deep learning textbook by Simon Prince with 68 fill-in-the-code notebooks and a set of interactive figures.

## Explorable explanations

Essays where the figures are live and the reader runs the experiment.

- [Bartosz Ciechanowski](https://ciechanow.ski/): Long essays on GPS, cameras, mechanical watches and airfoils where every figure is a 3D model you can drag.
- [AI Explorables (Google PAIR)](https://pair.withgoogle.com/explorables/): Short interactive pieces on memorisation versus generalisation, calibration, sparse autoencoders and fairness metrics.
- [Seeing Theory](https://seeing-theory.brown.edu/): Probability and statistics by simulation: roll dice, draw samples and watch the law of large numbers and Bayes' rule play out.
- [Explained Visually](https://setosa.io/ev/): Interactive explanations of PCA, eigenvectors, Markov chains, conditional probability and image kernels.
- [Distill](https://distill.pub/): The archive of the machine learning journal where articles were interactive essays on feature visualization, attention and optimisation.

## Computer vision

Vision and detection foundations, plus the tooling used in practice.

- [Hugging Face Community Computer Vision Course](https://huggingface.co/learn/computer-vision-course/unit0/welcome/welcome): A free course from CNN basics to vision transformers, detection, segmentation and multimodal models.
- [CS231n: Deep Learning for Computer Vision](https://cs231n.github.io/): Stanford's notes on image classification, backprop, CNNs and training practice, written to be read without the lectures.
- [Computer Vision: Algorithms and Applications](https://szeliski.org/Book/): Szeliski's free second-edition textbook covering image formation, features, motion estimation, stereo and recognition.
- [Ultralytics YOLO tracking mode](https://docs.ultralytics.com/modes/track): How to run BoT-SORT or ByteTrack on top of a YOLO detector, with the tracker config options explained.
- [supervision](https://github.com/roboflow/supervision): A Python library of reusable pieces for detection pipelines: box and mask annotators, zone counting, ByteTrack and dataset conversion.

## Tracking and estimation

Kalman filters, multi-target tracking, and how trackers are scored.

- [Kalman Filter Explained Through Examples](https://kalmanfilter.net/): Numerical examples, from weighing a gold bar to tracking a vehicle, that derive each Kalman filter equation before stating it.
- [How a Kalman filter works, in pictures](https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/): Gaussian blobs multiplied together show why predict and update give the Kalman equations.
- [Kalman and Bayesian Filters in Python](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python): A free book of notebooks that builds g-h, Kalman, EKF, UKF and particle filters from intuition, with simulations you can run.
- [Stone Soup tutorials](https://stonesoup.readthedocs.io/en/latest/auto_tutorials/index.html): Tutorials of an open tracking framework: Kalman and particle filters, data association, JPDA, GM-PHD and track metrics, with plots.
- [TrackEval](https://github.com/JonathonLuiten/TrackEval): The reference implementation of HOTA, CLEAR MOT and identity metrics used by the main tracking benchmarks.
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://arxiv.org/abs/2009.07736): Why MOTA over-weights detection and IDF1 over-weights association, and how HOTA splits the score into both.
- [MOTChallenge MOT17 (archive)](https://motchallenge.net/results/MOT17/): The archived MOT17 leaderboard with 219 methods, per-sequence results and a CSV of every metric.
- [ByteTrack](https://github.com/FoundationVision/ByteTrack): Code for the tracker that also associates low-confidence detections, which recovers occluded objects that score-thresholding drops.

## Roadmaps and courses

Structured paths, from a first map of the field to building a model from scratch.

- [AI Engineer Roadmap](https://roadmap.sh/ai-engineer): A clickable map of what an AI engineer is expected to know, from model APIs and embeddings to RAG and agents.
- [AI Agents Roadmap](https://roadmap.sh/ai-agents): A clickable map of agent topics: the agent loop, tools, memory, protocols, evaluation and security.
- [Neural Networks: Zero to Hero](https://karpathy.ai/zero-to-hero.html): Karpathy builds backprop, a character model, a tokenizer and GPT from an empty file, in video with code.
- [Build a Large Language Model (From Scratch)](https://github.com/rasbt/LLMs-from-scratch): Notebooks that implement a GPT-style model in PyTorch, then pretrain, fine-tune for classification and instruction-tune it.
- [Stanford CS336: Language Modeling from Scratch](https://cs336.stanford.edu/): Lectures and assignments that build tokenizer, transformer, training, scaling laws, data and alignment yourself.

## Writing worth following

People who publish careful, first-hand notes on building with ML and LLMs.

- [Simon Willison's Weblog](https://simonwillison.net/): Daily hands-on notes on new models, prompt injection, tools and what he actually built with them.
- [Lil'Log](https://lilianweng.github.io/): Long literature reviews on agents, hallucination, diffusion, RL and prompting, each with the full reference list.
- [Chip Huyen](https://huyenchip.com/blog/): Essays on ML systems in production, AI engineering and building LLM applications, from the author of 'AI Engineering'.
- [Eugene Yan](https://eugeneyan.com/): Applied notes on recommender systems, evals, LLM patterns and ML team practice from industry work.
- [What We Learned From A Year of Building with LLMs](https://applied-llms.org/): Tactical, operational and strategic lessons from six practitioners who shipped LLM products.
