Artifacts

Pick a GPU and a model to see whether it fits in memory for inference or training.

Calculates the memory a Hugging Face Hub model needs at different precisions for inference and training.

Estimates how much GPU memory a language model needs for a given size, quantisation and context length.

Predicts training memory use for a transformer from pasted config or manual architecture values.

Calculates VRAM needs for a GGUF model, including splitting layers across GPU and CPU.

Computes memory for a GGUF model from GPU layers and context size.

Web calculator for estimating and comparing LLM API costs across models.