Why does my 80 GB GPU report 81,920 MiB instead of 80 GB?

Because NVIDIA labels VRAM in binary. The 80 GB on an A100 is 80 GiB, which nvidia-smi reports as 81,920 MiB and which equals 85.9 decimal GB — roughly 7% more bytes than the label implies. The GB-vs-GiB gap is real, but on GPU capacity it runs in your favour rather than against you. Applying the storage conversion here makes every card look 7% smaller than it is.

How much of an 80 GiB card can a training run actually use?

Less than 80 GiB, but not because of units. ECC and reserved rows hold back a few hundred MiB (an H100 80GB reports 81,559 MiB against the A100's 81,920), the CUDA context costs 300 to 600 MiB per process, cuBLAS and cuDNN workspaces take 0.5 to 2 GiB depending on the algorithm selected, and the caching allocator fragments 5 to 15% under variable sequence shapes. A workable planning rule is to budget about 90% of the nvidia-smi total as the design ceiling.

Does a 7B model fit on a single 80 GiB GPU for fine-tuning?

Full fine-tuning with Adam does not. Mixed precision needs bf16 weights at 14 GB, gradients at 14 GB, fp32 master weights at 28 GB, and two Adam moments at 28 GB each — 112 GB in total, well past the roughly 72 GiB design ceiling. Switching to 8-bit Adam drops the optimizer state to 7 GB each, giving about 70 GB or 65 GiB, which fits with roughly 7 GiB left for activations. That is enough only for short sequences and modest batch sizes; sharded optimizers are the usual answer when it is not.

VRAM GB to GiB Converter — What Your GPU Actually Allocates

VRAM GB → GiB / MiB

An 80 GB A100 reports 81,920 MiB in nvidia-smi — about 7% more bytes than the label implies, because GPU vendors have always meant GiB when they print GB. The capacity that disappears before your first tensor is not a unit trick: it is ECC rows, the CUDA context, library workspace and allocator fragmentation. Get that arithmetic right and the 3 AM OOM stops being a mystery.

Bidirectional GB and GiB Converter

Converts a decimal-GB capacity into the binary GiB and MiB that filesystems, allocators and nvidia-smi report. Mind the direction: drive and RAM labels are decimal, but GPU VRAM labels are already binary — an 80 GB card is 80 GiB, not 74.5 GiB. The reference table below shows which is which.

Quoted Capacity (decimal GB)

Binary Equivalent (GiB)

Common GPU Reference Table

What the vendor prints, what nvidia-smi actually reports, and the same capacity expressed in decimal GB. Note the direction of the gap: every one of these delivers more bytes than the label implies, because the label is binary.

GPU	Label	nvidia-smi	Actual (GiB)	In decimal GB
T4	16 GB	15,360 MiB	15.00	16.11
A10	24 GB	23,028 MiB	22.49	24.15
L4	24 GB	23,034 MiB	22.49	24.15
RTX 4090	24 GB	24,564 MiB	23.99	25.76
A100 SXM4	40 GB	40,960 MiB	40.00	42.95
L40S	48 GB	46,068 MiB	44.99	48.31
H100 SXM	80 GB	81,559 MiB	79.65	85.52
A100 SXM4	80 GB	81,920 MiB	80.00	85.90
H200 SXM	141 GB	143,771 MiB	140.40	150.75

Why GB ≠ GiB Still Matters — Just Not Where You Expect

The decimal-versus-binary gap is real: 1 GB is 10⁹ bytes, 1 GiB is 2³⁰ bytes (1,073,741,824), and the difference is 6.87%. It wrecks capacity plans for drives and system RAM on a weekly basis. But the version of this story that circulates in ML circles — "your 80 GB GPU only gives you 74.5 GiB" — is backwards, and it is worth knowing why before you size a cluster against it.

GPU vendors print GB and mean GiB

nvidia-smi is the ground truth, and it reports in MiB. An A100 80GB reports exactly 81,920 MiB, which is 80 GiB — not 74.5. A T4 labelled 16 GB reports 15,360 MiB, exactly 15 GiB. An H200 labelled 141 GB reports 143,771 MiB, or 140.40 GiB. In every case the label is the binary figure, and in several cases the card physically carries more bytes than the decimal reading of its own label.

So the 0.9313 multiplier is the wrong tool here. Applied to a GPU label it understates capacity by about 7%, which is not a harmless rounding error at fleet scale: 64 cards at 80 GiB hold 5,120 GiB of real memory, and the same fleet written off as "64 × 74.5 = 4,768 GiB" understates it by 352 GiB — more than four cards' worth. Use the multiplier on drives, RAM sticks and data-sheet figures; read the GPU number straight off nvidia-smi.

Where the memory actually goes before your first tensor

Capacity is not the same as allocatable memory, and the distance between them has nothing to do with units. Four things take a cut:

ECC and reserved rows — an H100 80GB reports 81,559 MiB where the A100 reports 81,920, so 361 MiB never reaches the allocator.

CUDA context — 300 to 600 MiB per process, consumed the moment a context is created.

Library workspace — cuBLAS and cuDNN reserve 0.5 to 2 GiB depending on which algorithms are selected at runtime, and that selection shifts with input shape.

Caching-allocator fragmentation — 5 to 15% under variable sequence lengths, and the usual reason a run that fit yesterday does not fit today.

On a single-process job the first three are noise against 80 GiB. On a node running four MPS processes they are not: 4 × 500 MiB of context plus a gigabyte of workspace is roughly 3 GiB gone before any weights are loaded. The planning rule that survives contact with all of this is to design against 90% of the nvidia-smi total — 72 GiB on an 80 GiB card — and treat the remainder as the buffer that stops fragmentation from killing an overnight run.

Worked example: does 7B fine-tuning fit in 80 GiB?

Mixed-precision fine-tuning with Adam holds five things in memory at once. For a 7B model: bf16 weights 14 GB, bf16 gradients 14 GB, fp32 master weights 28 GB, and the two Adam moments at 28 GB each — 112 GB in total, which does not fit 80 GiB by a wide margin. That arithmetic decides the question, and no unit conversion changes it.

Swap in 8-bit Adam and the optimizer state falls to 7 GB per moment: 14 + 14 + 28 + 7 + 7 = 70 GB, or 65 GiB. Against the 72 GiB design ceiling that leaves roughly 7 GiB for activations — workable for short sequences and modest batch sizes, and not workable for long-context training, where the KV cache alone claims several gigabytes per sequence. When it does not fit, the answer is a sharded optimizer (ZeRO stage 2 or FSDP), which partitions exactly the three components that dominate that 112 GB — not a bigger card.