Skip to content
MUSEBOARD

GPU / L40S

NVIDIA L40S 48GB

Universal datacenter GPU for inference and fine-tuning

Plan on this GPU Compare All GPUs
Spec sheet
NVIDIA
Vendor
NVIDIA
Architecture
Ada Lovelace
Memory
48 GB GDDR6 ECC
Bandwidth
864 GB/s
Power
350 W
Class
Datacenter
Form factor
PCIe card
Interconnect
PCIe 4.0
Released
2023
Workloads
Inference · Fine-tuning
Model fit / Inference
8K ctx · batch 1 · ≥10% headroom
GPUs required per model and precision for NVIDIA L40S 48GB
ModelFP16INT8INT4
Llama 3.2 1B1×3.0 GB1×1.6 GB1×0.95 GB
Llama 3.2 3B1×8.0 GB1×4.5 GB1×2.7 GB
Llama 3.1 8B1×18.7 GB1×9.9 GB1×5.5 GB
Llama 3.1 70B4×158.0 GB2×80.3 GB1×41.5 GB
Llama 3.3 70B4×158.0 GB2×80.3 GB1×41.5 GB
Llama 3.1 405B24×897.2 GB16×450.7 GB8×227.5 GB
Qwen2.5 1.5B1×3.6 GB1×1.9 GB1×1.1 GB
Qwen2.5 3B1×7.1 GB1×3.7 GB1×2.0 GB
Qwen2.5 7B1×17.2 GB1×8.9 GB1×4.7 GB
Qwen2.5 14B1×34.0 GB1×17.8 GB1×9.7 GB
Qwen2.5 32B2×73.6 GB1×37.9 GB1×20.0 GB
Qwen2.5 72B4×162.6 GB2×82.7 GB1×42.7 GB
Mistral 7B1×17.0 GB1×9.0 GB1×5.1 GB
Mistral NeMo 12B1×28.2 GB1×14.8 GB1×8.1 GB
Mistral Small 24B2×53.3 GB1×27.3 GB1×14.3 GB
Mixtral 8x7B4×103.8 GB2×52.4 GB1×26.8 GB
Mixtral 8x22B8×312.1 GB4×157.0 GB2×79.4 GB
Gemma 2 9B1×23.1 GB1×13.0 GB1×7.9 GB
Gemma 2 27B2×62.9 GB1×33.0 GB1×18.0 GB
Phi-3 Mini 3.8B1×11.6 GB1×7.4 GB1×5.3 GB
Phi-3 Medium 14B1×32.5 GB1×17.1 GB1×9.4 GB
DeepSeek R1 Distill Qwen 32B2×74.3 GB1×38.2 GB1×20.2 GB
DeepSeek R1 Distill Llama 70B4×158.0 GB2×80.3 GB1×41.5 GB
DeepSeek V3 671B40×1,477 GB24×738.7 GB16×369.6 GB