GPU GUIDE · NVIDIA

Best AI models for the
NVIDIA L40

The NVIDIA L40 has 48.0GB of VRAM. Below are the top 30 open-source AI models that fit, ranked by composite benchmark score. Each row shows the best quantization that fits your hardware.

VRAM
48.0GB
Brand
nvidia
Models that fit
30
Generation
Data Center
Open full results filter →

Top 30 models for the NVIDIA L40

01

DeepSeek V4 Flash

37.0B
77.5
Best fit: Q8_0 · 40.3GBCan I run it? →
02

Qwen 3.6 27B

27.0B
76.4
Best fit: Q8_0 · 29.7GBCan I run it? →
03

Qwen 3.6 35B A3B

35.0B
72.5
Best fit: Q8_0 · 38.2GBCan I run it? →
04

Qwen3.5-27B

27.8B
70.1
Best fit: Q8_0 · 30.5GBCan I run it? →
05

Gemma 4 31B (free)

31.0B
65.3
Best fit: Q8_0 · 33.9GBCan I run it? →
06

Qwen 3.5 35B A3B

35.0B
61.9
Best fit: Q8_0 · 38.2GBCan I run it? →
07

Qwen 3.5 9B

9.0B
54.0
Best fit: fp16 · 19.0GBCan I run it? →
08

gemma 4 31B

32.7B
53.8
Best fit: Q8_0 · 35.7GBCan I run it? →
09

Gemma 4 26B A4B (free)

26.5B
52.0
Best fit: Q8_0 · 29.2GBCan I run it? →
10

Qwen 3.5 4B

4.0B
45.1
Best fit: fp16 · 9.0GBCan I run it? →
11

Qwen3 Next 80B A3B Thinking

81.3B
44.5
Best fit: Q4_K_S · 47.5GBCan I run it? →
12

GPT-OSS 20B

20.0B
40.8
Best fit: fp16 · 41.0GBCan I run it? →
13

Nemotron 3 Nano 30B A3B (free)

31.6B
40.5
Best fit: Q8_0 · 34.6GBCan I run it? →
14

Qwen3 30B A3B Thinking 2507

30.5B
37.4
Best fit: Q8_0 · 33.4GBCan I run it? →
15

Devstral 2

24.0B
36.7
Best fit: Q8_0 · 26.5GBCan I run it? →
16

Nemotron 3 Nano Omni 30B A3B Reasoning BF16

30.0B
35.7
Best fit: Q8_0 · 32.9GBCan I run it? →
17

Qwen3 Coder 30B A3B Instruct

30.5B
33.3
Best fit: Q8_0 · 33.4GBCan I run it? →
18

QwQ 32B

32.0B
32.9
Best fit: Q8_0 · 35.0GBCan I run it? →
19

Qwen3 VL 30B A3B Thinking

31.1B
32.8
Best fit: Q8_0 · 34.0GBCan I run it? →
20

Qwen3 Next 80B A3B Instruct (free)

81.3B
33.5
Best fit: Q4_K_S · 47.5GBCan I run it? →
21

Devstral Small 2 24B Instruct 2512

24.0B
32.4
Best fit: Q8_0 · 26.5GBCan I run it? →
22

Devstral Small 2

7.0B
31.7
Best fit: fp16 · 15.0GBCan I run it? →
23

gemma 4 E4B it

8.0B
31.3
Best fit: f32 · 33.0GBCan I run it? →
24

Llama 3.3 Nemotron Super 49B V1.5

49.9B
31.1
Best fit: Q6_K · 42.1GBCan I run it? →
25

Mistral Medium 3.5

70.0B
31.3
Best fit: Q4_K_M · 43.4GBCan I run it? →
26

Qwen3 4B Thinking 2507

4.0B
30.3
Best fit: fp16 · 9.0GBCan I run it? →
27

Qwen3 VL 32B Instruct

33.4B
28.7
Best fit: Q8_0 · 36.5GBCan I run it? →
28

DeepSeek R1 Distill Qwen 32B

32.8B
28.6
Best fit: Q8_0 · 35.9GBCan I run it? →
29

Qwen3 VL 8B Thinking

8.8B
27.8
Best fit: f32 · 36.2GBCan I run it? →
30

Qwen3 32B

32.8B
27.5
Best fit: Q8_0 · 35.9GBCan I run it? →
Advertisement

FAQ — running AI on the NVIDIA L40

How many AI models can the NVIDIA L40 run?

With 48.0GB of VRAM, the NVIDIA L40 can run 30+ open-source models from our database, including DeepSeek V4 Flash, Qwen 3.6 27B, Qwen 3.6 35B A3B.

What's the largest LLM I can run on a NVIDIA L40?

The biggest model that fits is approximately 81.3B. Larger models would need to be quantized further or won't fit at all.

Is 48.0GB of VRAM enough for local AI?

Yes — 48.0GB comfortably runs most popular open-source models including 30B-class LLMs at Q4_K_M.