We secured a $400M debt facility with Upper90 to scale inference compute.Read
benchmarksvision language modelllama 4qwenmultimodalinferencevlm

Vision-Language Models: Llama 4 Scout vs Qwen2-VL vs InternVL at Inference

General Compute·

Running a vision-language model in production is more complicated than running a text-only LLM of the same parameter count, because the latency budget has more moving parts. A 7B text model and a 7B VLM are not the same thing to serve. The VLM has an additional vision encoder that processes image tokens before the LLM ever sees them, an image-to-text projection step, and often a much longer effective context length due to the image patches being serialized into the token sequence. All of that adds time before the first output token appears.

This post compares three prominent open-source VLM families -- Llama 4 Scout, Qwen2-VL, and InternVL2 -- on both quality and inference speed, with notes on what drives the performance differences and which architecture makes sense for which workload.

How VLM Inference Differs From Text-Only LLMs

To understand the speed numbers, it helps to understand the pipeline. A typical VLM inference call has three distinct stages:

Vision encoder. The image is split into patches and processed by a vision transformer (ViT). This produces a sequence of visual feature vectors, one per patch. This stage does not depend on your text prompt and is entirely parallelizable, but it adds latency before the first autoregressive token.

Projection. The ViT output vectors are mapped into the LLM's embedding space using a linear layer or a small MLP. This step is fast.

LLM prefill and decode. The projected image tokens are concatenated with the text tokens and run through the LLM. The prefill processes the combined sequence, and then decoding proceeds token by token as usual.

The key variable is how many image tokens get injected into the LLM. If a model converts a 1024x1024 image into 1024 tokens, that adds 1024 tokens to every prefill. At 4,096 tokens of text, you're now doing a 5,120-token prefill. That's materially slower TTFT compared to a text-only 4,096-token prefill, and the difference scales with image resolution.

The Three Model Families

Llama 4 Scout

Llama 4 Scout is Meta's mid-tier multimodal model from the Llama 4 family (released April 2025). It uses a mixture-of-experts architecture with 17B active parameters out of 109B total, 16 experts per layer. The vision encoder produces image tokens that are interleaved with text tokens using early fusion -- the LLM sees image and text together from layer one, rather than having a separate vision-language connector grafted on.

Scout's 10 million token native context is the headline number, but for most single-image inference tasks you won't get close to that limit. The more relevant number for typical use is how many tokens a standard image consumes. Because Scout uses early fusion with a single high-resolution image encoder, it tends to produce fewer image tokens per image than tiling-based approaches, which keeps prefill times manageable.

The MoE architecture means Scout's active compute during decode is much closer to a 17B dense model than a 109B dense model, which matters a lot for throughput. You can fit Scout on a single H100 80GB in FP8 precision.

Qwen2-VL

Qwen2-VL (from Alibaba, the second-generation of Qwen-VL) introduced a technique called Naive Dynamic Resolution, which lets the model handle images at their native resolution rather than forcing everything into a fixed size. Images are divided into 14x14 pixel patches, and the number of tokens scales with the image resolution: a small 448x448 image produces 1,024 tokens; a larger 1792x1792 image produces up to 16,384 tokens.

The key innovation for position encoding is 2D-RoPE: instead of treating image tokens as a flat sequence, the model uses 2D rotary position embeddings that encode both the x and y position of each patch. This lets the model reason about spatial relationships more naturally than 1D position encoding applied to flattened patches.

Qwen2-VL comes in three sizes: 2B, 7B, and 72B. The 7B model has become a popular baseline for production deployments because it hits a reasonable quality/speed tradeoff and fits in 16GB VRAM in FP16. The 72B model is competitive with much larger models on document-heavy tasks.

For inference, the variable token count from dynamic resolution is the main thing to plan around. A high-resolution screenshot (2560x1440) can produce tens of thousands of image tokens. If your use case involves large documents or high-res images, your batch sizing and KV cache planning need to account for this.

InternVL2

InternVL2 (from Shanghai AI Lab) takes a different architectural approach. Rather than a small ViT proportional to the LLM size, InternVL2 uses InternViT-6B as the vision encoder across all variants in the family. Even the 8B LLM variant uses a 6B ViT. This is an unusually large vision encoder, and it shows up in benchmark scores for fine-grained visual tasks.

The model family spans a wide range: 1B, 2B, 4B, 8B, 14B, 26B, 40B, and 76B variants. The language backbone is either InternLM2 or Qwen2.5 depending on the variant. For inference purposes, the 8B and 26B models are the most commonly deployed sizes.

InternVL2 uses a dynamic high-resolution strategy with tiling: large images are divided into tiles, each processed independently by the ViT, and then concatenated. This means the effective number of image tokens can be much higher than models that process the image as a single crop. A 4096x4096 image with 4 tiles could produce 4,000+ image tokens. InternVL2.5 (the updated version) includes improvements to reduce this token count through pixel shuffle downsampling.

Quality Benchmarks

Here's a comparison across the main multimodal benchmarks. Numbers are approximate, sourced from published model cards and academic papers:

| Model | MMMU | MMBench | DocVQA | MathVista | |-------|------|---------|--------|-----------| | InternVL2-8B | ~51 | ~81 | ~91 | ~58 | | Qwen2-VL-7B | ~54 | ~83 | ~95 | ~58 | | Llama 4 Scout | ~70 | ~88 | ~89 | ~66 | | Qwen2-VL-72B | ~65 | ~89 | ~96 | ~67 | | InternVL2-76B | ~58 | ~89 | ~94 | ~65 |

A few things stand out:

Llama 4 Scout punches well above its weight on general reasoning (MMMU, MathVista). MMMU is a college-level multi-discipline benchmark with problems in medicine, law, chemistry, and other subjects. Scout's 70% score is notably higher than Qwen2-VL-72B's 65% and InternVL2-76B's 58%, despite Scout having fewer active parameters. The early fusion architecture likely helps here by letting the LLM layers reason directly over visual and textual information together.

Qwen2-VL leads on document understanding. DocVQA measures visual question answering over scanned documents, invoices, and forms. Qwen2-VL-7B at 95% and 72B at 96% outperform Scout's 89%, reflecting the benefit of dynamic resolution and the 2D-RoPE encoding for spatial document structure.

InternVL2 scales well across the family. The 8B model at 91% DocVQA and 81% MMBench is a strong result for its size. If you need a small, deployable model with solid performance across a range of tasks, InternVL2-8B is a reasonable choice.

Inference Speed: Image Tokens as the Primary Driver

Raw parameter count is a weak predictor of VLM inference speed. The number of image tokens injected into the LLM matters more for TTFT, and the active parameter count matters more for decode throughput.

Here's a rough token count comparison for a standard 1024x1024 input image:

| Model | Approx. image tokens (1024x1024) | Notes | |-------|----------------------------------|-------| | Llama 4 Scout | ~1,600 | Single encoder, no tiling | | Qwen2-VL-7B | ~4,096 | Dynamic resolution, scales with image size | | InternVL2-8B | ~2,048-4,096 | Depends on tiling factor |

These numbers shift significantly with image resolution. Qwen2-VL at full 2K resolution can generate 16K+ image tokens, which will dominate your prefill latency regardless of the text context length.

Practical throughput estimates (FP16, single A100 80GB, short text prompt + 1024x1024 image):

| Model | Approx. decode throughput | VRAM footprint | |-------|--------------------------|----------------| | Qwen2-VL-7B | ~45-60 tokens/sec | ~16GB | | InternVL2-8B | ~40-55 tokens/sec | ~18GB (ViT is large) | | Llama 4 Scout | ~25-35 tokens/sec | ~40GB (FP8) | | Qwen2-VL-72B | ~5-8 tokens/sec (single A100) | Needs multi-GPU | | InternVL2-76B | ~5-7 tokens/sec (single A100) | Needs multi-GPU |

InternVL2-8B's throughput is slightly lower than you'd expect from an 8B text model because of the 6B ViT running as a preprocessing step before every query. That ViT forward pass adds latency proportional to the image resolution, separate from the LLM decode cost.

Scout's decode throughput on a single GPU is modest compared to a 7B model because even at 17B active parameters, the KV cache and routing overhead add up. On multi-GPU setups where you can tensor-parallel across the active experts, Scout's effective throughput improves significantly.

Hardware Requirements by Use Case

Single GPU deployment, quality-first: Qwen2-VL-7B fits in 16GB VRAM with FP16 and delivers the strongest document/OCR performance in the 7B class. If your workload involves invoices, PDFs, or screenshots of text-heavy content, start here.

Single GPU deployment, reasoning-first: If your workload is more about understanding scenes, answering multi-step visual questions, or tasks that require reasoning across image and text (medical image analysis, chart interpretation), Llama 4 Scout on an H100 80GB is the stronger choice despite the higher VRAM requirement.

High-throughput batch processing: For workloads where you're processing large volumes of images with moderate resolution, InternVL2-8B's architecture -- ViT preprocessing in batch, then LLM decode -- maps well to batched GPU workflows. The ViT forward passes can be batched efficiently, and the LLM step is a standard 8B decode.

Multi-GPU production serving: Qwen2-VL-72B and InternVL2-76B both need multi-GPU setups. Qwen2-VL-72B tends to score higher on document tasks while InternVL2-76B is competitive on general VQA. For most teams, the choice here comes down to which benchmark set better represents your actual use case.

Choosing an Image Resolution Policy

One of the easiest ways to control inference cost with VLMs is to define explicit image resolution limits for your application, rather than passing images at their native resolution.

Qwen2-VL's dynamic resolution means a 4K image from a phone camera will generate roughly 16x the image tokens of a 1K image. For most tasks -- counting objects in a photo, describing a scene, answering questions about a diagram -- the quality difference between 1K and 4K input is small while the latency difference is large.

A sensible default for most applications:

  • Scale images down to at most 1024px on the longest edge before passing to the model
  • For document/OCR tasks, 1024-2048px on the longest edge is usually sufficient; beyond that the marginal token count yield diminishes relative to quality gain
  • For tasks involving very fine print or dense tables, allow up to 2048px and budget the latency accordingly

InternVL2's tiling strategy compounds this effect: a large image might be tiled into 4 or 9 patches, each of which is processed at full ViT resolution, multiplying your effective token count. Setting a maximum tile count (most implementations support this) is the practical lever to control this.

Serving VLMs With vLLM and SGLang

Both vLLM and SGLang support the major VLM families with native multimodal handling:

from openai import OpenAI
import base64

client = OpenAI(
    base_url="https://api.generalcompute.com/v1",
    api_key="YOUR_API_KEY"
)

# Read and encode the image
with open("image.jpg", "rb") as f:
    image_data = base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="qwen2-vl-7b",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{image_data}"
                    }
                },
                {
                    "type": "text",
                    "text": "What does this document show?"
                }
            ]
        }
    ]
)
print(response.choices[0].message.content)

All three model families work with the OpenAI vision API format (image_url content blocks). The main operational difference is that Qwen2-VL with dynamic resolution will produce variable token counts per image, so you'll want to monitor your actual token usage rather than estimating based on parameter count alone.

For batch workloads, pre-processing images to a fixed resolution before inference is simpler to reason about from a capacity planning standpoint. Dynamic resolution is most valuable for interactive, mixed-resolution workloads where you don't know the input size in advance.

Summary

If you're choosing between these three families:

  • Qwen2-VL-7B is the best default for document, OCR, and screenshot-heavy workloads. Its dynamic resolution handles diverse input formats well, and the 7B size fits in a single 16GB GPU.

  • Llama 4 Scout is the strongest single-model choice if your task requires reasoning over visual content: charts, diagrams, multi-step questions. The MoE architecture means it fits on a single H100 and delivers 70B-class reasoning at much lower decode latency.

  • InternVL2-8B is worth considering when you need a small, open-weight model with strong benchmark performance and a predictable deployment profile. The large vision encoder is its main differentiator and also its main cost driver.

For large-scale serving, General Compute supports all three families with the throughput infrastructure to handle variable-length multimodal contexts without batching headaches. If you want to run comparisons before committing to a model, the API is the fastest way to test quality on your actual data before spinning up dedicated GPU capacity.

ModeHumanAgent