festr2/kimi-k3-full-mxfp4-kld-reference-32x2048

Kimi K3 full-MXFP4 KLD reference logits

This dataset contains the canonical full-vocabulary reference logits for quantization comparisons of Kimi K3. The source is the original full MXFP4 checkpoint served as W4A16 on TP16 with vLLM dev/gg-k3, SparkInfer, and InstantTensor.

Contents

• 32 independent 2048-token windows
• 65,504 scored next-token positions (32 * 2047)
• vocabulary size 163,840
• one [2047, 163840] F32 safetensors tensor per window
• tensor key: logits
• total logit payload: 42,928,712,256 bytes (39.98 GiB)
• suite token hash: a6856e1d0504fd00d13c67a5515c081f349088664d7ea0894dc4d15db2c7d209

The corpus is intentionally not a 512-stride sliding window. Its independent, non-overlapping windows are spread over three pinned sources:

Domain	Windows	Source
prose	16	Salesforce/wikitext, wikitext-2-raw-v1, test
code	8	openai/openai_humaneval, test
instruction	8	databricks/databricks-dolly-15k, train

Exact source revisions, token-stream construction, window starts, token IDs, and per-window hashes are in suite-manifest.json and tokens/. Always send the stored token IDs directly to /v1/completions; do not reconstruct them through a chat template.

Layout

ref/logits_000.safetensors ... ref/logits_031.safetensors
ref/manifest.json
tokens/window-000-prose.json ... tokens/window-031-instruction.json
suite-manifest.json
capture-mixed32-reference.json
repeat-noise-window000.json
tools/

ref/manifest.json is authoritative for every file's SHA-256, tensor shape, source checkpoint identity, and exact runtime revisions.

Candidate capture

Apply tools/vllm-kld-capture.patch to the exact vLLM commit recorded in the manifest. Start the candidate with a fresh capture directory:

export VLLM_KLD_CAPTURE_DIR=/mnt/luke/kld/candidate/capture-chunks
export VLLM_TRITON_MLA_STATIC_KV_SPLITS=8
export VLLM_DCP_INDEXER_SHARDS=0

python -m vllm.entrypoints.cli.main serve /path/to/candidate \
  --served-model-name Kimi-K3 \
  --trust-remote-code \
  --host 0.0.0.0 --port 8000 \
  --tensor-parallel-size 16 \
  --max-model-len 4096 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 256 \
  --gpu-memory-utilization 0.982 \
  --compilation-config '{"mode":0,"cudagraph_mode":"PIECEWISE","cudagraph_capture_sizes":[1]}' \
  --load-format instanttensor

Capture the exact suite and merge the chunks:

python tools/capture-kimi-k3-kld-suite.py \
  --suite-dir . \
  --capture-dir /mnt/luke/kld/candidate/capture-chunks \
  --run-name candidate

python tools/finalize-kimi-k3-kld-suite.py \
  --suite-dir . \
  --capture-dir /mnt/luke/kld/candidate/capture-chunks \
  --run-name candidate \
  --output-dir /mnt/luke/kld/candidate/ref \
  --expected-vocab 163840

The hook is inactive unless VLLM_KLD_CAPTURE_DIR is set, runs only on global rank 0, captures raw logits before log-softmax, transfers low-precision logits to CPU before widening to F32, and writes 256-row chunks to avoid GPU OOM.

Compare against the reference

Download the reference and compare the same numbered files:

hf download festr2/kimi-k3-full-mxfp4-kld-reference-32x2048 \
  --repo-type dataset \
  --local-dir /mnt/luke/kld/kimi-k3-reference

python tools/compare-kimi-k3-kld-suite.py \
  --reference-dir /mnt/luke/kld/kimi-k3-reference/ref \
  --candidate-dir /mnt/luke/kld/candidate/ref \
  --suite-manifest /mnt/luke/kld/kimi-k3-reference/suite-manifest.json \
  --output /mnt/luke/kld/candidate/kld-vs-full-mxfp4.json

The primary metric is KL(reference || candidate). The comparator also reports per-token mean/median/P95/P99/max, Jensen-Shannon divergence, top-1 agreement, per-domain means, and a 10,000-sample 95% bootstrap confidence interval clustered by window. The window is the sampling unit; individual tokens must not be treated as independent observations.

For quick iteration add --stop-window 8 to capture, finalize, and compare so only windows 0-7 are required. Run all 32 windows for candidate selection. Capture the winner a second time if candidates differ by only a few thousandths.

Repeat noise and validation

Two earlier captures of the same model and first window measured KL=0.0035015. This 32-window capture's first window versus the earlier canonical run measured:

• KL(earlier || this) = 0.00294743
• JS = 0.000725872
• top-1 agreement = 99.0718%

This is within the observed runtime/kernel nondeterminism. Differences around 0.004 or below should not be treated as real without repeated captures.

All 32 files were read back with safetensors, their key/dtype/shape checked, and their complete contents hashed. The hashes are in ref/manifest.json.