Swift Bonsai 2

Quantizations

Quantization	File	Download size
1-bit / PTQ1_0	Swift-Bonsai-2-PTQ1_0.gguf	5.947 GB
2-bit / PQ2_0	Swift-Bonsai-2-PQ2_0.gguf	7.206 GB

Each file is a complete model with the Swift correction merged into the ternary weights. No adapter or runtime patch is needed. PTQ1_0 is the earlier release; PQ2_0 contains updated merged weights.

Evaluation

GPQA-Diamond and C-Eval compare the base model with the historical Swift runtime correction. IFBench and AIME 2025 compare the current base and Swift PQ2_0 files. Scores aggregate three GPQA-Diamond runs and five runs for each other benchmark. The historical GPQA and C-Eval evaluations have not been repeated on the current merged files.

Base Bonsai 2 vs Swift · historical GPQA and C-Eval; current PQ2_0 IFBench and AIME
Benchmark	Score		Mean tokens			Median tokens
Benchmark	Base	Swift	Base	Swift	Reduction	Reduction
General reasoning
GPQA-Diamond	84.18%	84.34%	20,066	16,245	↓ 19.0%	↓ 39.8%
C-Eval	81.37%	81.47%	2,123	1,732	↓ 18.4%	↓ 7.1%
IFBench loose	82.13%	82.60%	8,670	8,519	↓ 1.7%	↑ 0.4%
Mathematics
AIME 2025	92.00%	93.33%	22,158	23,101	↑ 4.3%	↑ 5.1%

Token statistics measure thinking tokens, except AIME, where they measure the full completion. IFBench uses official loose scoring. Up arrows mean the current 2-bit model used more tokens. Token changes are not direct measurements of latency or cost.

Benchmark methodology and reproduction settings

Sampling

Temperature 1.0, top-p 0.95, top-k 20, min-p 0, repetition penalty 1, presence penalty 0, and no additional inference-time logit penalty.

Benchmark configurations

Benchmark	Questions	Repetitions	Scored responses	Output cap
GPQA-Diamond	198	3	594	81,920
C-Eval validation, 5-shot	1,346	5	6,730	16,384
IFBench	300	5	1,500	81,920
AIME 2025	30	5	150	81,920

C-Eval covers the complete validation split, not the hidden test set. One duplicated prompt is cached per seed; all question IDs are scored and weighted separately.

The current AIME runs used identical corrected prompts and the archived Math-Verify 0.9.0 scorer, which checks the final answer and falls back to reasoning. It gives 92.00% base and 93.33% Swift. Final-answer-only scoring gives 90.67% for both.

IFBench uses official loose scoring for the current base and Swift PQ2_0 files. All 1,500 responses completed without request errors; prompt and sampler settings were spot-checked against the base run.

GPQA informed the choice of training settings and is not an untouched holdout. Named benchmark exclusions and exact-match checks were performed; a comprehensive near-overlap audit was not completed.

GPQA and C-Eval retain the original runtime-correction results; IFBench and AIME were refreshed on the current PQ2_0 file. PTQ1_0 remains the earlier merged release. Reproducibility can vary with runtime builds and hardware. Small score differences are not established improvements. Exact aggregates, truncation counts, and confidence intervals where computed are in benchmark_results.json.

Research status and limitations

This is an experimental research release. In our internal evaluations and practical testing, benchmark scores did not consistently translate into reliable general-purpose behaviour. Instruction following, tool use, and open-ended coding or agent tasks remain uneven. These observations concern the specific models, runtimes, and tests we used; they do not establish a general conclusion about ternary models.

We see ternary quantization as a promising direction for making larger models more accessible. Further advances in training, low-bit adaptation, and inference support may make the approach increasingly useful. This release shares a research result and its current limitations, rather than presenting a production-ready assistant.

Run Swift Bonsai 2

The merged GGUF files run on Prism ML's llama.cpp fork (tested at revision 1a07bfa5f) with no adapter, patch, or extra flag. Stock llama.cpp does not support these ternary tensor types. Linux prerequisites: Git, CMake, a C++ compiler, and the NVIDIA CUDA toolkit.

Build Prism's llama.cpp fork and run the 2-bit PQ2_0 model

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j
hf download ukisai/Swift-Bonsai-2-GGUF Swift-Bonsai-2-PQ2_0.gguf --local-dir .
./build/bin/llama-server -m Swift-Bonsai-2-PQ2_0.gguf --host 127.0.0.1 --port 8080 \
  -ngl 99 -c 32768 --flash-attn on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.0

Run the 1-bit PTQ1_0 model instead

hf download ukisai/Swift-Bonsai-2-GGUF Swift-Bonsai-2-PTQ1_0.gguf --local-dir .
./build/bin/llama-server -m Swift-Bonsai-2-PTQ1_0.gguf --host 127.0.0.1 --port 8080 \
  -ngl 99 -c 32768 --flash-attn on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.0

OpenAI-compatible API

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "bonsai-2-swift",
    "messages": [{"role": "user", "content": "Explain the difference between correlation and causation."}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "min_p": 0.0,
    "max_tokens": 4096,
    "chat_template_kwargs": {"enable_thinking": true, "reasoning_effort": "xhigh"}
  }'

The server binds to 127.0.0.1:8080 with a 32,768-token context; use -c 98304 for longer evaluation caps if memory permits. Runtime memory includes model weights and context caches, so download size is not total VRAM usage. Custom integrations should preserve the shown model settings.

License and attribution

Created using Bonsai by Prism ML. This is an independent UkisAI release, not an official Prism ML release or endorsement. The model is distributed under Apache-2.0; the upstream notice and attributions are retained.

For the Qwen3.8-based Swift models, see Swift 1.5.