Instructions to use PelaAI/KnowLine-4B-Gen4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PelaAI/KnowLine-4B-Gen4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="PelaAI/KnowLine-4B-Gen4")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("PelaAI/KnowLine-4B-Gen4") model = AutoModelForMultimodalLM.from_pretrained("PelaAI/KnowLine-4B-Gen4", device_map="auto") - Notebooks
- Google Colab
- Kaggle
KnowLine-4B-Gen4
The fourth release of PelaAI's 4B System One decision model.
Inference guide Β· δΈζθ―΄ζ Β· weights Apache-2.0 Β· previous version: KnowLine-4B-Gen3
Highlights:
- Decision Index 0.3 public suite: 64.90 (self-run), 1.79 points above Gen3 (63.11). For comparison, on the board Perplexity Decider v1.1 (27B) scores 62.25 and Jev 1.13 scores 57.96, and the best model of 5B or smaller has a public score of 50.82; see Comparison.
- Calibrated probabilities: a temperature, fitted on held-out data, is folded into the weights. Computed with the board's method, ECE is about 0.04-0.05 (board median: 0.084) and the Brier score is about 0.29-0.31, roughly 2nd-3rd among the 114 models on the board; see Calibration.
- Other public benchmarks: JevBench / JevBench-hard 86.1 / 72.1 (Gen3: 86.6 / 72.1), C-Eval 78.7 (79.2).
KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev.
What it does
You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:
- one forward pass per question, with no text generation;
- the output is the probability of each option's label token;
- existing Jev clients only need to change the base URL, and users of earlier versions only need to change the model name, because the interface and serving settings are unchanged.
| Base model | Qwen/Qwen3.5-4B (Apache-2.0) |
| Training | LoRA SFT (rank 32, alpha 64, language model only), merged into bf16 weights |
| Release format | bf16 weights, with a readout temperature T = 1.32 folded into the final norm weight (see Calibration). The vision tower and MTP head are unchanged from the base model; the config, tokenizer and chat template are identical to those of Gen1-Gen3. |
| Languages | English, Simplified Chinese, Traditional Chinese |
Quickstart
pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-4B-Gen4 0 8080 # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
"model": "m",
"state": "Customer: my order arrived broken, I want my money back.",
"questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
"tone": {"type": "choice", "instructions": "Customer tone?",
"criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
- Front end:
knowline_server.pyis a single file that depends only on transformers and requests. It is the same file as in Gen3. - Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.
Comparison
Scores on the Decision Index 0.3 public suite. The full 0.3 score also includes private tests that only the maintainers run (weighted 0.5 same-skill and 0.3 new-domain); our full score is not available yet.
| model | size | DI 0.3 public | DI 0.3 full | source |
|---|---|---|---|---|
| KnowLine-4B-Gen4 | 4B | 64.90 | not yet scored | self-run, official kit |
| KnowLine-4B-Gen3 | 4B | 63.11 | not yet scored | self-run, official kit |
| KnowLine-4B-Gen2 | 4B | 62.54 | not yet scored | self-run, official kit |
| KnowLine-4B-Gen1 | 4B | 60.47 | not yet scored | self-run, official kit |
| Perplexity Decider v1.1 | 27B | 62.25 | 62.75 | board |
| Clef | 27B | 61.71 | 53.08 | board |
| Jev 1.13 | (API) | 57.96 | 60.11 | board |
| RSI-Jev v6.1-VL | 4B | 50.98 | not listed | self-reported |
| ezjev 4B s2 | 4B | 50.82 | 46.95 | board |
| jiwo 4B | 4B | 45.76 | 42.86 | board |
| Nox 4B | 4B | 44.21 | 44.95 | board |
Public-suite scores by area (DI 0.3):
| area | KnowLine-4B-Gen4 | KnowLine-4B-Gen3 | Perplexity Decider v1.1 (27B) | Jev 1.13 |
|---|---|---|---|---|
| Knowledge & reasoning | 44.6 | 43.9 | 52.0 | 53.9 |
| Language | 65.5 | 64.8 | 67.5 | 59.2 |
| Retrieval & routing | 72.5 | 70.9 | 61.3 | 55.4 |
| Tools & agents | 87.6 | 86.6 | 78.9 | 75.1 |
| Arts & taste | 59.2 | 49.8 | 47.0 | 39.1 |
New versions will follow.
| model | date | DI 0.3 public | DI 0.2.1 | notes |
|---|---|---|---|---|
| KnowLine-4B-Gen4 (this model) | 2026-10-10 | 64.90 | β | fourth release |
| KnowLine-4B-Gen3 | 2026-10-09 | 63.11 | β | third release |
| KnowLine-4B-Gen2 | 2026-10-08 | 62.54 | β | second release |
| KnowLine-4B-Gen1 | 2026-10-07 | 60.47 | 60.92 | first release |
Since Gen2, we run only Decision Index 0.3, not 0.2.1.
Evaluation
All results are self-run and have not been verified by a third party.
Other public benchmarks
| suite | Gen4 | Gen3 |
|---|---|---|
| C-Eval (4 categories, macro) | 78.7 | 79.2 |
| JevBench / JevBench-hard | 86.1 / 72.1 | 86.6 / 72.1 |
Web operation (Mind2Web official test splits)
| split | Gen4 element selection | Gen4 operation (balanced) | Gen3 element selection | Gen3 operation (balanced) |
|---|---|---|---|---|
| test_task (Cross-Task) | 92.9 | 97.0 | 92.7 | 97.0 |
| test_website (Cross-Website) | 91.9 | 95.7 | 91.3 | 97.3 |
| test_domain (Cross-Domain) | 91.7 | 97.2 | 91.8 | 97.2 |
- About the same as Gen3. The lower operation score on test_website comes from 3 more misses among its 72 SELECT steps; the differences are within noise.
- So far, we use this only to explore the model for RPA-style automation and to check that it generalises to some degree.
- The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard.
Calibration
Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.
| metric | Gen4 | Gen3 | board median (114 models) | Jev 1.13 |
|---|---|---|---|---|
| ECE (lower is better) | 0.04-0.05 | 0.07-0.08 | 0.084 | 0.074 |
| Brier score (lower is better) | 0.29-0.31 | 0.31-0.33 | 0.49 | 0.36 |
| Confidence β₯95% but wrong | 1.7% | 2.7% | 1.3% | 2.1% |
| Mean confidence / accuracy | 0.82 / 0.77 | 0.84 / 0.76 | 0.81 / 0.74 |
- Temperature in the weights: the final norm weight includes a readout temperature T = 1.32, so every logit is divided by 1.32 before the softmax over the option labels. The top-ranked option does not change, so neither do the Decision Index scores; the probabilities are just less extreme. T was fitted on held-out data.
- Why ranges: the board does not publish which 32 benchmarks it uses. We checked our computation against three board models with public results; our ECE was within about 0.015 of the board's values, so we report ranges over the plausible benchmark sets.
- Still slightly overconfident: mean confidence is about 3-5 points above accuracy. Most of the gap comes from hard reasoning (HLE, CRUXEval, MuSR) and New Yorker caption matching.
- Recommendation: if your use depends on probability thresholds, check calibration on your own data.
Disclosures
- Decontamination:
- All training data of this round, new and replayed, was checked with a revised tool against every Decision Index 0.3 row and the public benchmarks we report (Intern-Decision, S1MB, JevBench, MMDM).
- Rows that nearly duplicate an evaluation item, repeat its question or claim, or contain its whole state were removed (1,909 rows). Rows that only share a source passage with an evaluation item, such as the same Wikipedia paragraph or tool description, were kept.
- No model outputs as labels: no output of Jev or any other decision model was used as a training label.
- Teacher-labelled synthetic data: LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all of them were checked by a human.
- Public suite: the Decision Index public suite is public; the private tests may show a lower score.
Limitations
- Knowledge-heavy reasoning: weaker than larger models. The knowledge area score is 44.6 (DI 0.3); MMLU-Pro barely moved (51.6), and HLE remains at chance level.
- Math: answers are given without step-by-step reasoning. The rebuilt 0.3 GSM8K scores 61.6, about the same as Gen3.
- Prompt injection: an instruction planted in the state can change the answer. Keep untrusted text clearly delimited.
- Images: there was no image training; the serving path accepts images, but image use is zero-shot, so check it on your own data before relying on it.
- Private tests: the 0.3 private tests may score lower than the public suite.
Citation
@misc{knowline4bgen4,
title = {KnowLine-4B-Gen4: a 4B decision model},
author = {PelaAI},
year = {2026},
url = {https://huggingface.co/PelaAI/KnowLine-4B-Gen4}
}
License
- Weights: Apache-2.0, the same as the base model.
- Code:
knowline_server.pyis MIT (see the file header).
- Downloads last month
- 86