Instructions to use PelaAI/KnowLine-4B-Gen2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PelaAI/KnowLine-4B-Gen2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="PelaAI/KnowLine-4B-Gen2")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("PelaAI/KnowLine-4B-Gen2") model = AutoModelForMultimodalLM.from_pretrained("PelaAI/KnowLine-4B-Gen2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
KnowLine-4B-Gen2
The second release of PelaAI's 4B System One decision model.
Inference guide Β· δΈζθ―΄ζ Β· weights Apache-2.0 Β· previous version: KnowLine-4B-Gen1
Newer version: KnowLine-4B-Gen3 is out, with a Decision Index 0.3 public score of 63.11.
Highlights:
- Decision Index 0.3, public suite: 62.54 (self-run), 2.07 above Gen1 (60.47). For comparison: Jev 1.13 scores 57.96 (board), and the best β€5B model on the board has a public score of 50.82; see Comparison.
- Better calibrated: computed the board's way, ECE is about 0.06-0.07 (Gen1 about 0.08-0.09, board median 0.084), and the Brier score is about 0.31-0.33, roughly 3rd of 113 models on the board; see Calibration.
KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev.
What it does
You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:
- one forward pass per question, with no generated text;
- the output is the probability of each option's label token;
- existing Jev clients only need a new base URL; Gen1 users only change the model name, since the interface and serving settings are the same.
| Base model | Qwen/Qwen3.5-4B (Apache-2.0) |
| Training | LoRA SFT (rank 32, alpha 64, language model only), merged into bf16 weights. |
| Release format | bf16 weights. The vision tower and MTP head are the base model's; config, tokenizer and chat template are identical to Gen1. |
| Languages | English, Simplified Chinese, Traditional Chinese |
Quickstart
pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-4B-Gen2 0 8080 # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
"model": "m",
"state": "Customer: my order arrived broken, I want my money back.",
"questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
"tone": {"type": "choice", "instructions": "Customer tone?",
"criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
- Without SGLang:
python knowline_server.py --model PelaAI/KnowLine-4B-Gen2 --backend hf --port 8080uses transformers only. It is slower and runs bf16. - Front end:
knowline_server.pyis a single file and needs only transformers and requests. It is the same file as in Gen1. - Full settings: the exact settings of our Decision Index run are in INFERENCE.md.
Comparison
Decision Index 0.3, public suite. The full 0.3 score adds private tests that only the maintainers run (0.5 same-skill, 0.3 new-domain); ours is not available yet.
| model | size | DI 0.3 public | DI 0.3 full | source |
|---|---|---|---|---|
| KnowLine-4B-Gen2 | 4B | 62.54 | not yet scored | self-run, official kit |
| KnowLine-4B-Gen1 | 4B | 60.47 | not yet scored | self-run, official kit |
| Clef | 27B | 61.71 | 53.08 | board |
| Jev 1.13 | (API) | 57.96 | 60.11 | board |
| RSI-Jev v6.1-VL | 4B | 50.98 | not listed | self-reported |
| ezjev 4B s2 | 4B | 50.82 | 46.95 | board |
| jiwo 4B | 4B | 45.76 | 42.86 | board |
| Nox 4B | 4B | 44.21 | 44.95 | board |
Public-suite scores by area:
| area | KnowLine-4B-Gen2 (0.3 public) | KnowLine-4B-Gen1 (0.3 public) | Jev 1.13 (0.2.1) |
|---|---|---|---|
| Knowledge & reasoning | 42.7 | 37.7 | 51.4 |
| Language | 63.3 | 60.8 | 62.0 |
| Retrieval & routing | 71.3 | 70.9 | 55.4 |
| Tools & agents | 86.6 | 86.8 | 75.1 |
| Arts & taste | 50.2 | 49.5 | 37.7 |
Releases
New versions will follow.
| model | date | DI 0.3 public | DI 0.2.1 | notes |
|---|---|---|---|---|
| KnowLine-4B-Gen4 | 2026-10-10 | 64.90 | β | fourth release |
| KnowLine-4B-Gen3 | 2026-10-09 | 63.11 | β | third release |
| KnowLine-4B-Gen2 (this model) | 2026-10-08 | 62.54 | β | second release |
| KnowLine-4B-Gen1 | 2026-10-07 | 60.47 | 60.92 | first release |
From Gen2 on we run Decision Index 0.3 only, not 0.2.1.
Evaluation
All results are self-run and not verified by a third party.
Other public benchmarks
| suite | Gen2 | Gen1 |
|---|---|---|
| C-Eval (4 categories, macro) | 78.6 | 77.9 |
Web operation (Mind2Web official test splits)
| split | Gen2 element selection | Gen2 operation (balanced) | Gen1 element selection | Gen1 operation (balanced) |
|---|---|---|---|---|
| test_task (Cross-Task) | 92.9 | 97.0 | 93.1 | 96.6 |
| test_website (Cross-Website) | 90.8 | 97.3 | 90.7 | 96.5 |
| test_domain (Cross-Domain) | 91.7 | 97.4 | 91.5 | 98.5 |
- About the same as Gen1; the differences are within noise.
- So far this is only used to explore the model in RPA-style automation and to check that it generalises to some degree.
- The task is to pick the target element among it and up to 5 other candidates sampled from the page. This is easier than the original Mind2Web protocol, so do not compare it with the Mind2Web leaderboard.
Calibration
Computed on the 0.3 public suite with the method the Decision Index board describes: each field is right or wrong, confidence is the probability on the chosen option, benchmarks are weighted equally, and there are 10 equal-width bins.
| metric | Gen2 | Gen1 | board median (112 models) | Jev 1.13 |
|---|---|---|---|---|
| ECE (lower is better) | 0.06-0.07 | 0.08-0.09 | 0.084 | 0.074 |
| Brier score (lower is better) | 0.31-0.33 | 0.34-0.36 | 0.49 | 0.36 |
| Confidence β₯95% but wrong | 2.2% | 3.3% | 1.3% | 2.1% |
| Mean confidence / accuracy | 0.83 / 0.76 | 0.84 / 0.74 | 0.81 / 0.74 |
- Why ranges: the board does not say which 32 benchmarks it uses. We checked our computation against three board models whose results are public. Our ECE was within about 0.015 of the board's, so we give ranges over the plausible benchmark sets.
- Still overconfident overall: mean confidence is about 7 points above accuracy. Most of the gap is on hard reasoning (HLE, CRUXEval), humour (Humicroedit) and colour judgements (cfcolor).
- Recommendation: if you act on probability thresholds, fit a temperature on your own data.
Disclosures
- Decontamination:
- Every component of a training row of at least 60 characters (a line or paragraph) was checked against the components of every Decision Index row and the public benchmarks we report.
- Rows with a component identical to an evaluation component, or with the same first 50 characters, were removed. Text that appears in more than 4 evaluation items counts as boilerplate and is not matched.
- No model outputs as labels: no output of Jev or any other decision model was used as a training label.
- Teacher-labelled synthetic data: LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all were checked by a human.
- Public suite: the Decision Index public suite is public; the private tests may show a lower score.
Limitations
- Knowledge-heavy reasoning: weaker than larger models. The knowledge area is 42.7 (0.3); MMLU-Pro and HLE are still close to the base model.
- Math: answered without reasoning. The rebuilt 0.3 GSM8K scores 58.0.
- Prompt injection: an instruction planted in the state can change the answer. Keep untrusted text clearly delimited.
- Calibration: overconfident overall; see Calibration.
- Private tests: the 0.3 private tests may score lower than the public suite.
Citation
@misc{knowline4bgen2,
title = {KnowLine-4B-Gen2: a 4B decision model},
author = {PelaAI},
year = {2026},
url = {https://huggingface.co/PelaAI/KnowLine-4B-Gen2}
}
License
- Weights: Apache-2.0, the same as the base model.
- Code:
knowline_server.pyis MIT (see the file header).
- Downloads last month
- 60