zli12321 commited on
Commit
eae403c
·
verified ·
1 Parent(s): f0e1c1e

Report LHTB as pass@0.95 (6.5) instead of mean reward

Browse files

Switches the reported LHTB metric from mean reward to pass@0.95, the share of the 46 tasks solved at reward >= 0.95 (3/46). The mean reward is retained in `notes`.

Files changed (1) hide show
  1. .eval_results/lhtb.yaml +2 -2
.eval_results/lhtb.yaml CHANGED
@@ -1,9 +1,9 @@
1
  - dataset:
2
  id: IntelligenceLab/Long-Horizon-Terminal-Bench
3
  task_id: lhtb
4
- value: 38.5
5
  date: '2026-07-16'
6
  source:
7
  url: https://zli12321.github.io/LHTB/leaderboard.html
8
  name: LHTB leaderboard
9
- notes: "mean reward x100 over 46 tasks (partial credit); solved@0.95=3/46; official LHTB Harbor harness"
 
1
  - dataset:
2
  id: IntelligenceLab/Long-Horizon-Terminal-Bench
3
  task_id: lhtb
4
+ value: 6.5
5
  date: '2026-07-16'
6
  source:
7
  url: https://zli12321.github.io/LHTB/leaderboard.html
8
  name: LHTB leaderboard
9
+ notes: "pass@0.95 = 3/46 tasks solved at reward >= 0.95; mean reward x100 = 38.5; official LHTB Harbor harness"