Hyper3-CLIP

For visual search and application development, we recommend Hyper3-CLIP V1.

Hyper3-CLIP is a hyperbolic version of CLIP from hyper³labs. It combines query-conditioned visual pooling with hyperbolic hierarchical representation learning.

Paper · Code · Hyper3-CLIP V1

Presented as an oral at the ECCV 2026 Beyond Euclidean workshop.

How it works

To train the model, we construct lightweight query hierarchies from text, comprising full captions, sentence fragments, localized part descriptions, and extracted phrases. Each query conditions the pooling of visual patches, and the resulting representations support image-text, whole-part, and parent-child entailment losses.

Query-conditioned pooling is active only during training; at inference, Hyper3-CLIP is a standard dual encoder, and the pooling module adds no inference-time computation.

Model

• ViT-B/16 image encoder and CLIP-B/32 text-encoder architecture, trained from scratch.
• Trained on GRIT for 500,000 steps.

Results

All downstream results are zero-shot evaluations of frozen checkpoints.

Image-text retrieval

Hyper3-CLIP improves over UNCHA across all eight considered COCO/Flickr retrieval settings.

Paper Table 2, ViT-B/16. Published baseline values are from UNCHA Table 2; Hyper3-CLIP is evaluated with the same retrieval and hierarchy metrics. Each pair reports R@5 / R@10. Higher is better.

Image → text

Model	COCO	Flickr30K
CLIP	71.4 / 81.5	93.6 / 96.9
MERU	72.3 / 82.0	93.5 / 96.2
ATMG	62.9 / 74.0	85.1 / 92.2
HyCoCLIP	72.0 / 82.0	92.6 / 95.4
UNCHA	72.7 / 82.7	91.4 / 95.9
Hyper3-CLIP	75.7 / 84.3	95.4 / 97.6

Text → image

Model	COCO	Flickr30K
CLIP	57.4 / 68.5	83.5 / 89.9
MERU	57.4 / 68.6	84.0 / 90.0
ATMG	51.2 / 62.6	78.0 / 85.3
HyCoCLIP	58.4 / 69.3	84.9 / 90.3
UNCHA	60.0 / 71.0	84.9 / 91.2
Hyper3-CLIP	63.0 / 73.2	86.4 / 91.4

ImageNet hierarchy

Paper Table 2, ViT-B/16. Higher is better except TIE and LCA.

Model	TIE ↓	LCA ↓	J ↑	H-P ↑	H-R ↑
CLIP	3.60	2.21	0.79	0.85	0.85
MERU	3.63	2.22	0.78	0.85	0.85
ATMG	4.19	2.48	0.75	0.83	0.83
HyCoCLIP	3.17	2.05	0.81	0.87	0.87
UNCHA	2.94	1.96	0.83	0.88	0.88
Hyper3-CLIP	3.16	2.08	0.82	0.87	0.88

Zero-shot classification

Paper Table 3. Values are mean-per-class accuracy. All models use ViT-B/16. Food-101, CUB, and Flowers, marked with *, use the common Photo prompt, a photo of a {}.; the remaining datasets use the baseline evaluation prompts.

Dataset	HyCoCLIP	UNCHA	PHyCLIP	Hyper3-CLIP
ImageNet	45.80	48.80	44.31	46.98
CIFAR-10	88.80	90.40	89.33	91.14
CIFAR-100	60.10	63.20	59.05	65.64
SUN397	57.20	57.70	55.32	59.39
Caltech-101	81.30	83.90	76.35	83.42
STL-10	95.00	95.70	94.84	96.56
Food-101*	61.14	64.10	62.40	67.31
CUB*	17.76	18.74	17.97	19.36
Cars	11.60	14.00	10.89	16.12
Aircraft	3.70	3.80	3.24	4.33
Pets	56.80	57.10	54.18	62.57
Flowers*	28.79	30.75	28.27	39.66
DTD	29.40	30.30	25.50	29.41
EuroSAT	35.80	41.30	36.29	28.52
RESISC45	45.60	52.70	48.22	48.94
Country211	6.50	6.10	5.56	6.06
Average	45.33	47.41	44.48	47.84

Prompt comparison (paper Table 6)

The 16-dataset average changes only the three prompt-sensitive columns; all other zero-shot columns are held fixed. Values are mean-per-class accuracy.

Food-101 / CUB / Flowers average

Model	Official	Photo	Δ
HyCoCLIP	33.09	35.90	+2.81
UNCHA	33.90	37.86	+3.96
Hyper3-CLIP	3.26	42.11	+38.85

16-dataset average

Model	Official	Photo	Δ
HyCoCLIP	44.82	45.33	+0.51
UNCHA	46.69	47.41	+0.72
Hyper3-CLIP	40.55	47.84	+7.29

Prompt templates and Table 6.

Multi-label classification

Under a shared evaluator, Hyper3-CLIP improves over UNCHA by 2.69 mAP on VOC and 3.35 mAP on COCO.

Paper Table 4, ViT-B/16. Values are mAP; Avg. is the mean of VOC and COCO. Higher is better.

Model	VOC	COCO	Avg.
CLIP	68.55	44.89	56.72
MERU	71.29	45.33	58.31
HyCoCLIP	74.89	50.73	62.81
UNCHA	75.51	50.93	63.22
PHyCLIP	73.29	49.64	61.47
Hyper3-CLIP	78.20	54.28	66.24

Query-conditioned objective ablation (paper Table 5)

To isolate their effects, each variant is first trained with only the base objective for 80k steps, then continues training to 100k steps with the query entailment terms specified in the table. Data, batch size, optimizer schedule, and seed are fixed; the control rows keep all query entailment terms enabled and modify only the named control.

Variant	Image-text	Visual hierarchy	Text hierarchy	AP	ΔAP	AUROC	ΔR@10
No query losses	–	–	–	76.21	0.00	98.34	0.00
Query image-text only	✓	–	–	75.34	-0.87	98.24	-0.46
Visual hierarchy only	–	✓	–	71.85	-4.36	98.01	-0.77
Text hierarchy only	–	–	✓	78.86	+2.65	98.44	-0.09
Full objective	✓	✓	✓	79.85	+3.64	98.55	+0.06
Mean-pooled visual nodes	✓	✓	✓	75.97	-0.25	98.32	+0.15
Reversed visual order (query → image)	✓	✓	✓	76.06	-0.15	98.29	-0.09
Shuffled text parents	✓	✓	✓	77.22	+1.00	98.40	-0.93

We report Average Precision (AP) and AUROC over 4,000 positive and 100,000 negative pairs from 1,000 images. Avg. R@10 is the mean of the four COCO/Flickr text- and image-retrieval R@10 values. ΔAP and ΔR@10 are measured relative to the no-query baseline.

Objective components and Table 5.

Citation

@inproceedings{mahmood2026hyper3clip,
  title     = {Hyper3-CLIP: Hierarchy-Conditioned Hyperbolic Vision-Language Training},
  author    = {Mahmood, Matin and Rueda-Toicen, Antonio and ElBassat, Mohamed and Elkerdany, Seifeldin and Wang, Weixing and de Melo, Gerard},
  booktitle = {Proceedings of the ECCV 2026 Workshop on Beyond Euclidean: Hyperbolic Deep Learning for Computer Vision},
  year      = {2026},
  eprint    = {2608.29313},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

License

OpenMDW-1.0 · NOTICE.