ethicalabs/Research-Intent-Curated

A curated, multi-source dataset for research paper intent classification — 5-class taxonomy trained into Echo-DSRN-v0.1.3-Research-Intent-CLF.

Each record pairs a paper's title and abstract with one of five research-intent labels, sourced from a tiered pipeline of gold, near-gold, and silver sources spanning PubMed, Semantic Scholar, Papers With Code, arXiv, and the OpenAIRE Graph.

---

Labels

Label	Meaning
Methodology	Introduces a new method, model, or algorithm
Dataset	Introduces or documents a dataset or benchmark
Review	Surveys or synthesises existing work
Applied	Applies existing methods to a domain problem
Theoretical	Mathematical or formal analysis without empirical evaluation

---

Dataset composition

Split	Records
Train	5,400
Validation	600
Total	6,000

Label distribution (perfectly balanced)

Label	Train	Val
Methodology	1,080	120
Dataset	1,080	120
Review	1,080	120
Applied	1,080	120
Theoretical	1,080	120

---

Data sources (tiered quality)

Records were drawn from 5 sources using a priority-ordered pipeline. Gold sources run first; silver sources fill remaining quota per class.

Source	Quality tier
arxiv	🥈 Silver (keyword + arXiv category prior, primary volume source)
arxiv_summarization	🥈 Near-Gold (keyword + arXiv category prior, clean abstracts)
llm_judge_corrections	🥇 Gold (11-model LLM consensus override)
openaire	🥉 Silver top-up (OpenAIRE Graph API keyword queries)
pubmed	🥇 Gold (NLM-indexed publication-type tags)

Per-source breakdown (train)

Source	Records	Labels contributed
arxiv	3,251	Methodology: 579, Dataset: 956, Review: 470, Applied: 455, Theoretical: 791
arxiv_summarization	73	Methodology: 11, Dataset: 1, Review: 6, Applied: 1, Theoretical: 54
llm_judge_corrections	1,161	Methodology: 490, Dataset: 114, Review: 157, Applied: 176, Theoretical: 224
openaire	23	Dataset: 9, Review: 1, Applied: 2, Theoretical: 11
pubmed	892	Review: 446, Applied: 446

Quality tier details

• PubMed E-utilities API — NLM indexer-curated publication-type tags: [pt]Review / [pt]Systematic Review → Review, [pt]Clinical Trial → Applied.
• Semantic Scholar /paper/search — three independent query banks (Methodology, Dataset, Theoretical), each cross-checked against keyword filters.
• Papers With Code API — /papers/ endpoint; papers returned are definitionally Dataset papers.
• LLM Judge Corrections — 11-model LLM consensus panel (Qwen, Gemma, GPT-OSS, DeepSeek, Nemotron, GLM families) validates or overrides silver labels where initial classification was low-confidence.
• ccdv/arxiv-summarization — near-gold: keyword silver-labeling + arXiv category prior on clean, long-form abstracts.
• gfissore/arxiv-abstracts-2021 — primary volume source: streamed up to 1M rows, keyword + category labeling.
• OpenAIRE Graph API — silver top-up: keyword queries per label (CC BY 4.0), last resort when all other sources are exhausted.

Silver-labeling methodology

For non-gold sources, labels are assigned via first-match keyword regex rules in four priority layers:

1. Theoretical — we prove, theoretically analyze, convergence rate, generalization bound, sample complexity, etc.
2. Review — survey, systematic review, literature review, meta-analysis, we survey, this paper provides an overview, etc.
3. Dataset — requires a release signal (we release, publicly available, github.com, huggingface.co, etc.) AND dataset-indicative title/body keywords (benchmark, corpus, manually annotated, leaderboard, etc.).
4. Applied — we apply ... to clinical/medical/legal, case study, real-world application, off-the-shelf, etc.
5. Methodology — we propose a new method/model/architecture, state-of-the-art ... our, we pre-train/fine-tune, etc.

arXiv category priors (cs.LG → Methodology, math.ST → Theoretical, etc.) serve as fallback when no keyword pattern matches.

Quality filters applied

• Titles must be ≥ 12 characters and not section headings (Introduction, 3.2 Methods, etc.).
• Abstracts must be ≥ 120 characters and start with an uppercase letter (reject sentence fragments).
• Abstracts containing LaTeX/markup artifacts (@xmath, @xcite, \textit, etc.) are rejected.
• MD5 fingerprint deduplication across all sources.

---

Text statistics (train, character counts)

Label	Title median	Abstract median	Abstract min	Abstract max	Short (<100)
Methodology	76	1,181	79	5,505	1
Dataset	76	1,126	123	1,919	0
Review	92	920	19	3,595	4
Applied	99	889	61	8,009	1
Theoretical	71	1,086	82	8,049	2

---

Language

100% English. Language was detected at ingestion via pycld2 (C extension) on the first 300 characters of each abstract.

---

Data format

Chat-style JSONL with messages, label, source, and lang fields:

{
  "messages": [
    {"role": "system", "content": "You are a multilingual research paper intent classifier. Given a paper title and abstract, classify its primary research intent as exactly one of: Methodology, Dataset, Review, Applied, Theoretical."},
    {"role": "user", "content": "Classify the research intent of this paper:\n\nTitle: Attention Is All You Need\nAbstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks..."},
    {"role": "assistant", "content": "Methodology"}
  ],
  "label": "Methodology",
  "source": "arxiv",
  "lang": "en"
}

Usage

from datasets import load_dataset

ds = load_dataset("ethicalabs/Research-Intent-Curated")
# ds["train"]  → 5,400 records
# ds["val"]    → 600 records

Provenance

Training data was curated from:

• PubMed — public domain / government work (US NLM)
• Semantic Scholar — ODC-BY license
• Papers With Code — CC BY-SA 4.0
• ccdv/arxiv-summarization — arXiv papers (various licenses)
• gfissore/arxiv-abstracts-2021 — arXiv papers (various licenses)
• OpenAIRE Graph API — CC BY 4.0

This dataset as a whole is released under CC BY 4.0.

OpenAIRE AI Hackathon 2026

Part of ethicalabs.ai's entry in the OpenAIRE AI Hackathon 2026, co-organised by OpenAIRE and Alien Intelligence.

🔴 Live Demo → openaire-2026.ethicalabs.ai

Citation

If you use this dataset, please cite:

Research-Intent-Curated — Multi-source, tiered-quality dataset for research paper intent classification. ethicalabs.ai, 2026. CC BY 4.0. huggingface.co/datasets/ethicalabs/Research-Intent-Curated

Related

• Echo-DSRN-v0.1.3-Research-Intent-CLF — model trained on this dataset
• OpenAIRE-AI-Research-Evaluator — source code
• Research-Intent-Collab — live platform annotations
• Research-Intent-Judge — LLM judge evaluation samples