Differential Transformer V2
•
47
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding
Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models
Articles
ChatBench Datasets and Simulators (same prompt + fine-tuning set-up) from the ChatBench paper.
A collection of SLMs based on Phi3.5-mini-instruct adapted to clinical natural language processing tasks: https://arxiv.org/abs/2505.10717
-
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Paper • 2505.10717 • Published • 5 -
microsoft/MediPhi-Instruct
Text Generation • 4B • Updated • 4.47k • 64 -
microsoft/MediPhi
Text Generation • 4B • Updated • 4.2k • 20 -
microsoft/MediPhi-PubMed
Text Generation • 4B • Updated • 144 • 10
NextCoder family of code-editing LMs developed with Selective Knowledge Transfer and its training data.
Phi-3 family of small language and multi-modal models. Language models are available in short- and long-context lengths.
-
microsoft/Phi-3.5-mini-instruct
Text Generation • 4B • Updated • 739k • 969 -
microsoft/Phi-3.5-MoE-instruct
Text Generation • Updated • 97.5k • 571 -
microsoft/Phi-3.5-vision-instruct
Image-Text-to-Text • Updated • 1.35M • 729 -
microsoft/Phi-3-mini-4k-instruct
Text Generation • 4B • Updated • 678k • 1.41k
Artifacts for the paper "Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements" (https://arxiv.org/abs/2410.08968)
MAI-DS-R1 is a DeepSeek-R1 reasoning model that has been post-trained by the Microsoft AI team.
The SpeechT5 framework consists of a shared seq2seq and six modal-specific (speech/text) pre/post-nets that can address a few audio-related tasks.
The Table Transformer (TATR) is a series of object detection models useful for table extraction from PDF images.
-
microsoft/table-transformer-detection
Object Detection • 28.8M • Updated • 3.12M • 412 -
microsoft/table-transformer-structure-recognition
Object Detection • 28.8M • Updated • 1.17M • 213 -
microsoft/table-transformer-structure-recognition-v1.1-all
Object Detection • 28.8M • Updated • 228k • 82 -
microsoft/table-transformer-structure-recognition-v1.1-fin
Object Detection • 28.8M • Updated • 705 • 2
Models for biomedical research applications, such as radiology report generation and biomedical language understanding.
UDOP is a general multimodal model for document AI
-
Unifying Vision, Text, and Layout for Universal Document Processing
Paper • 2212.02623 • Published • 12 -
microsoft/udop-large
Image-Text-to-Text • 0.7B • Updated • 98.6k • 123 -
microsoft/udop-large-512
Image-Text-to-Text • 0.7B • Updated • 27 • 6 -
microsoft/udop-large-512-300k
Image-Text-to-Text • 0.7B • Updated • 212 • 34
-
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
Paper • 2311.06242 • Published • 95 -
microsoft/Florence-2-large
Image-Text-to-Text • 0.8B • Updated • 1.22M • 1.79k -
microsoft/Florence-2-base
Image-Text-to-Text • 0.2B • Updated • 800k • 358 -
microsoft/Florence-2-large-ft
Image-Text-to-Text • 0.8B • Updated • 37.1k • 384
Locomotion policies for hundreds of simulated humanoid locomotion clips and demonstration data for training them.
Frontier Text-to-Speech Models https://microsoft.github.io/VibeVoice/
The models and datasets that comprise the Dayhoff Atlas
Paza is a collection of speech models & benchmarks for low resource languages by the Microsoft Research Africa - Nairobi Lab
Phi-4 family of small language, multi-modal and reasoning models.
Phi-1 family of small language models.
🔥BitNet family of large language models (1-bit LLMs).
-
microsoft/bitnet-b1.58-2B-4T
Text Generation • 0.8B • Updated • 17.5k • 1.42k -
microsoft/bitnet-b1.58-2B-4T-bf16
Text Generation • 2B • Updated • 6.69k • 38 -
microsoft/bitnet-b1.58-2B-4T-gguf
Text Generation • 2B • Updated • 35.8k • 259 -
BitNet b1.58 2B4T Technical Report
Paper • 2504.12285 • Published • 84
LLM2CLIP makes SOTA pretrained CLIP modal more SOTA ever.
-
microsoft/LLM2CLIP-EVA02-L-14-336
Zero-Shot Image Classification • Updated • 66 • 61 -
microsoft/LLM2CLIP-Openai-L-14-336
Zero-Shot Classification • 0.6B • Updated • 5.82k • 44 -
microsoft/LLM2CLIP-EVA02-B-16
Updated • 70 • 11 -
microsoft/LLM2CLIP-Openai-B-16
Zero-Shot Classification • 0.4B • Updated • 1.5k • 19
TAPEX is the state-of-the-art table pre-training models which can be used for table-based question answering and table-based fact verification.
-
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Paper • 2107.07653 • Published • 3 -
microsoft/tapex-large-finetuned-wtq
Table Question Answering • 0.4B • Updated • 693 • 78 -
microsoft/tapex-base-finetuned-wikisql
Table Question Answering • Updated • 896k • • 24 -
microsoft/tapex-large-sql-execution
Table Question Answering • 0.4B • Updated • 32 • 18
The LayoutLM series are Transformer encoders useful for document AI tasks such as invoice parsing, document image classification and DocVQA.
The Orca family of LMs developed by Microsoft.
GIT (Generative Image-to-text Transformer) is a model useful for vision-language tasks such as image/video captioning and question answering.
Industrial Foundation Models
ChatBench Datasets and Simulators (same prompt + fine-tuning set-up) from the ChatBench paper.
Frontier Text-to-Speech Models https://microsoft.github.io/VibeVoice/
A collection of SLMs based on Phi3.5-mini-instruct adapted to clinical natural language processing tasks: https://arxiv.org/abs/2505.10717
-
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Paper • 2505.10717 • Published • 5 -
microsoft/MediPhi-Instruct
Text Generation • 4B • Updated • 4.47k • 64 -
microsoft/MediPhi
Text Generation • 4B • Updated • 4.2k • 20 -
microsoft/MediPhi-PubMed
Text Generation • 4B • Updated • 144 • 10
The models and datasets that comprise the Dayhoff Atlas
Paza is a collection of speech models & benchmarks for low resource languages by the Microsoft Research Africa - Nairobi Lab
NextCoder family of code-editing LMs developed with Selective Knowledge Transfer and its training data.
Phi-4 family of small language, multi-modal and reasoning models.
Phi-3 family of small language and multi-modal models. Language models are available in short- and long-context lengths.
-
microsoft/Phi-3.5-mini-instruct
Text Generation • 4B • Updated • 739k • 969 -
microsoft/Phi-3.5-MoE-instruct
Text Generation • Updated • 97.5k • 571 -
microsoft/Phi-3.5-vision-instruct
Image-Text-to-Text • Updated • 1.35M • 729 -
microsoft/Phi-3-mini-4k-instruct
Text Generation • 4B • Updated • 678k • 1.41k
Phi-1 family of small language models.
Artifacts for the paper "Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements" (https://arxiv.org/abs/2410.08968)
🔥BitNet family of large language models (1-bit LLMs).
-
microsoft/bitnet-b1.58-2B-4T
Text Generation • 0.8B • Updated • 17.5k • 1.42k -
microsoft/bitnet-b1.58-2B-4T-bf16
Text Generation • 2B • Updated • 6.69k • 38 -
microsoft/bitnet-b1.58-2B-4T-gguf
Text Generation • 2B • Updated • 35.8k • 259 -
BitNet b1.58 2B4T Technical Report
Paper • 2504.12285 • Published • 84
MAI-DS-R1 is a DeepSeek-R1 reasoning model that has been post-trained by the Microsoft AI team.
LLM2CLIP makes SOTA pretrained CLIP modal more SOTA ever.
-
microsoft/LLM2CLIP-EVA02-L-14-336
Zero-Shot Image Classification • Updated • 66 • 61 -
microsoft/LLM2CLIP-Openai-L-14-336
Zero-Shot Classification • 0.6B • Updated • 5.82k • 44 -
microsoft/LLM2CLIP-EVA02-B-16
Updated • 70 • 11 -
microsoft/LLM2CLIP-Openai-B-16
Zero-Shot Classification • 0.4B • Updated • 1.5k • 19
The SpeechT5 framework consists of a shared seq2seq and six modal-specific (speech/text) pre/post-nets that can address a few audio-related tasks.
TAPEX is the state-of-the-art table pre-training models which can be used for table-based question answering and table-based fact verification.
-
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Paper • 2107.07653 • Published • 3 -
microsoft/tapex-large-finetuned-wtq
Table Question Answering • 0.4B • Updated • 693 • 78 -
microsoft/tapex-base-finetuned-wikisql
Table Question Answering • Updated • 896k • • 24 -
microsoft/tapex-large-sql-execution
Table Question Answering • 0.4B • Updated • 32 • 18
The Table Transformer (TATR) is a series of object detection models useful for table extraction from PDF images.
-
microsoft/table-transformer-detection
Object Detection • 28.8M • Updated • 3.12M • 412 -
microsoft/table-transformer-structure-recognition
Object Detection • 28.8M • Updated • 1.17M • 213 -
microsoft/table-transformer-structure-recognition-v1.1-all
Object Detection • 28.8M • Updated • 228k • 82 -
microsoft/table-transformer-structure-recognition-v1.1-fin
Object Detection • 28.8M • Updated • 705 • 2
The LayoutLM series are Transformer encoders useful for document AI tasks such as invoice parsing, document image classification and DocVQA.
Models for biomedical research applications, such as radiology report generation and biomedical language understanding.
The Orca family of LMs developed by Microsoft.
UDOP is a general multimodal model for document AI
-
Unifying Vision, Text, and Layout for Universal Document Processing
Paper • 2212.02623 • Published • 12 -
microsoft/udop-large
Image-Text-to-Text • 0.7B • Updated • 98.6k • 123 -
microsoft/udop-large-512
Image-Text-to-Text • 0.7B • Updated • 27 • 6 -
microsoft/udop-large-512-300k
Image-Text-to-Text • 0.7B • Updated • 212 • 34
GIT (Generative Image-to-text Transformer) is a model useful for vision-language tasks such as image/video captioning and question answering.
-
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
Paper • 2311.06242 • Published • 95 -
microsoft/Florence-2-large
Image-Text-to-Text • 0.8B • Updated • 1.22M • 1.79k -
microsoft/Florence-2-base
Image-Text-to-Text • 0.2B • Updated • 800k • 358 -
microsoft/Florence-2-large-ft
Image-Text-to-Text • 0.8B • Updated • 37.1k • 384
Industrial Foundation Models
Locomotion policies for hundreds of simulated humanoid locomotion clips and demonstration data for training them.