Hugging Face's logo

qualcomm
/
Zipformer

Automatic Speech Recognition
PyTorch
foundation
android
Model card Files
xet
 Community

Zipformer: Optimized for Qualcomm Devices

Zipformer streaming ASR (Automatic Speech Recognition) model is a state-of-the-art system designed for transcribing spoken language into written text streamingly. This model is based on the transformer architecture and has been optimized for edge inference by replacing linear layers with convolutional (conv) layers. It exhibits robust performance in realistic, noisy environments, making it highly reliable for real-world applications. Specifically, it excels in long-form transcription, capable of accurately transcribing audios. Time to the first token is the encoder's latency, while time to each additional token is joiner's latency, where we assume a max decoded length specified below.

This is based on the implementation of Zipformer found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
PRECOMPILED_QNN_ONNX float Snapdragon® X2 Elite QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Snapdragon® X Elite QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 3 Mobile QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 1 Mobile QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-8275 QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-9075 QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Mobile QAIRT 2.45, ONNX Runtime 1.27.1 Download
PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Gen 5 Mobile QAIRT 2.45, ONNX Runtime 1.27.1 Download
QNN_CONTEXT_BINARY float Snapdragon® X2 Elite QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Snapdragon® X Elite QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 3 Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 1 Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ QCS8550 (Proxy) QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® SA8775P QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-9075 QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® SA7255P QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Qualcomm® SA8295P QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Gen 5 Mobile QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Zipformer on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Zipformer on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.speech_recognition

Model Stats:

  • Input resolution: 80x71 (0.71 seconds audio)
  • Max decoded sequence length: 200 tokens
  • Model checkpoint: pfluo/k2fsa-zipformer-chinese-english-mixed
  • Model size (decoder) (float): 13.2 MB
  • Model size (encoder) (float): 242 MB
  • Model size (joiner) (float): 12.2 MB
  • Number of parameters (decoder): 3.47M
  • Number of parameters (encoder): 63.2M
  • Number of parameters (joiner): 3.21M

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder PRECOMPILED_QNN_ONNX float Snapdragon® X2 Elite 0.077 ms 1 - 1 MB NPU
decoder PRECOMPILED_QNN_ONNX float Snapdragon® X Elite 0.149 ms 7 - 7 MB NPU
decoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 3 Mobile 0.158 ms 0 - 7 MB NPU
decoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 1 Mobile 0.246 ms 0 - 13 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-8275 0.267 ms 0 - 4 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 0.196 ms 0 - 9 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® QCS8450 0.246 ms 0 - 13 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-9075 0.241 ms 0 - 3 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-X7181 0.149 ms 7 - 7 MB NPU
decoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ Q-8750 0.132 ms 0 - 7 MB NPU
decoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Mobile 0.132 ms 0 - 7 MB NPU
decoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Gen 5 Mobile 0.135 ms 0 - 9 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® X2 Elite 0.178 ms 1 - 1 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® X Elite 0.24 ms 1 - 1 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 3 Mobile 0.148 ms 0 - 8 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 1 Mobile 0.242 ms 0 - 9 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 0.253 ms 0 - 2 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 0.438 ms 0 - 9 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ QCS8550 (Proxy) 0.196 ms 0 - 2 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® SA8775P 0.324 ms 0 - 6 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® SA8650P 0.324 ms 0 - 6 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® SA8255P 0.324 ms 0 - 6 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® QCS8450 0.242 ms 0 - 9 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-9075 0.242 ms 0 - 2 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-X7181 0.24 ms 1 - 1 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ Q-8750 0.137 ms 0 - 13 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® SA7255P 0.438 ms 0 - 9 MB NPU
decoder QNN_CONTEXT_BINARY float Qualcomm® SA8295P 0.447 ms 0 - 6 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Mobile 0.137 ms 0 - 13 MB NPU
decoder QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Gen 5 Mobile 0.127 ms 0 - 8 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® X2 Elite 7.893 ms 10 - 10 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® X Elite 12.924 ms 143 - 143 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 3 Mobile 8.837 ms 9 - 17 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 1 Mobile 16.189 ms 9 - 22 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-8275 13.939 ms 8 - 13 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 12.369 ms 0 - 158 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® QCS8450 16.189 ms 9 - 22 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-9075 14.122 ms 8 - 12 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-X7181 12.924 ms 143 - 143 MB NPU
encoder PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ Q-8750 7.456 ms 8 - 20 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Mobile 7.456 ms 8 - 20 MB NPU
encoder PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Gen 5 Mobile 6.97 ms 10 - 23 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® X2 Elite 8.406 ms 3 - 3 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® X Elite 13.484 ms 2 - 2 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 3 Mobile 8.796 ms 2 - 11 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 1 Mobile 15.865 ms 2 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 13.141 ms 3 - 8 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 28.624 ms 3 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ QCS8550 (Proxy) 12.273 ms 3 - 4 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® SA8775P 14.158 ms 3 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® SA8650P 14.158 ms 3 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® SA8255P 14.158 ms 3 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® QCS8450 15.865 ms 2 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-9075 19.403 ms 5 - 9 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-X7181 13.484 ms 2 - 2 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ Q-8750 7.725 ms 3 - 15 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® SA7255P 28.624 ms 3 - 12 MB NPU
encoder QNN_CONTEXT_BINARY float Qualcomm® SA8295P 17.412 ms 0 - 6 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Mobile 7.725 ms 3 - 15 MB NPU
encoder QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Gen 5 Mobile 6.937 ms 3 - 12 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® X2 Elite 0.087 ms 1 - 1 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® X Elite 0.144 ms 6 - 6 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 3 Mobile 0.149 ms 0 - 7 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® 8 Gen 1 Mobile 0.238 ms 0 - 13 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-8275 0.268 ms 0 - 4 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 0.187 ms 0 - 9 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® QCS8450 0.238 ms 0 - 13 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-9075 0.271 ms 0 - 3 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ IQ-X7181 0.144 ms 6 - 6 MB NPU
joiner PRECOMPILED_QNN_ONNX float Qualcomm® Dragonwing™ Q-8750 0.134 ms 0 - 7 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Mobile 0.134 ms 0 - 7 MB NPU
joiner PRECOMPILED_QNN_ONNX float Snapdragon® 8 Elite Gen 5 Mobile 0.133 ms 0 - 7 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® X2 Elite 0.171 ms 0 - 0 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® X Elite 0.238 ms 0 - 0 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 3 Mobile 0.152 ms 0 - 8 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® 8 Gen 1 Mobile 0.228 ms 0 - 10 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 0.254 ms 0 - 2 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-8275 0.49 ms 0 - 8 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ QCS8550 (Proxy) 0.188 ms 0 - 1 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® SA8775P 0.328 ms 0 - 9 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® SA8650P 0.328 ms 0 - 9 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® SA8255P 0.328 ms 0 - 9 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® QCS8450 0.228 ms 0 - 10 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-9075 0.245 ms 2 - 4 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ IQ-X7181 0.238 ms 0 - 0 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® Dragonwing™ Q-8750 0.132 ms 0 - 13 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® SA7255P 0.49 ms 0 - 8 MB NPU
joiner QNN_CONTEXT_BINARY float Qualcomm® SA8295P 0.462 ms 0 - 6 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Mobile 0.132 ms 0 - 13 MB NPU
joiner QNN_CONTEXT_BINARY float Snapdragon® 8 Elite Gen 5 Mobile 0.132 ms 0 - 9 MB NPU

License

  • The license for the original implementation of Zipformer can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Automatic Speech Recognition
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/Zipformer