Instructions to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF",
	filename="Qwen3.6-35B-A3B-Uncensored-APEX-Compact.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
# Run inference directly in the terminal:
llama-cli -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
# Run inference directly in the terminal:
llama-cli -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
# Run inference directly in the terminal:
./llama-cli -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Use Docker

docker model run hf.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

LM Studio
Jan

vLLM

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Ollama
How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Ollama:
```
ollama run hf.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
```

Unsloth Studio

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF to start chatting

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Run Hermes

hermes

Docker Model Runner
How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Docker Model Runner:
```
docker model run hf.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M
```

Lemonade

How to use LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF:IQ3_M

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF-IQ3_M

List all available models

lemonade list

difference with LuffyTheFox/Qwen3.6-35B-A3B-Plus-Uncensored-Wasserstein-GGUF ?

by AI-Joe-git - opened Apr 22

Discussion

AI-Joe-git

Apr 22

Like the title said ahah

LuffyTheFox

Owner Apr 23

Like the title said ahah

This one has only tensor drift fix for ssm_conv1d layers. Qwen3.6-35B-A3B-Plus-Uncensored-Wasserstein-GGUF version MXFP4 quant contains a lot of fixed holes (blocks with zero weight) on per tensor and per neuron level.

aeronfx

Apr 23

Qwen3.6-35B-A3B-Plus-Uncensored-Q8_0 is smaller, what causes difference? is Qwen3.6-35B-A3B-Plus-Uncensored-Q8_0 same smart exactly like this?(Qwen3.6-35B-A3B-Uncensored.Q8_K_P) I have hard to decide which one is best. Thanks

LuffyTheFox

Owner Apr 23

Qwen3.6-35B-A3B-Plus-Uncensored-Q8_0 is smaller, what causes difference? is Qwen3.6-35B-A3B-Plus-Uncensored-Q8_0 same smart exactly like this?(Qwen3.6-35B-A3B-Uncensored.Q8_K_P) I have hard to decide which one is best. Thanks

Use plus version Q8_0 or MXFP4_MOE: https://huggingface.co/LuffyTheFox/Qwen3.6-35B-A3B-Plus-Uncensored-Wasserstein-GGUF/blob/main/Qwen3.6-35B-A3B-Plus-Uncensored-Q8_0.gguf

It's best what we currently have because i fixed dead neurons on per tensor and per neuron level.

aeronfx

Apr 23

Thanks, dropping this version now(Qwen3.6-35B-A3B-Uncensored.Q8_K_P)

wair1990

Apr 25

Thank you for the hard work. Why was the Plus version, Qwen3.6-35B-A3B-Plus-Uncensored-Wasserstein-GGUF, removed? Is there a problem with it?

LuffyTheFox

Owner Apr 25

Thank you for the hard work. Why was the Plus version, Qwen3.6-35B-A3B-Plus-Uncensored-Wasserstein-GGUF, removed? Is there a problem with it?

Users reported to me that it fails some benchmarks in consistency tests. Trying another way.

aeronfx

Apr 25

Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf - just fresh upload by you, does it supersede Qwen3.6-35B-A3B-Plus-Uncensored ? thanks.

LuffyTheFox

Owner Apr 25

Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf - just fresh upload by you, does it supersede Qwen3.6-35B-A3B-Plus-Uncensored ? thanks.

Yes it should be. Because instead of copying blocks, I recovered learning gradients in tensors via mathematical statistics effectively filling zero blocks.

Anduin1357

26 days ago

What's the model to get now?

LuffyTheFox

Owner 26 days ago

•

edited 26 days ago

What's the model to get now?

Wasserstein works for most people. I reccomend to use Wasserstein if you just started.

Genesis version is experimental, and contains more consistency during thinking mode in Chain-Of-Thought. Still in development and needs proper testing from community.

aeronfx

23 days ago

Strange, most other models are gone and this old (Qwen3.6-35B-A3B-Uncensored.Q8_K_P) what i dropped and Harmony(freshly release) are left. Harmony best now?

LuffyTheFox

Owner 23 days ago

•

edited 23 days ago

Strange, most other models are gone and this old (Qwen3.6-35B-A3B-Uncensored.Q8_K_P) what i dropped and Harmony(freshly release) are left. Harmony best now?

Harmony should be the best yes. Because I restored dead experts in it via simple tensor copy and harmonic Wasserstein.

aeronfx

14 days ago

Just wondering why you left that most oldest version up for download? i have Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P release from you but i have seen hallucinations

LuffyTheFox

Owner 14 days ago

•

edited 14 days ago

Just wondering why you left that most oldest version up for download? i have Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P release from you but i have seen hallucinations

I left it, because Wasserstein is the most stable. Wasserstein Transport can be even more stable, because I operate on block level instead of tensor level but it's still in development.

aeronfx

14 days ago

Just wondering why you left that most oldest version up for download? i have Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P release from you but i have seen hallucinations

I left it, because Wasserstein is the most stable. Wasserstein Transport can be even more stable, because I operate on block level instead of tensor level but it's still in development.

Thanks, i wait your update for this then in this series to replace my last one then.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment