Gabriel/diffu_sd35_128

Diffu — Swedish Handwritten-Line Generator (SD3.5 MMDiT, 128px)

🎨 Try it live: Diffu Studio — ZeroGPU Space · line + whole-page synthesis with GT labels.

Gabriel/diffu_sd35_128 is a latent-diffusion model that generates Swedish handwritten line images (modern + historical) from a text string and a writer-style reference crop. It exists to augment OCR/HTR training data for Swedish historical documents.

Given text (what to write) and a single-line crop of a target hand (style), it renders a tight, variable-width line image in that writer's style.

• Backbone: custom SD3.5-class MMDiT (transformer, 2D-RoPE, variable-length lines) — not a standard diffusers pipeline.
• Objective: rectified flow / flow matching.
• Guidance: classifier-free guidance (CFG); use cfg_scale ≈ 5 at inference.
• Line height: 128 px (this checkpoint; released with the CFG-null training fix).
• Sampling default: 24 steps.

"Diffu" is the package/code name. This repo holds config.json + model.safetensors (exported via the project's scripts/export_release.py).

---

What it is

A bespoke diffusion model assembling:

Part	Component
Backbone	SD3.5-class MMDiT transformer, 2D-RoPE, variable-length lines
Objective	Rectified flow / flow matching
VAE	16-channel f8 VAE (FLUX.1/SD3 class), decoder tuned for ink
Content / text	CANINE + glyph-line (GNU-Unifont) conditioning — Unicode-native å ä ö and historical glyphs
Style	DINOv3 ViT-L features → resampler (few-shot writer style)
Guidance	Classifier-free guidance, cfg_scale ≈ 5
Output	variable-width 128px line images

This checkpoint (step_120000 from the exp_sd35_128 run) was trained at 128px with both conditioning and text dropout (the "CFG-null" fix), which fixed the previously-broken unguided path and reached the guided CER floor roughly 5× faster than the earlier 64px release.

base_model note: the frontmatter cites SD3.5 as an architecture reference. The model is a custom MMDiT built to the SD3.5 design (2D-RoPE, ink-tuned 16-ch f8 VAE, CANINE+glyph content, DINOv3 style), assembled and trained from the diffu codebase — it is not a fine-tune of the SD3.5 checkpoint's weights.

---

Intended use

• Primary: generate synthetic Swedish handwritten line images to augment HTR/OCR training sets, especially for historical hands and Swedish-specific characters (å ä ö) that are scarce in real data.
• Composes into full synthetic pages via the companion diffu-page layout/compose pipeline.

Out of scope / not intended for: forgery or impersonation of real writers, general text-to-image generation, or non-Swedish scripts (trained on Swedish historical data only).

---

How to use

The model loads through the diffu package (it is not a diffusers pipeline).