Daniel Hiltgen 356c0b8e34 gemma4: add audio support with USM conformer encoder ...

Add audio encoding for Gemma 4 using the USM conformer architecture:
- Converter: audio tensor mapping, SSCP/conformer/embedder name replacements,
  softplus repacker for per_dim_scale, F32 enforcement for conv weights
- GGML backend: Conv1DDW and PadExt tensor ops
- Audio encoder: SSCP Conv2D, 12 conformer blocks (FFW + block-local
  attention with relative position embeddings + LightConv1d + FFW),
  output projection, audio-to-text embedding projector
- Audio preprocessing: WAV decode, mel spectrogram, FFT (pure Go)
- Model wiring: WAV detection, audio token handling, unified PostTokenize

Correctly transcribes "why is the sky blue" from test audio.

2026-04-01 15:24:17 -07:00

..

internal

docs: fix typos in repository documentation (#10683)

2025-11-15 20:22:29 -08:00

auth_test.go

server: reject unexpected auth hosts (#13738)

2026-01-16 14:10:36 -05:00

auth.go

server: reject unexpected auth hosts (#13738)

2026-01-16 14:10:36 -05:00

cloud_proxy_test.go

cloud_proxy: for the web_search legacy path, flush on newlines (#14897)

2026-03-17 13:30:17 -07:00

cloud_proxy.go

cloud_proxy: for the web_search legacy path, flush on newlines (#14897)

2026-03-17 13:30:17 -07:00

create_test.go

Clean up the manifest and modelpath (#13807)

2026-01-21 11:46:17 -08:00

create.go

gemma4: add Gemma 4 GGML model support

2026-04-01 15:23:10 -07:00

download.go

Clean up the manifest and modelpath (#13807)

2026-01-21 11:46:17 -08:00

fixblobs_test.go

server: replace blob prefix separator from ':' to '-' (#3146)

2024-03-14 20:18:06 -07:00

fixblobs.go

server: replace blob prefix separator from ':' to '-' (#3146)

2024-03-14 20:18:06 -07:00

images_test.go

server: preserve raw manifest bytes during pull (#15104)

2026-03-27 15:42:31 -07:00

images.go

server: preserve raw manifest bytes during pull (#15104)

2026-03-27 15:42:31 -07:00

inference_request_log.go

add ability to turn on debug request logging (#14106)

2026-03-19 17:08:17 -07:00

logprob.go

logprob: add bytes to logprobs (#13068)

2025-11-13 13:49:25 -08:00

model_resolver_test.go

Reapply "don't require pulling stubs for cloud models" again (#14608)

2026-03-06 14:27:47 -08:00

model_resolver.go

Reapply "don't require pulling stubs for cloud models" again (#14608)

2026-03-06 14:27:47 -08:00

model.go

Clean up the manifest and modelpath (#13807)

2026-01-21 11:46:17 -08:00

prompt_test.go

model/renderers: fix glm-ocr image tags in renderer prompts (#14584)

2026-03-03 12:51:34 -08:00

prompt.go

mlxrunner: Enforce model context limit

2026-02-27 17:29:47 -08:00

quantization_test.go

model: support for qwen3.5 architecture (#14378)

2026-02-24 20:08:05 -08:00

quantization.go

gemma4: add audio support with USM conformer encoder

2026-04-01 15:24:17 -07:00

routes_cloud_test.go

revert context length warnings change (#15121)

2026-03-28 16:43:59 -07:00

routes_create_test.go

Reapply "don't require pulling stubs for cloud models" again (#14608)

2026-03-06 14:27:47 -08:00

routes_debug_test.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

routes_delete_test.go

Reapply "don't require pulling stubs for cloud models" again (#14608)

2026-03-06 14:27:47 -08:00

routes_generate_renderer_test.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

routes_generate_test.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

routes_harmony_streaming_test.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

routes_list_test.go

Update the /api/create endpoint to use JSON (#7935)

2024-12-31 18:02:30 -08:00

routes_options_test.go

server: use tiered VRAM-based default context length

2026-02-02 10:47:09 -08:00

routes_request_log_test.go

add ability to turn on debug request logging (#14106)

2026-03-19 17:08:17 -07:00

routes_test.go

mlx: fix vision capability + min version (#15106)

2026-03-27 17:09:28 -07:00

routes_web_experimental_test.go

cloud_proxy: send ollama client version (#14769)

2026-03-10 15:53:25 -07:00

routes.go

revert context length warnings change (#15121)

2026-03-28 16:43:59 -07:00

sched_test.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

sched.go

sched: Model eviction for MLX

2026-03-16 17:40:29 -07:00

sparse_common.go

Don't hard fail on sparse setup error

2024-08-09 12:16:19 -07:00

sparse_windows.go

Don't hard fail on sparse setup error

2024-08-09 12:16:19 -07:00

test_home_test.go

add ability to disable cloud (#14221)

2026-02-12 15:47:00 -08:00

upload.go

Clean up the manifest and modelpath (#13807)

2026-01-21 11:46:17 -08:00