starred/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-04-26 02:36:09 +02:00
Code Issues Packages Projects Releases Wiki Activity

Gemma4 on MLX (#15244)

Browse Source 
* gemma4: implement Gemma 4 model for MLX (text-only runtime)

* gemma4: two MoE + SWA prefill perf fixes

Two performance optimizations in the gemma4 forward pass

1. Memoize the sliding-window prefill mask across layers.
2. Softmax only over the selected experts in Router.Forward.

* review comments
This commit is contained in:
Daniel Hiltgen
2026-04-13 16:36:51 -07:00
committed by GitHub
parent bf2a421727
commit 2cba7756c5
8 changed files with 2715 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
x/create/create.go
View File

@@ -634,6 +634,8 @@ var tensorImportTransformRegistry = map[string]tensorImportTransformFactory{
"Qwen3_5MoeForConditionalGeneration": newQwen35ImportTransform,
"Qwen3NextMoeForCausalLM": newQwen35ImportTransform,
"Qwen3NextMoeForConditionalGeneration": newQwen35ImportTransform,
"Gemma4ForCausalLM": newGemma4ImportTransform,
"Gemma4ForConditionalGeneration": newGemma4ImportTransform,
}
func newTensorImportTransform(modelDir string, cfg sourceModelConfig) (tensorImportTransform, error) {