Jesse Gross 074bac8447 kvcache: Log batch size if we can't find a slot ...

In some cases, we can't find a cache slot when using sliding window
attention. It would be helpful in this (and other cases) to know what
the batch size is.

Bug #10127

2025-05-01 16:26:36 -07:00

..

cache.go

ollamarunner: Preallocate worst case graph at startup

2025-04-08 10:01:28 -07:00

causal_test.go

chunked attention

2025-04-25 16:59:20 -07:00

causal.go

kvcache: Log batch size if we can't find a slot

2025-05-01 16:26:36 -07:00

encoder.go

ollamarunner: Preallocate worst case graph at startup

2025-04-08 10:01:28 -07:00

wrapper.go

ollamarunner: Preallocate worst case graph at startup

2025-04-08 10:01:28 -07:00