Files
ollama/llama/runner/README.md
2024-07-29 15:38:51 -07:00

16 lines
280 B
Markdown

# `runner`
A minimial runner for loading a model and running inference via a http web server.
```
./runner -model <model binary>
```
### Completion
```
curl -X POST -H "Content-Type: application/json" -d '{"prompt": "hi"}' http://localhost:8080/completion
```
### Embeddings