starred/nanoGPT-karpathy-1
1
0
Fork 0
mirror of https://github.com/karpathy/nanoGPT.git synced 2026-04-17 23:23:24 +02:00
Code Issues Packages Projects Releases Wiki Activity

add benchmarking script v0

Browse Source
This commit is contained in:
Andrej Karpathy
2022-12-28 23:55:43 +00:00
parent 5d2b4807bf
commit 70b5d93aee
2 changed files with 52 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -57,3 +57,7 @@ and observe the following losses on train and val:
| gpt2-xl | 1558M | 2.56 | 2.54 |
I briefly tried finetuning gpt2 a bit more on our OWT and didn't notice dramatic improvements, suggesting that OWT is not much much different from WT in terms of the data distribution, but this needs a bit more thorough attempt once the code is in a better place.
## benchmarking
For model benchmarking `bench.py` might be useful. It's identical what happens in `train.py` except we're measuring just the fwd+bwd+update time of the model on a fixed random batch of data.