diff --git a/README.md b/README.md index 3d0d1c6..c521e06 100644 --- a/README.md +++ b/README.md @@ -46,3 +46,14 @@ accelerate launch scripts/train.py --config config/dgx.py:aesthetic ``` If you want to run the LLaVA prompt-image alignment experiments, you need to dedicate a few GPUs to running LLaVA inference using [this repo](https://github.com/kvablack/LLaVA-server/). + +## Reward Curves + + + + + +As you can see with the aesthetic experiment, if you run for long enough the algorithm eventually experiences instability. This might be remedied by decaying the learning rate. Interestingly, however, the actual qualitative samples you get after the instability are mostly fine -- the drop in the mean is caused by a few low-scoring outliers. This is clear from the full reward histogram, which you can see if you go to an individual run in wandb. + + +