Source
Training a coding model to paint watercolours with TRL and OpenEnv
Sergio Paniego open-sourced a reproduction of Surya Narreddi's project to train a language model to paint watercolours via JavaScript code. The project explores whether reinforcement learning can be used to train a model based on aesthetic taste.
Why it matters
It shows how AI can be steered toward specific artistic styles by curating a reference pool, shifting the focus from generating images to defining what is beautiful. This allows individuals to instill personal preferences into smaller models.
The details
- Training used the Qwen3.5-35B-A3B model with LoRA applied to all linear layers. - The reference pool contains 178 model-generated paintings divided into "love" and "okay" tiers. - The model is restricted to ten specific p5.brush methods to preserve the watercolour look. - Three reward mixes were tested to compare personal taste against general aesthetic preferences.
What's next
Future attempts may include implementing a multi-step visual feedback loop and testing if smaller 4B models can achieve similar results.
Get the weekly recap
The stories like this one, picked and explained — once a week, straight to your inbox.