2019.09.20 04:12
[z Internetu] GPT-2 się wykoleił
Ciekawy fragment przeczytałem ostatnio w jednej pracy na temat deep learningu: https://arxiv.org/pdf/1909.08593.pdf (luźniejsze omówienie jest tu: https://openai.com/blog/fine-tuning-gpt-2/ ). Kiedy będziecie to czytać, zwróćcie uwagę na ten fragment, co brzmi, jakby go Lem napisał:
One of our code refactors introduced a bug which flipped the sign of the reward. Flipping the reward would usually produce incoherent text, but the same bug also flipped the sign of the KL penalty. The result was a model which optimized for negative sentiment while preserving natural language. Since our instructions told humans to give very low ratings to continuations with sexually explicit text, the model quickly learned to output only content of this form. This bug was remarkable since the result was not gibberish but maximally bad output. The authors were asleep during the training process, so the problem was noticed only once training had finished. A mechanism such as Toyota’s Andon cord could have prevented this, by allowing any labeler to stop a problematic training process.
komentarze:
powrot na strone glowna
RSS