Why Language Models Are So Hard To Understand

You don’t typically build a machine without understanding how it works. But for artificial intelligence researchers building large language models, understanding is about the one thing they haven’t achieved. In fact, sometimes their work feels more like gardening than engineering. “Put a tomato seed into the ground and you get a tomato plant,” said Martin Wattenberg, a language model researcher…
Source