Another thing I liked about this book is that at many places he admits that we just don’t know why certain aspects of ChatGPT work, in any deep theoretical sense. Elsewhere in the book, he points out how our understanding of ChatGPT is nowhere close to our understanding of how physical laws work. Then there are sections dealing with the inner working of ChatGPT, which I thought were the weakest; the concept of “attention” and “attention heads” wasn’t clear to me from his explanation.