It'll be interesting to see how Google integrates these cognitive science ideas into its products. The cognitive science aspects are an improvement on Transformers ("attention is all you need") but remember, these are also theories about how memory and association work. I didn't see a mention of the ability to retrieve years old experiences in an exact "as if today" recall. So, lol, forgetting isn't everything.

But it does point out that we're on a roll here. Transformer architecture, perhaps most alarmingly its quadratic compute hunger, shows that better architectures are and will be developed. Even a bumblebee's brain doesn't need that many megawatts of power to understand flowers and which way is home.

So good! We're just getting started with generative artificial intelligence architectures. I look forward to further articles from you. You're much better at the math than I will ever be. TY!