"Humans cannot retain perfect recall of all information, therefore they cannot learn" is what you're saying. Because recall is limited learning doesn't exist us a hell of a position.
Not really but okay. They don't self-improve because they don't keep any results of in-context learning. That's the point. Yes you can fake it with some sort of longer form memory but you still run into the context limit eventually, and summarized information doesn't have the same effect on inference as the full context anyway.
It's also simply ridiculous to compare it to humans since LLMs don't work the same way at all - and this just highlights my point, because in human brains learning results in reconfiguration of connections between neurons, i.e., long term self-improvement. The analogous form in LLMs would be reconfiguring their weight matrices run-to-run, which I already addressed (doesn't exist right now).
"Humans cannot retain perfect recall of all information, therefore they cannot learn" is what you're saying. Because recall is limited learning doesn't exist us a hell of a position.
Not really but okay. They don't self-improve because they don't keep any results of in-context learning. That's the point. Yes you can fake it with some sort of longer form memory but you still run into the context limit eventually, and summarized information doesn't have the same effect on inference as the full context anyway.
It's also simply ridiculous to compare it to humans since LLMs don't work the same way at all - and this just highlights my point, because in human brains learning results in reconfiguration of connections between neurons, i.e., long term self-improvement. The analogous form in LLMs would be reconfiguring their weight matrices run-to-run, which I already addressed (doesn't exist right now).