

Calling it a web crawler is just innacurate. You can give it access to a web search engine, which is how the “AI search engines” work, but LLMs can’t access the internet on their own. They’re completely self-contained unless you give them tools that let them do other things.
Yeah, I can see that. It’s definitely more like a search index than a web crawler. It’s not great at being a search index though, since it can synthesize ideas but can’t reliably tell you where it got them from in the first place.