Except I generally do know when I'm wrong because I'm working in a domain I am familiar with, and it will often create experiments on the fly unprompted (well, prompted, but generically in AGENTS.MD) to check itself. My experience actually using it for software is that it almost never makes up answers. The answer for hallucinations is fairly simple: give it facts and tools to ground itself.
> Except I generally do know when I'm wrong because I'm working in a domain I am familiar with, and … My experience actually using it for software is that it almost never makes up answers.
Yes I am certain that it feels that way. However empirical testing holds a lot more weight than anecdotes.
> The answer for hallucinations is fairly simple: give it facts and tools to ground itself.
The entire danger here is that it hallucinates when you don’t know the ground facts. After all, you don’t know what you don’t know.
Right now at least, if empirical testing shows poor results and anecdotal usage shows great ones, I'd say that in fact anecdotal evidence holds much more weight. The empirical testing is just bad.
So you make it demonstrate the ground facts. Show formal proofs, link to and reference specific documents and fragments in a document database. It wouldn't really be that hard to build a harness with specific support for these things (e.g. performing a tool call that then has UX to highlight fact-checked claims and link to details/references).
