PhillipOrdonez wrote: ↑22 Apr 2025
Gonna have to push back there against your claim of me hallucinating there. What you said on the quoted post (all of it not just the quotes I’m providing here, means those KGs are used by the LLMs to stay on script, or avoid errors (checking for errors).
Well, it might be easier if you learned how LLMs, neural networks, and KGs work (as I just might be doing a terrible job of explaining
).
Now, what I've written is not a complete description of how all of the systems work and work together, so it's inevitable that you MUST fill in the gaps between what I've written about the systems, and how the systems really work, right?
It's near impossible for your mental model to be correct based solely on what I've written, because I didn't use enough words (and diagrams) to fully communicate the intricacies of the systems.
Hallucination on your part is unavoidable. It's not a fault of your own, but a mere consequence of the communicative context (and the impracticality of people communicating verbosely enough to completely transmit a complete description of all involved phenomena).
But:
1. LLMs don't "use" KGs.
2. KGs don't necessarily "help them stay on script".
3. KGs aren't used to check for errors (or necessarily help them "avoid" errors).
What happens instead is that:
1. KGs are being used
with LLMs (though they don't have to be used with LLMs).
2. The typical approach to using them together is that what you write is
converted into queries into the Knowledge Graph database, a bunch of facts are spat out, those facts are presented to another LLM, which it then
summarises.
Think of it as a sleight of hand, or a changing of responsibilities. Instead of relying on the LLM to construct an answer, it is instead used as a tool to
find the answer (or possible answers), and then again to summarise the answer to you (but an LLM is less important at this stage, and more manual processes created by humans could be used).
But the error rate of LLMs in these tasks is much much lower than using LLMs on their own.
The real work is in building the Knowledge Graph (and in specific domains, they can be writen by humans or generated by LLMs and validated by humans).
For example, we had task to build data models from a long ass specification supplied to us in HTML4 documentation (usually done by hand and would have taken months). I wrote code to transform the HTML4 documentation into the data model. Errors in the HTML4 documentation (such as illegal characters, etc.) resulted in errors in the data model (plus some incompatibilities). A human (me) corrected those few errors manually.
While algorithms
could "hallucinate", it actually did a lot better than humans, who frequently made mistakes.
Similar things are done when building Knowledge Graphs in business domains. When the corpus is prohibitively large, human validation is hard limited, and used to correct edge cases, course correct the way the algorithm does the transformation (to reduce erroneous KG representations), and evaluate the error rate.
Before LLMs, chatbots were built with handwritten scripts and mailmerge-like placeholders.
Anyway, the increase in accuracy (and reduction in error) is a consequence of the LLM not being used entirely, and instead they are used to merely
translate the semantic knowledge to you as an answer - a task that yields fewer
errors.
Now, naive systems that uses only LLMs to produce KGs, produce KG queries, and then summarise them to you is just less erroneous than building an LLM based on that knowledge to answer your question directly.
This is because converting text to semantic knowledge, and then summarising that semantic knowledge back to text (after being queried) is an inherently less ambiguous task than constructing a statistical model to perform the same function on its own.
Still, what I've written is not a complete course in neural networks, knowledge graphs (though KGs are much simpler to explain and to intuitively grasp), and transformer models (or whatever variant is being used by the newest version of ChatGPT).
PhillipOrdonez wrote: ↑22 Apr 2025
Sure people misunderstand each other in communication all the time. You think that and an LLM hallucinating is the same?
How are they any different?
PhillipOrdonez wrote: ↑22 Apr 2025
If I upload a shopping receipt and the LLM starts turning it into a table, which is basically copying and pasting text, but the things it puts on the table are items I’ve never bought, how is that the same as someone misinterpreting my words????? If you have the same task to an assistant and they started doing the same thing, you would never think “oh, they’re misinterpreting my words”, would you? Really?
LLMs don't read receipts. They process words (well, technically they process "vector embeddings").
What happens is that ChatGPT hands your image over to some image analysis system, which spits out whatever it was designed to spit out.
So, you could be using a version of ChatGPT that uses an image recognition system that only spits out summaries, e.g. "this is a shopping receipt showing the date, a barcode, and a Walmart logo".
A newer version of ChatGPT (which you might need to pay more for or change settings) might use a different image recognition system that outputs character recognition data, which an LLM could use.
The LLM itself can only work with what it is provided, hence why I suggested using other tools designed for that.
♂️
I don’t understand how are businesses using it for anything useful other than content creation for marketing… which is simple shit already… gah. I guess the only ones benefiting from using this crap are teachers creating Images to aid in teaching… and I guess, as per Re8et, coding (?) other than that, this is lame.
Share and download patches!