I've been trying to come up with a simple explainer about how chatbots work, and I'm finding it really difficult to convey the non-deterministic aspects of the technology.
Let's say you're an alien from outer space. You don't understand human language, but you have several hundred years of intercepted radio signals from Earth in your supercomputer. When you finally visit Earth and meet your first humans, your supercomputer makes an educated guess as to what words to use to greet them. Unfortunately, neither you nor your computer know that "to serve mankind" can be understood as either assisting humans or cooking and eating them. Diastrous hilarity ensues.
That's sort of what a chatbot is. It's a computer program that can string words together into a convincing sentence by predicting which words to use. It can do this because it's studied the whole of human language and knows which words usually precede and follow other words.
Usually. Because sometimes it predicts the wrong word, and that single mistake might lead it down a path where the sentence it gives you is grammatically correct but factually wrong.
If you ask a chatbot for a summary of the life of Ben Franklin, it'll probably get most of it correct because the chatbot has studied all of Wikipedia. But because it's predicting what words to say to you, there's always the chance that it'll guess wrong and suddenly your report on Ben Franklin includes a section on how he appeared on season three of The Office.
It's unlikely a chatbot will reply "I don't know" because it doesn't actually know anything. It's not thinking about what it's saying. It can't think! It's designed to process your words and give back a reply. Not a true or accurate reply, just a reply.
That much alone is difficult enough to convey! And that's not even the worst part!