WTF is Retrieval Augmented Generation for AI chatbots and large language models?

AI-powered chatbots are scary smart, thanks to the large language models undergirding them. But they’re only as smart as the data they’re trained on.

Two major factors can inhibit the AI chatbots being created by brands and publishers: the timeliness of the bots’ training data and the inclusion of proprietary information in that data set. In the first case, a company would need a way to pass information in real time to an LLM powering its chatbot. And in the second case, the company would need to be able to safeguard that information so that the LLM is just provided whatever necessary context to respond someone’s prompt. In both cases, a system called “retrieval augmented generation” would provide a framework for bridging the gap, as explained in the video below.