tl;dr
We now utilize existing Public Loops when creating new Loops.
What is RAG?
Retrieval augmented generation, or RAG, is a technique for increasing the reliability and accuracy of large language models (LLMs) by including external information/data in the context of the LLM call.
How does this help create Magic Loops?
Previously, all Loop generation was one-off. That is, each time a user creates a Loop, they work with the Loop Creator to make a new Loop from scratch.
We’re pretty happy with how well this works today, but we’ve had RAG on our roadmap for many months now.
The idea is simple: once a user publishes (and makes public!) a Loop, we can then use that existing user-verified logic when creating new Loops.
Example
Let’s start with a simple (and popular) example Loop description:
send me the top 5 stories on hackers news
Before RAG
Previously, this request would frequently rely on a 3rd party API to grab Hacker News data:
This may work, but isn’t great. The 3rd party data may not be up-to-date, may go out of service, and generally shouldn’t be trusted.
After RAG
Now that we’ve implemented RAG, the Loop Creator finds an existing Loop that’s similar, and correctly scrapes the page instead:
Success! 🎉
How does it work?
We start by indexing all of the Public Loops into a vector database; we’re using Pinecone.
To index the Loops, we first need to get the vector embeddings for the Loop descriptions. To do this, we use OpenAI’s text-embedding-ada-002 model.
Once we have the vector embeddings, we send the Loop data to Pinecone.
When a user creates a new Loop, we then compare the new description embedding with similar embeddings within Pinecone.
For any matches (note: currently just the top match), we include the Loop details in the context of the Loop Creator’s LLM calls. This helps the Loop Creator see how a similar Loop was implemented.
How good is it?
Great question! There’s still a lot to be done and measured.
The initial implementation is about as simple as you can get. We actually created this feature at a hackathon, so although it works decently today, there are many tweaks we can make to improve the retrieval, improve the indexing, and improve the Loop Creator’s grasp of the additional context.
The good news is, the more Loops that get created, the better the Loop Creator will become!
Curious to see RAG in action? Create your first Loop today.