Colab: https://drp.li/T0SDK
My Links:
Twitter – https://twitter.com/Sam_Witteveen
Linkedin – https://www.linkedin.com/in/samwitteveen/
Github:
https://github.com/samwit/langchain-tutorials
https://github.com/samwit/llm-tutorials
00:00 Intro
00:47 Self Querying Diagram
01:55 Code Time
#langchain #openai #llm
30 Comments
Perfect. Thank you.
this + an additional chain that takes user input and does some kind of automatic entity extraction and inserts them back into the vector database = best second brain software ever
I implemented RNG to submit together queries to make my potty mouth AI bots even more toxic. It's almost like jailbreaking them. I will remember this video for more professional things tho. Very cool!
Fruity nodes? I thought it was fruity notes, but maybe AI has different opinions about that.
Another way of putting it is that you should only be using LLM as a classifier, in order to resolve ambiguity of NL queries, then translate them into normal database queries (which you can also understand and debug, unlike all the magic LLM output). While it is clearly taking a step back from the ridiculous attempts at placing AI in the driver seat, this approach actually works for real applications and brings added value… just like OCR, voice, or image recognition brings added value to tasks that require such capabilities. And when you need reasoning, idk, maybe use a FOL theorem prover or something.
I'm also not sure why you would use some dedicated weird vector db when you can just add vector functions to a production-ready db like ClickHouse.
is this a query builder? what is new here i don't get it? It's been there since gpt 3.5 went out.
The real challenge is the text search where when you have "apple" in your field, and the user searches for "fruit", and does it in sub 100ms.
Waiting for next video with open source model and with csv data
a plylist for this would be very useful
One major problem that cost me a great deal of time, I was trying to use the TogetherAI service to host the back end of my builds. Together is a fantastic service, but it can't do Langchain. It does not give you any kind of an error, it just does not work.
All of these demos are fine, but can we have a sample where function calling, these meta data works in tandem. Else these only work in isolation and not in reality. This will be video to watch. With real ingestion
I am building a pilot of Q&A chatbot for physicians based on RAG – thank you for video.
love your work
Great explanation on the why and how of Self Querying Retrieval!
My RAG app retrieves from 2 types of documents. I want the LLM to first look in document A, and then only look in document B if it can't find the answer. I was trying to insert meta fields to distinguish the document types, and came across Langchain's self-querying. But I think I'm barking up the wrong tree.
Any suggestions for my use case?
Great video Sam. Keen to uncover more of the hidden gems in langchain
Great Video❤
Video request or hint appreciated:
I would be interested in a video on a chatbot which builds up questions to narrow down search results on my unstructured data.
Whoa! I always thought that semantic searcher utilizes metadata automatically. Meaning, in your example, I thought that normal vectorstore.relevant_documents will automatically pick the relevant documents by looking to the content+metadata without help from LLM.
I assume that LangChain sends attribute info and ask LLM: make me filter with operators and used attributes in question I sent u. So again it is semantic query. Somehow example isn't crystal clear. It is blurry what should be treated as NOSQL query/metadata (lot of manual work included) as easily we can have flavour as meta data. Anyway I bet combined search will yield speed, even with relations databases as you can easily can translate semantic question to SQL query.
Excellent explanation. Thank you for sharing! I would like to know if Langchain is a good choice for production. Can you tell me, please?
hey sam have you seen the latest open source model CulturaX
Hi Sam, how would you implement semantic search with data reference when retrieving results over a large number of different documents?
Nice feature of langchain, but it works only if the metadata are not in the scattered in the documents itself…. Thanks for the video👍
How is this advanced??
Great video, as always.
Awesome content,please post a video on how to chat with google drive documents.
Great insight. Content like this helps a lot to keep the costs low. Thanks 🎉
Great video!
This was a fantastic walk through and explanation thanks
Awesome, video!! That's so useful!! Does Self Querying Retrieval perform sorting as well? For example: "What are the oldest wines from France?"
I think the topic of automatic metadata creation for Documents worth some time, for example Langchain's tagging.