Embedchain abstracts the entire process of loading a dataset, splitting it into parts, creating vector representations (embeddings), and storing them in a vector database.
All of this could previously be done using Python code and the OpenAI API, but it required a deep understanding of how embeddings work, calculating distances between vectors, etc. Embedchain simplifies the entire process, reducing it essentially to a code of a couple of dozen lines.
Imagine you want to create a Naval Ravikant bot based on information from 1 YouTube video, 1 PDF book, and 2 of his blog posts, as well as a few pairs of question-answer that you uploaded. All you need to do is add links to the video, PDF, and blog posts, as well as a couple of question-answer pairs, and Embedchain will create the bot for you.
Embedchain supports the following formats: YouTube videos, PDF files, web pages, text, and question-answer pairs.
The tech stack on which all this works:
- Langchain as the LLM framework for loading, splitting, and indexing data;
- OpenAI's Ada embedding model for creating embeddings;
- OpenAI's ChatGPT API as the LLM for getting answers, considering context;
- Chroma as the vector database for storing embeddings.
👨💻 Github: https://github.com/embedchain/embedchain
