How to use it with chroma db?

#4
by abdimussa87 - opened

I wanted to provide this model as an embedding function to chroma db. How can I do that?

juliuslipp changed discussion status to closed

Thank you

@juliuslipp One additional question I have is how I should add the prompt for retrieval. Should I add it to the documents I want to embed in chroma db as the first entry?

Mixedbread org

@juliuslipp One additional question I have is how I should add the prompt for retrieval. Should I add it to the documents I want to embed in chroma db as the first entry?

Hi @abdimussa87 , for the retrieval task, you need to add a prompt to the query, no need to add it to documents.

Hey @mixed-nlp , where should I add the prompt when I set up my retriever, like below:

vector_store = Chroma.from_documents(unique_docs, 
                           embedding=embedding_functions.SentenceTransformerEmbeddingFunction(model_name='mixedbread-ai/mxbai-embed-large-v1'),
                           persist_directory='../data/chroma-dbb')

retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
Mixedbread org

Hey @abdimussa87 , the prompt is added to the query. Here's an example:

prompt = "Represent this sentence for searching relevant passages: "
query = "What is mixedbreads fav. bread?"

# Pass this to retriever:
combined_query = prompt + query
Mixedbread org
This comment has been hidden

@juliuslipp I was using LCEL and this is my chain:

chain = (
            {"context": retriever, "question": RunnablePassthrough()}
            | self.prompt
            | self.llm
            | StrOutputParser()
        )
chain.invoke(combined_query)

If I pass the combined query as you described, it'll add that prompt to the LLM as well, which might mislead the LLM when generating an answer.

Sign up or log in to comment