Stanford DSPy
DSPy is the framework for solving advanced tasks with language models (LMs) and retrieval models (RMs). It unifies techniques for prompting and fine-tuning LMs — and approaches for reasoning, self-improvement, and augmentation with retrieval and tools.
Provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax.
Introduces an automatic compiler that teaches LMs how to conduct the declarative steps in your program.
Solvio can be used as a retrieval mechanism in the DSPy flow.
Installation
For the Solvio retrieval integration, include dspy-ai with the solvio extra:
pip install dspy-ai dspy-solvio
Usage
We can configure DSPy settings to use the Solvio retriever model like so:
import dspy
from dspy_solvio import SolvioRM
from solvio_client import SolvioClient
turbo = dspy.OpenAI(model="gpt-3.5-turbo")
solvio_client = SolvioClient() # Defaults to a local instance at http://localhost:6333/
solvio_retriever_model = SolvioRM("collection-name", solvio_client, k=3)
dspy.settings.configure(lm=turbo, rm=solvio_retriever_model)
Using the retriever is pretty simple. The dspy.Retrieve(k) module will search for the top-k passages that match a given query.
retrieve = dspy.Retrieve(k=3)
question = "Some question about my data"
topK_passages = retrieve(question).passages
print(f"Top {retrieve.k} passages for question: {question} \n", "\n")
for idx, passage in enumerate(topK_passages):
print(f"{idx+1}]", passage, "\n")
With Solvio configured as the retriever for contexts, you can set up a DSPy module like so:
class RAG(dspy.Module):
def __init__(self, num_passages=3):
super().__init__()
self.retrieve = dspy.Retrieve(k=num_passages)
...
def forward(self, question):
context = self.retrieve(question).passages
...
With the generic RAG blueprint now in place, you can add the many interactions offered by DSPy with context retrieval powered by Solvio.
Next steps
Find DSPy usage docs and examples here.