How to construct filters
This guide assumes familiarity with the following:
We may want to do query analysis to extract filters to pass into retrievers. One way we ask the LLM to represent these filters is as a Zod schema. There is then the issue of converting that Zod schema into a filter that can be passed into a retriever.
This can be done manually, but LangChain also provides some βTranslatorsβ that are able to translate from a common syntax into filters specific to each retriever. Here, we will cover how to use those translators.
Setupβ
Install dependenciesβ
- npm
- yarn
- pnpm
npm i zod
yarn add zod
pnpm add zod
In this example, year
and author
are both attributes to filter on.
import { z } from "zod";
const searchSchema = z.object({
query: z.string(),
startYear: z.number().optional(),
author: z.string().optional(),
});
const searchQuery: z.infer<typeof searchSchema> = {
query: "RAG",
startYear: 2022,
author: "LangChain",
};
import { Comparison, Comparator } from "langchain/chains/query_constructor/ir";
function constructComparisons(
query: z.infer<typeof searchSchema>
): Comparison[] {
const comparisons: Comparison[] = [];
if (query.startYear !== undefined) {
comparisons.push(
new Comparison("gt" as Comparator, "start_year", query.startYear)
);
}
if (query.author !== undefined) {
comparisons.push(
new Comparison("eq" as Comparator, "author", query.author)
);
}
return comparisons;
}
const comparisons = constructComparisons(searchQuery);
import { Operation, Operator } from "langchain/chains/query_constructor/ir";
const _filter = new Operation("and" as Operator, comparisons);
import { ChromaTranslator } from "langchain/retrievers/self_query/chroma";
new ChromaTranslator().visitOperation(_filter);
{
"$and": [
{ start_year: { "$gt": 2022 } },
{ author: { "$eq": "LangChain" } }
]
}
Next stepsβ
Youβve now learned how to create a specific filter from an arbitrary query.
Next, check out some of the other query analysis guides in this section, like how to use few-shotting to improve performance.