datalevin

Sigve 2025-05-08T06:43:10.053459Z

I am trying to build a search index like done in the https://github.com/juji-io/datalevin/blob/master/benchmarks/search-bench/src/datalevin/bench.clj, using a search-index-writer . How do i get the document from the kv store after getting the ref(s) from the d/search query?

Sigve 2025-05-12T07:43:41.662169Z

Thanks for the answers. I misunderstood and did not realise that the search-engine was separate in addition to the actual storage of the documents

Huahai 2025-05-09T16:13:06.417349Z

Don't understand what you are trying to do

Huahai 2025-05-09T16:14:58.442819Z

ref is whatever you put in to identify a document.

Huahai 2025-05-09T16:15:43.017179Z

by default, the document is not stored in the search engine, because we assume you know how to get the document given the ref.

Huahai 2025-05-09T16:16:17.642639Z

however, you can store the document in the search engine, by setting :include-text? true

Huahai 2025-05-09T16:18:20.615069Z

When that's the case, you can also return the matching documents in the search results by setting :display :texts in your search opts

Huahai 2025-05-09T16:19:49.393169Z

Of course, these are not default behaviors as they are slower. The default behavior expects that you know how to get the document given the ref. After all, it's you who come up with a ref for a document.

Huahai 2025-05-09T17:11:53.997709Z

our search benchmark took the default approach, i.e. we don't store documents and don't retrieve the documents either

Huahai 2025-05-09T17:12:49.113109Z

we only retrieve the refs, in this case, the wiki page URL.

Sigve 2025-05-08T06:43:52.093749Z

(d/get-value kv "datalevin/docs" ref) only returns 0. Am i missing something?