datalevin

mdiin 2024-11-14T07:13:32.847349Z

When querying a db with fulltext search, are the returned rows always returned in order of search relevance? And is that order stable across pagination? I’m experimenting with using datalevin as a simple file backed database for a product search service, and pagination support is quite high on the list of business requirements. 😊

Huahai 2024-11-14T07:16:04.352459Z

fulltextfunction does return in the order of relevance, however, Datalog query results are sets.

mdiin 2024-11-14T07:16:33.099419Z

I’m really enjoying how simple datalevin makes implementing search functionality, and database access in general. Next on my list of databases to try is SQLite, but that looks quite a lot more work for something that is ultimately less customisable.

āž• 1
Huahai 2024-11-14T07:16:56.301939Z

after the joins, the order may have changed. Unless there's no further joins.

Huahai 2024-11-14T07:18:41.650099Z

good thing is that fulltext function call normally execute late in the pipline, so it might be ok if you don't have other complicated query functions that involves multiple variables.

Huahai 2024-11-14T07:19:39.265369Z

so, it really depends on your query.

mdiin 2024-11-14T07:21:27.614759Z

Does that mean that it is important which position the fulltext has in the where clauses, or that I cannot control it? If I keep the query and inputs stable and only update the :offset value, the set of returned rows should preferably be the next page of search results as ordered by the fulltext search.

Huahai 2024-11-14T07:21:33.623679Z

if you don't use your fulltext result variables in other clauses, and just return it, you probably can preserve the order.

Huahai 2024-11-14T07:21:43.729929Z

no, clause order doesn't matter

Huahai 2024-11-14T07:21:50.687709Z

you don't have control

Huahai 2024-11-14T07:22:42.682839Z

fulltext function call will be inside variable bound clause, so it is always executed late.

mdiin 2024-11-14T07:23:04.420929Z

Thank you, I will have to ponder this and make some experiments.

Huahai 2024-11-14T07:23:06.055539Z

the question is, are you going to use those variables in other clauses or not

Huahai 2024-11-14T07:24:00.979579Z

:offset will be working on the cached results, so basically it's the same results

Huahai 2024-11-14T07:24:23.198039Z

unless you have a transaction happened in between

Huahai 2024-11-14T07:25:24.855849Z

then cache is destroyed. However, if data is the same, the output rows are in the same order

Huahai 2024-11-14T07:25:35.264539Z

there's no randomess

Huahai 2024-11-14T07:26:24.766199Z

just try it

mdiin 2024-11-14T07:28:54.756039Z

Nice! Will do. I think I want to use pull in the :find part of the query to get data about the entities returned by the fulltext, but from how I read your answers that should be fine, as long as I am not using the results from fulltext in other where clauses.

mdiin 2024-11-14T07:29:03.934159Z

Thank you for the speedy replies! šŸ™

Huahai 2024-11-14T07:30:10.773509Z

let me know how it goes.

šŸ‘ 1