So I’m trying to understand functions, queries and caching a bit better. I have two questions. *Question 1* I’m trying to filter for messages that have content longer than 200 characters:
(d/q '[:where
[?u :message/content ?content]
[(> (count ?content) 200)]
:find (count ?content).]
@*conn*)
=> nil
Doesn’t return anything. Why? I’m assuming you can’t nest functions?
Separating out the functions still doesn’t return anything either though:
(d/q '[:where
[?u :message/content ?content]
[(count ?content) ?n]
[(> ?n 200)]
:find (count ?content).]
@*conn*)
=> nil
If I implement my own greater than function this works:
(defn my> [a b]
(> a b))
(d/q '[:where
[?u :message/content ?content]
[(count ?content) ?n]
[(app.scratch/my2> ?n 200)]
:find (count ?content).]
@*conn*)
=> 5
Is this because > is special for range queries?
*Question 2*
If I implement a function for checking spam based on length like below:
(defn spam? [x] (> (count x) 200))
(d/q '[:where
[?u :message/content ?content]
[(app.scratch/spam? ?content)]
:find (count ?content).]
@*conn*)
=> 5
I get the expected result.
However, if I change the function and run the query again.
(defn spam? [x] (> (count x) 200000))
(d/q '[:where
[?u :message/content ?content]
[(app.scratch/spam? ?content)]
:find (count ?content).]
@*conn*)
=> 5
The above should return nil (as I have no content that length).
So some caching is happening, my question is what are the conditions for this sort of caching (also where’s the best place to look in the source code)? Is there a way to tell datalevin that the function has changed?Both issues create. I’ll see If I can create PRs for them in the next few days https://github.com/juji-io/datalevin/issues/287 https://github.com/juji-io/datalevin/issues/288
Question 1, the current built-in > does not resolve its arguments. Remember, we cannot call eval, so we need to walk the arguments to resolve them.
Question 2, the caching key is based on parsed-q data structure, looks like it should be based on the structure after the references inside are resolved.
There's no condition for caching, all query results are cached. Until a transaction happens to clear all cache.
the code is in query.clj
q-result function does the caching
could you file issues for these two?
PR for 288, but as mentioned this might not be something that is worth fixing (performance implications). https://github.com/juji-io/datalevin/pull/289
Merged
Thanks!