Clojurians Log v2

Can confirm on master the reproduction now gives me all three attributes!

if you believe there is a bug, please file an GitHub issue

will do; didn't know if this was a result of a mishandling on my part that was clear to someone more knowledgeable. edit: done https://github.com/datalevin/datalevin/issues/362

Thanks!

Let me know if there's anything you'd like me to run on my own (apparently cursed) environment to help debug- I'm still intermittently picking at different functions in the code-base to try and pin-point the breakdown: • I notice (di/fetch store (dd/datom 1 nil nil)) and (di/e-datoms store 1) as seen in db/-search returns the proper attributes when executed alone • Looking at wrap-cache itself, this form returns the `nil' attribute

(.get 
  (.get (deref #'ddb/caches) (di/dir store)) 
  [:search 1 nil nil])

These data points are very helpful. Will push a fix soon.

master branch has a fix. please test if it works for you.

Unable to reproduce in the 0.10.7 release, nor in current master branch

I can't say I'm surprised (the error seems so fundamental that I'd assume it'd be difficult to replicate, otherwise everyone would be sounding alarms on it). I have been bypassing it for now by putting every attribute in my db-schema, but it certainly is still happening- persisting through even machine restarts...super-odd. I'll instrument dtlv with flowstorm and see if anything jumps out at me edit: flowstorm starts to blow up fast with dtlv's background loop- that is going to be much harder to keep track of~

Hmm, it looks like the attribute might still be in the DB itself?

#datalevin.db.DB{
 :store #object[datalevin.storage.Store 0x4f8e7564 "datalevin.storage.Store@4f8e7564"], 
 :max-eid 0, :max-tx 1, 
 :eavt #{#datalevin/Datom [1 :company/home-page ""] #datalevin/Datom [1 :company/linkedin ""] #datalevin/Datom [1 :company/name "8AM Golf"]}, 
 :avet #{#datalevin/Datom [1 :company/home-page ""] #datalevin/Datom [1 :company/linkedin ""] #datalevin/Datom [1 :company/name "8AM Golf"]}, 
 :pull-patterns #object[datalevin.utl.LRUCache 0x70ab6a70 "datalevin.utl.LRUCache@70ab6a70"]}

confirming that :wal? false does not affect the error

sadly i can't easily introspect what's happening in the wrap-cache/`slice` macros (which is what actually returns the nil attribute)

looks to me a schema issue, can you show the schema?

just this, from above

(def db-schema {:company/name {:db/unique :db.unique/identity}
                :company/linkedin {:db/valueType :db.type/string}})

so home-page is not there, that's the bug

Datalevin is schema on write, apparently, this is somehow broken in your case.

Would need to have a repro to find the conditions for this to happen

ah im very sorry i misunderstood your question, i thought you were asking what i supplied as the initial schema

heres the output of d/schema

;; => {:company/linkedin #:db{:valueType :db.type/string, :aid 4},
;;     :company/name #:db{:unique :db.unique/identity, :aid 3},
;;     :db/created-at
;;     #:db{:valueType :db.type/long,
;;          :cardinality :db.cardinality/one,
;;          :aid 1},
;;     :db/ident
;;     #:db{:unique :db.unique/identity,
;;          :valueType :db.type/keyword,
;;          :aid 0},
;;     :db/updated-at
;;     #:db{:valueType :db.type/long,
;;          :cardinality :db.cardinality/one,
;;          :aid 2},
;;     :company/home-page #:db{:aid 5}}

oh, so home-page is there.

did you upgraded the db from prior version?

it should be as fresh as it comes, the path is a directory i made today

if this is fresh, then I don't know why it is not reproducible

definitely understand, im sure there's some ghost somewhere in my machine

any reason why we avoid running vld/validate-attr (which would throw an exception) in resolve-datom when the attribute is nil?

(defn- resolve-datom
  [db e a v default-e default-v]
  (when a (vld/validate-attr a (list 'resolve-datom 'db e a v default-e default-v)))
  (let [v? (some? v)]
    (datom
      (or (entid-some db e) default-e)  ;; e
      a                                 ;; a
      (if (and v? (ref? db a))          ;; v
        (entid-strict db v)
        (if v? v default-v)))))

because resolve-datom is for building patterns, it's possible a is nil for patterns, as nil means wildcard in patterns. We don't want to validate a nil attribute.

Otherwise, e _ _ pattern would blow up.

These patterns are used as index boundary (range query)

ahhh I see I see, hmm. Noticing the call below to slice alone (outside of wrap-cache) does seem to return the tuples I expect

(di/slice
    (->> @conn :store)
    :eav
    (#'ddb/components->pattern db :eav 1 nil nil 0 nil)
    (#'ddb/components->pattern db :eav 1 nil nil 0x7FFFFFFFFFFFFFFF nil))
  
;; => [#datalevin/Datom [1 :company/name "8AM Golf"]
;;     #datalevin/Datom [1 :company/linkedin "https: //www.linkedin.com/company/8am-golf/"] ; uri's edited to avoid slack embeds, not an error
;;     #datalevin/Datom [1 :company/home-page "https: //8amgolf.com/"]]

trying to call it with wrap-cache but REPL is complaining about caches being private

Ok, so it's wrap-cache which seems to be causing the voided attribute

(alter-meta! #'ddb/caches assoc :private false)
(ddb/wrap-cache
  (->> @conn :store)
  [:datoms :eav 1 nil nil]
  (di/slice
    (->> @conn :store)
    :eav
    (#'ddb/components->pattern db :eav 1 nil nil 0 nil)
    (#'ddb/components->pattern db :eav 1 nil nil 0x7FFFFFFFFFFFFFFF nil)))

;; => [#datalevin/Datom [1 :company/name "8AM Golf"]
;;     #datalevin/Datom [1 :company/linkedin ""]
;;     #datalevin/Datom [1 nil ""]]

👍 will take a look

another data-point: (d/touch (d/entity db 1)) shows that attribute nil'd out as well, so doubling down on 'maybe cache-related'; if the lone slice call didn't return the correct tuple I'd think the attribute really was gone in the DB

ah, the datalevin.db/-search in touch is what returns the nil attribute (edit: which I see also uses wrap-cache, which makes sense~)