Fork me on GitHub
#datasplash2021-08-17
domparry07:08:46

Hi datasplashers!

domparry07:08:25

I have a change in my forked version of the lib. It's for datastore, when creating entities.

domparry07:08:33

(defn- make-ds-entity-builder
  [raw-values {:keys [exclude-from-index] :as options}]
  (let [excluded-set (into #{} (map name exclude-from-index))
        ^Entity$Builder entity-builder (Entity/newBuilder)]
    (doseq [[v-key v-val] raw-values]
      (.put (.getMutableProperties entity-builder)
            (if (keyword? v-key) (name v-key) v-key)
            (let [^Value$Builder val-builder (make-ds-value-builder v-val)]
              (-> val-builder
                  (cond->
                      (or (excluded-set (name v-key))
                          (and (string? v-val) (> (alength (.getBytes v-val)) 1500)))
                      (.setExcludeFromIndexes true))
                  (.build)))))
    entity-builder))

domparry07:08:01

Currently you can pass in a set of fields to be excluded from indexes, but this only works for top level entities.

domparry07:08:33

So I have added a check to look for strings that are larger than the allowed 1500 bytes, and those are also excluded from indexes.

domparry07:08:18

Perhaps I can add this to a PR. But I'm thinking it should be opt in? Perhaps a flag that can be passed in.