Fork me on GitHub
#datomic
<
2017-02-08
>
rabbitthirtyeight03:02:21

I'm trying to use the new string temp id feature but keep getting this error:

:db.error/not-a-keyword Cannot interpret as a keyword: userid, no leading :
   {:db/error :db.error/not-a-keyword}
The transaction:
[{:db/id "userid"
 :user/email ""}

{:widget/purchased-by "userid"}]
The reftype is defined as:
:widget/purchased-by {:db/valueType :db.type/ref
                                        :db/cardinality :db.cardinality/one}
Does anyone see something obvious that I'm missing?

favila05:02:18

@rabbitthirtyeight what is the db/id for your second map?

rabbitthirtyeight05:02:54

@favila you mean in the transaction? At this point I've actually got the temporary string id working on the widget, so it's id is being set as

[{:db/id "userid"
 :user/email ""}

{:db/id "widget-id"
 :widget/purchased-by "userid"}

{:thing/has-widget "widget-id"]]
And I've gotten it to where "widget-id" is creating a temp id, but for some reason I still can't get it working with "userid" (I eventually just got the transaction working by reverting to the macro syntax for the user temp id)

rabbitthirtyeight05:02:23

So now I can use a string temp id and use it with this attribute:

:thing/has-widget {:db/valueType :db.type/ref
                :db/cardinality :db.cardinality/one}
But not this one:
:widget/purchased-by {:db/valueType :db.type/ref
                                        :db/cardinality :db.cardinality/one}

favila05:02:02

@rabbitthirtyeight is it any different if every map includes a :db/id pseudo-attr?

rabbitthirtyeight05:02:12

Hmm... good question.

favila05:02:34

so the third map in your last example, second one in your first example

rabbitthirtyeight05:02:32

That... fixed it!

rabbitthirtyeight05:02:30

Do you have any idea why that was the issue?

favila05:02:59

seems like a bug to me

favila05:02:57

but my hunch was that without a db/id to anchor the assertions it was not invoking the string-tempid-as-value semantics

favila05:02:53

and trying to parse as ref would normally be, i.e., a map, a tempid object, a lookup ref, or a keyword

rabbitthirtyeight05:02:04

Yeah that does look like what was happening. In the morning I'll see if I can reduce it to a simple reproducible case (my actual transaction had more noise in it)

favila05:02:10

Definitely seems like a bug to me, so file an issue on cognitect's zendesk support page if you can make a small test case

favila05:02:10

there were a number of really silly bugs related to string tempids and auto-tempids fixed in last two releases, would not be surprised if there are more

rabbitthirtyeight05:02:41

Will do. Now that that's settled I'm going to bed. Thanks again for your help!

pesterhazy08:02:52

It used to be that you would always need to specify :db/id. Is that changed in the new datomic version?

karol.adamiec09:02:56

@pesterhazy yes, it is inferred now

karol.adamiec09:02:08

same with db.install/alter

karol.adamiec09:02:39

section about tempids

souenzzo11:02:10

how to compare instant's? Example: I want all :user/name changed after login1. Some like [find ?e :in $ ?login :where [?e :user/name _ ?tx][?tx :db/txInstant ?inst][?login _ _ ?txLogin] [?txLogin :db/txInstant ?loginInst] [(> ?inst ?loginInst)]] But not sure on (> ?inst ?loginInst)

rauh12:02:05

@val_waeselynck I woudln't recommend doing this. With a proper transaction function you'll see exactly what has changed and have nice transaction, so you have a nice paper trail.

rauh12:02:47

I implemented this a while ago:

rauh12:02:10

It's for refs but can be modified for values.

karol.adamiec12:02:54

@val_waeselynck have the same. I need/want semantically have a collection, but instead of managing the items i always set in a new collction. I do that using a ref attr that is removed before update with ':db.fn/retractEntity’ , then new one is inserted. With careful management of isComponent works fine and no leftovers are in db, and retract entity does not remove too much as well 🙂. But if there is a nicer apttern i am all ears as well.

pesterhazy12:02:35

I've had the same issue in the past

pesterhazy12:02:25

in an admin interface, a product has 0..n features. A user can add and remove features. When she clicks Save, I compute the diff to the previous state, retract the obsolete ones and assert the novel ones.

pesterhazy12:02:10

I don't actually use a transactor fn for this, though using that would be safer I suppose

karol.adamiec12:02:41

think it depends on what the domain interest is. If one cares about data in collection to keep some kind of references to other entities (that are linked through entity id instead od domain keys like i.e. email) then one has to calculate the diffs and do minimal update. If the data in collection are not “linked” directly to any other part of system calculating the diff seems like accidental complexity. Then the delete/create seems more appropiate.

karol.adamiec12:02:28

but no idea about wider performance etc.. implications of this

rauh12:02:32

@karol.adamiec That's exactly what my db-fn above does. It even allows newly transacted items to be ref'd. Removing/re-adding means your history is messed up, which may or may not be bad depending on what your application is interested.

karol.adamiec12:02:18

@rauh yeah. i bookmarked that gist 😄. thanks. but still not a wizz in clojure enough to just swallow that bit of code 😄.

rauh12:02:33

@val_waeselynck To add why I would not do it like you suggest: You'll have non-used entities (the collection) after a while unless you carefully garbage collect.

karol.adamiec12:02:14

that is why isComponent in schema is crucial, and deleting using built in fn.

karol.adamiec12:02:03

but yeah, history will most likely be messed up

karol.adamiec12:02:26

@pesterhazy i think dbfn is a must for that. you are definitely at risk. when loads get high, mess will follow imo.

val_waeselynck12:02:32

thanks guys 🙂 don't hesitate to add these comments on SO for the future generations 😉

pesterhazy12:02:41

@karol.adamiec diffing is not unnecessary complexity in this particular case as you cannot retract and assert the same fact in the same tx

val_waeselynck12:02:23

@rauh I have implemented this fn too, and still find it limited. Could you elaborate on the drawbacks of the proposed approach?

val_waeselynck12:02:01

thanks hadn't seen your previous comment. Not sure garbage is that big an issue.

rauh12:02:18

@val_waeselynck I'd say your version is ok if you're fine with "quick 'n dirty", but the proper way (ie you have a proper history) is to do all this in a single transaction with all the proper minimal changes

rauh12:02:28

You do open yourself up to race conditions too.

rauh12:02:56

In what sense do you find the fn limiting?

val_waeselynck12:02:46

The fn kinda breaks the 'nested maps' writes for one thing

val_waeselynck12:02:13

it also has the limitation of running on the transactor, although I'm okay with that in this particular case.

val_waeselynck12:02:24

I don't see the race conditions you're mentioning.

rauh12:02:08

What do you mean with nested maps breaking? Can you give an example?

val_waeselynck12:02:42

@rauh going further with my example code:

pesterhazy12:02:31

but there could be an extended version

pesterhazy12:02:57

that accepts a vector of txs

rauh13:02:13

@val_waeselynck My version is for refs only, but there you def can use nested maps as much as you like, you just have to assign them temp-ids and pass them into the vector

val_waeselynck13:02:49

@pesterhazy which you'd have to parse in order to find the entity ids on which to perform the diffs... still not completely easy 🙂

pesterhazy13:02:52

and uses some special syntax marker to replace #{a b c} with #{:only! a b c}

pesterhazy13:02:39

not easy but it should be possible to implement this as a general reusable tx fn

pesterhazy13:02:27

that would be neat

pesterhazy13:02:57

not sure what to use as the syntax extension mechanism though

val_waeselynck13:02:06

Still, the tx-fn approach still feels a bit hacky to me. For some reason, I feel this is a contrived way of using a cardinality-many attribute - in my view cardinality-many attributes are design to model facts that don't collide with each other, not to model cohesive sets of entities

pesterhazy13:02:54

what would you use cardinality-many for then?

rauh13:02:44

@val_waeselynck Datomic very much encourages to use tx-fns. It's not hacky at all IMO. This isn't pg-sql.

val_waeselynck13:02:13

@rauh the db fn is not what disturbs me most

rauh13:02:40

If a user removes a favorite movie from his/her list then the proper transaction is to [:db/retract...] and not a complete rewrite IMO

val_waeselynck13:02:32

@rauh well it's not especially harder to retract from the list. That's what I like with this approach: that absolute and incremental operations are equally easy.

rauh13:02:39

Maybe down the road in a year you want to display the history of the user's favorite movies... You'll have to manually do the work with your approach.

val_waeselynck13:02:33

@rauh on contrary, I would argue that the intermediate-entity approach is more sound for that use case, because you don't have to resort to querying history, which has many limitations. The notion of "version" of your favorites movie list becomes a first-class citizen of your model (maybe at the cost of adding a couple more attributes).

val_waeselynck13:02:11

@pesterhazy In my view, there's a difference between "Stu likes pizza (among other things)" (for which cardinality-many attributes are well suited) and "There's a list of the things that Stu likes, which contains ..."

pesterhazy13:02:35

not sure I get the distinction

favila16:02:20

@rabbitthirtyeight I think this is the root of your problem last night:

(d/entid (d/db (d/connect "datomic:")) "str")
IllegalArgumentExceptionInfo :db.error/not-a-keyword Cannot interpret as a keyword: str, no leading :  datomic.error/arg (error.clj:57)
d/entid does not understand string tempids.

favila16:02:42

It should work the same way as tempids probably: (d/entid (d/db (d/connect "datomic:")) (d/tempid :db.part/user)) ;=> -9223350046623220288

rabbitthirtyeight16:02:57

Huh. Yep that's the error message.

favila16:02:45

Breaks one of my tx functions

nottmey17:02:45

Using the pull api with pattern [:release/country] I get {:release/country {:db/id 17592186045550}}. How do I get "enums" to automatically become either/or 1. {:release/country {:db/id :country/DE}} 2. {:release/country {:db/id 17592186045550 :db/ident :country/DE} 3. {:release/country :country/DE} I know that I could prefetch the schema and apply transformations on the pull result, but that seems counterintuitive.

favila17:02:36

@nottmey That is the only way

favila17:02:41

I retrieve :db/ident on the leaves and do a prewalk transformation

favila17:02:08

(->> (d/pull db '[{:release/country [:db/ident]}] eid)
     (clojure.walk/prewalk
       (fn [x]
         (if (and (map? x) (:db/ident x))
           (:db/ident x)
           x))))

nottmey17:02:42

ok ty, I see. There is no pattern to say “display all :db/ident regardless of which level of nesting” right?

nottmey17:02:06

Hmm I guess one could also walk over the finished pattern and insert :db/ident at each stage, before calling pull. Seems good enough.

favila17:02:47

yes, that's true

favila17:02:10

that is a schema-ignorant way of doing it (meaning you don't need to know which attrs you expect to have enum values)

favila17:02:44

I had been putting them in manually

favila17:02:54

based on which I expected to have enum values

nottmey17:02:12

I still need to know, which attributes are refs and * is still hiding ref attributes 😕 (but thats ok, I guess I need to do it the right way)

robert-stuttaford17:02:19

the reason it’s like this is because you can use :db/ident on any entity. it’s just useful as enums. pull allows you to work with idents for other-than-enums.

favila17:02:16

You can also include the db in the post-walk

favila17:02:25

that way all you need is a db id

favila17:02:30

(defn pull-enums [db pat eid]
  (->> (d/pull db pat eid)
       (clojure.walk/prewalk
         (fn [x]
           (cond
             (and (map? x) (:db/ident x)) (:db/ident x)
             (and (map? x) (:db/id x)) (or
                                         (d/ident db (:db/id x))
                                         x)
             :else x)))))

favila18:02:56

no need to change the pattern in this case

favila18:02:05

I think ident is atemporal though (looks at ident cache, not db snapshot)

nottmey18:02:21

@robert-stuttaford yea resolving anything else than an enum might not be a good idea, I exclude that corner case

favila18:02:51

maybe safer would be (:db/ident (d/entity db (:db/id x)))

nottmey18:02:26

the usecase is more like “pull entity tree with resolved enums” I like the “ask db again for ident” approach when it’s about very generic pulls. Would calling this for every leaf map be a bad idea?

favila18:02:08

if you include :db/ident in the pull, there is hardly any cost

favila18:02:26

if you don't you need a lookup which is likely to be in the object cache anyway

favila18:02:31

(and so fast)

favila18:02:39

so I doubt it makes a difference practically

robert-stuttaford18:02:07

as Rich says, why worry, when you can measure 🙂

favila18:02:13

(defn entity-pull
  "Like pull, but returns values consistent with d/entity, i.e.,
  entities with :db/ident are represented as keywords and sets are
  used instead of vectors."
  [db pat eid]
  (->> (d/pull db pat eid)
       (clojure.walk/prewalk
         (fn [x]
           (cond
             (and (not (map-entry? x)) (vector? x)) (set x)
             (and (map? x) (:db/ident x)) (:db/ident x)
             (and (map? x) (:db/id x)) (or
                                         (:db/ident (d/entity db (:db/id x)))
                                         x)
             :else x)))))

favila18:02:41

I've wanted that for other things, never thought about getting the ident after-the-fact

favila18:02:50

that technique makes it much easier to write

oscar20:02:03

I'm using the Datomic Client library and when I try adding the expression [(namespace ?ident) ?ns] as a where clause I get the error message "The following forms do not name predicates or fns: (namespace)". I tried namespacing the symbol but got the same error. When I run the same query in the Peer library, it works just fine. Does anyone have an idea of what I'm doing wrong?

favila20:02:34

clojure.core/namespace also does not work?

oscar20:02:23

Yeah. By the way Datomic Peer Server is a version old in case that has some significance.

az21:02:36

Hi all, can anyone point me in the right direction when it comes to building realtime web apps with clojure and datomic? Are there any frameworks I should look at?

zane23:02:01

I feel like Cognitect just announced something that fits this description. I'd check out their blog.

nottmey11:02:55

don't see anything fitting on their blog in the last months

nottmey19:02:24

ahh I know vase, is it supposed to cater to “realtime” apps?

zane19:02:03

I'm not sure! I haven't looked into it yet.

zane23:02:30

Hmm. datomic.api/pull-many is NPE-ing at me in one of my tests. Are there common gotchas I should be aware of when using it?

zane23:02:27

Oh damn. I think I might be hitting this issue:

zane23:02:05

> Datomic versions prior to 0.9.4699 cannot read adaptive indexes, and will fail with the following stacktrace: >

java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
at com.google.common.cache.LocalCache.getIfPresent(LocalCache.java:3988)
at com.google.common.cache.LocalCache$LocalManualCache.getIfPresent(LocalCache.java:4783)
at datomic.cache$eval2673$fn__2674.invoke(cache.clj:65)

zane23:02:09

Or something similar.

zane23:02:39

Our Datomic version is significantly newer than the one listed.