Fork me on GitHub
#datomic
<
2016-06-03
>
bendy10:06:31

hey, I’m very new to datomic, and I’m trying to set up an auto incrementing field (I need a serial id for reference). I get the general idea of database functions, but how can I set up a field to be auto incrementing? Can anyone point me to a beginner friendly resource? 😄

viniciushana12:06:11

depending on the constraints you need for this number, perhaps the db/id itself may suffice

viniciushana12:06:29

you could keep an entity to control the sequential generation, always keeping the current value, and calling inc in a transaction fn alongside with persisted the incremented value at the entity using it (and also updating the value at the control entity).

pesterhazy12:06:30

a function I use to generate order numbers: (let [new-number (d/q ' [:find (max ?number) . :in $ :where [_ :order/number ?number]] db)] [{:db/id eid, :order/number (inc (or new-number 0))}])

viniciushana12:06:18

my experience with using max for high insertion volumes is not so good 😕

viniciushana12:06:11

having a control entity helped tremendously - although it’s not sharding-proof, for this case we plan on using zookeeper.

pesterhazy12:06:24

@viniciushana: shouldn't that attribute be indexed so max would be pretty instant?

pesterhazy12:06:06

@bendy, the idea would be to transact a transaction that uses this fn, rather than setting :order/number directly

viniciushana12:06:14

even indexed, for hundreds of thousands of entities a day, it takes more than 20 seconds to run a max

viniciushana12:06:33

we keep those sequentials unique

pesterhazy12:06:04

the other option is to remember the highest order number in an entity somewhere

viniciushana12:06:21

yep, and I also recommend to keep it accessible by a :db/ident to ease retrieving it

pesterhazy13:06:07

that's what I would do too

viniciushana13:06:58

@bendy so in practical terms you’ll want the transactor fn to be like: 1 - lookup the control entity 2 - call inc on the attribute you keep as a sequential control 3 - return the tx vector asserting the business entity with the incremented sequential + a db.add for asserting the incremented sequential attribute at the control entity

bendy13:06:55

just to make sure, this is what I’ve come up with so far - is this what you’re suggesting?

bendy13:06:19

{:db/id #db/id[:db.part/db]
  :db/ident :invoice/next
  :db/valueType :db.type/long
  :db/cardinality :db.cardinality/one
  :db/doc "Next invoice number"
  :db/noHistory true
  :db.install/_attribute :db.part/db}

 {:db/id #db/id [:db.part/user]
  :invoice/next 1}

 {:db/id #db/id [:db.part/user]
  :db/ident :number
  :db/doc "Function that returns the next id for an invoice."
  :db/fn #db/fn {:lang "clojure"
                 :params [db id]
                 :code (let [[e n] (datomic.api/q '[:find [?e ?n]
                                                                      :where [?e :invoice/next ?n]]
                                                                    db)]
                         [[:db/add id :invoice/number n]
                          [:db/add e :invoice/next (inc n)]])}}

bendy13:06:28

(don’t worry this isn’t for a real billing system)

bendy13:06:02

I also ask because I’m getting an error and I can’t seem to figure out why… still debugging

bendy13:06:06

ah… well the issue is with my query, apparently you can’t do a :find ?a ?b ?c . 😁

bendy13:06:41

anyways, I think I’m doing what your recommending, so thanks for the confirmation!

bendy13:06:50

didn’t know whether this was the right track or not

Ben Kamphaus13:06:53

you can put them in [?a ?b ?c] . (omit the . actually)

bendy13:06:30

oh ok, that gave me a null pointer exception

bendy13:06:37

I guess there also another bug somewhere else!

bendy13:06:49

don’t mind me haha

Ben Kamphaus13:06:13

wait one second

Ben Kamphaus13:06:53

no ., but you do want find single tuple from here: http://docs.datomic.com/query.html#find-specifications

bendy13:06:51

ah, bingo!

stijn13:06:05

as soon as I add datomic as a dependency to the peer, logging in other parts of the application starts to break. I have configured leiningen as instructed here http://docs.datomic.com/configuring-logging.html

stijn13:06:22

but e.g. ExceptionInfo doesn't get logged with the data printed

stijn13:06:45

without datomic

(log/error (ex-info "oo" {:a 1 :b 3}) "aaa")
Jun 03, 2016 3:32:40 PM clojure.tools.logging$eval420$fn__425 invoke
SEVERE: aaa
clojure.lang.ExceptionInfo: oo {:a 1, :b 3}
	at clojure.core$ex_info.invokeStatic(core.clj:4617)
	at clojure.core$ex_info.invoke(core.clj:4617)

stijn13:06:23

with datomic & logback

(log/error (ex-info "oo" {:a 1 :b 3}) "aaa")
=> nil
19576820 2016-06-03 15:39:40,202 [nREPL-worker-1] ERROR user - aaa
 - clojure.lang.ExceptionInfo: oo
	at clojure.core$ex_info.invokeStatic(core.clj:4617)
	at clojure.core$ex_info.invoke(core.clj:4617)

stijn13:06:57

let's say I have zero experience with JVM style logging 🙂

stijn13:06:50

It is a problem with logback (pattern:

%date{ISO8601} [%thread] %-5level %logger{36} - %msg%n %ex{full}
) But why does a lein repl without logback print it properly then?

bendy13:06:23

got everything working, updated my snippet with the working code, thanks everyone for your help!

Lambda/Sierra17:06:46

I know that clojure.lang.ExceptionInfo includes data in its toString implementation but not its getMessage.

Lambda/Sierra18:06:40

It looks like Logback only uses getMessage and the stack trace in its default exception printer.

Lambda/Sierra18:06:52

@stijn: Whatever logging framework is active in your standalone REPL (maybe java.util.logging) must be printing the ExceptionInfo with toString.

stijn18:06:21

@stuartsierra: thanks I'll try with another logging backend, or get logback to use toString

stijn19:06:50

got it to work with logback by getting the formatted message, which displays exceptions the same way as a standard leiningen repl