This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-10-18
Channels
- # aleph (59)
- # beginners (21)
- # bigdata (1)
- # boot (110)
- # cider (7)
- # clara (1)
- # cljs-dev (160)
- # cljsjs (3)
- # clojars (10)
- # clojure (122)
- # clojure-czech (2)
- # clojure-dusseldorf (5)
- # clojure-france (1)
- # clojure-italy (4)
- # clojure-korea (5)
- # clojure-russia (13)
- # clojure-spec (15)
- # clojure-uk (78)
- # clojurebridge (1)
- # clojurescript (196)
- # core-async (6)
- # core-logic (27)
- # cursive (11)
- # data-science (2)
- # datomic (45)
- # dirac (9)
- # emacs (2)
- # funcool (8)
- # hoplon (16)
- # immutant (13)
- # jobs (1)
- # klipse (11)
- # lein-figwheel (1)
- # leiningen (1)
- # off-topic (3)
- # om (40)
- # onyx (31)
- # pedestal (25)
- # re-frame (55)
- # ring (1)
- # ring-swagger (1)
- # rum (4)
- # specter (1)
- # sql (2)
- # untangled (30)
- # vim (12)
- # yada (12)
http://docs.datomic.com/best-practices.html#use-pull-to-retrieve-attribute-values are there any performance hits/gains from using pull instead of {:find [?every ?attribute ?i ?am ?interested ?in] :where [[?e :blah/is 1] [?e :every ?every] ...]}
? I have a lot of queries using this syntax.
I'm guessing I might see some improvement simply because I can more easily re-order clases (http://docs.datomic.com/best-practices.html#most-selective-clauses-first)
i don’t think you’ll see a massive perf gain by doing that, but you will see cleaner code by using pull
q/q to find which things, and d/pull to look at them
either way, the query cache will almost certainly have your data in local memory by the time :find or d/pull operate
@robert-stuttaford As long as there isn't a significant hit, I'm happy. I'm trying to optimise a 20ms query that we currently run 2k times… Hopefully I can get it to run once — when we actually look up the entities. But I can't understand the queries to begin with, so this seems like a good first step.
that sounds like fun 🙂 what are you doing for measurement?
i’ve found https://github.com/ptaoussanis/tufte to be simply lovely
That's exactly what I am using! I have been wondering if this slipping under the radar isn't an excuse to have some kind of profiling for every request in dev, with significant spikes causing the speaker to notify at you.
i love that idea
i found us using a datalog query as a sort-by function using tufte -grin-
https://blog.codinghorror.com/performance-is-a-feature/ Stack Overflow have this MVC mini profiler, which is just awesome. It'd be pretty cool to generate one for every request.
oh man i’d love that
https://github.com/yeller/clojure-miniprofiler I did play with this a little bit, it was quite fun 🙂
you just put something on the very top of my list, sir
it has datomic measurements? i wonder how it does that...
It has a custom-timing function, which could be used for datomic:
(custom-timing "sql" "query" my-query-string
(execute-sql-query my-query))
ah, so it cheats 🙂
and makes you do the work
fair enough
does it deal with laziness sanely? or do you have to doall
the things yourself?
I know datomic can support a "tiny" number of databases with the same transactor. Is it also true for abandoned databases that are never connected to anymore? A case might be migrating the data to a new database and leaving the old one as is.
@yonatanel Every database running against a transactor will have to maintain a small amount of overhead. You can delete the abandoned database by calling d/delete database. To be clear, you could also leave abandoned databases out there as it is a very small amount of overhead.
Okay, I'm a little confused still, I have this query:
'{:find [...]
:in [$ ?input]
:where [[?input :e/name ?ea]
[?aaa :aaa/name ?ea ?tx]
[?aaa :aaa/locations ?location]
[?location :location/addresses ?address]]}
and it was performing OK (21ms mean avg) but I needed faster as it's running 2k+ times (If I can optimize that part, I'd love to, but can't see how right now...)
I figured the problem might be that I am doing an AVET
lookup from ?ea
on the :aaa/name
value, so I added an index there, but haven't seen any performance improvement (I did sync-schema).
In case it's relevant, this query is being done on a as-of
db.I'd like to know how to most efficiently figure out what the expensive lookup is, and how to optimize that part. I might be into difficult territory, in which case I think it's better to re-evaluate if I need to do this, but that'll require me cleaning up an even larger query..
@dominicm Datalog is super nice for query de-composing. I recommend dropping each clause and confirming you have the optimized clause order.
The biggest hit was 15ms from the second clause. I believe it's due to https://github.com/Datomic/day-of-datomic/blob/master/tutorial/decomposing_a_query.clj#L81 I tried to get round it by adding an index.
I'm finding a relationship through a shared, non-normalised value. I guess that's the hit, but I thought that was the purpose of an index. Or have I misunderstood?
{:db/ident :aaa/name
:db/index false
:db/id #db/id [:db.part/db]
:db.alter/_attribute :db.part/db}
Does anyone know if there are any other ports that I need to open up besides 4334-4336 to connect peers to a Datomic dev transactor (4336 is open to see the h2 database)?
I've got a transactor running on one box and when trying to connect from a peer on another box I can see how many databases I have from the peer (`(d/get-database-names "datomic:dev://<transactor-box-ip>:4334/*")`) – so the peer is connected – but if I try to create a new db (`(d/create-database "datomic:dev://<transactor-box-ip>:4334/test-db-two")`) I get an exception (`ActiveMQNotConnectedException AMQ119007: Cannot connect to server(s). Tried with all available servers. org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory (ServerLocatorImpl.java:799)`).
However, running the exact same create-database
call on the transactor box succeeds. So it's only an issue with the peer.
I'm wondering if it has something to do with a port I haven't opened that should be and that's why the peer doesn't succeed. Couldn't find anyone with a similar issue on Google. Thanks in advance for any help.