Fork me on GitHub
#datomic
<
2015-08-18
>
bhagany02:08:34

@clojuregeek: using the raw REST api isn't too terrible. I'm doing it from Python.

bhagany02:08:02

@clojuregeek: maybe I'm just very motivated to think about it positively, though. When I look at it a bit more objectively, I am doing quite a bit of string interpolation. But I do get to use datomic simple_smile

bhagany02:08:26

Also, datomic is my way of getting clojure in the door, so… in the fullness of time...

clojuregeek02:08:58

@bhagany: what data store you using?

bhagany02:08:56

our own - I'm porting an existing system. It's basically a row/column based page designer, for ecommerce

bhagany02:08:43

@clojuregeek - I think I saw on twitter that you're doing the music brainz examples?

clojuregeek02:08:28

i loaded it up

clojuregeek02:08:24

i just finished all the day of datomic training videos from the site

bhagany02:08:31

I see. From my experience, doing things in clojure transfers pretty well to using the REST api from non-jvm-land, if that's where you're headed. Either way would probably result in some good learning simple_smile

bhagany02:08:13

or, if you're feeling really ambitious, update the gem!

sdegutis12:08:34

@bhagany: even using the in-memory db?

sdegutis12:08:17

Good morning everyone.

sdegutis12:08:47

I've been considering making a function which wraps (d/transact ...), takes the :db-after key from the result, and swaps it out in a global atom/ref/var/whatever, so that I don't need to pass the most recent database value everywhere. What do you think of this idea? Would it be terrible on performance?

borkdude13:08:59

I'm trying to run a transactor, but I get: java.lang.IllegalArgumentException: :db.error/not-enough-memory (datomic.objectCacheMax + datomic.memoryIndexMax) exceeds 75% of JVM RAM

borkdude13:08:04

which values should I change

borkdude13:08:21

I increased Xms and Xmx to 2G and that seems to work

bhagany13:08:06

@sdegutis: I've never used the in-memory storage with the rest api, but I don't know of any reason why it would be different

bhagany13:08:52

@sdegutis: also, I'm no expert, but having a global db atom would defeat one of the big benefits I get from datomic - a consistent view of time

bhagany13:08:05

to expand, I want to be able to arbitrarily compose functions that all use the same db value, so as not to get inconsistent results from updates that happen in other threads, etc. This can't happen if those functions are constantly referencing the most recent db-after

sdegutis14:08:23

@bhagany: The docs say "The memory system is included in the Datomic peer library." which seems to imply that it needs a full-fledged Clojure Peer process?

sdegutis14:08:22

@bhagany: Also the majority of the time I want to see updates that happen in all threads as soon as possible, since this is a live web app it's used in

bhagany14:08:29

@sdegutis: The rest process is a peer

bhagany14:08:15

@sdegutis: In a web app context, you'll get inconsistent results within a single request, if you don't use a consistent db value

sdegutis14:08:55

@bhagany: How do you figure?

bhagany14:08:11

@sdegutis: contrived example: your request is composed of 3 functions: func1 gets a count of some entities, func2 transacts some data and resets your db-after atom, func3 gets a list of the entities that func1 counted,

bhagany14:08:40

@sdegutis: in another thread, after func1 runs, but before func2, another entity is added

bhagany14:08:04

@sdegutis: now the count from func1 and the list from func3 are inconsistent

sdegutis14:08:36

@bhagany: That seems like an expected type of inconsistency within the context of web apps

bhagany14:08:46

@sdegutis: heh, I suppose you could say it happens all the time. I'd hesitate to call it good, and avoiding it is one of the benefits of using datomic.

sdegutis14:08:04

@bhagany: But even when passing database values around, the user will seem to get somewhat inconsistent results. Consider that if a route /count-things is called, and immediately afterwards /insert-thing is called and returns before /count-things returns, then /count-things will give an outdated number, inconsistent with the effects of /insert-thing.

sdegutis14:08:20

Which I guess isn't better or worse, just different.

bhagany14:08:35

@sdegutis: yes, this is true. If these are something like rest endpoints and the results will be displayed in a single view, I would use an as-of filter, so that the results are consistent

bhagany14:08:18

@sdegutis: if these are two different page views, I think that the user's expectation is generally that time has passed in between page loads.

sdegutis14:08:09

Good point!

bhagany14:08:38

I don't have a specific attribution, but that's almost certainly not originally my point simple_smile

bhagany14:08:44

but thanks

tcrayford14:08:09

the fact that you can stuff as-of into all ajax requests (in theory anyway) is a super good idea 😄

tcrayford14:08:44

@sdegutis: typically ring/datomic apps have a middleware that derefs the database and stuffs it into the request somewhere for this stuff

sdegutis14:08:06

@tcrayford: Yeah that's what we're doing now. But that means every function that my handlers call which need access to the database will need to take a database parameter.

tcrayford14:08:27

yep, and that's a good thing

sdegutis14:08:51

@tcrayford: But since we also do extensive testing, all our functions return a new database value too. So return values often look like [db user] or [db cart item]

tcrayford14:08:07

@sdegutis: so maybe have the thing inside the request be an atom and mutate that somewhere? My apps tend to do very minimal mutation/transactions anyway, and rarely does rendering the response require the updated database value

sdegutis14:08:44

@tcrayford: Hmm that's an interesting idea. I'm not sure how much it will clean up my API though since they'd all still need to take the db atom.

tcrayford14:08:18

for the record: ~40 route webapp, maybe 5k lines of code all told or something. Been doing stuff like this for nearly 3 years now, and not had any problems with it. I typically only test transactional things by having functions that take in a request and return the transaction data, then just test them with d/with. I don't worry about testing the actual call to transact via unit tests

sdegutis14:08:27

@tcrayford: Hmm interesting technique.

sdegutis14:08:25

@tcrayford: I haven't been testing the calls to transact either, I look at that as an implementation detail, but I have been testing the returned "presenter" values. Like, (create-user db "myname") should return [db user] where user is {:name "myname", :confirmed false} etc. which necessitates returning the value from the database (and in some cases the updated database itself for querying against in my tests).

a.espolov16:08:39

(s/schema inspection (s/fields [time :long] [status :enum [:rejected :unverified :confirmed]] [targets :ref :many] [comment :ref :many] [outlet :ref :one] [promo :ref :one] [user :ref :one])) Guys how to find all "inspection" for sample by user ids [... ... ...] and by outlet ids [... ... ... ] in datomic database?

a.espolov16:08:20

I have to use and or or in a query?

val_waeselynck18:08:59

@a.espolov: sorry, I don't understand the question :s

sdegutis18:08:11

Just came up with a pattern I'm liking and wanted to share: https://gist.github.com/sdegutis/7f75d257abbf037e8e48

sdegutis18:08:25

It's a push-based schema updating technique, that's friendly for code-reloading too.