Fork me on GitHub
#datomic
<
2016-01-11
>
jimmy06:01:18

hi guys, what is a good way to query the current schema in user partition ? thanks

andrewboltachev15:01:35

Hi. Are there any migration tools for Datomic? Say, I'm in development and I'm changing my schema arbitrarily, and I want to preserve all the data though?

timgilbert15:01:39

@andrewboltachev: I’ve been using https://github.com/rkneufeld/conformity for that (I’m a newbie though)

timgilbert15:01:15

No prob. Seems to cover most of what I was using lobos for in pgsql-world. There are a few other similar tools around if you google a bit

andrewboltachev15:01:25

Main problem is Datomic's support around schema changes, e.g. fulltext attribute once it's set

andrewboltachev15:01:41

@timgilbert: so, would that library, say help me to (1) take all the data (2) drop existing and create a new schema (3) import preserved data, and apply transformations to it where appropriate?

timgilbert15:01:37

Uh, sort of except for (2), to my understanding

timgilbert15:01:11

It just basically lets you run a set of transactions on your database, and then it keeps track of which ones have already been run

timgilbert15:01:45

But as I understand it, you don’t really drop old schema changes, since everything is immutable

andrewboltachev15:01:20

Like I said, there are cases where you can't apply a change to the schema

timgilbert15:01:24

So transactions can include schema updates, and oalso adding new data from somewhere

timgilbert15:01:48

Not sure about that, I’ll defer to someone with more expertise

andrewboltachev15:01:08

Ok, thanks anyway!

timgilbert15:01:15

No prob, good luck

andrewboltachev16:01:19

My real situation is like this: (0) I have an idea about an (web) app I want to build (1) I'm starting to with "categories". And I want to have schema to save categories and make categories widgets first (2) I realize that there would be "collapsed" flag for each category (in category widget, is it collapsed or not) — need to add new attribute (3) I'm finished with the widget and I want to add "items" entity. (4) Adding users to my project (5) Say, categories list (or tree, or graph ['cause yep, there's no tree]) is common for all users, but each has own ones collapsed-or-not. So, thus "collapsed" flag must be turned to some relation then. Could I achieve (5) (and all of the above) without creating new DB?

andrewboltachev16:01:54

And, have old data preserved (say, I added bunch of categories)?

andrewboltachev16:01:24

Also, interesting thing is that this way of development I just described is sort of "Agile", i.e. categories, items and users are "features". According to Rich Hickey we should be solving problems, not building features. So, is my way fundamentally incorrect, i.e. I should first imagine what my app would operate (design it) and then implement it (so no changes to DB like that would arise)? Would much appreciate any answers/opinions.

jonahbenton16:01:31

@andrewboltachev: hey andrew- definitely don't go in the "waterfall" direction of having heavy "design" and "implementation" cycles. instead, tune your workflow to make it easy to try, discard, and try again. use datomic in mem mode, or even datascript, to iterate on a data model, and separately, if you need to "seed" your data model to test your app, keep the seed datoms in an edn file and just transact them at app startup

andrewboltachev16:01:19

@jonahbenton: so, you say I'll be able to switch to Datomic when finally ready to production?

andrewboltachev16:01:50

Well, seems like easiest approach

andrewboltachev16:01:59

w/o need for external tools

andrewboltachev16:01:36

One big thing is confidence, though. Whoever said that building and deploying an app must not be harder than in 1-click.

jonahbenton16:01:22

yes, you can use Datomic in a "disposable" mode as you iterate through design and implementation, and even when you first show it to users in 1-on-1s. getting positive user feedback, and being comfortable with your feature set, helps build confidence. when you have confidence to capture user data durably, then you can switch Datomic to a durable storage

jonahbenton16:01:51

but you don't want to saddle yourself with unnecessary workflow obstacles while getting to that place, because then you will lose momentum

andrewboltachev16:01:27

Well, seems I have an idea (and some thesis) for a library (but I'll probably build it for SQL, at least first). 'Cause migrations is what I missed in Clojure world, when switching from Python/Django. Thanks for help @jonahbenton ! I'll take into account what you've answered

sdegutis22:01:45

Is there a way to specify inside a :where clause that a the number of ?things must be more than a given number, e.g. (> (count ?things) 6) ?

Ben Kamphaus22:01:53

@sdegutis: if you want to get it done in a single query, you can do so with a subquery, as in this example: https://groups.google.com/d/msg/datomic/5849yVrza2M/31--4xcdxOMJ

sdegutis22:01:49

Oh wow I never knew about subqueries until now.

sdegutis22:01:12

Hmm. That seems a bit hacky though doesn't it?

Ben Kamphaus22:01:25

main use case (at least re: performance) is REST, on JVM may be as or more efficient to just chain query results into another query or sequence manipulation.

sdegutis22:01:33

Ahh interesting.

sdegutis22:01:33

Slack needs a "the more you know" reaction.

Ben Kamphaus22:01:12

I’d say only mildly hacky, maybe the fully qualified namespace stands out a little : ) It is a use case it’s designed to support. I.e., chaining queries together or using subquery are both means of composing queries. Composing queries is one reason datalog query in Datomic takes in a set of relations and returns a set of relations as its default (when return not overridden by find specification).

sdegutis22:01:16

Hmm, I wonder if it can be cleaned up using that function-storing feature Datomic has.

alwaysbcoding22:01:39

Does anyone have an example of querying the Datomic Rest API through clojure, I can't figure out how to do it...

alwaysbcoding22:01:12

If I try something like that ^ it just gives me an error because of the formatting of all the ?eid fields

alwaysbcoding22:01:42

Putting a quote in front of the query doesn't work either

alwaysbcoding22:01:35

the documentation just shows this {:q [:find …] :args [{:db/alias … }]} with no information about how to deal with the ?eid syntax