Fork me on GitHub
#clojure-uk
<
2017-01-12
>
maleghast06:01:35

Morning… Does anyone have any solid recommendations for Datomic learning resources? (asking for reasons)

maleghast06:01:22

Also, does anyone have any smart hacks around the in-memory only status of the completely free version of Datomic?

maleghast06:01:57

(clearly I know where the Datomic website is, I mean non-Cognitect resources that are warts and all and aimed at the neophyte, in case that was not clear 😉 )

agile_geek07:01:55

@maleghast I'm going to be completely useless as usual and say I didn't find anything beyond the odd blog post and Cognitect's docs. I struggled to learn from them. Not sure why, just didn't suit my learning style.

maleghast07:01:29

@agile_geek - Thanks for trying, honestly 🙂 Your answer is rather hence my question, i.e. I’ve tried to read the Cognitect stuff about Datomic, and it’s REALLY not clicking… I know people that use it, that are very happy with it, and I am sure that if I could get my foot in the Datomic door I would love it too, plus I have a (potential) use-case for it that might be really useful, but I need to find something that I click with in terms of learning...

maleghast07:01:38

I probably need to get my head around Onyx as well for related but different reasons… sorry for the vague-slacking 🙂

agile_geek07:01:23

I found Onyx much easier to grok. Also @michaeldrogalis videos of various talks are awesome resources about how it's architected.

maleghast07:01:51

@agile_geek - Thanks, I will - if I EVER get time… 😉

maleghast07:01:47

Current @TODO list outside of work: 1. Grok Boot 2. Grok Datomic 3. Grok Onyx 4. Build Super Secret Proof of Concept using #1-3 5. Spend time with wife and children 6. Relax, Read, watch Man in the High Castle 7. Sleep (maybe) 8. Exercise 9. Finish at least one of my creative projects before the end of 2017

maleghast07:01:45

I am fairly sure that I need to scale it down a bit or take some time out of day-to-day work...

maleghast07:01:49

we shall see...

seancorfield07:01:13

I'll take 1 & 6 out of those...

agile_geek07:01:16

I would say don't skimp on 5, 7 or 8 otherwise none of the rest are going to happen

thomas08:01:54

@maleghast your todo list looks vaguely familiar... have you considered cloning yourself ?

mariusz_jachimowicz08:01:21

a lot of todos, might require quit the job, take your own savings for some time, do cool stuff, find new client

mariusz_jachimowicz08:01:46

this is my way 😄

maleghast08:01:13

@mariusz_jachimowicz - This may well happen...

maleghast08:01:37

@agile_geek - You are quite right there

maleghast08:01:53

(it is not in order of priority, but in order of how came out of my head in the moment)

maleghast08:01:21

@thomas - I have considered it, yes, but then I realised that I would have a new @TODO that was “Grok cloning technology” and another “Clone Self” and another “nurture clone to self-sufficient age” and another “educate clone sufficiently to assist me"

maleghast08:01:49

That was the point at which I realised it was an interesting idea but unlikely to help me with my time and resource management 😉

jonpither09:01:26

@maleghast the #datomic channel is a friendly place. Could also direct any ultra newbie questions my way (and I will route them accordingly)

maleghast09:01:52

@jonpither - Thanks very much 🙂 I will peruse / lurk the channel for a day or two and then ask some questions… In the meantime, how do you get around the whole “it’s only in memory” thing with the non-Pro version, or am I just worrying about nothing?

maleghast09:01:38

@seancorfield - Are you offering to take some of my @TODOs on..?

jonpither09:01:51

I think you are mistaken. You can run datomic against a persistent store without a pro license

maleghast09:01:37

Oh? That’s cool, but this also speaks volumes about what I was inferring above - that Cognitect’s website is confusing and weird...

maleghast09:01:46

I spent some time on the site last night and it seemed__ to be telling me that the non-Pro version was in-mem only, and then if I was prepared to give them my details and get a "starter key” I could get Pro (for a year) at no £ / $ cost and that would work with persistent storage.

maleghast09:01:15

Based on what I was reading I didn’t want to start the countdown clock to needing $5k… 😉

jonpither09:01:31

I would defo encourage you to find the misleading text and report it.... I havent read it minutely, but I thought the recent changes to http://www.datomic.com/get-datomic.html made it clearer

maleghast09:01:31

Datomic Free Datomic Free is a good fit for open source applications. Datomic Free enables seamless migration of your data to Datomic Pro, and your code is completely compatible. Datomic Free includes a memory database and embedded Datomic Datalog. The Datomic Free transactor is limited to 2 simultaneous peers and embedded storage and does not support Datomic Clients. Because the components are redistributable, it's great for applications that you want to share. You can download Datomic Free and use it right away - neither you nor your users need to sign up with us.

maleghast09:01:19

I think that’s the point and maybe there is no countdown clock on “Starter” but I do need to register with Cognitect and get a key to get "Support Supports all Storages"

maleghast09:01:21

The 1 year is maintenance and updates, though the licence is perpetual, so basically if one wants updates past the 1 year mark then one needs to ask nicely for a new licence - I get it now...

jonpither09:01:49

Where does it say supports all storages? The big recent change is for that to be across all versions

maleghast09:01:58

I think it may have more to do with reading at 0230h than anything else…

maleghast09:01:33

@jonpither At the top above the grid that lays out the licences, Starter, Pro and Enterprise

jonpither09:01:11

Yeah. ALL versions have unlimited peers and all storages

maleghast09:01:42

whereas Datomic Free still exists but is still limited to 2 peers, no clients and no persistent storage (bottom of the page)

maleghast09:01:56

but requires no registration with Cognitect

jonpither09:01:26

So starter is new basically

maleghast09:01:33

It would appear

agile_geek09:01:35

I must admit to liking the idea of Datomic for a while but the licensing puts me off using it for any kind of start up idea.

jonpither09:01:00

@agile_geek I would say starter is appealing tho?

maleghast09:01:32

$5k / annum is not a lot of money in an annual IT budget, and a lot less than some other paid versions of other datastores - don’t get me started about Commercial Neo4J licences...

agile_geek09:01:15

@jonpither not really. Unless you can be certain you are going to make enough money in year two to stump up $5000 you would be stuck with no upgrades and with a newish product there are bound to be issues you need upgrades for.

maleghast09:01:19

I mean as long as the business / platform you are building needs a db like Datomic, that is - it’s clearly not a requirement__ for a lot of ideas.

agile_geek09:01:09

@maleghast for large orgs yes. But as I said it would dissuade me from using it in a startup

jonpither09:01:11

@agile_geek interesting. TBH I havent scrutinised it properly until now

agile_geek09:01:40

..and I'm an old cynic and wary of vendor lockin

jonpither09:01:17

Be interesting the official response to that

maleghast09:01:39

@agile_geek - You can just ask for another key after 12 months, for starter, that is.

maleghast09:01:31

Once your business has either the load / traffic need for Pro, or the money to afford it and “play fair” ‘cos you are making money you upgrade and get the other benefits, but in the meantime just get a new key every year?

agile_geek09:01:52

@maleghast OK wasn't aware of that but if the new key is free and includes upgrades then why have a year restriction at all?

maleghast09:01:19

To make people come back and ask, so that they get a bite of the sales cherry, I expect

maleghast09:01:42

fair enough in my head, but YMMV and I would have no judgement either way.

agile_geek09:01:31

I'd still not consider it for a startup unless it saved me a lot of time then it might be worth the investment and lockin

maleghast09:01:45

@agile_geek - The lock-in is a concern, but I (may) need a Time Series DB, and I feel happier subjecting myself to vendor lock-in from Cognitect than from Influx or with RiakTSD, at the moment, all things being equal.

agile_geek09:01:16

True. Like I said I'd just need a good justification. TBH I am slightly jaded by the whole No-SQL thing. It became a band wagon to jump on without a good reason in an awful lot of orgs.

agile_geek09:01:19

Maybe my very painful experience with HBase has scarred me for life 😞

maleghast09:01:27

@agile_geek - Oh Hell yes, I am the first one to say things like “why don’t we use Postgres for now and if we decide to store JSON documents as well as RDBMS tables we can” and “right tool for right job noSQL fanboi!"

maleghast09:01:12

If this idea “happens” the stack will include RDBMS and Document Store(s) as well as a TSD, for reasons - sorry again for the vagueslacking

maleghast09:01:53

@agile_geek - Also, you are a braver man than I; HBase scares me and I’ve been dodging it and urging other choices for 3 jobs now… I know I am a coward, but I’ve heard the war stories.

agile_geek09:01:50

@maleghast not my choice. I demonstrated it's extreme fragility to the "Enterprise Architect" at my last client with loads of evidence to back it up but they went ahead with it anyway as they'd bought a package support deal from HortonWorks and HBase was part of it. We are in a "post-truth" era so why was I expecting evidence to matter?

thomas09:01:16

@agile_geek your assessment wasn't fair...

agile_geek09:01:23

I spent almost 3 months on a PoC that conclusively proved you couldn't get the resilience or the performance they required without a lot of work and about 5 times as many region servers as HW recommended

maleghast09:01:39

Yikes, that is a new war story, but just as frightening, if not more so than the stuff I had heard about Hbase being very inflexible and hard to work with in general.

agile_geek09:01:11

to paraphrase Nick Cage in "Gone in 60 Seconds" they just were just "..self-Indulgent wieners with too much bloody money!"

jasonbell09:01:59

HBase, so good in theory. 🙂

maleghast10:01:22

What like low-fat, high fibre diets, running and voting Liberal Democrat? 😉

glenjamin10:01:23

$5k/year is roughly 2 months salary of an employee?

glenjamin10:01:50

@maleghast The datomic talk from Strangeloop 2015 I found illuminating

glenjamin10:01:02

i think i did anyway, haven’t actually used it since

maleghast10:01:18

@glenjamin Depends on the employee - not many code-centric businesses paying their employees $30k / annum

maleghast10:01:34

Re Strangeloop talk, I will source it - thanks 🙂

glenjamin10:01:36

oh right, nearer 1 month then

glenjamin10:01:14

for a “startup”, whatever that actually means these days

maleghast10:01:17

Well, somewhere between 2weeks and a month depending on the role, their experience and whatnot, I imagine

maleghast10:01:26

Either way, @jonpither managed to decode the Cognitect / Datomic site enough for me to realise that I was panicking about nothing and so Rich and Stuart have my data now, and I have a 12month licence key and a copy of Datomic Pro. I will be playing with it soon...

mccraigmccraig10:01:40

i like datastax's startup licensing - it's free until your revenues exceed $5M or your raises exceed $50M

davesnowdon10:01:57

Good morning folks

davesnowdon10:01:29

Just catching up. Can completely relate to @maleghast 's TODO list

thomas10:01:47

@mccraigmccraig that seems very generous IMHO.

maleghast10:01:07

@mccraigmccraig - That’s remarkably sensible 🙂 That’s commercial Cassandra, right?

maleghast10:01:32

@davesnowdon - Good morning; it’s good to know I am not alone; thanks 🙂 *fist-bump of solidarity*

mccraigmccraig10:01:43

@maleghast yeah, commercial cassandra - i'm currently using it because of the spark integration which isn't nearly so useful without the commercial components

maleghast10:01:18

@mccraigmccraig - Cool 🙂 I’ve done one prototype project with Cassandra and I really don’t think I was getting the DB to notice it was being used tbh… I nearly had a brush with it via a mad architect at my last job trying to get us to build a Customer 360 system using Atlas, but I’ve really never used it in anger at all. Do you rate it, and how nicely does it play with Clojure (just to keep us slightly on topic 😉 )?

glenjamin10:01:35

the homepage seems to make a lot of promises

glenjamin10:01:07

iirc it’s a “columnar” store? whatever that means

mccraigmccraig10:01:15

@maleghast i like cassandra - it's a big change from a relational db and takes some getting used to, but it's behaved very well for me so far

mccraigmccraig10:01:54

@glenjamin all the current docs are on the datastax site... there's a tonne of out of date stuff from the old days on different sites out there

maleghast10:01:54

That seems like as ringing an endorsement as I’ve ever heard a coder give to a DB of any stripe to be honest 🙂

mccraigmccraig10:01:34

@maleghast it plays very well with clojure too - i'm using https://github.com/mpenet/alia which is great

mccraigmccraig10:01:51

i have a higher-level client built on that too, which is open-source, but i need to push a tonne of changes from our product repo back out to the open-source repo

glenjamin10:01:57

ah ok, i think i’m following this

glenjamin10:01:24

it’s fairly similar to a normal table-based RDBMS, but you can’t join and storage is always partitioned?

mccraigmccraig10:01:44

@glenjamin it presents a query interface which looks like an RDBMS, without joins - there are strict limits on the queries you can do against a given table though

glenjamin10:01:09

the tables seem to map more closely to indexes you’d have in an RDBMS from a modelling pov?

mccraigmccraig10:01:23

@glenjamin correct - you often create tables to reflect a query you want to do, and denormalize to those tables

glenjamin10:01:43

are there any tools in cassandra itself to do that denorming, or is that up to you?

mccraigmccraig10:01:57

recently there are materialized views which will do lots of denorming for you

mccraigmccraig10:01:29

though if you want to denorm a collection column by unrolling it then you are still on your own

mccraigmccraig10:01:01

a lot of what my higher-level client does is look after the denormalizations for you

maleghast10:01:18

So, I am assuming that the up-side is speed and consistency of writes..?

mccraigmccraig10:01:09

@maleghast linear throughput scalability and multi-datacenter replication are the big wins

mccraigmccraig10:01:58

roughly, it doesn't do anything that doesn't scale linearly

maleghast10:01:00

Right, so you can make it eat high volumes of data and you can pull ‘em out as well and you can do to or from the nearest DC

mccraigmccraig10:01:25

yes, and you can nuke one or more of your DCs and your app keeps on rolling.... and if you suddenly double your customer base, all you have to do is double your node count

mccraigmccraig10:01:54

until it all falls apart of course, but c'est la vie... there are deployments out there with many thousands of nodes, which is quite impressive

maleghast11:01:43

Yes, that is very impressive, truly. 1000s of nodes is both impressive and rather intimidating, but wow!

mccraigmccraig11:01:11

the really nice thing (for me) that the datastax commercial distribution offers over the open-source cassandra is a hive metastore for the spark integration - it means that i can point tableau at cassandra and do interactive analytic queries just like it was a vanilla RDBMS

maleghast11:01:36

Oh wow, that is VERY cool

otfrom16:01:16

so my big question on datomic is: is it open source? I'm happy to pay for a supported database (like c*) I'm not happy to let the vendor have the ability to take it away

minimal16:01:39

it’s not open source

otfrom16:01:11

Paradox, dBase, FoxPro

otfrom16:01:23

I've had too many databases shot out from underneath me

jasonbell16:01:40

dBase III+ -> Clipper oh those were the days.... 1989 🙂

jonpither19:01:36

Reminder about drinks tomorrow evening at the Pack and Carriage nr Euston for those around

mattford19:01:42

hey all 🙂 new to clojure and the community

mattford19:01:49

I've been writing some simple stuff - I'd like to replace some of my tooling with clojure versions. It's all CLI stuff to help me in my day job.

mattford19:01:33

But it seems very old.

mattford19:01:49

Any pointers to what I might use?

jonpither19:01:25

gosh I'm not clued up on the CLI at all

mattford19:01:09

That's what I looked at. It's just that the plugin hasn't been touched in 4 years.

jonpither19:01:51

could be because it just works and is relatively small in terms of what it does

mattford19:01:06

Clojure is that robust 🙂

mattford19:01:21

Well I'll try and see.

otfrom21:01:51

mattford there is some cljs node stuff that might be better for cli tools

otfrom21:01:01

depends on what you are doing obviously