Fork me on GitHub
#clojure-hk2015-11-20
>
onetom15:11:55

@macropolo: hello! do u live in hong kong or just curious about whats going on here?

macropolo15:11:21

@onetom: hi unfortunately not, however I do visit every once in a while, so I was interested in seeing what is going on wrt clojure in hk

macropolo15:11:37

@onetom: great city btw, first came in 2011 and fell in love simple_smile

onetom15:11:08

most people u see here are from https://exiconglobal.com and we are developing our platform in clojure http://hoplon.io and datomic

macropolo16:11:46

datomic is great, I have the pleasure of using it in one of my projects

onetom16:11:14

hey everyone, it would be great if you could introduce yourself

onetom16:11:20

we should have a meetup too. my ex-colleague has created this months ago: http://www.meetup.com/clojure-hk/ but we never had a meeting yet

onetom16:11:40

macropolo: what kind of data do u store in datomic? we just started using it actively recently and we store mobile app and app store data in it, besides our user and org data

macropolo16:11:20

@onetom: the project is concerned with geocoding and address processing, so, naturally, most of the data are hierarchical address information plus some request logging/job control/etc.

macropolo16:11:06

datasets are smallish, tens to hundreds of millions of datoms

onetom16:11:53

that sounds like a gigabyte range db already

onetom16:11:15

also it's an interesting type of data. why did u choose datomic? r u using its time travelling functionality? are u annotating your transactions with the user's or software subsystem's id for audit purposes?

macropolo16:11:47

gigabytes don't seem so big these days simple_smile

onetom16:11:03

well... i was just dealing with a datomic db which occupied 4.6GB on dynamodb (by accident really, because we left of some uniqueness constraints and over a few months it has accumlated a lot of duplicate datoms simple_smile

onetom16:11:03

it deleted that db the other day and ran bin/datomic gc-deleted-dbs on it and it took 70minutes to complete it

onetom16:11:24

and even after that there was ~1.8GB garbage left over

macropolo16:11:50

there are no reqirements for time travel/strong audit capabilities per se, however datomic is quite suited for hierarchical data (vs say sql which will work but would look rather ugly) and read/write asymmetry

onetom16:11:42

then i tried to rename the latest backup of it on s3 by aws s3 mv and that took 2.5hours and ~700'000 aws api operations.

macropolo16:11:58

we don't use dynamo, so disk space/iops are almost free

onetom16:11:14

i should have run that from an EC2 machine probably, but still...

onetom16:11:36

what storage do u use then?

macropolo16:11:15

local one since we currently can get away with it

onetom16:11:28

yeah, hierarchical data is definitely a sweet spot for datoimc

onetom16:11:51

u mean the built-in h2 datomic:dev:// protocol?

macropolo16:11:37

on an unrelated note, bulk-load performance was somewhat of a problem (required index tuning), however I think it improved in recent releases

onetom16:11:01

i was entertaining that idea too for a while but since we run on aws im not sure how much can i trust EBS to store it and even than the uptime might be not so nice

onetom16:11:33

which version are u using?

macropolo16:11:09

i agree, if using aws dynamodb is a better idea

onetom16:11:37

we just bumped up to the latest 0.9.5327 because the .5132 one had some disconnect error when we were creating and dropping in-memory dbs too quickly or something like that

macropolo16:11:53

0.9.5302 currently

onetom16:11:13

otherwise ror/nodejs/angular/meteor is still the mainstream requirement in most jobs

onetom16:11:34

there are quite a few companies who are experimenting with go and react too

macropolo16:11:48

the hesitation about developer availability is quite understandable, however i think currently there are more clojure developers than clojure jobs and also at least some of them are prepared to take a pay cut or compromise on other concerns just to use it in their workplace instead of whatever they are allowed now

macropolo16:11:28

as for go among other recently popular languages it is pretty much the anti-clojure simple_smile

macropolo16:11:49

some speculate it was google's elaborate attempt to ensure none of its competitors ever get ahead sorry if that offended anyone simple_smile

onetom17:11:28

"anti-clojure" simple_smile my 1st association was anti-christ

onetom17:11:57

so if u say there is a clojure dev surplus, what do u think how can i find them?

onetom17:11:40

hmmm... i see there are some job channels on clojurians...

onetom17:11:35

regarding paycut, i also made that compromise, but i dont really regret it (yet simple_smile it's really rewarding to work in clojure after node, ruby, etc...

macropolo17:11:10

i think the usual suspects (channels here, other communities and various job sites) would produce some results my statement is mostly based on "i like/learn/use clojure for my projects but my job doesn't let me do the same" i heard more than once simple_smile not tried to hire anyone yet however if you want clojure devs in a specific location and don't allow remote people, this it of course is more complicated since mean clojure developer density is obviously much lower

macropolo17:11:40

anyway, being able to give people cool toys (clojure in this case) may be an useful advantage when competing for developers with other organizations

onetom17:11:30

that's what i thought too, but failed twice already in the past 9 months... in those cases ppl valued salary more 😞

macropolo17:11:59

did you look in hk only or for remote devs too?