Fork me on GitHub
#clojure-uk
<
2018-02-02
>
thomas08:02:32

moin moin morning

otfrom08:02:12

@mccraigmccraig your sorted stream joins give me ideas on doing joins on other bits of file munging I do too. I'm excited by @reborg’s parallel work https://github.com/reborg/parallel

otfrom08:02:40

he's got an external parallel sort that I could use with this

mccraigmccraig08:02:26

+1 p/external-sort : nice @reborg !

rickmoynihan10:02:40

@reborg: With p/external-sort, am I right that it assumes fetchf returns printable clojure data, and that you’ll need enough RAM to store (( chunk-size-as-str + chunk-size-as-clj-data) * Ncores)?

rickmoynihan10:02:12

So you need relatively small chunks? maybe 10s of mb in size?

reborg10:02:56

Yes @U06HHF230, at the moment is storing chunks assuming edn/text. Somewhat naively perhaps 🙂 It could be relaxed. Do you have a specific use case in mind? That might help to better shape some future work. I'm sure it needs chunk x ncores space, but I don't think the code mandates the two representations in memory at once. I guess it depends from your fetchf. fetchf only contract should be to return something that the comparator can understand. It's true that I'm using "pr" to serialize, something to relax as well.

guy10:02:43

morning all 🙂

thomas10:02:47

I had to change 12 files just to add three fields to a webpage (and these fields are saved in an SQL DB).... :thinking_face::thinking_face::thinking_face:

mccraigmccraig11:02:39

that seems a little excessive @thomas! i count 6 for a simple field addition in our system: migration, schema||spec, model, view, view-model-subs, view-event-handlers - what else do you have?

yogidevbear11:02:17

Fun and games... Windows update seems to be taking a rather long time to restart. Guessing this is related to an update for Spectre and Meltdown

mccraigmccraig11:02:52

i had to install windows and visual studio about 5 times each a couple of weeks ago @yogidevbear, searching for the magic combination of versions that would work with cordova... although in the end my problem turned out to be something to do with msbuild not working with vmware or parallels sharing mounts. i am scarred

sundarj11:02:03

that is some A-grade complexity. i feel your pain

thomas11:02:43

@mccraigmccraig the page it self, some java code where the data comes from, several DTO's, some factories, an SQL update file and a new class for the list of options.

thomas11:02:48

and this is a rather 'simple' web app done in GWT with just 1 DB and nothing else.

thomas12:02:17

yes... it !@#$s big time IMHO.

mccraigmccraig12:02:37

big hindsight lol @ DTOs

thomas12:02:18

I don't get them to be honest. But then it isn't entirely clear to me why I'd want to use an ORM in the first place. (I kinda understand what they do, but as documented in plenty of places on the interweb, there is an impedance mismatch)

thomas12:02:35

seems to complect things.

chrjs15:02:51

In case folks hadn’t noticed, it’s State of Clojure survey time: http://surveymonkey.com/r/clojure2018 🙂