Fork me on GitHub
#lumo
<
2017-11-23
>
pesterhazy12:11:10

@rauh, your snippet is interesting. I've had problems with using transit/read with regular JSON in the past. When values (not just keys) start with ^, the result is corrupted

pesterhazy12:11:17

does the snippet fix that?

pesterhazy12:11:53

we really need a fast "JSON-string => clj-data-structure" implementation

pesterhazy12:11:23

js->clj is a real perf killer in my experience

rauh12:11:13

@pesterhazy Yeah probably would also corrupt it. But if you delete these handler then it should be safe: https://gist.github.com/rauhs/301f59e0e2f94db4f22a4724fe50bd5f#file-parse-json-with-transit-cljs-L27-L36

rauh12:11:40

IMO it would be pretty easy to write a super fast js->clj implementation. I just personally don't use it much so never bothered with it

pesterhazy12:11:31

cool, thanks for the pointer

rauh12:11:33

All you need to do is manually create a for look and create PAM/PV's from it.

pesterhazy12:11:14

might be a good weekend project

pesterhazy12:11:56

some people dislike js->clj but IMO there's no alternative when you're consuming JSON data from a server as clj data structures

rauh12:11:01

One start would be to speed up keyword, though it's probably not the main cost for js->clj: https://dev.clojure.org/jira/browse/CLJS-2120

pesterhazy13:11:46

I'm sure it's part of it

pesterhazy14:11:17

@rauh, what do you mean by "not for node"?

rauh14:11:57

@pesterhazy It caches any key it sees, so somebody can attack your server and just send nonsense JSON with lots of random keys.

pesterhazy14:11:10

ah because of security considerations

rauh14:11:12

In CLJ they're weak refs and will be GC'd. In JS that doesn't exist

rauh14:11:36

That's also the reason core doesn't do it. IIRC the first implementation actually did cache them

pesterhazy14:11:03

the cache needs to have limit of N elements

rauh14:11:00

Yeah, but that's pretty complicated to do. and might negate any perf benefits

pesterhazy14:11:21

shouldn't be too hard

pesterhazy14:11:40

you could have n Objects with m keys max

pesterhazy14:11:12

once you reach n*m you can clear the first one again

pesterhazy14:11:27

that might work for a lot of situations

rauh14:11:50

I don't think it's that easy. NOw you have linear search. You'd need to store access tiem every time you access it. -> Overhead

rauh14:11:17

And then walk it every time since JS doesn't give you a possibility to count the keys of an object (fast).

rauh14:11:33

And now I'm already faster with a simple new keyword creation.

pesterhazy14:11:35

Object.keys(o).length is not constant time?

mfikes16:11:10

@rauh FWIW, when I wrote http://blog.fikesfarm.com/posts/2017-11-09-avoid-converting-javascript-objects.html, I checked if :keywordize-keys true was affecting js->clj perf. It was not (a 1.1 speedup by disabling that). Compared that to to the overall speedup ranging between 26 and 68 by ditching js->clj as a whole and using goog.object/getValueByKeys). (See the post for details.)

mfikes16:11:43

Well, it was affecting perf, but small compared to the gross perf stuff going on in js->clj

rauh16:11:49

@mfikes Yeah my guess is that for and (vec (map ..)) are the main culprits. Just getting rid of those would likely be much faster

mfikes16:11:55

It gets rid of for as you suggest

mfikes16:11:56

Perhaps the (vec (map ...)) could be another fruitful perf avenue to pursue

rauh16:11:35

@mfikes I wish we had a js-for to generate native for loops 😕

borkdude20:11:14

@richiardiandrea FWIW, we have this kind of error message in our cljs build regularly: > Attempting to call unbound fn: #’clojure.spec.alpha/macroexpand-check at line 2529, column 11 in file /Users/Borkdude/.boot/cache/tmp/Users/Borkdude/dre/…/app/r0g/-tnd3jd/public/js/app.out/cljs/pprint.cljs I think I saw you talking about something similar here.

borkdude20:11:31

It’s not reproducible. When we touch a source file, it recompiles and it’s fine.

richiardiandrea20:11:42

@borkdude there is something going on there, it's been hard to track down and I haven't yet

borkdude20:11:18

@richiardiandrea Do you perhaps have parallel build on, like we do?

borkdude20:11:25

I wonder if it’s a race condition of some sort

richiardiandrea20:11:57

talking about lumo only, where no parallel builds are done...so there might be some other race condition

richiardiandrea20:11:43

(maybe in common with cljs)

borkdude20:11:34

The error I posted here didn’t come from lumo, but I saw you posting something similar, so just sharing this fwiw

richiardiandrea20:11:45

that's good thanks a lot, and you are not the first btw I think to notice that