Fork me on GitHub
#clojure
<
2021-03-24
>
cjohansen08:03:04

What’s the recommended way to turn Clojure into EDN? I ran into some issues with pr-str. It works fine with pure data, but less so if data contains lazy seqs. I tried to make naive edn-response middleware for ring. I gave it some lazy seqs (a map containing something that had passed through for ) and this resulted in a string of log messages bundled up with the stringified data :thinking_face: Interestingly, the ring-json-middleware does not have this problem, as cheshire/generate-string does not use printing. Is there a way to produce EDN data that isn’t based on printing, or am I barking up the wrong tree?

delaguardo08:03:44

https://clojuredocs.org/clojure.core/print-method there is a way to adjust printing for some data based on type in general there is nothing wrong with using pr-str to produce edn. cheshire is not using printing because the serialized form of data not generaly compatible with json specification

delaguardo08:03:41

I just tried to reproduce and it works just fine for the case like that:

(def x (for [i (range 100)] i))

  (type x)

  (pr-str x)
could you share a sample that give you a headache?

magnars09:03:37

The problem is lazy sequences that do side effects, like logging. When creating a generic tool (like an edn-middleware for Ring), you have no control over what crazy shenanigans the lazy data structures you’re passed will do. So, using pr-str to generate EDN, might include log statements printed to `*out*` .

cjohansen09:03:14

Yes, logging is my exact problem 🙂

cjohansen09:03:18

This reproduces it:

(pr-str
 (for [i (range 10)]
   (do
     (prn i)
     i)))

cjohansen09:03:32

;;=> "(0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n0 1 2 3 4 5 6 7 8 9)"

didibus16:03:31

It's impossible to serialize a lazy-seq without realizing it, unless you don't want to serialize the data contained in it?

didibus16:03:47

And so realizing it will always trigger the side effect

didibus16:03:13

But to get the serialized string without the printed stuff that should be possible. Hum... Let me check

didibus16:03:11

Right, you can use pr-on instead

didibus16:03:52

Though that seems to be a private fn

didibus17:03:00

You should upvote the issue if you want it fixed, the core team does take into account votes for priorities

cjohansen07:03:22

👍 Thanks!

rickmoynihan09:03:12

FYI you can work around this by first realising the sequence, whilst holding onto its head. This will apply all its effects. Then simply rewalk the structure printing it. Clojure caches the returned values of lazy seqs, so this should work. Obviously it might not be viable for huge datastructures though.

didibus17:03:30

If you go that route, you need to deep-walk it, probably an easy hack around is to actually print it first, and then call pr-str again on it

emccue15:03:03

Does anyone have a snippet that enables http/2 on jetty without using info.sunng/ring-jetty9-adapter

emccue15:03:24

Jetty's official docs are very "xml focused", which does not make it easy

emccue15:03:29

and i really don't want to stray off the "official" path to get something like this working

Vikas Gautam20:03:33

Hi Clojurians, I wrote a list multiplication program which multiplies two numbers represented as lists using standard multiplication algorithm taught in school. https://www.mycompiler.io/view/E5DB2XN is the first program I wrote. This program helped me pass a bunch of problems on Hackerrank, until it started giving me timeouts at https://www.hackerrank.com/contests/projecteuler/challenges/euler016/problem project euler problem. So, I tried improving the functions with https://www.mycompiler.io/view/4owO1yk shorter but slower attempt. I am not able to figure out why the latter program is slower than the first one. Can anyone help?

p-himik21:03:43

In the new attempt there's this code:

(reduce (partial map +) ls)
It doesn't look right.

p-himik21:03:24

Oh, ls is a list of lists, I see.

Vikas Gautam21:03:26

@U2FRKM4TW Yes sir, that is list of products.

p-himik21:03:33

It's almost night here, so I'm rather lazy to go through the code right now. But at the very least, that particular part should be written as (apply map + ls).

p-himik21:03:58

This way, it won't create a (dec (count ls)) of lazy collections.

Vikas Gautam21:03:00

@U2FRKM4TW yes, I already tried that. That was even slower. (apply map + ls)

p-himik21:03:56

I can't offer a more constructive advice right now other than "try profiling". Then you'll know for certain.

Vikas Gautam21:03:21

@U2FRKM4TW thank you Sir. Appreciate that. I will look into profiling.

Vikas Gautam22:03:52

@U2FRKM4TW I found the cause. It was map function retuning lazy sequences which delays the execution. I used mapv at two places and it made it twice faster. Thanks for putting me on the path and letting me explore this lazy sequence angle.

👍 3
dpsutton21:03:57

what's the state of core.typed right now? Is anyone using it after its renaissance? is it ready?

jjttjj21:03:52

The author is interviewed fairly recently here: https://youtu.be/8A8EKTBlFRM which I watched when it came out but unfortunately I somehow don't remember what the summary of the current state is. I know he talks about what he's been working on and I believe has been actively working on core.typed but I'm not sure if those changes are available yet

jjttjj21:03:00

Actually the changes I'm talking about are probably just the renaissance you refer to

emccue21:03:41

(unrelated) maybe a malli thing might work well. Like, a subset of what malli can specify might be translatable to typedclojure

emccue21:03:36

oh wow the video here is almost along those lines, just talking about spec

nilern09:03:39

It should be pretty easy to translate (lossily) between types and schemas. But inferring either is the really interesting part.

didibus03:03:27

In theory, it can be used to type check Clojure pretty well, but the user ergonomics might annoy you

didibus03:03:59

Since it doesn't have global inference, it just does local inference, and typing Clojure functions often involve complex type defs, since Clojure functions tend to be quite powerful in input

didibus03:03:37

And the core isn't typed, and all libs won't be typed either, so you've always got this type/untyped boundary as well

nilern10:03:11

I'm a type theory expert and I find the types too complicated 😅 But a lot of people like Typescript so 🤷

emccue22:03:44

I don't think its unreasonable to want static analysis of the parts of a spec that can be statically analyzed

emccue22:03:52

> And the core isn't typed, and all libs won't be typed either, so you've always got this type/untyped boundary as well  Yeah, but core will be spec'd and typescript balances that fairly well so there is some way forward

emccue22:03:01

and if the whole community weren't osborne effect'ed on spec-2035 it might be possible to get some core ones on board

didibus03:03:39

I don't think you can do a lot of typing with spec no? Cause the predicates rarely map to a logical type?

didibus03:03:03

It would have been nice if maybe spec had done something like: type + pred

didibus03:03:40

So like the broader type + a refinement pred

didibus03:03:24

And if I understand Typescript properly, a hord of devs jumped on it and retrofitted types for a ton of existing JavaScript. I mean this could happen for core.type as well but did not

didibus03:03:16

Also personally, not sure how useful types would be without having abstract data types as the standard compound data-structure.

didibus03:03:16

All maps would still just be of type map. So the issue of like what keys are in this? Am I using a key that doesn't exist? Etc. Would still be there

dpsutton03:03:32

I think type script has row type polymorphism that can handle this. Not sure what it’s like in practice though

didibus03:03:30

I think row type just means that any superset is automatically valid, even if not explicitly defined as is-a relationship.

didibus03:03:55

But it would still require a definition of the fields on the type no?

didibus03:03:53

Like I think you can do row polymorpism with Clojure records.

dpsutton04:03:56

Yes that’s my understanding. Don’t know if core typed has this kind of analysis but would love it

didibus04:03:40

Ya I think that's correct. Basically it just let's you say this function works on any type which have at minimum these keys of some given types. But it still requires the type checker to know what keys of what types the object has. So for it to work with maps, it would need that somehow it could infer for any given map at any point in the program what are the keys it would now contain and what types those keys would be of. Which seems like really hard to do. And that ignores that it's possible to have a dynamic key too, like where the key name is coming at run-time from external input

didibus04:03:57

But records would work well with it. Cause records already force you to define the mandatory keys, you'd only need to also define their types and you could then do row polymorpism type checking, if your variables and function arguments specific row type constraints.