Fork me on GitHub
#beginners
<
2022-04-07
>
emilaasa03:04:12

Is there a neat way to make cheshire or clojure.data.json gracefully ignore objects it cannot serialize (functions mostly)? I have a large configuration map I want to persist to disk as json, but there's some functions sent along with it.

Alex Miller (Clojure team)04:04:21

data.json uses a protocol for writing that dispatches on type so you could extend to the type you don't care about and then not write

emilaasa04:04:25

Thanks Alex, it's something like that I've ended up with unfortunately. 🙂

Alex Miller (Clojure team)12:04:03

If you overwrite the Object extension that covers the fall through case and that alone might be sufficient

Lycheese11:04:27

If I have a key :test/mark that can be at arbitrary depth and position and can appear multiple times inside a map, is there a way to target the map the key is inside and apply an update fn to it? E.g with the map:

{:test/a [{:test/b {:test/mark :id
                    :test/other :val}}]
 :test/z [{:test/different {:test/path [{:test/c {:test/mark :id
                                                  :test/other :val}}]}}]}
I would like to call an update fn on the map passing {:test/mark :id ...} as the argument and replace it with the result without specifying both paths manually. Is there a cljs-compatible library that implements a nice API for doing this or do I need to write it myself (presumably something tree recursive)?

Lycheese12:04:06

I solved it with https://github.com/redplanetlabs/specter and its recursive-path macro:

(def MARKED-MAPS (sp/recursive-path [] p
                                 (sp/cond-path vector?
                                               [sp/ALL p]

                                               #(and (map? %)
                                                     (not (contains? % :test/mark)))
                                               [sp/MAP-VALS p]

                                               :else sp/STAY)))
  (sp/transform MARKED-MAPS (fn [_] {:tested true})
             {:test/a [{:test/b {:test/mark :id
                                 :test/other :val}}]
              :test/z [{:test/different {:test/path [{:test/c {:test/mark :id
                                                               :test/other :val}}]}}]})
Inside :require :
[com.rpl.specter :as sp :refer-macros [select recursive-path]]

manas_marthi12:04:24

How can I add clojure code into an existing spring boot app and repl into it while running it in eclipse

manas_marthi12:04:14

Does the spring boot auto config reload take eefect within nrepl session as well?

Mark13:04:43

I'm sure this has been asked a million times, but what's the difference between (a :two) and (:two a)?

user=> (def a {:one "one" :two "two"})
#'user/a
user=> a
{:one "one", :two "two"}
user=> (a :three)
nil
user=> (a :two)
"two"
user=> (:two a)
"two"

dpsutton13:04:41

You need to be careful with the ( versus {. {:a :two} and {:two :a} are maps, the first with key a and value two, the second with key two and value a. But in your example you are using (a :two) and (:two a) (note ( vs {. Keywords and maps are also functions that do the “obvious” thing. A keyword invoked as a function will look up the value associated to that key in an associative structure. A map used as a function will act as a function from key to value, or lookup the value of the key provided.

Mark13:04:25

Sorry, yeah I mean (

Mark13:04:08

Is there a preference for map as function vs. key as function?

Ferdinand Beyer13:04:06

Prefer the keyword, as it handles nil gracefully

(def a nil)

(:key a) ; => nil
(a :key) ; => NullPointerException
See also: https://github.com/bbatsov/clojure-style-guide#keywords-as-fn-to-get-map-values

1
Mark13:04:06

Ah that makes sense. Thanks.

Ferdinand Beyer13:04:47

Just for completeness: Being able to use maps as functions is actually quite nice, and super obvious once you realise that a map is the prototype of a pure function. Keys in, values out. A “mapping” in the mathematical definition. (Rich also mentioned that in one of his talks)

1
Roee Mazor13:04:34

hey hey, good clojurians, how can I force a clojure hashmap to become an arraymap? for reitit body response purposes I am getting:

IllegalArgumentException: No implementation of method: :write-body-to-stream of protocol: #'ring.core.protocols/StreamableResponseBody found for class: clojure.lang.PersistentHashMap

oxalorg (Mitesh)13:04:19

ArrayMap seems to have nothing to do with this. What do you expect to happen when you return a hash-map as an HTTP body? There are 2 ways to go about fixing this: 1. If you want to print the hash map as a string in the HTTP response, then return (str <your-map-here>) 2. If you have some way to stream hash-maps, you can do something like

(extend-protocol StreamableResponseBody 
  clojure.lang.PersistentHashMap
  (write-body-to-stream [...] ...))

Roee Mazor13:04:15

but array maps work fine

Roee Mazor13:04:46

(muuntaja/format-response-middleware this middleware may be behind this magic?)

oxalorg (Mitesh)13:04:54

Yes I don't think array-maps should be supported by default :thinking_face:, adding in a middleware which handles this makes more sense to me!

Roee Mazor14:04:06

How can I turn my hashmap to arraymap so my middleware will work? 😄 I assume writing my own middleware can be avoided?

oxalorg (Mitesh)14:04:30

Do you have format-negotiate middleware or exceptions added?

oxalorg (Mitesh)14:04:35

json should work out of the box with that middleware :thinking_face:

Ferdinand Beyer14:04:22

This definitely has nothing to do with hasmap or arraymap. You might confuse the fact that small maps in clojure are implemented by the ArrayMap class, and these get promoted to hashmaps as they grow.

oxalorg (Mitesh)14:04:58

Muntaaja automatically decodes returned values based on requests "Content-Type" headers

👍 1
Ferdinand Beyer14:04:29

Maps are not valid Ring responses. You will need some middleware to encode the map to something like JSON or EDN or whatnot, for example Muuntaja as you write.

1
Roee Mazor14:04:57

Well, I am using the same middleware. a request with a hashmap doesn’t work, arraymap does work

Ferdinand Beyer14:04:54

Very sure the error lies somewhere else

Ferdinand Beyer14:04:09

You can see that PersistentArrayMap gets auto-promoted to PersistentHashMap for more than 8 keys:

user=> (class (into (array-map) (zipmap (range) (range 8))))
clojure.lang.PersistentArrayMap

user=> (class (into (array-map) (zipmap (range) (range 9))))
clojure.lang.PersistentHashMap
It is merely an optimisation for very small maps. You should not need to care about these low-level details and just consider both persistent maps.

Ferdinand Beyer14:04:40

Can you share some minimal code to reproduce your issue?

1
Roee Mazor14:04:48

ok, looks like you are right, it seems that my middleware probably fails because there is a nested list under that map

Roee Mazor15:04:07

Thanks guys, I just switched over to jsonista for now and I gave up on the response coercion and response documenting features for now. I will get back to it eventually I guess.

icemanmelting14:04:04

If this is a problem of sorting the hasmap, why not use a sorted map instead?

Roee Mazor14:04:22

@U9BQ06G6T , it is not, it is just me not using reitit and the middleware properly. Still haven’t gotten to understand whats going on there and how it should be used tho :)

icemanmelting14:04:10

I thought it was an ordering issue, as I have had that before, Array-maps tend to keep the order of insertions, but you can’t convert from one type to the other in a transparent way. Regarding reitit, I have never used it before 😄

lepistane14:04:24

I started using "1.11.0" version of Clojure but i am getting a lot of warnings about overshadowed vars (expected because of abs, update-keys, update-vals etc... ) is there anyway to hide these particular messages?

seancorfield17:04:54

Where these warnings appear in library code I've created GitHub issues against the various projects: medley, java-time, zprint, lacinia, selmer...

seancorfield17:04:20

Mostly the maintainers have already fixed the issues but not all of them have released new versions yet.

lepistane21:04:53

good idea! i didn't even think of that. Very nice. Thanks!

Alex Miller (Clojure team)14:04:07

in the namespace that shadows, you can (:refer-clojure :exclude [abs whatever]) in the ns declaration

🙌 1
Alex Miller (Clojure team)14:04:31

there is no other way to hide the warnings

👍 1
Jon Olick16:04:19

implementing flatten lazily is a bit tricky…

Jon Olick16:04:57

need to implement recursion without recursion over essentially as little state as possible

dpsutton16:04:59

not sure if you’re aware but flatten is already lazy. And its a pretty neat way to accomplish it with the general tree-seq

noisesmith16:04:40

late follow up - it is recursion, a special kind where the recursive call is baked into a thunk

JohnJ17:04:21

Is there a vim plugin that works with the built-in socket repl?

clj.max17:04:36

Try in #vim

kishima18:04:29

neovim conjure!

flowthing18:04:17

I believe Conjure is built on nREPL these days, no?

kishima19:04:33

oops, that’s correct, my bad

kishima18:04:12

Hello, is there a function to unqualify a namespaced keyword? Something like:

(unnamespace :qualified/banana)
=> :banana

Martin Půda18:04:37

(keyword (name :qualified/banana))
=> :banana

😃 1
kishima18:04:54

Ooooh thanks!

seancorfield18:04:46

@UUA5X8NRL Q: why do you want to unqualify a keyword? Qualified keywords are idiomatic in Clojure -- and if you need to produce JSON, the various JSON libraries have options to remove the qualifier when producing JSON objects.

kishima19:04:20

Hmmm, I was producing a JSON, actually, I’ll check for this option here, thanks!

seancorfield19:04:20

We use org.clojure/data.json at work and it does that automatically:

dev=> (require '[clojure.data.json :as json])
nil
dev=> (json/write-str {:a/b 1 :b/c 2})
"{\"b\":1,\"c\":2}"

seancorfield19:04:00

With Cheshire, you need to specify a :key-fn:

dev=> (require '[cheshire.core :as cc])
nil
dev=> (cc/generate-string {:a/b 1 :b/c 2})
"{\"a/b\":1,\"b/c\":2}"
dev=> (cc/generate-string {:a/b 1 :b/c 2} {:key-fn name})
"{\"b\":1,\"c\":2}"

seancorfield19:04:12

Not sure about any others.

seancorfield19:04:04

(I recommend clojure.data.json because it doesn't drag in the Jackson libraries which are notorious for both version conflicts and a number of well-known CVEs)

naxels18:04:16

Aaron and myself are having the craziest problem our project is upgraded to 1.11.1 and the deps.edn + deps-graph tool reflect that much. we even deleted the .cpcache folder however.. every time we start clojure or clj, it will kick back to 1.10.3 (I removed every Clojure version from the .m2 folder that is older than 1.11.1 even)

naxels18:04:33

it turns out that when we turn off just 1 dependency, it will actually work

naxels18:04:40

but when we leave it on, it will kick back

naxels18:04:43

very strange

naxels18:04:12

anyone has an idea where Clojure ultimately decided which Clojure version to use?

naxels18:04:50

i compared with a new project and using honeysql which has a clojure 1.9.0 dependency, but it will start 1.11.1 just fine.. very strange behavior!

naxels18:04:51

we caught this while trying to upgrade our app https://github.com/naxels/youtube-channel-data to use (iteration from 1.11

naxels18:04:37

Thanks for your help 🙂

naxels18:04:58

This is the Clojar link to the dependency that pulls us back to 1.10.3: https://clojars.org/org.clojars.crowbrammer/csv-exporter and here’s the code: https://github.com/Crowbrammer/csv-exporter

hiredman18:04:15

The dependency is likely badly packaged, and includes the clojure class files in it's jar

hiredman18:04:36

A library packaged as an uberjar

naxels18:04:51

that’s what we are starting to think as well, but is there any way to check for this in the JAR itself? I unzipped both csv-exporter and honeysql JAR files and found dependency for older Clojure in the pom files

naxels18:04:03

except that the issue doesn’t occur with honeysql

hiredman18:04:24

It wouldn't be in the pom, like if you unzip the jar it it should spill out clojure class files like clojure.lang.RT

hiredman18:04:41

clojure/lang/RT.class

naxels18:04:36

thanks, will check!

naxels18:04:06

jup, it’s got clojure in there

hiredman18:04:34

Yeah, badly packaged library

🙏 1
naxels18:04:26

yes, Aaron is going to resolve this

naxels18:04:31

it was our first time haha

Søren Sjørup20:04:03

I wonder if the standard library or other popular libraries has a function that does this:

(pairwise [1 2 3]) ;=> ([1 2] [2 3])
have not been able to find it in the standard lib or medley.

kj20:04:09

Maybe partition ?

user=> (partition 2 1 [1 2 3])
((1 2) (2 3))

Søren Sjørup20:04:56

Exactly what I needed! Thank you!

👍 1
pithyless21:04:23

@U021UJJ3CQ6 - something you should be aware of with partition (vs partition-all). They behave differently when you don't enough elements:

user=> (partition 2 [1 2 3 4 5])
((1 2) (3 4))

user=> (partition-all 2 [1 2 3 4 5])
((1 2) (3 4) (5))

🙌 1
Søren Sjørup17:04:43

Thanks a lot @U05476190 I had seen this before but forgotten about it!

James Amberger20:04:12

Hello. Trying to port a shell script. What should I do if I need to yield to interactive program/command?

noisesmith16:04:48

if you use java.lang.ProcessBuilder and the inherit option for stdio, it just works

Clojure 1.10.1
(-> ["vim"]
    (ProcessBuilder.)
    (.inheritIO)
    (.start)
    (.waitFor))
0
what doesn't show up in that snippet is that it immediately launched vim, and 0 was the vim exit code

noisesmith16:04:41

one thing to be careful of - if you don't call waitFor the repl and the sub-process will be trying to read the same content from stdin, with weird results

noisesmith16:04:44

wait - I might have misunderstood what you need here - if you just need to exit early to the calling script, use (System/exit 0)