Fork me on GitHub
#clojure
<
2020-07-10
>
datran03:07:09

I'd like to create my own edn tag, are there any good guides for doing that? The edn specification mentions that you can do it but I can't find a reference for how to do it

Alex Miller (Clojure team)03:07:38

just use any tag you want in edn data. I assume the question is how to read it?

Alex Miller (Clojure team)03:07:19

clojure.edn/read and clojure.edn/read-string take an opts map that includes a :readers map from tag symbol to data reader function https://clojure.github.io/clojure/clojure.edn-api.html#clojure.edn/read

datran04:07:15

ok, thanks, that's really simple. Now, if I want to provide this tag in a library that others can consume, how do I hook into the caller's reader?

Alex Miller (Clojure team)04:07:16

if you're talking about edn reading, then it's up to the caller

Alex Miller (Clojure team)04:07:10

if you're talking about tags in code, you can provide readers in a lib in a data_readers.clj resource

datran04:07:39

thanks! That is exactly what I was looking for : )

datran04:07:37

and it works perfectly! :thumbsup:

Romit Gandhi06:07:45

Does anyone developed any chat app using ejabberd or openfire using clojure? Please message also if anyone developed with other than above 2.

jrychter09:07:53

I'm looking at a code base which has lots of tests where the order is swapped (as opposed to what clojure.test expects): (is (= form result)) instead of (is (= result form)). Does anybody know of a tool that will help me swap all those around?

yuhan10:07:45

This is quite simple in Emacs, define a macro or command that searches for "(is (=", and then forward-sexp and transpose-sexps, repeat till end of file

jrychter12:07:49

Thanks — transpose-sexps is what I was looking for. I wrote an Emacs function and it does 99% of the work.

👍 6
noisesmith15:07:00

clojure.test doesn't care about the order, it doesn't even care if you use =, it just tells what form failed and what was actually present

noisesmith15:07:13

(ins)user=> (is (= :a :b))

FAIL in () (NO_SOURCE_FILE:1)
expected: (= :a :b)
  actual: (not (= :a :b))
false
(ins)user=> (let [some-random-value nil] (is some-random-value))

FAIL in () (NO_SOURCE_FILE:1)
expected: some-random-value
  actual: nil
nil

noisesmith15:07:23

perhaps some other tool you are using is making extra assumptions?

vemv16:07:19

https://clojure.github.io/clojure/clojure.test-api.html There's a notion of expected and actual. API usage is assumed to be = expected actual and not the opposite Failure to follow this convention can lead to confusing reports, if a test fails

vemv16:07:11

There's further evidence of this fact in that https://github.com/magnars/kaocha-noyoda exists.

noisesmith16:07:13

expected and actual are your input form and the result this has nothing to do with =, which is has no special cases for koacha cares, clojure.test doesn't

noisesmith16:07:49

you still don't get it: expected and actual are the same form, before and after evaluation

noisesmith16:07:03

(ins)user=> (let [some-random-value nil] (is some-random-value))

FAIL in () (NO_SOURCE_FILE:1)
expected: some-random-value
  actual: nil

noisesmith16:07:07

expected: the form (`some-random-value`) actual, the evaluated result (`nil`) - clojure.test isn't using = and doesn't care which part of an equal form was your input vs. a constant, they could both be inputs and no constants for higher order testing

noisesmith16:07:32

your mental model of clojure.test is wrong

noisesmith16:07:10

Some people (probably people coming from imperative languages where = is assignments) have previously learned "yoda testing" as it avoided accidental assignment by putting an rvalue on the left. This is a style question and has nothing to do with what clojure.test expects.

vemv16:07:24

IDK. Could be, but also different people have recommended = expected actual order to me over the years. I'm also quite sure that some failure reports were confusing when not honoring this order. (...maybe the confusion has more to do with the person reading it than with any reporting/macroexpansion)

noisesmith16:07:36

the failure report treats the whole (= x y) as a single element, if it's hard to read it's because you and the author disagree about where x and y go

noisesmith16:07:12

it's a style question and has nothing to do with the machinery of clojure.test, which will happily do its job even if you never call = at all

👍 3
noisesmith16:07:08

I have tests that do (let [a (f x) b (g x)] (is (= (h a) (h b)) "f and g are h-compatible")) - clojure.test doesn't care (but a reader might find them weird) - neither side of the comparison is constant

noisesmith16:07:50

a more understandable example of the above

(ins)user=> (require '[clojure.test :refer [is]])
nil
(let [a (keyword "x")
      b (symbol "x")]
  (is (= (name a)
         (name b))
      "keyword and symbol are name-compatible"))
true
and failing case
(let [a (keyword "x")
      b (symbol "x")]
  (is (= (str a)
         (str b))
      "keyword and symbol are str-compatible"))

FAIL in () (NO_SOURCE_FILE:3)
keyword and symbol are str-compatible
expected: (= (str a) (str b))
  actual: (not (= ":x" "x"))
false

Ramon Rios15:07:49

Hello, i 'm trying to replace keys in a map for strings, Anybody has some insight of how to do it?

emccue15:07:18

@ramonp.rios If you don't mind including a library or copy pasting code

emccue15:07:28

then (map-keys m name)

noisesmith15:07:58

the code to do it by hand also works, and is a one liner

noisesmith15:07:25

(into {} (map #(update % 0 name)) m)

noisesmith15:07:56

I'd even argue it's idiomatic

noisesmith15:07:39

that's nice for the recursive unconditional case, yeah

👍 6
euccastro15:07:41

it does that recursively though

Ramon Rios15:07:46

thank you all for the insights 😄

noisesmith15:07:13

one way that stringify-keys is better than map-keys or my one liner, is it ignores all the non-keyword keys and leaves them as is

Ramon Rios15:07:08

Stringify-keys resolves my problem

Ramon Rios15:07:32

I was creating a excel file but the headers are filled with keywords instead of the name of the keyword

noisesmith15:07:25

another option is to disable the option that created the keywords in the first place, IMHO clojure devs are over-eager to turn outside input into keywords

💯 6
Daniel Stephens16:07:17

Anyone know what's going wrong with my integrant configuration here. I use aero with integrant and I am having trouble doing a merge of a normal map with an integrant key.

{:ig/system
 {:worker-operator.client/client {}
  :worker-operator.monitoring/web-server #merge [{:port 8081}
                                                 #ig/ref :worker-operator.client/client]}}
:worker-operator.client/client successfully builds a map on startup containing {:client ...} Unfortunately I can't get the #merge to work. Seemingly in the order displayed, I get the map {:port 8081, :key :worker-operator.client/client} as config to monitoring/web-server. If I swap the order [#ig/ref :worker-operator.client/client {:port 8081}] then I get {:client ...} into monitoring/web-server as I want but it's then missing the port key.

Daniel Stephens16:07:42

Might be important that I have

(defmethod aero/reader 'ig/ref [_ _ value]
  (ig/ref value))
as for that ig/ref tag

Daniel Stephens16:07:54

realising it probably isn't possible to combine them in this fashion since one requires the information upfront and the other is resolved later, probably should have been in #beginners mb!

tcrawley20:07:18

Hiya! Does anyone know if there is a way to have test.check report which generator failed in this case?

ERROR in (some-test) (generators.cljc:435)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries.
{:pred #object[clojure.spec.alpha$gensub$fn__1876 0x56930e8b "[email protected]"], :gen #clojure.test.check.generators.Generator{:gen #object[clojure.test.check.generators$such_that$fn__804 0xda5ceeb "[email protected]"]}, :max-tries 100}
 at clojure.test.check.generators$fn__801.invokeStatic (generators.cljc:435)
    clojure.test.check.generators/fn (generators.cljc:434)
    clojure.test.check.generators$such_that_helper.invokeStatic (generators.cljc:425)
    clojure.test.check.generators$such_that_helper.invoke (generators.cljc:419)
    clojure.test.check.generators$such_that$fn__804.invoke (generators.cljc:478)
    clojure.test.check.generators$gen_fmap$fn__674.invoke (generators.cljc:59)
    clojure.test.check.generators$call_gen.invokeStatic (generators.cljc:43)
    clojure.test.check.generators$call_gen.invoke (generators.cljc:39)
    clojure.test.check.generators$such_that_helper.invokeStatic (generators.cljc:427)
    clojure.test.check.generators$such_that_helper.invoke (generators.cljc:419)
That test uses a tree of generators, and fails < 1% of the time, so tracking down the failing generator is difficult.

noisesmith20:07:36

the name of the fn is there, but it's auto-generted - probably #(...) or (fn [] ...) - maybe you could strategically rename (fn [] ...) to (fn some-name [] ...)

noisesmith20:07:33

oh wait no, it was generated inside gen-sub

noisesmith20:07:35

I read it wrong

tcrawley20:07:49

Yeah, that was the comment I was about to make :)

tcrawley20:07:37

I haven't dug into the test.check source yet, but maybe I can modify it somewhat to give me more info

noisesmith20:07:41

there might also be some trick for adding tap> to each generator (and attach an identity generator with an identify fmap as needed), then using an add-tap that stores the contexts in order, and see which one came in last at time of error

noisesmith20:07:10

but that's still indirect, and quite hacky

tcrawley20:07:33

That's not a bad idea! I don't mind a hack if I can find the offending test. If that works, I'd have this in my toolbox for next time. Thanks @noisesmith! Also, good to talk to you, it has been a while :)

datran21:07:08

I've implemented my own datastructure, a multiset, and I've learned a lot about tagged literals and implementing clojure protocols by doing it.

datran21:07:08

Everything works exactly how I'd like, except when I try to use my datastructure from a new namespace

datran21:07:40

here's the repl example:

(ns my.multiset)
  ;; => nil
  #mset [1 2 3]
  ;; => #mset [1 2 3]
  (multiset [1 2 3])
  ;; => #mset [1 2 3]
  (ns my.other)
  ;; => nil
  (require '[my.multiset :as mset])
  ;; => nil
  #mset [1 2 3]
  ;; => Syntax error (IllegalArgumentException) compiling fn* at (*cider-repl projects/paragon:localhost:33015(clj)*:0:0). Unable to resolve classname: IPersistentMap

datran21:07:21

(mset/multiset [1 2 3])
;; => #mset [1 2 3]

datran21:07:59

Where is that syntax error come from? The only place I reference IPersistentMap is in the deftype:

(deftype Multiset [^IPersistentMap meta
                     ^IPersistentMap multiplicities
                     ^int size]
    ,,,)

hiredman21:07:04

there is an error in the data reader for the #mset tag

datran21:07:16

And my reader function is really simple, it's just (defn multiset-reader [x] (multiset x))

datran21:07:51

here's my data_readers.cljc: {mset my.multiset/multiset-reader}

hiredman21:07:03

I would look at the metadata on the multiset function

hiredman21:07:03

I forget if this is still a thing with current versions of clojure, but older versions for a long time required type hints for return values to be fully qualified to work correctly

datran21:07:52

(meta mset/multiset) ;; => nil

noisesmith21:07:17

the metadata would be on the var

dpsutton21:07:37

(contrast (meta map) with (meta #'map))

datran21:07:26

ah, there it is `

datran21:07:37

{:arglists ([] [s]),
 :doc "Create a multiset from a sequence",
 :line 124,
 :column 1,
 :file "/my/multiset.clj",
 :name multiset,
 :ns #namespace[my.multiset]}

datran21:07:18

what sort of stuff should I be looking for? This is my first time peering behind the curtain of clojure datastructures, so I'm fumbling through

datran21:07:35

(I'm pretty impressed with how simple it's been so far, honestly)

hiredman21:07:49

my next guess would something nrepl middleware related

hiredman21:07:41

you could try evaling something like (do #mset [1 2 3] nil)

hiredman21:07:36

the idea is you want to isolate which part of the repl the error is coming from: read, eval, or print

hiredman21:07:50

so the above returns nil to be printed, so if you don't get an error with that, but you do with #mset [1 2 3] you know the error comes from printing (or even technically after printing, which would be where some middleware might be trying to mess with things)

datran22:07:02

Ok, so I bounced my repl for the first time in a few days and found some issues. Those are cleared up now and I'm still getting the same error. Running your do-block above blows up, so I think the conclusion there is that it is a problem with the reader

seancorfield22:07:59

@datran Since you have CIDER / nREPL in play, I would try the exact same REPL session with a plain ol' console REPL -- using clj rather than lein -- just to see if this is a problem with nREPL or CIDER.

seancorfield22:07:36

If the same REPL session that fails via CIDER works via plain lein repl with you typing stuff in, it's a CIDER issue. If it fails in lein repl but works in clj then it's nREPL.

hiredman22:07:47

the way to isolate that would be something like (read-string "#mset [1 2 3]") but you'll need to make sure read-string understands your tag and I forget the details of doing that

datran22:07:53

When I run it through the edn reader, things are fine: (clojure.edn/read-string {:readers {'mset mset/multiset-reader}} "#mset [1 2 3]") ;; => #mset [1 2 3]

datran22:07:12

When I run in a plain clj repl, it still blows up:

clj
Clojure 1.10.1
user=> (require '[paragon.multiset :as mset])
nil
user=> #mset [1 2 3]
Syntax error (IllegalArgumentException) compiling fn* at (REPL:0:0).
Unable to resolve classname: IPersistentMap

hiredman22:07:29

so the next thing would be (eval #mset [1 2 3])

datran22:07:47

read-string is fine too

hiredman22:07:35

which, come to thing of it, that must be it

seancorfield22:07:39

@datran did you try @hiredman’s earlier suggestion of changing the type hints to fully-qualified versions, with clojure.lang.IPersistentMap etc?

hiredman22:07:15

the compiler, when figuring out how to embed your type in bytecode, is choking on the type hints

hiredman22:07:21

(similar to function return type hinting thing I mentioned earlier)

hiredman22:07:21

I forget again, but deftype in general might not like it if you type hint non-primitive fields, but it has been a while

datran22:07:32

ah, I was thrown by your reference to hinting a "return value", since the only type hints I had were for the inputs

datran22:07:32

And making those fully qualified appears to fix it!

datran22:07:05

Thanks everyone, I would never have figured that out on my own :thumbsup:

dpsutton22:07:17

did you include the type hints because you were correcting reflection warnings or just adding them?

datran22:07:37

I was just adding them. The example I was following had them, so 🤷

datran22:07:29

I guess step one should have been removing them to see what happened

hiredman22:07:03

https://clojure.atlassian.net/browse/CLJ-252 must be what I am thinking of, not exactly related

lilactown22:07:29

is there any info about doing concurrent work inside of a transaction w/ Clojure STM?

lilactown22:07:06

like would it be reasonable to schedule work within a transaction on a thread pool or something similar to that?

Alex Miller (Clojure team)22:07:53

The stm is specifically designed to work with agent sends - it delays sends until the transaction completes

lilactown22:07:34

oh? interesting

Alex Miller (Clojure team)22:07:43

Generally, txns will be retried so you should not do any other kind of io

lilactown22:07:04

that's very useful for me actually, but not quite what I was originally thinking of

lilactown22:07:10

what I'd like to do is split up the work done to calculate the ref values within a transaction, across some threads

lilactown22:07:33

so say I have 20 refs that I want to alter in a transaction. I split that work up across 4 threads, and if any of the alterations fail then it should retry or abort all of them

Alex Miller (Clojure team)22:07:38

That’s not going to work

Alex Miller (Clojure team)22:07:23

Or at least I wouldn’t attempt it

lilactown22:07:29

I’m interested in learning why, if you have the time to explain

Alex Miller (Clojure team)23:07:21

You’re way outside the design space for stm

Alex Miller (Clojure team)23:07:59

Txns should be short and retryable - doing anything multithreaded seems like you’re doing this wrong. There are way more failure modes

lilactown23:07:00

that makes sense

Alex Miller (Clojure team)23:07:32

I’m not saying it won’t work :)

lilactown00:07:15

I guess, the part that attracted me to STM is the transactionality. the properties that I need for my use case are: • an alteration isn’t “committed” until the end of the transaction • that a failure causes the whole transaction to fail • safe to use concurrently (for some definition of “safe”) in that way some of the specifics of the STM are useful but not necessary (e.g. retry on conflict)

lilactown00:07:18

the use case i’m working with is this idea of a graph of computations whose inputs can be changed, and then the graph is stabilized by running each computation and storing their result inside of a ref. this way, a result is only stored if the whole stabilization succeeds. one of my goals is to allow parallelization of computations. but maybe i’ll punt on that for now

isak22:07:02

What do Clojure people think of how scopes work in https://janet-lang.org/docs/specials.html? (You can add new bindings without increasing nesting):

# Prints 1
(do
 (def a 1)
 (print a))

# a is not defined here, so fails
a

lilactown23:07:18

personally I like the distinction between global definition (`def`) and local binding (`let`), but that feels like an aesthetic preference

3
💯 3
yedi23:07:30

hey yall, is there a way to create a io/writer that writes to a string? or does that not make any sense? I'm trying to use csv/write-csv to output a valid CSV line as string. I'm not using str/join to do it because I need write-csv to handle escaping commas for me

dpsutton23:07:35

Bytebufferoutputstream is what you want. (Correction java.io.ByteArrayOutputStream)

phronmophobic23:07:54

I'm in the same boat as @lilactown. additionally, I'll sometimes temporarily group top level defs in a do so that they all reevaluate together when I do eval-top-level-form in my editor. for example:

(do
  (defn my-view [] ...)
  (defn uses-my-view [] (my-view)))

lilactown23:07:54

I've definitely used it to great success when writing macros

6
3
lilactown23:07:17

(defthing ,,,) can emit multiple defs by wrapping them in a (do ,,,)