This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-12-12
Channels
- # adventofcode (135)
- # announcements (1)
- # beginners (192)
- # boot (5)
- # calva (130)
- # cider (42)
- # cljdoc (4)
- # cljs-dev (6)
- # cljsrn (3)
- # clojure (222)
- # clojure-europe (2)
- # clojure-greece (5)
- # clojure-italy (24)
- # clojure-nl (23)
- # clojure-russia (1)
- # clojure-spec (6)
- # clojure-uk (67)
- # clojurescript (35)
- # cursive (39)
- # datomic (61)
- # emacs (17)
- # figwheel (3)
- # figwheel-main (2)
- # fulcro (12)
- # hyperfiddle (10)
- # juxt (3)
- # leiningen (10)
- # nrepl (35)
- # off-topic (34)
- # onyx (3)
- # pathom (6)
- # quil (5)
- # re-frame (29)
- # reitit (6)
- # ring (1)
- # ring-swagger (8)
- # shadow-cljs (37)
- # spacemacs (9)
- # sql (9)
- # tools-deps (24)
- # unrepl (1)
- # vim (1)
What's the best way to deal with the problem of recursively fetching something via rest api? so I need to walk folder structure in Box using its api. You can get a folder and its content in a single request, If a folder has 300 sub-folders, you need to send 300 requests. My recursive implementation using pmap and reducers/fold is way too slow. Can someone suggest any good blogposts or whatever? I'm thinking if core.async would be any faster here? Can someone suggest good approach (explain your idea at a high level), e.g. you send first request, then store the results in an atom, then map over that list, etc... etc...
Pathom is a library for that https://medium.com/@wilkerlucio/implementing-custom-om-next-parsers-f20ca6db1664
you may have the impression that Pathom is for om.next, but actually it can be used independently
it has support for async operations, too
I haven’t used pathom - would be interested in reading how it can be used independently thouhg
What is the slow part? I'd place my bets on the HTTP calls being the bottleneck, and then it makes little difference (on speed) whether you're using threads, channels or pathom. Just make sure your threads/channels aren't all waiting/parked and you should be near your max throughput.
How long does 1 API call take, and how long does 300 take? Is this “way to slow”?
but basically, start a go block that creates a bunch of other go block, then waits on results from each
it’s a flag you set on invocation: https://clojure.org/reference/repl_and_main#_launching_a_socket_server
I was showing one of the other students at my college clojure and they ran into one of the 1.10 error messages (def vs defn) almost immediately
If you can recreate, would be curious to see
And thanks for the feedback
is it possible to read lazy stream twice? like something from http stream. Or I better is to read everything to a string and use multiple times?
Reading it consumes it (it is stateful) so you can’t read it twice
how do I use named parameters in clojure.java.jdbc? Do I have to repeat the variable in the sqlvec if I want to use it twice?
I think #sql better place for this question
About clojure erros in 1.10
:
if you put a random macro in middle of some file (example: defn
) then load this file to repl:
Syntax error compiling at (user.clj:1:1).
Can't take value of a macro: #'clojure.core/defn
if you have a repro, would like to see it
is the user.clj in that namespace really a user.clj being side-loaded by the runtime or is it just a file with a longer namespace like foo/user.clj
[~]> cat test.clj
(ns test)
(defn foo [])
defn
[~]> clj
Clojure 1.10.0-master-SNAPSHOT
user=> (require 'test)
Syntax error compiling at (test.clj:1:1).
Can't take value of a macro: #'clojure.core/defn
I believe it’s accurate, asking whether this is a user.clj being loaded by the RT or if it is just from a normal require (which are different code/error report paths)
but I think that example is enough to answer that
@alexmiller is it out of the question that clojure might one day allow docstrings on keywords?
I'm finding myself attaching (sometimes quite subtle) meaning to namespaced keys in our programs, and I get the feeling that sometimes this meaning would be expressed better in a sentence than a name
::this-keyword-is-for-function-X-and-it-means-that-we-should-trigger-function-Y-after-it-finishs
keywords can't carry metadata because they are interned - symbols can have docstrings as they can have metadata
I mean I guess in theory a given keyword could have a docstring, and every instance of that keyword would have the same doc, but that would likely be a big change
that's fair
Exactly. I did that (and other questionable stuff) in https://github.com/mpenet/spex
as others stated above, yes it’s out of the question
independent context-specific registry keyed on keywords would be better
cool thanks for the answer 🙂 that makes sense
but is it out of the question a centralised registry for namespaced keywords that worked for doc
?
😄 my next question @bronsa !
doc already works for keywords
by looking them up in the spec registry
user=> (doc :clojure.core.specs.alpha/params+body)
-------------------------
:clojure.core.specs.alpha/params+body
Spec
(cat :params :clojure.core.specs.alpha/param-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
what is missing is the ability to specify doc or other meta for spec keywords
but we have every intention of adding that
ah and this is already a ticket, I remember seeing that
it is the highest voted ticket in jira :)
democracy.txt
When using Cheshire JSON lib, is there a way to tell it to drop map keys where value is null?
I can walk the structure after and do it myself, but it's not the most performant option.
you can't AFAIK @roklenarcic. I have a old library that can support that. https://github.com/ghadishayban/tinyjson/blob/master/src/ls/tinyjson/json.clj#L16-L22
Was surprised today to find that in 1.10 keyword
can only take symbols as values when passed 1 argument, not 2:
clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-RC5"}}}'
user=> (keyword 'foo)
:foo
user=> (keyword 'foo 'bar)
Execution error (ClassCastException) at user/eval3 (REPL:1).
clojure.lang.Symbol cannot be cast to java.lang.String
Converting a structure of entities with field names to a map with namespaced keywords:
[Person [first-name]] => {:person/first-name}
Wasn't able to find the Jira for the change where keyword
now accepts symbols :man-shrugging:
Thinking about it more, there'd have to be checks whether the symbol has namespaces, so I guess this makes sense.
I.e. what would happen when passing (keyword 'foo/bar foo/bar)
?
(! 577)-> clj
Clojure 1.9.0
user=> (keyword 'foo)
:foo
user=> (keyword 'foo 'bar)
ClassCastException clojure.lang.Symbol cannot be cast to java.lang.String clojure.core/keyword (core.clj:606)
user=>
Same behavior in 1.9.I think it's been that way for a long time.
oh? I thought keywords accepting symbols was new? I must be thinking of something else
No JIRA associated, but here's Rich's commit https://github.com/clojure/clojure/commit/d29219f78e51da66daf1c66108ebebb97c68442f
Clojure 1.9.0
user=> (symbol :foo)
ClassCastException clojure.lang.Keyword cannot be cast to java.lang.String clojure.core/symbol (core.clj:579)
I’m sure I saw some work being done there recently (the name demunging part), but it seems it doesn’t work in this particular case?
that’s in a different place (error message printing in main)
is there a specific ticket that he is “waiting” for?
I am not aware of one, happy to have one or more tickets on clojure.test reporting
hard to say “waiting for Clojure.core” is the right phrase when no one has logged or worked on it? there is no resistance to improving this.
Twitter sucks. Sorry for bringing this up in such a way, I could have filtered the message instead of just passing it along.
those error messages aren't great. perhaps his patch can land in the actual lib? twitter seems to have worked here?
I mean, great that that person is happy with what they have, but everytime I have used some kind of stracktrace manipulating(printing,formatting,filtering) library that someone else wrote it has driven me nuts
@orestis no worries, happy to see it as I’m not actively looking at twitter these days
I think this is a good way to stay sane :)
For a long time I’ve had my twitter in notification-only mode, where I only see DMs and mentions. It’s a huge life improvement.
I am a huge fan of https://github.com/stuartsierra/stacktrace.raw
there are at least 3 ideas bound up in that change above
1) focusing on root cause in chain - endorse
2) filtering stack head to remove test infrastructure - endorse (and we’re already doing that - this is just more aggressive)
3) demunging from Java stack to Clojure function - I’m a bit more ambivalent about this one but not necessarily opposed in this context
glad that you guys are looking at error reporting!
but the way to approach all of these is by backing up to principles and goals, like we did with repl error reporting
and determine the set of changes needed to improve what it does
so that it satisfies the goals in line with the principles
filtering out and prettying up of stacktraces is all well and good, until you get an exception where the parts you want are prettied away, then it is infuriating
eliding the top stack frames that are the test infrastructure itself is generally something I favor, and that is already being done
cursive does something similar when debugging, just not showing stack frames in clojures internals
I think there is actually a ticket in the ballpark of part of #2 in https://dev.clojure.org/jira/browse/CLJ-1811
I’m not a big fan of removing intermediate or bottom stack frames as I find those too often have info I want, but having some kind of a switch or pluggable filter for that kind of thing is a possibility
the thing above is about clojure.test reporting
similar but different issues in pst
huh, I find test failures one of the more straight forward errors as it usually says which test case broke on what assertion
Sophisticated environments like CIDER offer just that – ability to toggle "uninteresting"/infrastructural frames. If the tool is on the simpler side (e.g., a bare clj
REPL), it's better to err on the side of overtelling, at least for me.
All those "improving" libraries actually don't do any good in the long run. Particularly, because of aviso I now can never tell which end I should start reading the exception from.
on websites like rollbar, they shrink/collapse the non-app stacktraces, but let you open them all up. it's very nice to switch between
idk how easy that is on the command line
I agree that filtering stacktraces is, well, depends on a situation and prob. shouldn’t be in core
But I think no one would argue that datascript.query_v3$resolve_clause.invoke
is less readable than datascript.query-v3/resolve-clause
. And it seems we can do this sort of transition back to “what was actually called” pretty reliably?
also two frames (.invoke and .invokeStatic) per clojure call aren’t helping anyone I believe
wouldn’t it be nice if all this was translated back to clojure-like names, so you don’t have to be familiar with how Clojure code is compiled to read stack traces?
make a ticket
final-draft-really-final-v3
nothing has changed there
we didn’t touch pst in 1.10
I see that now. I compared to 1.10-0-beta3 which for some reason has (pst) results unmunged. Nevermind 🙂
I would not expect there to be any difference in any 1.10 release in pst
we are demunging the location you see in the first line of the exception message in many more cases (the user/mydiv), so that’s different in 1.10
Right, I confused demunging traces with https://dev.clojure.org/jira/browse/CLJ-2438
#pre-git-life
too late for 1.10
if they implement the right interface, sure
right
that's the interface you would need
(for getting meta - I assume setting it is defined elsewhere)
@alexmiller Re: clojure.test
-- it hasn't had any love for a long time (only one commit -- for 1.10 -- since December 2015): is that because Cognitect folks don't use it much? Or it's considered "good enough" by them? Or just because no one is really advocating for improving clojure.test
?
Just general lack of attention I think
But also there is less complaining about it than lots of other things
Fair enough. I'm planning to have another run at Expectations at some point soon to get it to a 2.0 gold release and that should give me more insight into clojure.test
so I may start picking over some of the open JIRA issues for it. Or adding some new ones, with patches, or at least kicking off discussions... Maybe.
We wrote circleci.test to work around issues with clojure.test. Most acutely, the direct linking in 1.8 broke a lot of our test tooling. We also added the concept of global fixtures that are applied once per run of a test suite. https://github.com/circleci/circleci.test
I've seen a few complaints about fixtures not being flexible enough...
No tickets I’m aware of
Too much effort, apparently 😆
I definitely feel that clojure.test
fixtures could be more useful.
Being able to run a fixture once for a whole run is definitely one thing.
In the past I’ve also found it frustrating that fixtures are configured at a namespace level; either once for the ns or for all deftests
in a namespace. Sometimes it would be nice to be able to exclude a few deftests from being within a fixture etc. Perhaps with a metadata flag.
Usually though it’s been easy enough to work around by just wrapping the deftest form itself in a with-fixture
, so it’s not been something I’ve felt too strongly about. However I have many projects now that implement a similar thing in slightly different ways.
with clojure.test, the namespace is a meaningful “group” of tests. rather than exclude some, put them in a different namespace.
but having a per-run fixture is a reasonable idea
I think having built-in selector support would be useful though, like what lein or test-runner provides
> with clojure.test, the namespace is a meaningful “group” of tests.
That is certainly one way to arrange it.
The other which in my experience is better supported with editor tooling is having a foo.bar-test
ns for each foo.bar
ns under test.
Granted the 1-1 mapping is also an artificial source of annoyance; but if you follow that convention editors can jump you to a test definition by convention.
Ultimately I think the conventions here are a problem though; and that this sort of thing is probably better being data driven from an edn file or metadata forms which editor tools could read. I think kaocha seems to be taking this approach.
but mapping test ns’s to sut ns’s can be a problem
imo, the 1-1 thing is too dogmatic
I do agree with that. It’s just a shame you can’t have your cake and eat it
it’s a good general starting point but I find many reasons to deviate from it
is jump-to-test really that critical? can’t say I ever do that, but maybe I’m an outlier
As a project scales I definitely feel the urge to diverge from the 1-1 mapping. Though you can partly take that as an indication to restructure your apps namespaces too, so maintaining the 1-1 mapping isn’t a problem. Regarding jump-to-test etc… It’s not just jump-to-test; though I find that very useful; it’s also having the editor create a test ns file for you in the right place according to convention. It can be frustrating when you want to add a small test to have to build directories etc and name a file and then setup the ns requires etc…
I suppose
Not a massive problem I grant you; but friction here can lead to some tests not being written.
I guess what would be helpful for me (in the big picture) is to have a prioritized list of the worst pain points for clojure.test
Yeah… FWIW we use clojure.test inspite of its flaws; it’s certainly good enough. It’s nice to just have something that’s there - even if it has some rough edges. The above issues are pretty minor really; I feel there are perhaps some deeper issues too though. Writing custom assertions, and sometimes getting useful test failure messages out has been a pain for me in the past. I may well have just been doing it wrong. Also perhaps a bigger issue is integrating clojure.spec generative tests into a clojure.test suite. I’ve written some infrastructure (outside of clojure.test) that does this in some projects - but more native support would be good.
Where do you think we should assemble such a list for prioritising?
some shared web location :) google spreadsheet, design wiki (https://dev.clojure.org/display/design/Home), etc. there may even be some prior lists on the design wiki already.
don’t see any lists there
me neither
I’m glad to hear that splitting out clojure.test is being considered. I would like to see this taken one step further, and the test runner being split from the assertions library.
https://github.com/cognitect-labs/test-runner is that to some degree
I think if we’re talking about changing what’s “in the box” with clojure.test, the changes need to be non-breaking or additive. if you want to do something substantively different, the only reasonable thing to do is put it in a new library (which many people have already done). So I don’t see how splitting the runner out of clojure.test helps anything. Either incrementally improve what’s there, or make a new thing, and if you’re making a new thing, don’t put it in core, and at that point nothing is blocking you from moving forward, and plenty of people have already done it.
does that make sense? I’m hashing this out as I type :)
What is the argument for splitting out the runner? Is it that the runner is more of a common infrastructure concern, and should therefore be closer to core (for tooling support etc?) than the choice of how you express assertions??
what I’m actually fishing for is in CLJS, how do I add metadata to a ReactElement. It seems that it’s just a bare JS obj, it has no special prototype it inherits from
but I’m loathe to implement IMeta / IWithMeta for object
since I have no idea what that might do
(I did a lot of work on Expectations, getting it compatible with clojure.test
, so that all the tooling out there would work better with Expectations, but the failure reporting is really basic and, as @tonsky mentioned, stacktraces are pretty horrible when a test does blow up)
re: stacktrace filtering, Cursive does it using code folding, which means it’s hidden by default but you can undo it if you need to. That’s tooling-dependent obviously, but it works well.
There are libraries to "improve" clojure.test
failure reporting but they all rely on extending the test reporting machinery as I recall and they can easily step on each other if you accidentally combine them.
I like expectations and covered it in Clojure Applied, along with clojure.test and test.check
I find in Cursive’s folding that the first thing I end up doing is unfolding most of it. That is, I find it more irritating than useful.
But in a couple dimensions I assume I am off the norm
One thing we have batted around is pulling clojure.test out of core into a lib (like spec, so still available through a dep) to allow it to move faster
@alexmiller Settings->Languages & Frameworks->Clojure->Prettify stacktrace elements using folding & Fold stacktrace lines from outside project
Thx :). I am so lazy at configuring my tools
Not sure if those are as useful to split out
Are any of those 3 likely to receive improvements? Or perhaps better asked as - are there pending improvements that are not being looked at?
There are things out there for all of them and complete rewrites of both floating around
clojure.test seems like the most likely candidate since others have been inspired to write e.g. eftest
Hasn't there been a metadata bug in walk for a while? i'm thinking of ridley and metadata in macro expansion i think
clojure.walk
finally got a fix for pre walking hash maps, as I recall. Was that 1.10 or in 1.9?
postwalk
and, yes, 1.10.
CLJ-2031
Say, anybody got a link to a good tutorial on zippers? Ideally Clojure, but any language is cool, I think I'm missing some basic concepts
I wrote https://www.ibm.com/developerworks/library/j-treevisit/ a billion years ago
Maybe I can list some or all of these on the Clojure survey for possible improvement areas

Glad folks are finding it useful.
Those both look great, thanks guys