Fork me on GitHub
#clojure
<
2017-11-08
>
aaelony00:11:53

@seancorfield the cause isn't that helpful but I'll post it:

Exception in thread "main" java.lang.RuntimeException: Problem parsing near line 1 < [taoensso.timbre :refer [info warn error] :as timbre]> original reported cause is java.lang.NoSuchMethodException: clojure.lang.LispReader.matchSymbol(java.lang.Str
ing) -- java.lang.NoSuchMethodException: clojure.lang.LispReader.matchSymbol(java.lang.String), compiling:(/tmp/form-init450743688382126384.clj:1:72)

aaelony00:11:11

looks ns related though, what is the issue with illegal ns syntax in 1.9?

seancorfield00:11:43

1.9 checks ns syntax more stringently than 1.8 did

hiredman00:11:44

that is a reader error, before any macros would see anything

seancorfield00:11:06

Yeah, that's what I was thinking having seen the error.

seancorfield00:11:24

Or maybe an AOT issue?

hiredman00:11:39

nosuchmethod is rather odd

hiredman00:11:00

yeah, a library aot'ed against a different version of clojure would do it

seancorfield00:11:37

@aaelony So it's probably specific to your code base setup rather than something in Marginalia per se...

hiredman00:11:28

timbre can be really bad for that, because people will write extensions to whatever java logging framework and depend on timbre for whatever reason, but they need to be aot compiled, so you can end up with an aot'ed version of timbre coming in from some other jar

seancorfield00:11:52

Repro'd

Exception in thread "main" java.lang.RuntimeException: Problem parsing near line 1 < [taoensso.timbre :refer [info error warn] :as timbre])> original reported cause is java.lang.NoSuchMethodException: clojure.lang.LispReader.matchSymbol(java.lang.String) -- java.lang.NoSuchMethodException: clojure.lang.LispReader.matchSymbol(java.lang.String), compiling:(/private/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/form-init7849628237322743095.clj:1:125)

aaelony00:11:57

well, I can change anything. No issue though for lein marg under 1.8.0. I don't even have to use lein marg, but why not?

seancorfield00:11:52

Calls the LispReader directly -- I suspect the calling arity changed in 1.9?

seancorfield00:11:28

Ah, there we go. Looks like matchSymbol expects two arguments now? A string and a resolver...

seancorfield00:11:28

I added a comment to that issue. Maybe @gdeer81 will be able to chime in?

aaelony00:11:07

thank-you, Sean

bcbradley01:11:29

I wonder if Rich would be ok with clojure 2.0 having breaking changes to remove warts from the core language, or if he wants clojure to be eternally backwards compatible regardless of whether the occasional bad design decision aggregates or not

bcbradley01:11:05

I mean clojure is a very well designed language but there are a few ugly parts that could use a do-over

bcbradley01:11:34

i know he mentioned that if he could do it all over again reduce would require an initial value

noisesmith01:11:28

and rename for to something like sequence-comprehension

noisesmith01:11:37

(my suggestion, not anybody elses)

bcbradley01:11:03

and I tend to think that lazy evaluation of the elements of a sequence and the actual transformation performed over a sequence are orthogonal concepts. its unfortunate that transducers are complected with the idea of immediate evaluation.

noisesmith01:11:25

are they? you can transduce lazily

bcbradley01:11:12

(transduce ...) isn't truly lazy

noisesmith01:11:29

but that’s not the only transducing context - and right, transduce is not lazy

bcbradley01:11:18

what i'm really getting at is that the core api is more complex than it needs to be simply because it was necessary to keep it backwards compatible and not make a breaking change

bcbradley01:11:53

what you'd really want to do is be able to define a set of transformations (probably a sequence of transformations actually)

noisesmith01:11:09

sequence is lazy, and eduction

noisesmith01:11:34

and defining a transducing function is exactly what you describe

bcbradley01:11:02

then you'd want to separately define how those transformations are applied to a sequence of data (lazy, not lazy, across multiple cores or not, etc)

bfabry01:11:06

transducers are literally a way to define a set of transformations without worrying about representation. the transduce function is perhaps a bit misleading as it's a way of applying transducers

bcbradley01:11:31

right now we have reducers in the core library for handling the parallel case

noisesmith01:11:36

@bcbradley you are describing exactly how transducers are used

bcbradley01:11:06

let me be more specific then

bcbradley01:11:16

what i'm saying isn't that something should exist

bcbradley01:11:22

its that something shouldn't exist

bcbradley01:11:32

there shouldn't be an api for doing things "without transducers"

bcbradley01:11:48

transducers are the bottom, simplest (most decomplected) case

bfabry01:11:03

eh, that's a position to take. it'd make the language less approachable though

madstap01:11:12

@bcbradley Do you know where Rich said that about reduce requiring an init val? I remember that as well, but I tried looking for it once and couldn't find it.

madstap01:11:48

And I think he has said that clojure 2.0 just isn't going to happen...

bcbradley01:11:00

there is something else i find a bit puzzling though-- we use transducers to convey what is basically loop unrolling to the compiler-- we really don't NEED then to pass around the idea of doing a specific sequence of transformations, because we could just use apply to apply a sequence of transformations or we could pass around an s-expression that represents the transformations

sundarj01:11:04

@madstap he mentions it in his effective programs talk

madstap02:11:22

Yeah, I remember hearing/reading it longer ago than the conj.... I have a crappy memory though, so you never know ¯\(ツ)

sundarj09:11:41

sorry, i was mistaken - it's the Inside Transducers talk

sundarj09:11:36

>>>Who knows what the semantics of reduce are when you call it with a collection and no initial value? [Audience response] No one, right. No one knows. It's a ridiculous, complex rule. It's one of the worst things I ever copied from Common Lisp was definitely the semantics of reduce. It's very complex. If there's nothing, it does one thing. If there's one thing, it does a different thing. If there's more than one thing, it does another thing. It's much more straightforward to have it be monoidal and just use f to create the initial value. That's what transduce does, so transduce says, "If you don't supply me any information, f with no arguments better give me an initial value."

seancorfield16:11:27

Yeah, reduce is ... weird. When I first implemented reducible-query in clojure.java.jdbc I got the no-init arity of reduce wrong because I assumed the semantics of transduce. The reduce docstring is very clear -- the behavior is just a bit non-intuitive.

bcbradley01:11:31

basically i've been thinking and i'm trying to understand whether or not transducers are really ACTUALLY simple, or whether or not they are just an implementation detail

bcbradley01:11:07

wouldn't a sufficiently intelligent compiler be able to inspect an s expression before evaluating it and rearrange its subexressions algebraically to do what transducers are effectively doing?

bcbradley01:11:46

or heck, a macro could do that

bcbradley01:11:01

the async library has really magical macros that tear apart s expressions

madstap01:11:21

A macro is a compiler, in a way

bcbradley01:11:45

i'm just thinking that transducers, while they capture the idea of performing a computation independent of the type of the collection or data source, and in that sense are a basic fundamental thing, realistically aren't NECESSARY to extract the information about what sequence of transformations are actually taking place, since this is a lisp after all

bcbradley01:11:21

i'm really not sure how I feel about it, its either transducers (and the initial difficulty in learning them) or the magic of a macro

bcbradley01:11:03

imagine that if instead of having go channels take a transducer you just had something like (magic (map f (filter g chan))), where magic is responsible for doing what transducers essentially do for chans

bcbradley01:11:43

it would do all the unrolling itself, and ensure that you didn't have intermediate collections or channels

bcbradley01:11:01

furthermore you can extend magic beyond what transducers are strictly capable of

bcbradley01:11:13

you can use it with other (non transducible) forms

ghadi01:11:46

If you want to use transducers on a lazy source call (sequence xfn coll)

ghadi02:11:34

(where xfn is the transducer)

bcbradley02:11:01

i'm aware of that @ghadi i was just criticizing some of clojure's design

madstap02:11:26

I think there is an option somewhere in between those. I was thinking about that the other day. Transducers are kind of ... fragile?... they have all these rules like don't touch the result so far, call the next rf in the chain in the cleanup arity, zero arity just calls (rf), etc. Could a macro kind of like fn take care of all that stuff instead of us having to write all that boilerplate?

ghadi02:11:08

Rich has made it really clear that despite some regrets, compatibility will be preserved. There will not be any attempt to "fix mistakes" and break everyone's code. This is why it's important to commit to and promise less

ghadi02:11:08

(see the "speculation" keynote)

ghadi02:11:33

As always though, the future holds interesting possibilities...

phronmophobic02:11:09

@madstap, there really aren’t that many types of transducers. it seems like it’s pretty uncommon to create a “new” transducer. out of curiosity, what kinds of transducers have you or others been creating that aren’t map, filter, cat, dedupe, etc?

madstap02:11:33

Wouldn't your magic macro have to know about all possible transducers though? How would it know if something can be made into a transducer?

bcbradley02:11:40

I mean i understand that one of the really great things about transducers is that you can construct your own transducers and then use them with any transducible process. My criticism about it is that it seems to expose some limitation in the implementation; its not purely about separating the sequence of transformations from the sequence (or channel or whatever) of data-- you could make that separation without creating the idea of a transducer, by just passing around function that is composed of the data processing functions for instance, then apply that function directly to the data.

phronmophobic02:11:27

just for my understanding, how does this compare to (comp (partition-by consecutive-fn) (map merge-fn))?

madstap02:11:10

The merge-fn is reduced over each partition instead of given a partition as an argument.

madstap02:11:17

Does that make sense?

phronmophobic02:11:11

so more like (comp (partition-by consecutive-fn) (map #(reduce merge-fn %))?

madstap02:11:23

I think that would be equivalent. Nice one!

phronmophobic02:11:47

although, I think your example wouldn’t have to hold a whole partition in memory at the same time

phronmophobic02:11:03

so if you had large partitions

madstap02:11:20

That's true

phronmophobic02:11:46

i’m trying to figure out if the same transducer could be created from two simpler transducers, but my brains a little fried at the moment

phronmophobic02:11:55

too much philosophizing about tranducers 😛

madstap02:11:51

I find them fun and tiring to write 🙂

phronmophobic02:11:01

that’s kinda why i’m hoping you could make more complicated transducers by just composing simpler ones

phronmophobic02:11:33

because, as you mentioned, there are a bunch of rules that you have to follow

madstap02:11:43

Yes, exactly. I think that you mostly can do that, but sometimes there are just basic building blocks that aren't compositions of other transducers. And sometimes, like I did, you make a new one cause you don't have the imagination to combine two existing ones.

madstap02:11:59

Take a look at source of the xforms library some time, some of those are basic building blocks. (Fair warning, trying to grok that is headache inducing.)

phronmophobic02:11:09

yea, it seems neat

phronmophobic02:11:17

thanks for the link!

phronmophobic02:11:26

@bcbradley, (map inc), (filter even?), etc. seem like fairly minimal descriptions of a “step”

phronmophobic02:11:44

which part of (map inc) doesn’t need to be part of the transducer?

bcbradley02:11:05

why even bother me with that though?

bcbradley02:11:24

why can't i say (map inc (filter even? (map inc foo)))

bcbradley02:11:32

why can't clojure do algebra to reorganize it

madstap02:11:37

@bcbradley But how would you do filter in that scheme? A function that returns nothing, as something separate from nil?

phronmophobic02:11:27

how would you like it organized? currently, you can do (comp (map inc) (filter even?) (map inc))

ghadi02:11:31

Doing algebra in the compiler has a cost... That's what Haskell does

ghadi02:11:06

There is also no clear definition in an impure language around what is pure vs impure, where can I move things around, etc

ghadi02:11:37

Transducers aren't perfect, but the cost-benefit / leverage is off the charts high

ghadi02:11:05

They were also implemented in user space, in a library initially

ghadi02:11:17

No involvement of compiler

bcbradley02:11:51

basically, (comp (map inc) (filter even?) (map inc)) is no better a representation of the process from an apparent point of view than the s-expression ((map inc) (filter even?) (map inc)), or the s expression (map inc (filter even? (map inc)))

bcbradley02:11:08

the idea of transducers were to separate the definition of what was being done from the thing upon which it was being done

bcbradley02:11:13

i posit that we could already do this

bcbradley02:11:15

because this is lisp

bcbradley02:11:55

the real reason transducers are around is because clojure see's (map inc (filter even? (map inc x))) and makes three sequences

bcbradley02:11:05

it isn't smart enough to know it can unravel for you

ghadi02:11:08

I don't really know what you're arguing for tbh

ghadi02:11:48

Transducers are independent of collections. They don't compare to a collection pipeline

ghadi02:11:10

But sure, can make a macro that does a lot of things. Transducers as they are in core hit a sweet spot with expressivity and leverage

ghadi02:11:45

The examples being bandied about (map inc) are nice because they capture only essential detail

ghadi02:11:05

Hard to improve on that

ghadi02:11:02

There would have to be a huge benefit to incur additional complexity into the compiler or macro. Not saying that's a no, but it's unlikely to be sufficiently compelling.

ghadi02:11:42

Esp to overcome the inertia of adding an opt phase in the compiler

bcbradley02:11:34

my biggest gripe is that transducer's inherent value seems to be that you can describe a process to be done on the elements of a data source (channel, collection, sequence, iterable, whatever else) by "composing that process out of transducers"-- but you could already DESCRIBE that process anyway, without resorting to composing functions; you could just compose data in S-expressions that literally represent what you are doing to whatever thing you are doing it to; and if you want it to be independent of the thing you have that choice already! just omit it from the S-expression (its always going to be the most nested, rightmost element)

bcbradley02:11:24

so i'm wracking my head trying to think, what do transducers really actually do? They don't necessarily allow me to communicate anything about the process of transformation on data moreso than i could without them, and they don't allow me to separate the process from the thing being processed moreso than i could do without them

bcbradley02:11:22

i can only think that what transducers really do that the nested S-expressions don't is to capture the idea of doing a bunch of processes together at once in one pass on whatever the thing is

phronmophobic02:11:26

I think you could say that about functions in clojure in general

phronmophobic02:11:34

once you do (def myfn (fn [x] (+ x 1)))

phronmophobic02:11:44

you don’t pass around the s-expr

phronmophobic02:11:49

you pass around the function

phronmophobic02:11:55

and you can’t really take it apart later

phronmophobic02:11:17

if you hand somebody myfn, they can’t tell if it’s made up for complicated stuff or that it just returns a constant

bcbradley02:11:36

if the idea of transducers was to separate what is being done from the thing is is being done to, isn't that just the definition of a function?

bcbradley02:11:49

doesn't myfn do precisely that?

bcbradley02:11:08

why should i need transducers when i have (fn ...)

phronmophobic02:11:24

i mean, you can take a look at the definitions of the different transducers

phronmophobic02:11:29

and they are “just functions”

phronmophobic02:11:43

I think it’s useful to say more about them and give them a name like transducers

phronmophobic02:11:04

for these types of functions, map, filter, dedupe, etc

phronmophobic02:11:29

and having the tranducers as a separate thing has already paid off

phronmophobic02:11:45

originally, core async had their own map, filter, dedupe functions

phronmophobic02:11:07

but now you can reuse these core functions in core async or use them with lazy sequences

phronmophobic02:11:45

it seems kinda obvious that it should be possible, but if you look around at other ecosystems, they do have have a map for Rx, and then map for collections

bcbradley02:11:21

i don't mean to imply that i think we should have a bunch of different maps for different data types

bcbradley02:11:28

i'm trying to say that i don't think we need transducers to avoid that

bcbradley02:11:40

think about what you are actually feeding a chan for instance when you provide a xf argument

bcbradley02:11:53

you are giving it a function that is composed of transducers (other functions)

bcbradley02:11:01

it has to be composed of transducers, nothing else

bcbradley02:11:45

you can't for instance, give it just a general expression that expects a value in a particular place and can use any function ("transducible" or otherwise)

bcbradley02:11:57

you also can't just give it any old function

bcbradley02:11:13

and expect it to work, even if the dimensionality and data types match up

bcbradley02:11:18

atleast not as exepcted

bcbradley02:11:41

IF the transducible process had the macro capability to look inside what you gave it

bcbradley02:11:04

THEN it could decide for itself what it can do, whether it has to break up some things or rearrange others into what we are calling "transducible process"

bcbradley02:11:21

it could even rearrange things or break things up in more particular ways that are specific to the thing being worked on

bcbradley02:11:26

chan in this instance

bcbradley02:11:48

this is what async/go actually does

bcbradley02:11:00

it breaks up your s-expression and creates a state machine out of it

bcbradley02:11:34

transducers were a great way to follow DRY, but i feel like it was a missed opportunity to just fix the real issue

phronmophobic02:11:08

can you give an example of what you would like the code to look like?

phronmophobic02:11:14

i feel like it’s hard to be more concise than (chan 1 (map inc)),

bcbradley02:11:04

i'd like to say (map dec (filter even? (reduce conj [] (foo bar (xan zoo WHATEVER))))) and have clojure know how to optimally unravel it so that it doesn't create more intermediate sequences than necessary, can do what is effectively known as a "transducible process" (doing things in one pass) on either side of the reduce, and can even do things in parallel if it can use the runtime to judge that doing so doesn't alter the semantics of what you are doing (it can't if you are just using immutable values) and if doing so would result in faster computation even after taking into account the overhead involved with concurrency.

bcbradley02:11:40

in other words, the details exposed to us in the transducer api shouldn't really have to be exposed to us-- if that is how things need to work between "transducible processes" well ok, but why should i be bothered with it?

bcbradley02:11:51

that sort of thing belongs in the implementation of eval

bcbradley02:11:17

the fact that making new transducers means they have to do this and this and that, why do i care?

bcbradley02:11:27

can't eval look at what i'm giving it and verify that i've done these things?

bcbradley02:11:02

if i want to express the idea of a computation without the thing being computed, i can just say (map dec (filter even? (reduce conj [] (foo bar (xan zoo)))))

bcbradley02:11:06

notice the lack of WHATEVER

bcbradley02:11:30

i just pass it as an s-expression and someone can inject whatever they want into it and then evaluate it

bcbradley02:11:41

imho transducers just don't feel very lispy

bcbradley02:11:01

they work well for what they try to do, but it sort of misses the point of lisp

madstap02:11:10

Verify that I've done these things is different than doing them automatically, though.

madstap02:11:31

Imho what you're describing sounds more haskelly than clojurey

phronmophobic02:11:44

it would be cool if the compiler would take my clojure code and optimize/parallelize it for me, but that seems like a separate issue than the design of transducers

bcbradley02:11:57

yeah its sort of a bigger issue

bcbradley02:11:56

i guess i tend to think that ideally programs should just say what it is, semantically, that you want to do

bcbradley02:11:12

the question of how that is done or how to make it performant is a completely separate and orthogonal concern

bcbradley02:11:23

usually best solved in the complier or evalutor

bcbradley02:11:45

i don't like it when programming languages ask you to make them faster by instructing them on how to do the compiler's job

madstap02:11:17

I think that's kind of not the clojure way, though. Like the difference between last and peek on a vector.

madstap02:11:29

Semantically they're the same, but one's way faster

bcbradley02:11:47

yeah i guess i just have a different opinion on the matter

bcbradley02:11:51

you are right it probably isn't the clojure way

bcbradley02:11:01

heck it probably isn't even a very popular view on things

bcbradley02:11:06

oh well i'll stop ranting

bcbradley02:11:11

thanks for hearing me out though

madstap03:11:38

I think it's very valid to question things 🙂 Rich is not a messiah and blindly following him is not good for anyone.

madstap03:11:40

What you're saying about a sufficiently smart compiler/macro really reminds me of both criticism and praise about haskell though. Where it can do some really cool optimizations, but that makes it hard to reason about perf.

bcbradley17:11:23

honestly the jvm can be hard to reason about anyway, especially when you add clojure's runtime on top of that.

seancorfield03:11:24

Re: writing your own transducing functions -- I wrote one that turned flat sequences of maps into sequences of threaded sequences of maps (based on certain fields acting as keys and back references). It turned out to be a very elegant solution to creating threaded conversations out of raw sequences of messages between members. It hid the complexity of the problem and it was easily composable with any other processing I wanted to do on messages (before transformation) or conversations (after transformation).

tbaldridge04:11:04

@bcbradley there's a lot of compiler tech out there that does this sort of thing. PyPy does it out-of-the-box at runtime, infact PyPy actually runs transducers over int arrays faster than Clojure. ClojureScript does as well. Clang can do this sort of stuff but it would require all your pipelines to be defined up-front and fully static

tbaldridge04:11:52

The beauty of the Clojure model, is that none of this took compiler support. It just works as a library.

tbaldridge04:11:35

That's the problem with a lot of these approaches. PyPy took 3-4 people 10 years to engineer. The JS JITs took less time, but larger teams. Transducers were mostly designed in about 3 weeks by one person.

tbaldridge04:11:11

And you'll see that pattern throughout Clojure, never over-engineer. If you can accomplish what you need with your existing tools, do it that way.

tbaldridge04:11:38

Not saying these other methods aren't valid, but they are much more complicated.

tbaldridge05:11:00

And for anyone here interested, here's pretty much the state-of-the-art for this sort of thing. From the Scala world, but the paper goes over Lisp and the like as well: https://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf

the2bears05:11:44

That's a big paper! Won't start it tonight 🙂

tbaldridge05:11:50

yeah the first 50 pages or so are an overview of why we need this thing

qqq05:11:37

@tbaldridge: can you tell https://www.youtube.com/channel/UC6yONKYeoE2P3bsahDtsimg/videos to create a video walkthrough of the paper, step by step, implementing a minimal setup in clojure ?

tbaldridge06:11:38

Heh, most of it leans really heavily on static typing. How to adapt it to more dynamic languages is still a WIP

brainfreeze06:11:07

Were Clojure's transducers only able to be made so quickly because of the way the language is, or does putting them in the compiler just take longer? Since it's lispy i'm just curious if impossible to do it the library way in other languages.

sundarj10:11:48

well, transducers are very simple, language-agnostic concept - there are (library) implementations for javascript, ruby, python, and java as well

sundarj10:11:02

tim's point was just that transducers were invented in a very short time, and required no compiler support at all

Mudge07:11:02

It would be nice if there was something like defmacro that did not evaluate its arguments but that could return a value instead of code that gets evaluated. Is such a thing possible?

val_waeselynck08:11:57

@nick319 well, it is possible as long as the value can be expressed as code... Isn't this what comment does ?

(defmacro comment [& body] nil)

igrishaev09:11:05

Could anybody help me with fuzzy multi-method dispatching? Say, I have maps with :os and :version fields. Basically, I need to dispatch them by OS, but for some specific versions I need to perform some other logic. What I’m trying to reach is:

;; let's say my dispatcher function is (juxt :os :version)
[:mac <any-version>] ==> the standard iOS algorithm
[:win <any-version>] ==>> the standard Windows algorithm
;; but!
[:mac :11.22.33] == > some Mac-specific algorithm for version 11.22.33
So once I’ve got some another buggy version, I just add another (defmethod...) definition for that case. Probably, later I’ll need to add more minor keys to dispatch, say, browser name or something else. I’ve tried a bit derive and custom hierarchies but without success.

igrishaev09:11:32

yes I’ve seen that example but cannot understand it completely so it prevents me from using it.

rauh09:11:49

@igrishaev What's the question? I wrote that example 🙂

rauh09:11:34

Basically: 1. If no 100% match, find the default method in the :default fallback. 2. Add this fallback to the defmulti

igrishaev09:11:26

@rauh makes sense. Another solution might be to derive any version over ::default value once I’ve got that map.

chrisbetz10:11:48

@igrishaev Why not use something like this:

(derive :mac/high-sierra :mac/default)
=> nil
(isa? :mac/high-sierra :mac/default)
=> true
`

chrisbetz10:11:11

and derive all the os-versions from a default version.

igrishaev10:11:07

@chrisbetz yes, it came to me afterwards. I’m going to derive version once I get a map to dispatch.

chrisbetz10:11:58

@igrishaev 🙂 Happy hacking.

eggsyntax14:11:39

@alexmiller or other Cognitect folks -- I noticed that the instructions in the readme.txt in the Clojure repo are no longer complete and accurate -- I can build with ant or maven, but java -cp clojure-${VERSION}.jar clojure.main no longer works; it throws an exception because of the spec dependency. Maybe worth adding a note to readme that spec has to be built and installed first? Note: I'm assuming that's what has to be done to fix; I haven't actually tried it at this point. Didn't seem worth filing a ticket, since it's so minor, but a suggestion for the final 1.9 release.

Alex Miller (Clojure team)15:11:23

There are several things in that readme that are lagging and need updating. We don’t plan to make any more changes for 1.9 though.

eggsyntax14:11:24

Just seems like it could trip up new users who are naively trying to follow the instructions in the readme.

noisesmith14:11:17

@eggsyntax this is a known issue, and being addressed with the clj tool

noisesmith14:11:49

it will be integrated (including the readme) when 1.9 is the stable release, is what I heard

eggsyntax14:11:07

Right, I'm aware of the new CLI tool -- just suggesting that the readme be updated.

eggsyntax14:11:22

Cool, good to know; just wanted to be sure it hadn't been overlooked.

nooga14:11:14

what new CLI tool?

noisesmith14:11:45

the tool is called clj, it's clojure.core plus a dependency manager to bootstrap

noisesmith14:11:54

since clojure.core now needs other libraries to run

nooga14:11:31

ah, I see

danm15:11:12

Why were those other libs not merged into clojure.core to make sure it could run standalone?

Alex Miller (Clojure team)15:11:46

they were originally in core and we split them out to allow them to be updated independently from core

danm15:11:59

Fair enough

jerger15:11:09

Does anyone know a way how to genereate documentation out of plumatic/schema ?

bja15:11:01

@jerger what kind of documentation are you looking for?

bja15:11:57

@jerger

user=> (s/defn foo :- Bar
  #_=>   [qux :- s/Str, quux :- [s/Keyword]])
#'user/foo
user=> (doc foo)
-------------------------
user/foo
([qux quux])
  Inputs: [qux :- s/Str quux :- [s/Keyword]]
  Returns: Bar

lvh17:11:15

How do you idiomatically spell (is (thrown-with-msg? ...)) except also binding the exception to a name so I can ex-data it?

noisesmith17:11:39

quoting doc of is:

(is (thrown? c body)) checks that an instance of c is thrown from
  body, fails if not; then returns the thing thrown.
- so at the very least, thrown? lets you do that

noisesmith17:11:07

@U07QKGF9P - looks like it works with thrown-with-msg? too

kingfisher.core=> (clojure.test/is (thrown-with-msg? Exception #"foo" (/ 1 0)))

FAIL in () (form-init2223331148129063339.clj:1)
expected: (thrown-with-msg? Exception #"foo" (/ 1 0))
  actual: #error {
 :cause "Divide by zero"
 :via
 [{:type java.lang.ArithmeticException
   :message "Divide by zero"
   :at [clojure.lang.Numbers divide "Numbers.java" 158]}]

 :trace
 ...
kingfisher.core=> (type *1)
java.lang.ArithmeticException

noisesmith17:11:52

so the pattern would be to put the is in a let block so you can make other assertions about the ex-data

lvh17:11:28

I totally read over that, oops

lvh17:11:33

that makes sense. Thanks!

noisesmith17:11:02

it only explicitly documents the return-value of the thrown? check, I guess it’s implied that thrown-with-msg? would similarly return e

jerger17:11:35

like

(def ServerTestConfig {
 (optional-key :netcat-test)
 {Keyword {:reachable? Bool}},          ; keyword is used to match test against fact
 (optional-key :netcat-fact)            ; parsed result of "nc [host] -w [timeout] && echo $?"
 {Keyword {:port Num,
           :host Str,                   ; may be ip or fqdn
           :timeout Num}},              ; timeout given in seconds
 (optional-key :netstat-test)

jerger17:11:09

But the datatypes described here are splittet across several modules, so we will need a way to 1. add documentation as meta to elements like 'keyword' or 'defschema' 2. a way to render documentation on schema endpoints (sth. like explain)

roti18:11:18

Does anyone have an idea what could be wrong here?

(defn testex []
  (go
    (let [response (try (<! (http/get "" {:with-credentials? false}))
                        (catch :default e
                          (println "error" e)))]
      (println response))))
The URL returns HTTP 500 with content-type application/json but not a JSON body, so http/get throws a parse error, but I can't figure out why I can't catch it. I tried putting the try everywhere.

noisesmith18:11:52

you can’t catch it because channel ops are lifted into core.async’s state machine

noisesmith18:11:36

so the fact that http/get returns a channel means that you can’t catch its error - in order to handle that error you need to use the facilities your http library defines, and if they define none the library is broken

noisesmith18:11:40

normal clj-http at least takes a :throw-exceptions arg you can set to false - I don’t know what library http is but with any luck they accept that arg and return the error response instead of throwing an error(?) - I have no idea why that wouldn’t be the default in an async lib frankly

roti18:11:36

I use this one: https://github.com/r0man/cljs-http . Are there alternatives out there that also use core.async?

roti18:11:16

so this means that any code that returns a channel should never throw an error?

hiredman18:11:29

:default likely doesn't work correctly inside clojurescript core.async go blocks anyway

bfabry18:11:43

asynchronous code shouldn't throw errors. who's gunna catch it?

bfabry18:11:46

not just core.async

noisesmith18:11:07

@roti it can throw an error if its non recoverable, what would be the point? but yeah, it shouldn’t throw exceptions because there’s no reasonable block of code that can catch it

bfabry18:11:37

anyway looking at that code it doesn't seem like it would throw exceptions to me, it looks like it returns some sort of structured error

noisesmith18:11:22

@roti looking at that lib it isn’t the one throwing, yeah

noisesmith18:11:37

but really, you can just use xhr via interop, and use core.async inside the callback

noisesmith18:11:47

that way you can attach your own error handler inside the callback

noisesmith18:11:57

wrapper libs are severely overrated

roti19:11:17

@noisesmith

ioc_helpers.cljs?rel=1509880107319:42 Uncaught SyntaxError: Unexpected token S in JSON at position 0
    at JSON.parse (<anonymous>)
    at cljs_http$util$json_decode (util.cljs?rel=1509894674732:63)
    at core.cljs:4829
    at Function.cljs.core.update_in.cljs$core$IFn$_invoke$arity$3 (core.cljs:4829)
    at cljs$core$update_in (core.cljs:4820)
    at cljs_http$client$decode_body (client.cljs?rel=1509894676467:83)
    at Function.<anonymous> (client.cljs?rel=1509894676467:181)
    at Function.cljs.core.apply.cljs$core$IFn$_invoke$arity$2 (core.cljs:3685)
    at cljs$core$apply (core.cljs:3676)
    at async.cljs?rel=1509880110795:712

noisesmith19:11:47

right, so your server is giving your garbage that isn’t json, and cljs-http isn’t set up to catch that for you

noisesmith19:11:09

you need to either replace wrap-json with something that handles the error nicely, or just use xhr and js/JSON yourself directly

noisesmith19:11:22

my bet is that in the long term that last option is the least trouble

roti19:11:13

well, I was using JulianBirch/cljs-ajax before, which is ok, but I wanted a solution where my logic in not split in event handlers

roti19:11:14

for simple stuff there should be no problem, but as soon as you have something more complex, like several ajax requests in sequence it gets messy

noisesmith19:11:31

right, you can use core.async in your callback - then you’re in clojure land

roti19:11:20

regarding "who should catch the error", my answer would be "the code inside go". core.async gives you the illusion to have sequential code when its actually asynchronous, right? so that is where errors should handled as well. ironically throwing an exception/error is actually a break in a sequential code. i don't know how it works internally, i just recently learned that they are actually not the same as continuations, so it may not be easy or possible, but that question definitely has an answer

Mudge19:11:36

Do people prefer to use for list comprehensions or filter and map?

noisesmith19:11:47

@roti but that’s impossible - your go block can’t see errors inside another go block, and that’s what is happening in your code when you call http/get

noisesmith19:11:31

this is a hazard of using library code that creates go blocks you can’t control

roti19:11:46

@noisesmith it may be technically impossible (though I'm not so sure about that), but not logically. my point is that asynchronous code should have a generic way of dealing with errors, like sequential code has. blaming the library for not following a convention and not having a way to deal with that is not good 🙂

noisesmith19:11:42

OK - you would need to re-implement core.async to implement this

noisesmith19:11:56

it’s a core structural problem, not a superficial oversight

roti19:11:15

hmm, i'm going to have to get to know core.async better to understand that. I come from JS promises, where errors are part of the "promise system", i.e. when you have a promise chain, and one of the steps throws an error it is caught by the "promise system" and the closest error handler (sort of like a catch) is executed. granted it's not perfect, for example errors while creating the promise chain are not caught.

roti19:11:24

i'm going to follow your recommendation and switch back to JulianBirch/cljs-ajax

noisesmith19:11:36

this might be a better discussion for #core-async - and perhaps I’m wrong and there’s a way to put in a more generic exception handler for go blocks- but if that existed I’d think we would have seen it in action already

noisesmith19:11:17

to be clear, I recommended using xhr and JSON via interop, but if that works and is easier that’s cool too 😄

roti19:11:58

why would you favour xhr vs a cljs library?

roti19:11:56

@nick319 I would choose what looks best aesthetically (by which I mean what code is more understandable), but I guess performance may also matter in some cases. why do you ask?

noisesmith19:11:33

@roti because wrapper libs are limiting, and interop is strightforward

noisesmith19:11:58

using the js lib directly will be the most flexible option, and least likely to get you stuck

Mudge19:11:52

@roti cool, I am learning how to write better code