Fork me on GitHub
#clojure
<
2017-08-31
>
dinoleif01:08:18

Has anyone ever encountered tricky issues with deadlocking at startup with the clojure.lang.RT class intializer? (Here’s a stack trace that illustrates what I’m talking about https://pastebin.com/1w2bqLun) Here’s what’s going on: Main thread defines an top-level var (which is initialized in the RT class loader) whose value is taken from spawning another thread and blocking on it. That second thread tries to use clojure.lang.RT in normal use (in this case, invoking contains?) which blocks on the RT.java class loader completing. Which won’t ever happen because the two threads are blocking on each other.

dinoleif01:08:45

Figured this out. TLDR: “user” namespace is treated specially by clojure. Swapping repl init namespace to something else (e.g. foo.user) loads it outside of the clojure.lang.RT class loader and breaks the deadlock.

bfabry02:08:11

yes, this has bitten me a bunch of times

bfabry02:08:16

be careful what you put in user.clj

bfabry02:08:43

my usual solution has been to take user.clj off of the classpath and have a post-init task in leiningen load it manually

seancorfield02:08:56

(or use Boot which seems to subvert the user.clj loading altogether -- which some may consider a bug 🙂 )

didibus02:08:02

@dinoleif I kind of have a related issue in jira I made about this: https://dev.clojure.org/jira/browse/CLJ-2185

theikkila02:08:24

what are you using as a http server with clojure nowadays when http-kit is kinda dead project?

greywolve05:08:31

immutant web

tbaldridge02:08:25

Pedestal, data driven webservers ftw! 🙂

didibus04:08:47

Is there a reduce that works more in the style of doseq and for?

donyorm04:08:31

So I'd like to call a public, static java method from a symbol similiar to calling a clojure a method. That is how could I pass 'Integer/parseInt (possible read from an .edn file) to a function and have it execute? resolve apparently doesn't work on java interop

didibus05:08:20

Hum, I think there was a function that could wrap java methods as clojure functions, so you could pass them around

didibus05:08:56

Try this form:

(. Classname staticMethod args*)

donyorm05:08:26

that works but not if you use symbols instead of the actual class names

didibus05:08:48

Hum, the class seems problematic. I was thinking of memfn, but that won't work on static

donyorm05:08:07

basically I'm creating a program that accepts Java plugins, and I need to load them from a .edn file. The easiest way to do this would be to have my main program call a static method in the plugin (that's basically what I do with clojure plugins). Unfortunately this seems complicated.

didibus05:08:26

This works:

((eval (read-string "#(Integer/parseInt %)")) "12")

donyorm05:08:17

That will. using eval on user input seems...wrong. But I may end up doing that.

didibus05:08:29

Its wrong, if those are not trusted plugins

donyorm05:08:24

I'm letting them run arbitrary code anyway, so I'm probably shouldn't be worried about using eval. Thanks

didibus05:08:19

Okay, this also works

didibus05:08:22

And its safer

didibus05:08:37

well, I'm not sure about being safer

didibus05:08:45

but it works and uses the edn reader

didibus05:08:52

(defmacro run
  [f & args]
  `(~(edn/read-string f) ~@args))
(run "(fn [%] (Integer/parseInt %))" "12")

didibus05:08:27

Sorry, I meant:

(defmacro run
  [f & args]
  `(~(edn/read-string f) ~@args))
(run "Math/max" 12 34)

didibus05:08:18

You can also have it this way if you prefer to give it a symbol instead of a string:

(defmacro run
  [symbol & args]
  `(~symbol ~@args))
(run (edn/read-string "Math/max") 12 34)

donyorm04:09:08

So I've been working on this some more, and whenver I run this code: (run (edn/read-string "java.util.Date.")), I get this exception: clojure.lang.ArityException: Wrong number of args (0) passed to: Symbol. Why is that? It doesn't seem to be very different.

donyorm05:09:55

I did get it to work with eval, though I would be curious to know why this example doesn't work, but no stress if you don't have time.

donyorm05:08:30

that works

donyorm05:08:32

thanks again

vinai06:08:04

Is there a way to tell environ to load changes from a project profiles.clj without restarting the repl?

tap06:08:49

The author suggested not to structure the app that way https://github.com/weavejester/environ/issues/16#issuecomment-54326836

tap06:08:26

I had some what similar need. He pointed me to that explanation https://github.com/weavejester/environ/issues/24

qqq11:08:58

in defmulti/defmethod, is there a way to NOT call the multi-method, but to call the dispatch function, to know what it's resolving it?

Alex Miller (Clojure team)12:08:26

It's often a good idea to pull the dispatch function out as an actual function so you can test it independently

mbjarland12:08:17

I have a basic style question on clojure functions. I find it common to need to check a number of preconditions (arg validation etc) before actually executing the body of a function. In languages like groovy/java/x I prefer a number of early if (!condition) return statements in lieu of deep nested ifs for readability. In clojure (with my limited knowledge of it) I tend to end up with just this kind of nested if structure using if and when. I am aware of {:pre x} but pre throws an assertion exception which is not always desirable, the method might just come to the conclusion the data is not applicable but that we are still not at an error condition, i.e. the desirable behavior would in my case be to just skip executing the method. What would be the idiomatic way in clojure to perform these kinds of precondition checks? Or are nested conditional statements it?

Alex Miller (Clojure team)12:08:09

There are a bunch of libraries that do this kind of thing

Alex Miller (Clojure team)12:08:33

The upcoming spec in core can be used for this too

qqq12:08:49

@alexmiller : the problem I'm having with -is reloading / redefihning defmulti / defmethod at the repl

qqq12:08:11

so I 'know' what dispatch function should be called, but I need to test if it actually is being called

Alex Miller (Clojure team)12:08:04

I think there Java API methods you could poke on the Multimethod instance

mbjarland12:08:35

@alexmiller ok, spec could be an option...but wouldn't a spec :pre still throw an assertion exception?

Alex Miller (Clojure team)12:08:23

Yes - you would want to call s/valid? In the code

mbjarland12:08:33

ah, missed the thread. Thank you, I'm just about familiar enough with spec to do this

Alex Miller (Clojure team)12:08:00

There is also s/assert but maybe that's the same as pre for you

nicoschneider12:08:55

@mbjarland no, you can get plain data from it’s functions

nicoschneider12:08:25

it’s a detailed map with explanation info where the spec failed

nicoschneider12:08:40

it’s really worth looking into, still trying to wrap my head around it 🙂

mbjarland12:08:26

@nicoschneider right, but you would still end up with logic within your method body with an if (or similar) to check that the data returned from the spec is valid?

nicoschneider12:08:55

there’s a way in spec to constrain the relationships between function input and output

hmaurer12:08:11

@mbjarland what do you mean by “skip execution of the method”? you would need to return some values to the caller

nicoschneider12:08:11

it’s really that general so any validation you could think of can take place

mbjarland12:08:58

@hmaurer good point, in the particular case I'm on right now this is a side effecting method printing things to the console, so the returned value is secondary, but yes in a general case true and then we end up with an if or when (returning nil)

misha12:08:09

why last does not has a special case for vectors? It is already (linearly) slow.

mpenet12:08:08

Same issue with first if I recall. nth 0 prolly is much faster than first. That's...odd, to say the least.

misha13:08:54

@mpenet the first will become slower for everyone, if you add extra if there, last - will not. So I can agree with first in this regard.

misha12:08:17

to make "working with a vectors" in code as explicit as possible?

Alex Miller (Clojure team)12:08:56

last in general should give you slow (linear) expectations

Alex Miller (Clojure team)12:08:18

In general, we don't have "special cases". We have traits and collections that implement those traits. The API lib is built on those traits

Alex Miller (Clojure team)12:08:39

last is a sequence function that traverses the sequence

Alex Miller (Clojure team)12:08:51

If you want fast access to the insertion point, use the stack API (peek, pop, etc)

misha12:08:15

@alexmiller what is the difference between last and reduce from the "generality" point of view?

misha12:08:01

(`reduce` is a bunch of special cases combined, as opposed to none in the last)

misha12:08:36

is there an intuition rule to categorize function either as general (last), or special (reduce), w/o looking up the source code?

misha12:08:52

sort of similar to "functions operating on collections, take them as last arg, functions for scalars - as first arg"

Alex Miller (Clojure team)12:08:16

I don’t see reduce as a bunch of special cases

Alex Miller (Clojure team)13:08:33

I really see it as two cases - you are something that can be sequentially traversed or you know how to reduce yourself

Alex Miller (Clojure team)13:08:10

both of those have many implementations, but they are the same operation

mpenet13:08:33

Let s put it this way: when would anyone want to use first/last on vectors then? Makes the impl a bit pointless

Alex Miller (Clojure team)13:08:50

transient support (again, something trait-like, although not done as well) does add a special case to reduce for perf

Alex Miller (Clojure team)13:08:25

reduce (being the bottom of many things) has more pressure on it than most other functions

mpenet13:08:25

I get that using last on a vector is odd... But first...

Alex Miller (Clojure team)13:08:10

both first and last are part of the sequence abstraction. sequences are logical lists. you should reason from there.

Alex Miller (Clojure team)13:08:10

generally, most code shouldn’t use either first or last. you should think in collections.

misha13:08:51

@alexmiller I understand "usually don't use last", can you elaborate about first? Do you mean "use destructuring (in loop)/reduce instead"?

mpenet13:08:48

Sure but that's counter intuitive to have poor impls of these for vectors. Would it have any bad implications/cost to do something reasonable in that case?

Alex Miller (Clojure team)13:08:00

if you particularly need access to the ends of a collection, it’s usually because you are using it as a stack or a queue, etc. choose a data structure that’s good at those things, then use the ops that are designed for that usage.

Alex Miller (Clojure team)13:08:18

it is absolutely not counter intuitive. it is exactly about making operations have intuitive meanings and performance implications.

bronsa13:08:34

i agree with you but I can't help but see the irony in having to say "it is absolutely not counter intuitive" :)

Alex Miller (Clojure team)13:08:07

yeah, I regret writing that already :)

leonoel13:08:27

coming from scala, where every possible operator is implemented for every possible collection, I found clojure's consistency on performance very refreshing

mpenet13:08:29

As I said I can somewhat understand why last is like this, doesnt make much sense to use it on a vec. But first is way more common/widespread. Ultimately lot of people are probably using it the wrong way and just expect the impl to be more forgiving

Alex Miller (Clojure team)13:08:39

To be clear, I am not arguing that vector should be slow on first. What I care about is the expectations you should form about operations.

Alex Miller (Clojure team)13:08:29

and that everything should start from a consistent model of operations and collection traits

Alex Miller (Clojure team)13:08:02

in that framework, there is opportunity to provide polymorphic implementations with better performance

bronsa13:08:36

@alexmiller playing devil's advocate, if the claim is "last is defined to be linear", constant time is technically O(n)

Alex Miller (Clojure team)13:08:54

which is covered by what I just said - you should have expectations based on the op, but no one is going to complain if you’re faster

bronsa13:08:28

so it wouldn't be incorrect to have last work constant time in vectors and still claim that it's linear, altho I understand the rationale of worse is better to encourage users to not rely on non promised performance optimizations

Alex Miller (Clojure team)13:08:52

I’m not claiming worse perf is better

bronsa13:08:12

sorry, didn't mean to imply you did

bronsa13:08:28

that's how i rationalize last being linear on vectors in my head

Alex Miller (Clojure team)13:08:35

I’m claiming that having a strong conceptual model and an implementation based on traits/interfaces is tremendously valuable

Alex Miller (Clojure team)13:08:10

and I’d say that’s more important than having an op be faster than the expectation you should have for that op

Alex Miller (Clojure team)13:08:31

to give an example where this has come up in a very nuanced way - we rewrote range to be reducible in 1.7

Alex Miller (Clojure team)13:08:51

because the case of all long start/end/step is a) the 99% use case and b) highly optimizable, we did so in LongRange. In those constraints, it’s possible to compute the count of a LongRange in constant time (which is what the Counted interface implies). We struggled over whether to make LongRange implement Counted (normal Range is not - too many special cases) and ultimately did. I can justify that via implementing the extended trait of Counted.

misha13:08:33

oh, reduce is "bunch of special cases" in clojurescript:

(cond
       (implements? IReduce coll)
       (-reduce ^not-native coll f)

       (array? coll)
       (array-reduce coll f)

       (string? coll)
       (array-reduce coll f)

       (native-satisfies? IReduce coll)
       (-reduce coll f)

       (iterable? coll)
       (iter-reduce coll f)

       :else
       (seq-reduce f coll)))

Alex Miller (Clojure team)13:08:45

and those similar things exist in Clojure - but I consider them to be polymorphic implementations under an abstraction, not special case. but maybe that’s splitting semantic hairs.

misha13:08:54

@mpenet the first will become slower for everyone, if you add extra if there, last - will not. So I can agree with first in this regard.

misha13:08:51

@alexmiller I understand "usually don't use last", can you elaborate about first? Do you mean "use destructuring (in loop)/reduce instead"?

Alex Miller (Clojure team)13:08:47

no, I mean usually you have a collection and you transform it through a series of sequence or transducer operations to another collection.

Alex Miller (Clojure team)13:08:26

if you’re accessing or updating individual items in a collection, you should use a collection operation (get, nth, contains?, update, etc). first/last are only really collection operations on a list or a seq, not on a vector.

Alex Miller (Clojure team)13:08:40

so, first is interesting b/c its part of ISeq which includes first/next/more

Alex Miller (Clojure team)13:08:17

where more is really derivative of next

Alex Miller (Clojure team)13:08:11

it is tempting to say that the perfect number of operations in an interface is 1 and this interface has several. ISeq could conceptually be a composite of “has a first item” and “has more items” which could be separate smaller interfaces

Alex Miller (Clojure team)13:08:34

and vectors could provide a more optimized implementation of the smaller “has a first item” interface

Alex Miller (Clojure team)13:08:51

is all that worth doing to save a few ns? that’s a matter of taste and design.

misha13:08:55

coming back to intuition and readability question: operating on/from the sequence head, random (key/index) element access, and operating on/from tail should use different data structures, and it should be obvious by looking at the code? Basically, you need to pick one of those 3 realms, and only then appropriate abstraction's interface: eg. make it obvious you are working with the tail by using peek, and then choose vector or queue, etc. under the hood all you want.

Alex Miller (Clojure team)13:08:21

I would start with: what ops do I need to do on a data structure. Then, choose a data structure that can do those things well. Then, use the best expression of those ops (peek, not last).

Alex Miller (Clojure team)13:08:05

if you followed that, you would never get to the point of asking why first is not faster on a vector

Alex Miller (Clojure team)13:08:19

and I would say the “realms” are a little different: indexed access, list/seq like access (first/rest), FIFO, FILO, etc

Alex Miller (Clojure team)13:08:54

both lists and vectors work great for FILO (stack) ops using peek/pop

Alex Miller (Clojure team)13:08:06

but lists have a head insertion point and vectors have tail insertion point

Alex Miller (Clojure team)13:08:52

I actually don’t know which of those is faster off the top of my head. both are fast. :)

bronsa13:08:18

depends on how big the vector is

bronsa13:08:37

yeah IIRC if the vector has to grow a node then cons is faster

bronsa13:08:45

but I've benchmarked this a long time ago

bronsa13:08:50

I might be mistaken

Alex Miller (Clojure team)13:08:36

well I will agree that vector has the potential for more variability

bronsa13:08:41

and by cons i mean conj on lists

misha13:08:54

TIL peek works on lists dafuq

Alex Miller (Clojure team)13:08:28

based on a quick test, I’d say lists are faster as stacks :)

misha14:08:30

If I need to reverse vector, what is the best way to do it? Size is < 100, or even < 10. Does it even matter?

misha14:08:08

I need to traverse seq till about half way from head, and what's left - from tail

bronsa14:08:12

you can use a vector for that

bronsa14:08:21

rseq reverses a vector in constant time

bronsa14:08:45

gives you back a sequence not a vector but for linear traversal it should be fine

tbaldridge14:08:09

or call nth a decrementing range

misha14:08:53

sweet, thanks

florinbraghis14:08:02

Hi, is there a shorter/nicer way to achieve (and num (pos? num)) ?

lxsameer14:08:55

i don't know about shorter but clojure.spec ?

dpsutton14:08:56

(fnil pos? -1)

ghadi14:08:52

1.9 has pos-int? which is close

dpsutton14:08:04

i always forget about some->

ska14:08:01

Hi. Anyone familiar with clj-http? I want to dump the certificate chain when connecting to an HTTPS server and can't find a clue how to do that. I guess, I need to somehow get at the connection object and then do sth like https://stackoverflow.com/questions/7199129/how-to-get-server-certificate-chain-then-verify-its-valid-and-trusted-in-java but how to get the connection? Or is there a better way?

iku00088815:08:29

My coworker released a library for making life a bit easier with verbose Hiccup forms https://github.com/ayato-p/kuuga . Thought it may be interesting.

leonoel16:08:04

what is a globally accepted word to designate types which can be derefed and add-watch/`remove-watch`ed ?

leonoel16:08:23

reference types ?

leonoel16:08:28

identity types ?

leonoel16:08:44

thanks, that is indeed the least common divisor, but the name kind of overlaps with refs (the concrete type)

leonoel16:08:57

I will stick with reference, it seems to be the most commonly used in what I've read so far

didibus16:08:52

I think using the interface name is the least ambiguous. It depends on your audience I guess.

didibus16:08:35

But one thing I often find with names, is that you fall easily in the cognitive dissonance trap. Where a programming constructs is called with a word which metaphorically describes it, but concretely doesn't. Reference is overloaded, people come to the table with their own preconception of a reference. If you say IRef, people say what's that? And they go look it up, and now have a concrete name for a concrete clojure construct, which is not overloaded in their head.

leonoel17:08:43

good point

leonoel17:08:10

after all, it's the hardest problem in computer science for a reason

Garrett Hopper20:08:32

Is there an easy way to allow function calls in arguments to a macro? (e.g. reduce to immutable types)

Garrett Hopper20:08:31

E.g. reduce '({:a 1} (do-stuff 2)) to the result: '({:a 1} {:stuff 2})

hiredman20:08:57

for exceedingly simple cases you can have your macro call eval on its arguments, but that will (depending on use of the macro) very quickly run in to issues

hiredman20:08:11

if you need evaluated arguments, write a function

Garrett Hopper20:08:49

Yeah, it won't work in this case. That's what I'll need to do. I was hoping I could get this to happen at compile time, but that's alright.

didibus07:09:57

eval will execute within the context that exists at the time of macro-expansion. If you want to have it eval at that time, it works.

didibus07:09:36

But, that's pretty weird, and very unexpected from people who will use your macro.

didibus07:09:13

You can have the macro return a form which executes the argument. Like a macro could return a form which has normal evaluation order, only useful if you want a sort of DSL, otherwise there's not point and a normal function is best.

didibus07:09:18

(def a 120)
(defmacro tt [a b]
  `(~a ~(eval b)))

(macroexpand-1 '(tt + a))

didibus07:09:13

Like here, you can see that at macroexpansion, a is replaced with 120

didibus07:09:40

But you could also write

(def a 120)
(defmacro tt [a b]
  `(~a ~b))

(macroexpand-1 '(tt + a))

didibus07:09:29

Now you see it expands to (+ a) instead of (+ 120). But the end result is the same, when you run (tt + a) you get 120 in both cases.

didibus07:09:46

I can't really think of a reason when evaluation at macroexpansion instead would be useful

hmaurer21:08:37

Is it considered non-idiomatic to have a function with arity 2 and 3, like this:

(a b)
(a c b)

hmaurer21:08:53

(e.g. arity 3 pushes the last argument to the end, and inserts one in betwen

joshjones21:08:08

in general it's a bit strange and a potential pitfall for callers of the function, but clojure.core/reduce uses this approach when inserting an initial argument https://clojuredocs.org/clojure.core/reduce

noisesmith21:08:02

there’s a few others too, like make-array

hmaurer21:08:07

@joshjones thank you! I just realised even defn takes that approach with docstrings

noisesmith21:08:25

or fn, with optional names

noisesmith21:08:50

or condp, with the magic :>> arg

kenny22:08:18

Does anyone using cljfmt know if there is a way to align assoc like so:

(assoc {} :x "x"
          :a "z")
Where the default is:
(assoc {} :x "x" 
       :a "z")
I don't see any way of getting this indentation with :inner or :block. Am I missing something?

kenny22:08:02

I think there needs to be a way to control the amount of indentation, rather than where the indentation occurs.

noisesmith23:08:45

my workaround is just to move :x down to the next line

noisesmith23:08:07

I don’t think cljfmt as is is clever enough for the one you want