Fork me on GitHub
#clojure
<
2018-05-16
>
thheller06:05:39

@zentrope (-> (.parse parser str) (LocalDateTime/from)) should also work

👍 4
zentrope07:05:25

Oh, thanks. That makes a lot of sense.

roklenarcic12:05:37

there's so many HTTP routing libraries it's hard to pick one

dominicm13:05:07

bidi (I work for juxt, totally biased :D)

roklenarcic13:05:53

Bidi has nice featureset

roklenarcic13:05:11

Pedestal has nice terse syntax and performance optimizations

mpenet13:05:39

+1 for bidi

12
joaohgomes13:05:05

In which HTTP Server/stack are you planning to use bidi?

roklenarcic13:05:53

httpkit

👍 8
edannenberg13:05:23

hello, i'm working on clj wrapper for a java lib. the java api has a couple methods like .addPath that can be called multiple times to add more paths before finally executing some action. to make things a bit nicer for users the clj wrapper should accecpt a seq for those params and handle the rest internally. so i wrote a little generic helper function that takes some java object, a fn like #(.addPath %1 %2) and a seq of paths to be added which calls the passed fn for each path before finally returning the passed java object so the helper can be easily used in threading macros. no issues there, however to avoid reflection warnings this approach requires return type hinting the helper function whenever it is used in a threading macro, which I'd like to avoid. turning the helper function into macro seems to be the obvious solution but after some tinkering i'm not sure this is doable, but then again my macro fu is fairly basic. any pointers if this feasable or not would be greatly appreciated.

moxaj13:05:14

@edannenberg can't you just put the type hint on the function itself (ie. the args list)?

edannenberg14:05:41

@moxaj the return type depends on the passed java object, so kinda hard with a regular function unless they are created dynamically which i don't find very appealing

moxaj14:05:38

oh, both the object and the function are arbitrary

igrishaev14:05:37

moved to another channel

moxaj14:05:01

@edannenberg here's a way to do it: make the helper function a macro, pass an additional symbol (the name of the java class or a primitive type hint like int)

moxaj14:05:54

then wrap your macro body with (vary-meta <body> assoc :tag <the symbol argument>)

👍 4
edannenberg14:05:12

@moxaj i think i can get the type symbol via (symbol (.getName (class somejavathing) so hopeful i don't need pass that in, however wrapping the macro body with (vary-meta) might just be the thing i missed, thanks!

moxaj14:05:48

that won't work, since the macro only sees the symbol, not your actual java object

moxaj14:05:18

while you can retrieve the type information at runtime, you need to apply the meta at compile time

edannenberg14:05:50

ah bummer, not sure it's even worth it then, probably not. anyways thanks for the hint, gonna play with that a bit more later and see how it turns out

huthayfa15:05:13

hello guys, does clojure spec support file upload multipart validation

ghadi15:05:41

@huthayfa.ainqawi anything you can write an ordinary function for (a predicate), you can write a spec with. All you have to do is write an ordinary predicate on the inbound request (assuming it's ring)

👍 4
johnj15:05:23

with Java getting new features (java >8), does APIs that use stuff like functional interfaces/lambdas or APIs that use static/default interface methods make interop less smooth for Clojure or things just keep working the same?

ghadi15:05:24

clojure can still participate in that stuff @lockdown- through reify

Alex Miller (Clojure team)15:05:00

we’re planning to move to a Java 8 baseline in Clojure 1.10 and that will allow us to patch a few things in more automatically too

Alex Miller (Clojure team)15:05:31

like implementing java.util.Function or whatever

mg15:05:39

@alexmiller out of curiosity… I saw Toby Crawley’s talk on Java 9 last year, and my biggest takeaway was that Java modules are a catastrophe for Clojure. Has this situation improved?

Alex Miller (Clojure team)15:05:52

I don’t think that’s the correct takeaway

ghadi15:05:59

it's not a catastrophe

Alex Miller (Clojure team)15:05:06

My takeaway was that Java modules are a catastrophe :)

😆 8
👍 4
ghadi15:05:37

modules just are a move in the static direction, and we like dynamicity simple_smile

Alex Miller (Clojure team)15:05:02

you can use modules with Clojure and most stuff will do the same thing they would be in Java

mg15:05:03

more specifically, the way they break Clojure reflection unless you (at that time) passed some immediately-deprecated JVM startup flags

ghadi15:05:09

it doesn't really offer benefits to us.

Alex Miller (Clojure team)15:05:33

that breakage only happens if you are using Clojure that relies on reflection. it is pretty easy to introduce type hints to avoid that in most cases

mg15:05:37

lots of Clojure relies on reflection, whether you’re writing it or not. Turning on reflection warnings in almost any large project is a sobering moment, in my experience

Alex Miller (Clojure team)15:05:37

that’s not my experience, but maybe I’m just better at avoiding it in the first place

bronsa15:05:53

I’d say relying heavily on reflection is a pretty bad code smell

Alex Miller (Clojure team)15:05:58

For example, javax.xml.bind issue that people run into is easily fixed by adding that module

Alex Miller (Clojure team)15:05:19

And the most common xml-related reflection issue (coming out of data.xml, but now fixed there) is easily solved with a single type hint

Alex Miller (Clojure team)15:05:46

to force the use of the public interface rather than the non-visible implementation class

4
mg15:05:54

in other words, it’s not my code that’s triggering the warnings, but there are a lot of widely-used libraries out there that do

Alex Miller (Clojure team)15:05:27

the vast majority of those warnings are also not an issue on Java 9+

Alex Miller (Clojure team)15:05:00

there’s a pretty narrow set of cases that trigger the issue, and I expect we will have updates to the reflector in clojure 1.10 to address those too

Alex Miller (Clojure team)15:05:23

it’s just a matter of altering the reflection search algorithm to find only the visible stuff

mg15:05:32

ok. I think that’s the answer I was looking for, thanks

Alex Miller (Clojure team)15:05:43

“just” = hiding a lot of nastiness :)

👍 4
mg15:05:30

I remember that period of crazy drama last year when the module proposal got voted down at the last minute, which was a huge sense of relief, only to have it suddenly green-lit again a few weeks later (with no apparent significant changes)

Alex Miller (Clojure team)15:05:31

the tooling-related issues related to the bootclasspath were a bigger impact to users than most of the stuff above

johnj16:05:15

@bronsa isn't that a hard feat? Avoiding reflection, for example, just calling .toString causes it

bronsa16:05:56

not at all

bronsa16:05:04

type hinting is super easy

johnj16:05:47

@bronsa is reflection mostly a performance problem?

johnj16:05:41

Cause only when doing interop correct?

bronsa16:05:48

right

👍 4
sobel16:05:39

hrm, does adding type metadata improve that?

andy.fingerhut16:05:15

Adding the correct kind of type metadata to a Clojure program that is currently using reflection, can avoid its use of reflection after those changes, yes.

johnj17:05:15

Is reflection often ignored until it becomes a performance issue or it's recommended to avoid it from the start?

Alex Miller (Clojure team)17:05:43

both approaches are valid depending on the project and/or part of the project you are working on

Alex Miller (Clojure team)17:05:25

if I am working on code I know will be performance sensitive, I often turn on reflection warnings in that namespace and make a proactive effort to avoid reflection

Alex Miller (Clojure team)17:05:47

if I’m in something that’s called once or rarely, I might just ignore it

Alex Miller (Clojure team)17:05:12

if I’m writing a lib for others to use, I might take more care in avoiding reflection

Alex Miller (Clojure team)17:05:18

because the clojure compiler does type inference, often a single type hint on an arg or a return value can drastically reduce the reflection

johnj17:05:43

Got it, thanks

Alex Miller (Clojure team)17:05:25

(defn capitalize [s]
    (if (.isEmpty s)
      s
      (let [up (.toUpperCase (.substring s 0 1))
            down (.toLowerCase (.substring s 1))]
        (.concat up down))))

Alex Miller (Clojure team)17:05:48

uses a bunch of reflection, but you can just type-hint the s arg to ^String and the compiler then figures out the rest

johnj17:05:46

Oh nice, yeah that makes it non tedious

Alex Miller (Clojure team)17:05:03

and similarly, if you type-hint the return here to ^String then all down-stream callers can leverage that automatically

Alex Miller (Clojure team)17:05:16

(defn capitalize ^String [^String s] ...

Alex Miller (Clojure team)17:05:47

in most cases, I am unable to help doing at least this as I write the initial code

johnj17:05:06

I''ll look into it 👍

johnj17:05:30

So what does clojure used for runtime dispatch, it has its own stiff?

Alex Miller (Clojure team)17:05:18

it’s bytecode that runs on the jvm, so same as Java and Scala and Groovy and …

Alex Miller (Clojure team)17:05:53

that is, it just emits the proper bytecode for method invocation

hiredman17:05:13

the context of the question isn't clear to me. if you are asking how clojure does runtime dispatch in the context of the discussion about type hints, the anser is basically it doesn't, the type hints tell the compiler what types things are, the compiler generates bytecode with the types filled in, the jvm does any dispatch

hiredman17:05:30

if the question is a general, how would do polymorphism in clojure? the answer is by far the most common data passed around in clojure is some kind of map, so dispatch is usually something that chooses what do to based on a key in a map, and a common thing for that is a multimethod. clojure also has protocols and defrecord which give you something which is sort of between multimethods and java interfaces

hiredman17:05:19

the way the jvm does runtime dispatch is through a type hierarchy or interfaces, all of which still apply to type hints, you can type hint something with an interface, and it will compile to a non-reflective method call where the jvm machinery figures out which method to call based on which implementation of the interface is passed in (just like interfaces work in java)

johnj17:05:30

Your first paragraph answers my question.

johnj17:05:41

About multimethods thing, i guess clojure really does its own stuff here and pays a penalty correct?

hiredman17:05:35

I am not sure it is meaningfully comparable, you can only represent a small subset of multimethods usage as java type dispatch

hiredman17:05:32

and if you want to do java type dispatch, you can, so to say clojure pays a penalty for it, doesn't really track

johnj18:05:13

Helpful, thanks

ghaskins18:05:58

I continue on my journey wrestling with the dot special form

ghaskins18:05:07

(defmacro parsefn [sym]
  `(fn [is#]
     (. is# ~(eval `(symbol ~sym)))))

(defn- ->parsefn [type]
  (let [name (-> (str "cis->" type)
                 symbol)
        foo (str "read" "Enum")
        f (parsefn foo)]
    (intern *ns* name f)))

ghaskins18:05:27

this blows up whenever I call (parsefn) with a variable at compile time

ghaskins18:05:51

e.g. (parsefn “readEnum”) works fine

ghaskins18:05:29

but anything involving a variable blows up (at compile time, but works fine in the REPL)

ghaskins18:05:33

I dont understand why

scriptor18:05:05

have you tried calling macroexpand on the parsefn call

ghaskins18:05:13

in the repl, yeah

ghaskins18:05:21

in the repl, expanded looks correct, and it runs

hiredman18:05:02

what do you expect the macro expanded result to be?

ghaskins18:05:33

(defmacro parsefn [sym]
  `(fn [is#]
     (. is# ~(eval `(symbol ~sym)))))

=> #'user/parsefn
(macroexpand-1 '(parsefn (str "to" t)))
=> (clojure.core/fn [is__9937__auto__] (. is__9937__auto__ toString))

hiredman18:05:45

your eval call there is super weird

scriptor18:05:58

what is the output of macroexpand when you pass it a binding

hiredman18:05:59

not what does it expand to

ghaskins18:05:00

heh, i know…the dot special form is killing me

scriptor18:05:05

like you have in your earlier snippet

hiredman18:05:06

what do you want it to expand to?

scriptor18:05:22

(def foo (str "a" "b"))
(macroexpand-1 '(parsefn foo))

hiredman18:05:55

a macro is a transform from X to Y, so you need to know what your X looks like and what your Y looks like

ghaskins18:05:01

i need to basically dynamically compose a symbol at compile time and get it to the dot special form

ghaskins18:05:30

(def foo (str "a" "b"))
(macroexpand-1 '(parsefn foo))
=> #'user/foo
=> (clojure.core/fn [is__9937__auto__] (. is__9937__auto__ ab))

ghaskins18:05:39

and that is what I expect

hiredman18:05:25

the problem is almost certainly the call to eval

hiredman18:05:58

eval always evals in the global scope, not in any surrounding lexical scope

ghaskins18:05:07

the problem seems to be all the variants of dot special form require a symbol

hiredman18:05:08

but the eval is not needed anyway

ghaskins18:05:14

im open to how to accomplish that

noisesmith18:05:15

also . doesn't evaluate that method name arg and that auto generated method name is surely wrong

hiredman18:05:22

(eval (quote x)) == x

ghaskins18:05:05

to be clear, I am looking for something like (fn [obj] (. obj toString))

ghaskins18:05:31

but I need to assemble “toString” from parts

hiredman18:05:40

just do the algebra

ghaskins18:05:47

not following

ghaskins18:05:02

oh above, yes yes

hiredman18:05:16

~(eval `(symbol ~sym)) == ~`(symbol ~sym) ;; evaling a quoted form is just the form

ghaskins18:05:53

i think I tried that and it doesnt work, but let me try again

noisesmith18:05:03

ah, right, the method is the second arg when . is used that way, my misunderstanding

hiredman18:05:04

~(symbol sym) ;; unquoting and then immediately quoteing is kind of nonsense

ghaskins18:05:28

@hiredman yeah, that doesnt work

ghaskins18:05:32

you get

(defmacro parsefn [sym]
  `(fn [is#]
     (. is# ~`(symbol ~sym))))

=> #'user/parsefn
(macroexpand-1 '(parsefn (str "to" t)))
=> (clojure.core/fn [is__9971__auto__] (. is__9971__auto__ (clojure.core/symbol (str "to" t))))

hiredman18:05:40

(defmacro parsefn [sym]
 `(fn [is#]
    (. is# ~(symbol ~sym))))

ghaskins18:05:54

I need the (symbol) form evaluated before the (.) form

ghaskins18:05:17

so the ~(eval ..) malarky was a way I was trying to force that

hiredman18:05:18

yeah, you can't do that with dot

hiredman18:05:28

what you are talking about is reflection

hiredman18:05:46

not a macro at all

hiredman18:05:20

there are two ways to do the reflection, either use the java reflection api, or call eval and let clojure do it for you

ghaskins18:05:43

basically im trying to do a bunch of interop that has the same pattern against a class like this: https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/CodedInputStream

ghaskins18:05:52

e.g. .readString .readInt32, etc

ghaskins18:05:04

but I was trying to get a macro that could generate the set

hiredman18:05:18

(defn parsefn [sym] (eval `(fn [obj] (. obj ~(symbol sym)))))

ghaskins18:05:34

ah, thats an angle I havent tried

ghaskins18:05:36

let me try that

hiredman19:05:11

given something like that, instead of using reflection when invoking methods, I would use reflection to examine the class to get a list of accessors I want to generate, then have a macro use that to generate the accessors

ghaskins19:05:14

Id be curious to hear your thoughts here. I cant claim to fully understand what you are suggesting but I suspect its a different approach than the (eval `(fn ..)) one you already suggested

hiredman19:05:29

something like

(defmacro something []
  (cons 'do
        (for [method (.getDeclaredMethods com.google.protobuf.CodedInputStream)
              :let [argc (count (.getParameterTypes method))
                    args (repeatedly argc gensym)]]
          `(defn ~(symbol (.getName method)) [obj# ~@args]
             (. obj# (~(symbol (.getName method)) ~@args))))))

hiredman19:05:23

there is also clojure.reflect which attempts to expose the java reflection stuff as clojure datastructures, which might be nicer to work with

ghaskins19:05:29

so far, looking promising

ghaskins19:05:36

let me play a bit more, thank you

ghaskins19:05:44

out of curiosity, why did it work fine in the REPL?

ghaskins19:05:53

thats really confusing to me

hiredman19:05:41

because in the repl you were trying it with a var (a global)

hiredman19:05:49

and eval is always in the global scope

ghaskins19:05:08

ok, i think I somewhat understand

ghaskins19:05:13

@hiredman bingo, it all works

ghaskins19:05:17

you guys rock, thanks for the help

benzap21:05:47

So i've been looking over clojure/core/server.clj, and I have a few questions in regard to implementation : clojure/src/clj/clojure/core/server.clj - ~line 26 - Why does it use a var to store servers? It's also using a locking mechanism to ensure there are no concurrency issues. Why not just use a ref in this situation? - ~line 38 - Why does it use a Thread directly? Why isn't it using an agent? I only ask because i'm trying to get familiar with the code so I can write my own socket repl for a pet project, and I was going to use ref and agent over the implementation presented here. Is there a particular reason why this would be a bad idea?

Alex Miller (Clojure team)21:05:25

re 26 - refs are good for data, locks are better for thread coordination

Alex Miller (Clojure team)21:05:50

re 38 - wanted a separate and identifiable thread pool for this

benzap21:05:31

I see, interesting

ghadi21:05:33

I think @benzap is asking why the lock is needed at all @alexmiller

ghadi21:05:05

i think the socket manipulation (open/close etc.) necessitates it, but not necessarily the data accounting

benzap22:05:13

My guess is to prevent any possible headaches down the road, they chose to using a lock to ensure there isn't any potentially hard to debug code in the future

benzap22:05:43

It looks like it's used in any situation where the servers var is used

benzap22:05:09

at which point, it probably resembles more of an atom

noisesmith22:05:26

you don't want to create a stateful resource in any operation that might retry

noisesmith22:05:05

which means some kind of lock somewhere, or no concurrency (and even no concurrency is enforced with ... a lock)