Fork me on GitHub
#clojure
<
2015-06-17
>
devn03:06:06

Having trouble setting the logback.configurationFile via :jvm-opts in a profile within my project.clj

devn03:06:37

it disobeys me unless I export JAVA_OPTS=... && lein ring server

devn03:06:51

any idea how to get it to play nice?

devn03:06:07

id like to have a separate file per environment

danielcompton03:06:49

@devn IIRC there is different metadata you can attach to keys in project.clj to control how they’re merged with default ones

devn04:06:24

im not sure that's the issue here

stijn09:06:57

does anyone know why partition-all has an ArrayList managing the state in the transducer version? https://github.com/clojure/clojure/blob/clojure-1.7.0-RC1/src/clj/clojure/core.clj#L6943

stijn09:06:34

all the other stateful transducers are making use of volatiles...

ragge10:06:26

@stijn: probably depends on the type of state they need to maintain

ragge10:06:47

@stijn: partition-all and partition-by need to hang on to actual things being reduced

ragge10:06:07

@stijn: take otoh only needs to keep a counter

stijn10:06:35

distinct keeps a collection in a volatile too

stijn10:06:10

as far as I understand it wouldn’t be safe to use partition-all in a core.async context e.g.

ragge11:06:34

aren't channel ops executed in serial, under a lock

rauh11:06:48

@stijn: partition-by also uses ArrayList, I was wondering the same and re-wrote it with volatile a few weeks ago: The volatile was about half as fast

rauh11:06:41

@stijn: All the volatile usages for transducers don't hold a collection but often only a flag or a counter. Only distinct holds a volatile set.

stijn12:06:35

hmm, I see, but that means that if your operation gets scheduled on another thread, it might not see the correct value of the partition built so far

rauh12:06:08

@stijn: Not sure what you mean with that. Transducers are not parallel at all currently. And you still get a (vec) view of it when you actually get the result.

stijn12:06:25

@rauh: no they’re not parallel, but if you put a transducer on a core.async channel, is there a guarantee that it’s always executed on the same thread?

stijn12:06:14

@rauh: otherwise it might be the case the value of your arraylist is not visible on the other thread (when running on multiple cores)

stijn12:06:06

a volatile synchronizes the write between threads, a java Vector does the same, but ArrayList doesn't

stijn12:06:15

but maybe I’m missing something obvious here simple_smile

rauh12:06:35

@stijn: You arent' allowed to share the transducer like that. I believe it's noted in Rich's "inside transducer" video

stijn12:06:11

@rauh: it’s not sharing the transducer, you set the transducer on a channel, which will instantiate it. the only process having access to the transducer state is that particular channel.

stijn12:06:14

but in between receiving values which are stored in the arraylist the executing thread might change

stijn12:06:37

it’s not about concurrent access, it’s about having the value visible to the currently executing thread on the jvm

rauh12:06:28

Yeah I understand what you mean, but once you instantiate the transducer with the reducing function you'll have to keep it to yourself. Check out the last sentence on: http://clojure.org/transducers

stijn12:06:42

meaning that partition-by and partition-all cannot be used on a core.async chan then?

alexmiller12:06:04

no, they can

alexmiller12:06:24

the ArrayList reference doesn't change, so it doesn't need to be volatile

alexmiller12:06:42

that is, it is the stateful container

alexmiller12:06:05

an alternative would be a (volatile []) but that would be much slower

stijn12:06:53

@alexmiller: and a write to the list is being flushed to memory always?

alexmiller12:06:10

yes, although I think the question you're asking is whether visibility is guaranteed across threads right?

alexmiller12:06:39

I would say yes and if not it's a bug :)

alexmiller12:06:18

in the case of a channel transducer, the transducer is being applied inside the channel, under a lock and that lock will force happens-before

stijn12:06:59

ok, I see

stijn13:06:33

@alexmiller: is there any particular reason why distinct uses a volatile set instead of a mutable java set?

alexmiller13:06:55

java sets use java hashing

alexmiller13:06:01

but we want clojure hashing

alexmiller13:06:54

the first impl did (incorrectly) use java hashing :)

stijn13:06:26

well thanks for the explanation. very interesting simple_smile

stijn13:06:15

we had a discussion at yesterday’s BeClojure meetup where the ArrayList came up (as I copied the code to create a partition-all that allows stepping)

escherize13:06:37

How should I think about the fact that

(= false (= #{1} [1]))

stig13:06:30

escherize: probably best not to think about it. Or at least make sure you sit down when you do.

dnolen13:06:45

@escherize: sets are unordered, so it makes sense that won’t work

escherize13:06:52

just as a thought exercise, what would be the best way to check that collections contain the same elements? #(= (vec %1) (vec %2)) #(= (set %1) (set %2))

stig13:06:49

I think that depends whether hashing the items or a straight compare is quicker…?

escherize13:06:15

I would think vec is faster

stig13:06:38

especially if there’s a lot of differences

dnolen13:06:49

stig: well, first is a seq operation not a set operation

escherize13:06:09

wait vec isn't going to work for that above

escherize13:06:18

since sets are unordered.

stig13:06:20

escherize: doh! you’re right

stig13:06:23

dnolen: i know, I was more thinking that it’s not immediately apparent because it’s not an error to do that, despite it not making much sense. and for the toy data example the expected result (if it had been a sequence) was found

stig13:06:35

s/found/returned/

stig13:06:58

escherize: then I think you have your answer simple_smile

escherize13:06:31

One must check that the initial sizes were equal too

em15:06:38

Hi, I want to provide a java wrapper class for my clojure project. I'm wondering if leiningen supports building and unit testing the java wrapper class. Would this be simple to do? No, I haven't tried yet (sorry).

stuartsierra15:06:19

@em Leiningen can compile Java sources (`:java-source-paths` in project.clj I think). Not sure what testing would involve.

em15:06:02

Thanks @stuartsierra. Would the dependency (and thus the build order) from java code to clojure be a problem? All samples I've seen so far have the dependency the other way.

stuartsierra15:06:39

@em I think the default is to compile the Java first. Don't know if that can be changed.

alexmiller15:06:24

you can hack it with :prep-tasks in leiningen I think

em15:06:55

Thanks! I'll give that a try

txus15:06:29

@juhoteperi: it was my fault in the end — got it working today and it’s AMAZING!

txus15:06:40

thank you! 😄

aengelberg17:06:00

Question: if I write a library using cljc files, will that library be completely incompatible when consumed by projects with clojure <= 1.6? Or will some compilation step during the jarring / deploying eliminate that specific concern?

alexmiller18:06:05

it is incompatbile

alexmiller18:06:40

cljc and reader conditionals will only work on 1.7+

alexmiller18:06:36

someone somewhere (forget where now) suggested backporting (only) reader conditionals to a 1.6.1. I'm not eager to take that on but if there was a widespread desire for it, that might be an option.

aengelberg18:06:59

Man. I was hoping that while deploying something to clojars, it would compile to .class files, at which point it wouldn't matter whether it came from clj or cljc

alexmiller18:06:17

if you aot to .class, then you should actually be ok

alexmiller18:06:38

.class is preferred to clj or cljc during load

aengelberg18:06:01

Does that happen automatically during lein deploy clojars? (noob jar question)

alexmiller18:06:15

you would have to set up your lein project for aot

alexmiller18:06:34

"lein sample" and grep for aot

aengelberg18:06:12

Any prominent downsides of doing that? Lamer stack traces, etc?

alexmiller18:06:24

well, you'd be compiling with 1.7

alexmiller18:06:42

it's possible that you would create code that depended on 1.7 at runtime as well

alexmiller18:06:54

that is, calling into functions that don't exist in the 1.6 runtime

aengelberg18:06:53

Wouldn't such issues appear anyway if I just distribute source?

alexmiller18:06:21

generally, no if you avoid using functions that are new in 1.7

stuartsierra18:06:27

AOT-compiled .class files contain references to internal methods in the Clojure runtime.

alexmiller18:06:25

calls to RT etc

aengelberg18:06:43

OK, that makes sense.

alexmiller18:06:22

we do make an effort not to break or remove existing methods as we move forward, but of course if you compile to new things that are added, then the old RT can

alexmiller18:06:27

t have those

alexmiller18:06:00

so old compiled .class can run on new runtime but new compiled .class can't necessarily run on old runtime

aengelberg18:06:40

Thanks Alex (+ Stuart cameo)

alexmiller18:06:15

that said, the diffs in this area are small and you might get away with it :)

alexmiller18:06:26

but we don't test for it

aengelberg18:06:27

I'm trying to figure out the best way to target clj and cljs from instaparse.

aengelberg18:06:53

With cljx apparently now deprecated, I'm not sure what's left (since cljc is incompatible with <1.7 and that compatibility is important to us)

stuartsierra18:06:38

cljx is a source-code transformation, so I expect it will continue to work for a while.

alexmiller18:06:28

I looked at the diffs from 1.6 to 1.7 again and I think there is almost nothing that would trip up .class code from 1.7 running on 1.6. The trickiest likely issue would be the classloading changes from CLJ-979 but it's hard for me to know for sure without doing a bunch more testing.

danielcompton21:06:55

I suggested a cljc backport option in 1.6.1, or a tool like cljx which creates cljs and clj files

ej21:06:13

what’s the most used auth library?

ej21:06:41

I want to integrate sessions and persisted users with compojure + sente.

danielcompton22:06:00

Friend has plenty of examples, but none integrating with Sente that I’ve seen

bostonaholic22:06:22

Is than an equivalent to *clojure-version* to get the java version?

jcf22:06:07

@bostonaholic: have you tried (System/getProperty "java.version") ?

jcf22:06:55

@ej: I've hooked Sente up to Buddy (not Friend but similar). The trick is to store your current user's identity in the session, and look that up when a websocket connection comes in. There's an example of doing auth inside the Sente example app. https://github.com/ptaoussanis/sente/blob/master/example-project/src/example/my_app.cljx

danielcompton22:06:43

@ej I’ve got some open issues on Sente about alternative auth mechanisms, as session based isn’t what we’re after

ej22:06:36

@danielcompton: what do you propose?

danielcompton22:06:58

Allowing the user to pass headers with auth tokens

ej22:06:38

yeah got a boot setup here too, index.html + re-frame js compiled

ej22:06:42

to be deployed to a cdn

danielcompton22:06:20

@ej, I’m using re-frame too simple_smile

devn22:06:56

I'm having trouble getting JVM properties to pass through from leiningen. In my project.clj for a specific profile dev, I have :jvm-opts ["-Dfoo.bar=abc"]. I do not see this when running lein ring server in System/getenv or System/getProperties, nor do I see it when running leiningen with -m my.ns with a -main in it.

devn22:06:00

If I do java -Dfoo.bar=abc -cp myjar.jar clojure.main -m foobar.namespace, it gets picked up.

devn23:06:47

profiles.clj in the project directory was the issue

ej23:06:37

(defn index
  "Handle index page request. Injects session uid if needed."
  [req]
  {:status 200
   :session (if (session-uid req)
              (:session req)
              (assoc (:session req) :uid (unique-id)))
   :body (-> "index.html"
             io/resource
             slurp)})

(defroutes my-routes
  (GET  "/"  req (index req))
  (GET  "/chsk"  req ((:ring-ajax-get-or-ws-handshake (:sente system)) req))
  (POST "/chsk"  req ((:ring-ajax-post (:sente system)) req))
  ;;(POST "/login" req (login! req))
  ;;
  (route/not-found "<h1>Page not found</h1>"))

(def my-ring-handler
  (let [ring-defaults-config
        (-> site-defaults
            (assoc-in [:static :resources] "/")
            (assoc-in [:security :anti-forgery] {:read-token (fn [req] (-> req :params :csrf-token))}))]
    (ring.middleware.defaults/wrap-defaults my-routes ring-defaults-config)))
Any idea why this returns the raw file? I can’t seem to override the content-header from application/octet-stream surely the file should be slurped?

ej23:06:38

it just opens a download prompt in my browser to the index.html file