This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-08-27
Channels
- # admin-announcements (42)
- # aws (15)
- # beginners (8)
- # boot (102)
- # cider (7)
- # clojure (141)
- # clojure-italy (10)
- # clojure-japan (4)
- # clojure-russia (26)
- # clojure-seattle (1)
- # clojurescript (239)
- # core-matrix (13)
- # cursive (19)
- # datascript (54)
- # datomic (21)
- # editors (2)
- # events (1)
- # hoplon (125)
- # instaparse (10)
- # jobs (3)
- # ldnclj (13)
- # ldnproclodo (1)
- # om (1)
- # onyx (2)
- # rdf (206)
- # re-frame (30)
- # reagent (7)
- # yada (4)
Has anyone ever seen a diffing tool for Clojure/lisp that highlights diffs based on semantic changes, not just whitespace?
danielcompton: have you tried https://janestreet.github.io/patdiff.html ?
So I haven’t tested this … but maybe somebody already knows the answer: say I want to invoke a Java instance method by String, I could use clojure.lang.Reflector, or I could write a multimethod that dispatches to a set of (.foo x) functions; which would likely be faster?
Java reflection is slow, but if you can keep hold of the java.lang.reflect.Method instance, invoking that will be fast
@joelkuiper: the multi-method approach would be faster, if you know up-front which methods you'll need to call
@joelkuiper: disclaimer: I am not a profiler
hah, I could memoize Reflector
that’s a stupid trick 😛
why are there so many versioning schemes used? Latest Clojure is 1.8.0-alpha4, latest Clojurescript is 1.7.107, latest core.async is 0.1.346.0-17112a-alpha (which is ridiculously long)
I tend to approach versions as mostly meaningless strings with no discernible (partial) ordering for that reason 😛
with that mindset it doesn’t bother me 😉, might as well use git-sha’s for all I care
if one wants to use sha's ok, but these are release versions which carry some meaning. The core.async one is longer than an average safe password
in the case of core.async I think it’s generated by pinning SNAPSHOT versions in maven
M-x cider-jump-to-var
I’m having a problem with ring.mock.request - the header function adds the header I pass to the :headers map, but doesn’t put the header in the request as well. This means I can’t set :remote-addr like this. Is this a bug or am I missing something?
There are specific functions for content-type and content-length that do add to both - is it the case that not all headers should be added to the request map?
Actually, never mind, I just figured out I can use middleware on my handler to do the same thing.
@nicola: I think its normally bound . M-.
Is ring multi-threaded?
I have a Clojure web app using ring and compojure and so far I didn’t care about threading because Clojure shields me from it (immutable data for the win) but now I want to share an Java object between requests that is not multi-thread-safe and it’s mutable. How do I shield about two threads using the same one at the same time?
you could wrap the Java object in an atom, and use swap! with a function that does some defensive copying
pbalduino: I could put it in an atom, yes, when would I use swap!?
use swap! when you need to mutate the object, but create a copy first and mutate that, then return that
pupeno: putting a mutable object inside an atom is not generally safe, since the point of swap! is that it can retry instead of commit if there has been parallel alteration. Nothing about that mechanism prevents your mutable object from being punched repeatedly and in the wrong order.
yes, just making sure that distinction is clear - and the fact that retries will happen
thanks to retries, just using a mutable object inside an atom is likely less safe than using it naked would be (unless you have some reasonable copying scheme)
pbostrom: I pick up on this now in particular because we discovered people on my team were kind of cargo culting atoms, and doing mutation and derefs inside swap! operations - leading to things breaking in very weird ways of course
though I would seriously reconsider doing things that way
heh 😛 I just don't want my routing definitions to worry about how I've decided to split up my handlers.
yeah, import-vars from potemkin will take care of that then
but my complaint about these kinds of schemes is it makes finding things in your code harder (we get used to being able to find defs in the namespaces they belong to after all)
@noisesmith: I agree. And I'm kind of deciding over whether it's going to cause issues or not.
I just feel awkward doing (:require [bans.routes.auth :as auth-routes] [bans.routes.x :as x-routes])
yeah, import-vars will make it easy to construct a top level so consumers don't need to worry about your namespace layout
@pupeno Ring itself is single-threaded, as it turns into a chain of function calls. However, the adapter, usually for Jetty, is multi-threaded, so individual requests will be executing on multiple threads; Jetty defines the thread pool for handling such requests.
I did a bit of work to integrate Ring with core.async ... in practice, it added a lot of complexity and made it much harder to debug, and didn't seem to affect performance in any measurable way.
Ultimately, we stripped the core.async support back out (this was in earlier versions of https://github.com/AvisoNovate/rook).
hlship: so, the functions that are called by the routes don’t need to be thread safe?
hlship: I’m using nashorn, it’s not thread safe.
But no, just the opposite; those functions are going to be called by many threads simultaneously.
so if you are, say, invoking methods on non-threadsafe Java objects, you may see some race conditions
But if jetty is creating new threads, then each of those threads should have their own copy of the object.
But if your Ring request handler functions are pure, or semi-pure (a term that roughly translates to "pure, except it hits the database") you should be fine
My problem is that I need to create an object, a scriptengine, and re-use it, but never in two threads at once.
perhaps make the var special (thread-local) so that each thread gets their own instance of it?
just make sure that each thread initializes the thread-local variable is nil, and make its root binding nil...
maybe, maybe that's foolish
Each thread should be using it’s own render-app. Does it make sense?
sure, in that case making the var a dynamic binding makes sense maybe
unless you are using a ring-adaptor that moves a single request between threads
in that case a better solution is a middleware that attaches an exclusive script-engine instance to the request object itself?
a middleware feels like an overkill, but I haven’t found a solution that works yet.
@ztellman: https://github.com/ztellman/potemkin/issues/30 I think this macro is great, just ftr.
pupeno: middlewares are so easy to write, and they can be very lightweight
(defn attach-script-engine [handler] (fn [request] (handler (assoc-request (get-engine pool))))
noisesmith: ah… but then I need to implement a pool of engines.
well I thought that was a pre-requisite for your plan no matter what
I mean replace that with making a fresh one each time if you prefer
noisesmith: I’m not sure about that yet.
What I am trying to avoid is making a fresh one. That is the problem I’m trying to fix, as they are expensive.
another option (defn attach-script-engine [handler] (fn [request] (handler (assoc request :engine (make-script-engine)))))
sure, OK, so if you can't make fresh ones, and threads can't share them, you need a pool
but I'm suggesting that regardless of those things, a middleware is a lightweight way to provide them concretely to the request handler
noisesmith: not really. Jetty is creating a pool of threads already, each thread needs 1 engine, so, I don’t need a pool.
pupeno: OK if you only use jetty that's that
I just need to create the engines so it follows the semantics of whatever container is running my code.
pupeno: I suggested the other option because some very good servers don't attach a request to a single thread
but you can carry a resource in the request (which led to the middleware suggestion)
What’s the situation in which a server doesn’t attach a request to a single thread?
netty for example, has non-blocking async stuff - great for performance, but the cost is that your request might be passed between threads and/or share a thread with other concurrent requests
so you can't count on the 1-1 thread to request mapping
aleph and http-kit iirc are based on netty
But on each thread there can be only 1 request at any one time.
err, I don't think aleph does use netty actually, but it does break the 1-1 mapping concept
pupeno: depends on what you mean by "at any time" - the thread could swap which request it is working on before the other request has finished
think core.async style parking
I believe that is fine for me.
If I were to use a pool, I can just request it from the pool when I need it, no need to attach it to the request. 1 request will use the script engine once and only once and even if it uses twice, it’s ok if they are different ones. The only thing I have to prevent is the same script engine being used at the same time by two different threads.
Would each thread run (defroutes …)
dominicm: I appreciate the vote, but that means that when someone is referencing a var in a namespace, there’s no explicit reference to it anywhere in the namespace
pupeno: yes, when a request comes in, a thread is used off the pool to run the handler
@ztellman: That's true, but it makes it easier for me to just "collect" namespaces. One less place to remember to update, when I modify one of the imported namespaces.
my only consolation is that it’s super easy to roll your own, as evidenced in the issue
@ztellman: Yep. Very easy, although I don't know if I could have got there without the help. Just figured I'd register interest without bumping a dead issue.
but my gut feeling is that it’s just a little over the threshold of “too much magic” for me
So, if each thread is executing (defroutes …) then here: https://gist.github.com/pupeno/0aca674623f4f3530d8a each thread should automatically get their own js-engine, correct?
Is there a preferred way to say “an infinite lazy seq containing only this object”? (cycle [x])
? (repeatedly (constantly x))
?
(repeat x)
I never know which is which, repeatedly, repeat and iterate
@pesterhazy: I just think of it like this, repeatedly
takes a function (an action) like (repeatedly increment)
you want to "repeatedly increment" something
@bostonaholic: I think iterate
would fit that example better
depends on the implementation of increment
but, I was just trying to explain how I remember which function of the three take which arguments
What are the benefits and drawbacks of putting :aot :all at the root of your project map in project.clj?
But are there other hidden disadvantages, such as vars not having metadata available at runtime?
I don't understand how that can be since it's in the project.clj file which is only seen by Leiningen.
no, creating jars is different. It just means that you have a .class that you can run via java, once you’ve set the classpath
ah, I misread the original question… I was distracted. When I saw “root” I was thinking that you were only talking about doing AOT on the namespace with -main in it
If you’re doing AOT on everything then it’s useful for providing classes to frameworks that need them (since everything is already compiled). It’s also a little bit quicker to start (no parse/compile needed). But the only reason I’ve usually seen is if you’re writing commercial code and you don’t want to be distributing your source code to everyone
Does anyone know how I can figure out the namespace of a file read in with load-script? (https://clojuredocs.org/clojure.main/load-script)
I've tried using (all-ns) before and after, but that all gets anything that's a dep of the file that's been read. (afaict)
Hey all is there a way to get Instaparse to use JavaCC type files? like https://github.com/apache/jena/blob/31a8cce9e26e46526613c2ca550c900c3dcb4b34/jena-arq/Grammar/Final/tokens_11.txt