Fork me on GitHub
#clojure
<
2015-08-27
>
danielcompton02:08:39

Has anyone ever seen a diffing tool for Clojure/lisp that highlights diffs based on semantic changes, not just whitespace?

plexus07:08:47

that's funny someone mentioned patdiff to me just yesterday 😉

cfleming07:08:01

Is anyone using Kerodon for interaction testing Ring apps? It looks very nice.

plexus07:08:46

looks nice indeed. the name and API suggest it's inspired by Capybara

joelkuiper09:08:01

So I haven’t tested this … but maybe somebody already knows the answer: say I want to invoke a Java instance method by String, I could use clojure.lang.Reflector, or I could write a multimethod that dispatches to a set of (.foo x) functions; which would likely be faster?

malcolmsparks09:08:06

Java reflection is slow, but if you can keep hold of the java.lang.reflect.Method instance, invoking that will be fast

malcolmsparks09:08:00

@joelkuiper: the multi-method approach would be faster, if you know up-front which methods you'll need to call

malcolmsparks09:08:20

@joelkuiper: disclaimer: I am not a profiler

joelkuiper09:08:18

hah, I could memoize Reflector

joelkuiper09:08:22

that’s a stupid trick 😛

xificurC09:08:30

why are there so many versioning schemes used? Latest Clojure is 1.8.0-alpha4, latest Clojurescript is 1.7.107, latest core.async is 0.1.346.0-17112a-alpha (which is ridiculously long)

joelkuiper09:08:59

I tend to approach versions as mostly meaningless strings with no discernible (partial) ordering for that reason 😛

joelkuiper09:08:31

with that mindset it doesn’t bother me 😉, might as well use git-sha’s for all I care

xificurC09:08:20

if one wants to use sha's ok, but these are release versions which carry some meaning. The core.async one is longer than an average safe password

xificurC09:08:54

like 2-3 times actually

joelkuiper11:08:40

in the case of core.async I think it’s generated by pinning SNAPSHOT versions in maven

niquola12:08:54

hey, cider users - how to jump to java source from emacs?

delaguardo13:08:12

@nicola:

M-x cider-jump-to-var

cfleming13:08:24

I’m having a problem with ring.mock.request - the header function adds the header I pass to the :headers map, but doesn’t put the header in the request as well. This means I can’t set :remote-addr like this. Is this a bug or am I missing something?

cfleming13:08:05

There are specific functions for content-type and content-length that do add to both - is it the case that not all headers should be added to the request map?

cfleming13:08:15

Actually, never mind, I just figured out I can use middleware on my handler to do the same thing.

cfleming14:08:40

@xeqi: Kerodon is lovely, thanks

rickmoynihan14:08:20

@nicola: I think its normally bound . M-.

Pablo Fernandez14:08:32

Is ring multi-threaded?

Pablo Fernandez15:08:35

I have a Clojure web app using ring and compojure and so far I didn’t care about threading because Clojure shields me from it (immutable data for the win) but now I want to share an Java object between requests that is not multi-thread-safe and it’s mutable. How do I shield about two threads using the same one at the same time?

pbostrom15:08:18

you could wrap the Java object in an atom, and use swap! with a function that does some defensive copying

Pablo Fernandez15:08:39

pbalduino: I could put it in an atom, yes, when would I use swap!?

pbostrom15:08:37

pupeno: I assumed you would be mutating the Java object inside your ring handler

pbostrom15:08:10

use swap! when you need to mutate the object, but create a copy first and mutate that, then return that

rauh15:08:57

@pupeno: You can do java locking with (locking obj ...)

noisesmith15:08:21

pupeno: putting a mutable object inside an atom is not generally safe, since the point of swap! is that it can retry instead of commit if there has been parallel alteration. Nothing about that mechanism prevents your mutable object from being punched repeatedly and in the wrong order.

pbostrom15:08:03

my advice was to make a defensive copy inside swap!

noisesmith15:08:24

yes, just making sure that distinction is clear - and the fact that retries will happen

pbostrom15:08:31

I think locking macro is probably better for this case

noisesmith15:08:02

thanks to retries, just using a mutable object inside an atom is likely less safe than using it naked would be (unless you have some reasonable copying scheme)

noisesmith15:08:43

pbostrom: I pick up on this now in particular because we discovered people on my team were kind of cargo culting atoms, and doing mutation and derefs inside swap! operations - leading to things breaking in very weird ways of course

dominicm15:08:09

Is there a way to expose something I've imported?

dominicm15:08:33

Basically, I want to collect a bunch of namespaces into a single namespace.

noisesmith15:08:02

though I would seriously reconsider doing things that way

ztellman15:08:06

ah, yes, the mandatory disclaimer

dominicm15:08:29

heh 😛 I just don't want my routing definitions to worry about how I've decided to split up my handlers.

noisesmith15:08:48

yeah, import-vars from potemkin will take care of that then

noisesmith15:08:25

but my complaint about these kinds of schemes is it makes finding things in your code harder (we get used to being able to find defs in the namespaces they belong to after all)

dominicm15:08:06

@noisesmith: I agree. And I'm kind of deciding over whether it's going to cause issues or not.

dominicm15:08:45

I just feel awkward doing (:require [bans.routes.auth :as auth-routes] [bans.routes.x :as x-routes])

noisesmith15:08:22

yeah, import-vars will make it easy to construct a top level so consumers don't need to worry about your namespace layout

hlship16:08:50

@pupeno Ring itself is single-threaded, as it turns into a chain of function calls. However, the adapter, usually for Jetty, is multi-threaded, so individual requests will be executing on multiple threads; Jetty defines the thread pool for handling such requests.

hlship16:08:33

I did a bit of work to integrate Ring with core.async ... in practice, it added a lot of complexity and made it much harder to debug, and didn't seem to affect performance in any measurable way.

hlship16:08:00

Ultimately, we stripped the core.async support back out (this was in earlier versions of https://github.com/AvisoNovate/rook).

Pablo Fernandez16:08:01

hlship: so, the functions that are called by the routes don’t need to be thread safe?

hlship16:08:36

Well, you have to go out of your way in Clojure to not be thread safe.

Pablo Fernandez16:08:54

hlship: I’m using nashorn, it’s not thread safe.

hlship16:08:54

But no, just the opposite; those functions are going to be called by many threads simultaneously.

hlship16:08:12

so if you are, say, invoking methods on non-threadsafe Java objects, you may see some race conditions

Pablo Fernandez16:08:42

But if jetty is creating new threads, then each of those threads should have their own copy of the object.

hlship16:08:03

But if your Ring request handler functions are pure, or semi-pure (a term that roughly translates to "pure, except it hits the database") you should be fine

Pablo Fernandez16:08:10

My problem is that I need to create an object, a scriptengine, and re-use it, but never in two threads at once.

noisesmith16:08:58

perhaps make the var special (thread-local) so that each thread gets their own instance of it?

noisesmith16:08:22

just make sure that each thread initializes the thread-local variable is nil, and make its root binding nil...

noisesmith16:08:28

maybe, maybe that's foolish

Pablo Fernandez16:08:47

Each thread should be using it’s own render-app. Does it make sense?

noisesmith16:08:36

sure, in that case making the var a dynamic binding makes sense maybe

noisesmith16:08:52

unless you are using a ring-adaptor that moves a single request between threads

noisesmith16:08:34

in that case a better solution is a middleware that attaches an exclusive script-engine instance to the request object itself?

Pablo Fernandez16:08:32

a middleware feels like an overkill, but I haven’t found a solution that works yet.

noisesmith16:08:32

pupeno: middlewares are so easy to write, and they can be very lightweight

noisesmith16:08:29

(defn attach-script-engine [handler] (fn [request] (handler (assoc-request (get-engine pool))))

Pablo Fernandez16:08:49

noisesmith: ah… but then I need to implement a pool of engines.

noisesmith16:08:04

well I thought that was a pre-requisite for your plan no matter what

noisesmith16:08:15

I mean replace that with making a fresh one each time if you prefer

Pablo Fernandez16:08:16

noisesmith: I’m not sure about that yet.

Pablo Fernandez16:08:48

What I am trying to avoid is making a fresh one. That is the problem I’m trying to fix, as they are expensive.

noisesmith16:08:03

another option (defn attach-script-engine [handler] (fn [request] (handler (assoc request :engine (make-script-engine)))))

noisesmith16:08:27

sure, OK, so if you can't make fresh ones, and threads can't share them, you need a pool

noisesmith16:08:57

but I'm suggesting that regardless of those things, a middleware is a lightweight way to provide them concretely to the request handler

Pablo Fernandez16:08:13

noisesmith: not really. Jetty is creating a pool of threads already, each thread needs 1 engine, so, I don’t need a pool.

noisesmith16:08:28

pupeno: OK if you only use jetty that's that

Pablo Fernandez16:08:47

I just need to create the engines so it follows the semantics of whatever container is running my code.

noisesmith16:08:49

pupeno: I suggested the other option because some very good servers don't attach a request to a single thread

noisesmith16:08:11

but you can carry a resource in the request (which led to the middleware suggestion)

Pablo Fernandez16:08:02

What’s the situation in which a server doesn’t attach a request to a single thread?

noisesmith16:08:13

netty for example, has non-blocking async stuff - great for performance, but the cost is that your request might be passed between threads and/or share a thread with other concurrent requests

noisesmith16:08:23

so you can't count on the 1-1 thread to request mapping

noisesmith16:08:51

aleph and http-kit iirc are based on netty

Pablo Fernandez16:08:23

But on each thread there can be only 1 request at any one time.

noisesmith16:08:47

err, I don't think aleph does use netty actually, but it does break the 1-1 mapping concept

noisesmith16:08:14

pupeno: depends on what you mean by "at any time" - the thread could swap which request it is working on before the other request has finished

noisesmith16:08:31

think core.async style parking

Pablo Fernandez16:08:59

I believe that is fine for me.

Pablo Fernandez16:08:22

If I were to use a pool, I can just request it from the pool when I need it, no need to attach it to the request. 1 request will use the script engine once and only once and even if it uses twice, it’s ok if they are different ones. The only thing I have to prevent is the same script engine being used at the same time by two different threads.

ztellman16:08:27

http-kit doesn’t use netty

ztellman16:08:48

and in aleph, by default, there’s a separate thread pool for requests

ztellman16:08:51

so blocking is okay

Pablo Fernandez16:08:56

Would each thread run (defroutes …)

ztellman16:08:24

dominicm: I appreciate the vote, but that means that when someone is referencing a var in a namespace, there’s no explicit reference to it anywhere in the namespace

ztellman16:08:43

pupeno: yes, when a request comes in, a thread is used off the pool to run the handler

ztellman16:08:01

if the handler blocks, it won’t cause things around it to hang

dominicm16:08:27

@ztellman: That's true, but it makes it easier for me to just "collect" namespaces. One less place to remember to update, when I modify one of the imported namespaces.

ztellman16:08:44

yeah, I can see that

ztellman16:08:02

my only consolation is that it’s super easy to roll your own, as evidenced in the issue

dominicm16:08:10

@ztellman: Yep. Very easy, although I don't know if I could have got there without the help. Just figured I'd register interest without bumping a dead issue.

ztellman16:08:45

again, I appreciate it, I’ll give it some more thought

ztellman16:08:17

but my gut feeling is that it’s just a little over the threshold of “too much magic” for me

dominicm16:08:28

That's fair enough. Thanks simple_smile

Pablo Fernandez16:08:50

So, if each thread is executing (defroutes …) then here: https://gist.github.com/pupeno/0aca674623f4f3530d8a each thread should automatically get their own js-engine, correct?

lvh16:08:54

Is there a preferred way to say “an infinite lazy seq containing only this object”? (cycle [x])? (repeatedly (constantly x))?

lvh16:08:02

I knew I was missing one. Thanks simple_smile

pesterhazy17:08:19

I never know which is which, repeatedly, repeat and iterate

bostonaholic17:08:44

@pesterhazy: I just think of it like this, repeatedly takes a function (an action) like (repeatedly increment)

bostonaholic17:08:24

you want to "repeatedly increment" something

pesterhazy17:08:44

@bostonaholic: I think iterate would fit that example better simple_smile

bostonaholic17:08:43

depends on the implementation of increment

bostonaholic17:08:12

but, I was just trying to explain how I remember which function of the three take which arguments

exupero17:08:01

- repeatedly call f - repeat the value x - iterate f over x

andrewhr17:08:08

iterate could be though as a lazy reduce?

sdegutis21:08:24

What are the benefits and drawbacks of putting :aot :all at the root of your project map in project.clj?

sdegutis21:08:27

I imagine it would terribly mess with dynamic code reloading at runtime right?

sdegutis21:08:31

But are there other hidden disadvantages, such as vars not having metadata available at runtime?

quoll21:08:00

one benefit is not needing to run your project via lein

sdegutis21:08:25

Meaning it produces a jar file?

sdegutis21:08:48

I don't understand how that can be since it's in the project.clj file which is only seen by Leiningen.

quoll21:08:48

no, creating jars is different. It just means that you have a .class that you can run via java, once you’ve set the classpath

sdegutis21:08:12

I don't see what the disadvantages are then.

quoll22:08:59

ah, I misread the original question… I was distracted. When I saw “root” I was thinking that you were only talking about doing AOT on the namespace with -main in it

quoll22:08:00

If you’re doing AOT on everything then it’s useful for providing classes to frameworks that need them (since everything is already compiled). It’s also a little bit quicker to start (no parse/compile needed). But the only reason I’ve usually seen is if you’re writing commercial code and you don’t want to be distributing your source code to everyone

lfn322:08:47

Does anyone know how I can figure out the namespace of a file read in with load-script? (https://clojuredocs.org/clojure.main/load-script)

lfn322:08:44

I've tried using (all-ns) before and after, but that all gets anything that's a dep of the file that's been read. (afaict)