Fork me on GitHub

> Likewise, bindings created with binding can be assigned to, which provides a means for a nested context to communicate with code before it on the call stack.


can anyone produce an example where you would want to do this?


side channels for extra information


so you wouldn’t have any data escape the top-level binding


it’s a thread-bound stack, so your changes wouldn’t escape upwards


i guess i don’t see why you wouldn’t just return the “extra information” with your regular information


but perhaps that is just a lisp idiom?


one example where I used it was a multiplexing codec implementation


it reads and writes headers to the byte data to label the codec to use


@alexstokes that's the clean way to do it, but it also requires changing every function in between to carry that contextual data


but the encode/decode functions are value-focused - you get back the encoded bytes or the decoded value


instead, I had the codec implementations record the headers written or read to a dynamic var on the side


and sometimes you are not at liberty to change that data


@greg316 link to this code?


so the consumer could bind the *headers* var and check at the end of the operation


sure, one sec


I have since rewritten it and gotten rid of that bit, but it did the job for a while 🙂


@alexstokes especially consider the situation where you might have multiple threads doing the operation, so it isn't safe to use a top level var either - and code that has to be "in the middle" can't pass the value in or out


it is definitely not the simplest way to do things, but sometimes it’s the only option you have


I think in every option where you do that you could also use an instance of an object that is contextually shared by multiple higher order functions - but that's much messier


eg. you'd have a codec-context-factory that creates a codec-context, and it would give you a decoder function and a driver function that you'd use together - I'm sure you can imagine how clumsy that would be to use


if that architecture makes sense to you you're probably happily using spring with no use for clojure 😄


Anyone know what might be causing this? Basically, there's a nested dependency in this codax library for an old version of encore. Using lein's :managed-dependencies for a newer version of encore or using :exlcusions on the codax dependency and manually a newer version of nippy doesn't work. The only luck I have is by cloning the codax repo and changing it to use a newer version of nippy which no longer depends on an old version of encore. Shouldn't that be the same thing as doing [com.taoensso/timbre :exclusions [com.taoensso/nippy]]?


I'm testing some kafka streams code, but the relevant part is about the reference to a function (a var). I have code like below and the problem is that when used directly at the pipeline (at .filter), this pipeline sees the any change at the function, but when used like the commented line below (at kf/filter), it doesn't

 (.stream builder new-electric-data-topic)
 (kf/map-values avro->clj)
 (.filter (reify Predicate (test [_ k v] (voltage-event? k v)))) ;;; THIS WORKS when changing the function at REPL
#_ (kf/filter voltage-event?) ;; BUT WHEN I USE THIS, I don't get the change, have to restart the stream
 (kf/peek peek-test))


The function that I'm changing is voltage-event? It's just one-line with true or false for this test


And kf/filter is below

(defn filter
  [stream f]
  (.filter stream (reify Predicate (test [_ k v] (f k v)))))


The problem could be that I return a different stream every time I call this function (because of immutability, maybe I should use a macro?)


Or a function calling your defined function doesn't retain the reference to the original var, but I doubt it


is it too insane to change all (fn [...]) to (fn some-name [...]) for the sole purpose that it'll be easier to read stack frames ?


good idea uhaeuaehueh I'll start to use this


I do that, it's great - it also acts as a brief code documentation


i do that often


I need a macro which does this:

then for any printlns executed in that region, it gets indented by 2 spaces ... this would drastically simplify prnitln debugging in making things easier to read


@pfeodrippe that happens because vars used inside a function are looked up when used, but when you pass a function by name to another function or method, it's looked up once on that invocation and reused - which is an issue for long running things


@pfeodrippe you can work around this by passing a var in instead of the function itself eg. #'voltage-event?


@qqq I think with-out-str could help you


well, pprint already takes a writer, that could be a string-writer

noisesmith13:01:01> (let [sw (] (pprint {:a 0 :b 1 :c 2 :d [1 2 3] :e (range 20)} sw) sw)
#object[ 0x232b391e "{:a 0,\n :b 1,\n :c 2,\n :d [1 2 3],\n :e (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)}\n"]> (doseq [line (clojure.string/split-lines (str *1))] (println "  " line))
   {:a 0,
    :b 1,
    :c 2,
    :d [1 2 3],
    :e (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)}


@noisesmith Understood, thanks! I've changed it to be a defmacro instead of a defn (just a few changes at the body), so the user doesn't need to pass a var every time, but now I know if I need it again 😃


@moxaj: hmm, so grab out-str, parse the \n, and manuall insert the indendataion


yes, that's the idea


with-out-str does more than you need though, since pprint already takes a writer arg


@noisesmith but I'd guess he wouldn't want to modify the code, just sprinkle in some indent-println-by-2


oh you wanted println not pprint? never mind


@noisesmith, @moxaj: right, what moxaj said, I dont want to modify underlying code


then you need to mess with *out*


more than just println, anything that dumps out things to the console


basically, I want to specify an int, and after every \n or \r, have it insert 'n' spaces


i can't believe no one else has run into this issue / solved it already


OK, then you need with-out-str I guess - be careful, this will totally break with lazy processing


you need to force all laziness that prints before exiting that block, or else the indentation won't apply


but you already signed up for pain if you use IO in laziness


and I think that's why nobody has "resolved this issue" - laziness makes it very messy and we use lazy things a lot


you're saying, it's possible that (foo) runs fine, but it returns a thunk, which when executed, throws an exception / does println


I'm saying that if the lazy tail of foo prints, it won't use the indentation you wrapped around it's construction


unless forced inside that scope


and, if some other lazy tail is forced in that scope, it uses that scope's indentation


which might be fine for your usage, but would get a lot of complaints and broken results if provided as a general indented printer facility


I want to drive an interactive program from Clojure; anyone familiar with code for doing that, or should I be just generally looking at transducers to interact asynchronously with InputStreams and OutputStreams?


I'm not sure that transducers make sense, but what you're describing is essentially REPL (just IS/OS not Reader/Writer)


Nope, I'm talking about something that will start an instance of bash, ssh into another machine, and run commands there 🙂


I wanna have something vaguely like but no tcl/tk


hugo duncan did a whole lot of work around this 4-5 years ago


had a clojure-lite -> bash compiler


and used it for server management


might be some useful libs in there


nifty! thank you!


no prob. there are other very useful things in that org

Al Baker17:01:59

yeah, pallet works through ssh'ing in and running a series of shell commands to move the system through its init states


@benjaminster I've had best luck with using ProcessBuilder and Process to do streaming IO to shells


yeah, it looks like that plus some way to turn an InputStream into something more easily digested by pure functional code would be one way to go


be very careful about that transformation - for example don't use laziness it just makes things more complicated


that's what I was saying in the other message about conch - an abstraction that forces non-io code to care about io concerns is worse than no abstraction at all




I tried using conch but found that it added complexity without actually simplifying my task which is matched pretty well by the built in classes (fundamentally at that layer I want to do imperative IO, lazy-seqs only add bugs at that level)


come to think of it you could make something that converts an IO channel to an IReduceInit though...


conch has some interesting ideas but is (as you pointed out) aimed at a different target...


I think I have a gist of doing this - digging


I want to have timeouts and such so... I think something that gobbles up an inputstream and bleeds it over a core.async channel might be a good funnel to get the pipeline going


We have a requirement to allow users to upload a JSON file, parse the records, and stream to kafka. I'd like to harden the process so it doesn't easily cause out-of-memory exceptions when the file is somehow goofy. Currently planning to use parsed-seq from cheshire, not clear how/if it protects against this scenario. Or am I worrying too much?


that gist is super simplistic, but I think it has most of the moving parts you need to do something interesting with streaming input / output from a task. The annoying thing is the JVM gives you no straightforward way to get the PID of your task - you get an abstraction that lets you shut it down or check if it's still running but no PID 😦


thanks! oh gosh, that's a classic Java API with misnamed methods... .getOutputStream gives you what most would call the process' stdin and .getInputStream gives you stdout


haha yeah - as far as Java is concerned it's the OutputStream because that's where java can put output haha


that's the only way it makes sense at least


it helps my mental model if I don't think of a Process object as the OS process, but rather the ambassador for one inside the jvm, so it's outputstream is the stream that sends your output to the process input - that's a better fit for what's going on on a low level anyway


also don't forget to join the error stream with the input stream, or set up handlers for both


but any pure JVM solution is going to have that problem (unless it was fixed in 1.9 ?)


@dave.dixon I would put some kind of cap on the size of the json file, and setup cheshire to not turn keys in to keywords, I doubt parsed-seq will do anything there


@hiredman Thanks. Theoretically the amount of data we need to accept could be large. I suppose I could cap it and make the user upload multiple files, though it complicates the application a bit.


the problem with json is it is a context free language, ideally you would want some kind of regular language


it is possible to get an actual pid from Java with 1.9


awsome! great to hear


fun fact: years ago, the way the jvm started new processes on linux was implemented in such a way as to require (at least for an initial instant) the same amount of memory for the child process as the jvm required, which meant if you had a jvm with a 7 gig heap on a machine with 8 gigs of memory (and no swap) it would fail if you tried to System/exec


wow - yeah the naiive way to spawn a process under linux would do that if you didn't have overcommit enabled


Hi all, Does anyone know how to run a function after all tests are complete? Sort of like a fixture around all tests?


the simplest thing would be to make your own function invoking clojure.test (that wouldn't help if you are using eg. lein test though)


I would like to be able to use lein test


Would creating my own test runner be an option?


I'm not sure - I guess worst case you fork lein test as your own plugin


Yeah, I can look into that. The actual use case is that: my tests start a selenium browser and the tests go much slower if I have to restart the browser between namespaces. I'll look into it thanks


@johnnyillinois you could use an alias in your project, something like this (using lein-shell):

:aliases {"test" ["do" ["shell" "start-browser-selenium"] ["test" ":selenium"] ["shell" "stop-browser-selenium"]]}


oh yeah - that's a good call


hey, greg can you help me out


there's also :prep-tasks (or is that only for :uberjar ?)


the same process that runs the tests, needs to be able to access the selenium browser


that’s meant for builds


With the clj and clojure CLI tools, what’s the practical purpose for having both -R and -C? When would I want them to be different?


Doesn't that command have the selenium browser running if in a different process?


Why do you need it to run in the same process? Wouldn’t the browser always be in a different process anyway?


I understand the usefulness of having separate resolve-deps and make-classpath functions in the code, but I’m not clear on why they should be exposed to the user.


When playing around with clj and deps.edn I had the same feeling.


boot-tools-deps ( provides -A to specify a list of aliases to provide to both.


@moxaj thanks for the link! Voted!


That's a good point @grzm. I guess I'm using: so I can look into how to configure it to use an already running driver


@johnnyillinois I’m not sure I follow. I kinda jumped into the middle of your conversation with a completely unrelated topic. Apologies if that caused some confusion. Looks like you’re discussing Leiningen config?


Yeah, don't worry about thanks grzm


I’m looking to sample from a probability distribution, i.e. either take n from a vector of probabilities like [0.2 0.4 0.3 0.1] (where I’d get 1 40% of the time) or do the same for a map like {:a 0.2 :b 04. :c 0.3 :d 0.1} — is this already implemented somewhere?


there's a random-sample function which iirc was derived for a goofy thing I did on IRC once that was accidentally useful


+justin@fuckmonkey:~/clojure/peregrine$ rlwrap bench
Clojure 1.9.0
+user=> (doc random-sample)
([prob] [prob coll])
  Returns items from coll with random probability of prob (0.0 -
  1.0).  Returns a transducer when no collection is provided.
+user=> (random-sample 0.3 (range 100))
(2 4 5 7 9 18 28 30 32 34 39 58 59 63 64 66 69 71 73 74 75 76 80 89 98)


Yeah, I came across that, but I have a pre-existing probability distribution, rather than the sample probability for each element and it didn’t look like I could specify that there.


check the code for random-sample - it's very simple and you can make something new using map (map takes multiple collection args, so the first could be probabilities and the second elements)


s/map/mapcat/ (since we are arbitrarily dropping items use mapcat)


something like (mapcat (fn [p e] (when (< (rand) p) [e])) probs elts)


Aha, cool, will do, thanks.


actually I think that code might just be it!


+user=> (defn psample [probs elts] (mapcat (fn [p e] (when (< (rand) p) [e])) probs elts))
+user=> (psample [0.0 1.0 0.0 1.0] (range))
(1 3)
looks good to me