Fork me on GitHub

A reducer is a function like (a -> r -> r). Technically you also have it’s “default” form, but we can handle that in the notation by just passing a default r value along too


a reducer is (a -> r -> r, r)


that’s exactly what you pass to reduce, too


a transducer is a function between reducers


(a -> r -> r) -> (a -> r -> r)


technically you can change the type of the a if you like (and more things)


so you might write Transducer a b = (b -> r -> r) -> (a -> r -> r) where I’ve written it to demonstrate the “backwardness” of it


identity is a function with a type like x -> x for any choice of x


so it can definitely be a transducer


a Transducer a a


by letting x be filled in as (a -> r -> r)


I worked some of this stuff out a while back


very Haskell-y which makes various things harder to do


but the idea is very clear if you know how to read the types


the clever bit if you want to skip much of the haskell


is that a transducer is essentially “as powerful as” mapcat


a “pure” function of the form (b -> r -> r) -> (a -> r -> r) is equivalent to a -> [b]


but transducers do two or three tricks extra


they let you intermingle effects, they compose functions and therefore avoid construction of intermediate data structures, and they wedged an early termination effect inside


Howdy. How do I realize a LazySeq returned to me by Korma? doall didn't appear to do it, so I simply used (list thing). What's the 'right' way?


According to the docs, in-ns doesn't include clojure.core, so a lot of basic functions will not be available. I'm curious why that's the case, but at least it's documented. There's something that confused me a bit more, though: I have a test namespace:

(ns foo.test.db.core
  :require [clojure.test :refer :all]) ; various requires omitted
...but when I use (in-ns 'foo.test.db.core) from the REPL, I can't access the clojure.test stuff either. Particularly (run-tests). If I execute the full ns form in the REPL, I get access to all the stuff. Is this expected? Or am I doing something weird?


Whoops, just figured it out. I'll leave this question up in case it ever helps some other duffer. The answer is that I was in-ns-ing to a namespace that had not been loaded yet! That is, since none of its code had been exercised as part of the standard REPL startup, it didn't "exist." It is necessary to execute the entire contents of the namespace, actually. 😦

Pablo Fernandez10:09:30

I need to have a jar file somewhere predictable when I deploy, because I need to pass it as a command line. Is it possible to copy a jar from dependencies to the uberjar when it’s being built?


@pupeno: you could put that jar into resources as part of your build process

Pablo Fernandez10:09:45

@martinklepsch: that sounds like a good approach, any ideas what to search for?


Not really tbh


you could always write some shell scripts or manually copy it


I assume you’re using Leiningen?

Pablo Fernandez11:09:22

That’s still not trivial to do. I don’t know what’s the appropriate way to invoke shell scripts either and I don’t know how to write a shell script that can run on both unix and windows and reach out to the maven repositories, wherever they happen to be on the computer and copy the files.

Pablo Fernandez11:09:26

Yes, I’m using Leiningen.

Pablo Fernandez11:09:48

But maybe I need some plug in.


Maybe you could download it during startup using something like

Pablo Fernandez11:09:51

@nberger: indeed, that looks like what I’m looking for, I don’t know how to refer to my maven repo and deal with version numbers.


yeah, I don't know how to do that either... have you exhausted all the other options and are you really sure you need to do the manual copy?

Pablo Fernandez11:09:25

nberger: I don’t know, I rather not do the manual copying, but my other options didn’t work:


@pupeno for the reference, Yeller has a few jars like this, and I just throw them in the filesystem in my deploy tool (which grabs em from s3)


on heroku you might be able to do something similar with a custom buildpack


(specifically: you want the multi buildpack, combining the default clojure buildpack and another custom buildpack that downloads the new relic jar)

Pablo Fernandez11:09:16

tcrayford: I generally prefer more self-contained solutions. I’d like my app to be more portable. My biggest problem with having the jar in a build-pack is that lein ancient won’t upgrade that jar.


as far as I know it's a limitation of the jvm that you can't uberjar an agent

Pablo Fernandez11:09:37

That jar will be stuck in that version until someone happens to remember to check for an upgraded version.


but that's a guess 😉

Pablo Fernandez11:09:36

tcrayford: oh, but I already did it. I put the jar in my source control and the passed -javaagent:newrelic/newrelic.jar and it worked. I don’t want the jar in my source control for the same reasons, it’s out of the build system and thus won’t get updated.


right - I meant "uberjar in the same jar as your app" 😉

Pablo Fernandez11:09:05

I just need something to automatically copy it from $MAVEN_REPO/om/newrelic/agent/java/newrelic-agent/$LATEST_VERSION/newrelic-agent-$LATEST_VERSION.jar to $SOME_DIR/newrelic-agent.jar at uberjar build time.


yeah, so a buildpack or a lein plugin could do that for sure (pretty sure java can copy some files on the filesystem), but you'd have to write either of those yourself I think

Pablo Fernandez11:09:07

I find it surprising that New Relic itself advocates having a jar on the source tree. In Ruby/Rails it’s a gem that gets updated with all the other gems, it’s part for the checks for security vulnerabilities, etc.


it's a java agent limitation thing - agents have to be seperate jars from the app

Pablo Fernandez11:09:40

tcrayford: lein plug in seems to be the way to go. looked promising, but it’s not working.


(and if they didn't write it as an agent it couldn't get at certain information because of the jvm's security model)


fork it and fixit 😉

Pablo Fernandez11:09:37

@tcrayford: of course. Two things are stopping me: I don’t understand the error yet (when the plug in is present, my code fails to compile) and I’m not sure it’s actually a viable solution as it’s not clear whether it can copy files from the maven repository.

Pablo Fernandez11:09:31

Oh, Heroku is not picking it up from the uberjar, it’s picking it up from a copy of the source code 😞


Hey clojurians, anyone managed to get a web service running with ring that accepts gzipped request bodies? Wondering if it only works in our Dropwizard services because they implemented it themselves... (


Sorry I'm not following everything but perhaps :scope "provided" would help in the sense that it won't be able to "pick it from the source code"?

Pablo Fernandez12:09:01

nberger: in Heroku, in my app, I can see the source code of my app with a target directory with uberjar. When the jar is my source control, it’s naturally on my Heroku app as well, but I’m trying to avoid that.

Pablo Fernandez12:09:11

nberger: does that make sense?

Pablo Fernandez13:09:32

I almost got my jar-copying plug in.


any reason why you can’t pull the jar from maven?


Or push the jar to your own registry using something like artifactory or nexus?

Pablo Fernandez13:09:39

roberto: I could download it from maven, but maven already downloaded it to the local repo, why should I download it again and figure out how to talk to maven? About the second question, I’m not sure I follow, do you mean my own maven repository?


hmmm, I don’t think I understand what the issue is, if you are already using maven to download the jar


why are you copying it around?

Pablo Fernandez13:09:32

roberto: I don’t want to have my own repository, I want to pick the jar from the central maven repository. One of the reasons why I’m doing all of this is to make sure I’m picking the latest jar, so that lein ancient upgrades it. Besides, having my own repository still leaves me figuring out everything else (doesn’t solve the problem of having the jar in a well known location at runtime)

Pablo Fernandez13:09:24

roberto: I need the jar to be present in my production server in a well known location so I can pass -javaagent:path-to-the-library.jar so that the JVM knows where it is to pick it up and load it as an agent.

Pablo Fernandez13:09:45

This code feels horrible: It essentially checks that :jar-copier {:java-agents: true, :destination “blah”} exists in the project.clj. Any advice on how to make it more idiomatic, easier to read, etc?

Pablo Fernandez14:09:23

I'm doing lein release for this library that I just wrote: and I'm getting the error gpg: skipped "J. Pablo Fernández <[email protected]>": secret key not available, which is confusing, because that’s not the email address in my key which I specified in project.clj:; Any ideas what’s going on?


suggestions how to generate PDFs in clojure from html, similar to ?


I was thinking of calling myself


unless someone has a nicer solution


@pupeno: have you uploaded your key to clojars?

Pablo Fernandez15:09:23

It’s failing well beyond that, as far as I know. It’s not finding the private key when signing because it’s looking for one that doesn’t exist.


@robert-stuttaford: that doesn't really use html. I need to render the same html/css as in the browser (sort of) but to pdf. I'm now trying


@borkdude I think some people also used enlive with clj-pdf


@yogthos: what is the advantage of this library compared to others?


I find using hiccup style syntax works really well for describing PDF docs


@yogthos: I can also write hiccup and render to pdf using html. Does your approach offer anything else that I'm missing?


@yogthos: I'll try and see how far I get with both libs


clj-pdf is explicit about what gets generated, but you can just look at the examples in the readme to see if it's a good fit or not :-)


@borkdude: I’ve used wkhtmltopdf by simply calling the executable. Has worked well so far.


I find myself writing this a lot:

(if (vector? y)
  (into [] (map #(assoc {} :x %) y))
  (assoc {} :x y))


Basically I want the apply the same operation to each element in y if y is a collection as to y itself if not.. Anyone got a better idea than using if?


(into base-map xform source-collection) is the idiomatic way to apply a transducer to a source-collection and, assuming that the transducer yields map entries, yields a new map based on the given base-map. This will overwrite any vals in base-map that share keys with any entries that the transducer outputs. But what if I want to merge vals in base-map with those in the entries that the transducer outputs, rather then overwriting them? I could write a custom reducing function (one that uses transients, like into does) and use transduce, but I suspect that there’s an easier, more idiomatic way.


@jstaffans: I haven't used it , but I wonder if Spectre would help. Can't wait to use it myself


@cigitia: maybe producing a map from transducer and then using merge-with?


but writing custom reducing function would be probably better optimized


@jstaffans: why don’t you make a helper function out of that snippet?


#(assoc {} :x %) can be an op parameter


That may give you some ideas for abstracting it. @darwin's suggestion is good.


I could use some advice on converting from clj-time to SQL timestamps when using YeSQL. It's easy to convert to the timestamp format:

(let [timestamp (clj-time.core/now)]
  (db/create-user! {<other data>, :created_at (clj-time.coerce/to-sql-time timestamp)}))
...but since YeSQL generates a get-user function, the only way I see to get the corresponding from-sql-time is to write a bunch of functions wrapping the ones automatically generated by YeSQL, which seems to kind of defeat the purpose. I have a feeling I'm missing something that would make my problem go away.


@cigitia: something like? (merge base-map (into {} xform source-collection))


but you want it to merge during the transducing, not as a separate step


It’s fine; I wrote my own reducing function. Thanks for the help!


wait, you want to merge values when there are duplicate keys, right?


Yes, that’s right.


yeah, seems like a custom reducing function is the way to go


since what it means to "merge" the duplicate values needs to be determined somehow




I was thinking that update might be useful


Then you just supply update with a custom function that knows how to do the merging.


And since update was added to 1.7 that might be a good sign that it's useful as the reducing function in tranducers


@cigitia: Have you looked at update?


@exupero: I believe that update is the reducing function form that corresponds to merge-with which operates on the entire map.


But now I'm not so sure...


Just when you think you've got things all figured out... 😞


This really shouldn't be that hard to figure out since this is clearly the bailiwick of transducers.


I'm so used to just relying on conj as the reducing function.


Now I believe I'm wrong, and update is merely filling in the gap of needing a single-level version of update-in


But the question remains: how to get update/update-in functionality within a tranducable process without having to write a custom reducing function?


Which is what merge-with would do, and I'm just being stupid, right?


I just hope there is some entertainment value in all the stupid things I say or do...


where should one look if you get an error 500 from a ring application, but there is no exception printed in the console, nor do I get it back in html?


I'll try it, thanks @akiva


I still haven't cracked this nut. Anyone got a better idea? This didn't work:

(transduce identity (partial merge-with +) {:a 1 :b 2} {:b 3 :c 4})
ClassCastException clojure.lang.Keyword cannot be cast to java.util.Map$Entry  clojure.core/key (core.clj:1496)


I found the exception by adding ring.middleware.stacktrace to the mix.


in lib-noir I got that one for free I think, never thought about it


borkdude: I use timbre logging to print to the console, stacktraces or any other messages I want printed


@ecelis: ring.middleware.stacktrace does that too


maybe I should take a look at that too


well, it's better to turn off ring.middleware.stacktrace in production, so your solution is probably better


@meow: how about (transduce identity (partial merge-with +) {:a 1 :b 2} [{:b 3 :c 4}])


Recall that reduce and transduce take a list of elements to reduce onto the starting value


@aengelberg: aha! thank you! simple_smile


(transduce identity conj {:a 1 :b 2} [{:b 3 :c 4}])
=> {:a 1, :b 3, :c 4}
(transduce identity (partial merge-with +) {:a 1 :b 2} [{:b 3 :c 4}])
=> {:a 1, :b 5, :c 4}


What threw me was this worked:

(transduce identity conj {:a 1 :b 2} {:b 3 :c 4})
=> {:a 1, :b 3, :c 4}


More fun with transducers. They keep getting easier.


One of the reasons I don't see why conj can operate on tuples or maps.


Rather, conj can operate on maps and tuples, or maps and maps.


i regularily ask myself that, find the answer and forget


all i remember is that there is a good reason for that


is it really the case that some and every?


or is there something I’m missing


tel: I didn’t follow your question


some and every? seem to be parallel functions


despite the difference in naming convention


whereas some? is different?


also some-fn and every-pred


some? is more recent


they are different


are you wondering about the question mark?


some stops as soon as one value is true


every? if all of them


and just the general naming convention


@tel some returns a value, every? returns a boolean


that’s the logic


Looking for the best clojure finite state machine library, any advice?


Is anyone here using RabbitMQ with Clojure? Any library recommendations (or warmings)?