A reducer is a function like (a -> r -> r). Technically you also have it’s “default” form, but we can handle that in the notation by just passing a default r value along too

so

a reducer is (a -> r -> r, r)

that’s exactly what you pass to reduce, too

a transducer is a function between reducers

(a -> r -> r) -> (a -> r -> r)

technically you can change the type of the a if you like (and more things)

so you might write Transducer a b = (b -> r -> r) -> (a -> r -> r) where I’ve written it to demonstrate the “backwardness” of it

identity is a function with a type like x -> x for any choice of x

so it can definitely be a transducer

a Transducer a a

by letting x be filled in as (a -> r -> r)

I worked some of this stuff out a while back

https://tel.github.io/posts/typing-transducers/

very Haskell-y which makes various things harder to do

but the idea is very clear if you know how to read the types

the clever bit if you want to skip much of the haskell

is that a transducer is essentially “as powerful as” mapcat

a “pure” function of the form (b -> r -> r) -> (a -> r -> r) is equivalent to a -> [b]

but transducers do two or three tricks extra

they let you intermingle effects, they compose functions and therefore avoid construction of intermediate data structures, and they wedged an early termination effect inside

Howdy. How do I realize a LazySeq returned to me by Korma? doall didn't appear to do it, so I simply used (list thing). What's the 'right' way?

According to the docs, in-ns doesn't include clojure.core, so a lot of basic functions will not be available. I'm curious why that's the case, but at least it's documented. There's something that confused me a bit more, though: I have a test namespace:

(ns foo.test.db.core
  :require [clojure.test :refer :all]) ; various requires omitted

Whoops, just figured it out. I'll leave this question up in case it ever helps some other duffer. The answer is that I was in-ns-ing to a namespace that had not been loaded yet! That is, since none of its code had been exercised as part of the standard REPL startup, it didn't "exist." It is necessary to execute the entire contents of the namespace, actually. 😦

I need to have a jar file somewhere predictable when I deploy, because I need to pass it as a command line. Is it possible to copy a jar from dependencies to the uberjar when it’s being built?

@pupeno: you could put that jar into resources as part of your build process

@martinklepsch: that sounds like a good approach, any ideas what to search for?

Not really tbh

you could always write some shell scripts or manually copy it

I assume you’re using Leiningen?

That’s still not trivial to do. I don’t know what’s the appropriate way to invoke shell scripts either and I don’t know how to write a shell script that can run on both unix and windows and reach out to the maven repositories, wherever they happen to be on the computer and copy the files.

Yes, I’m using Leiningen.

I’m looking at this https://github.com/technomancy/leiningen/blob/master/sample.project.clj

But maybe I need some plug in.

@pupeno: seems like https://github.com/m0smith/lein-resource might be of help...

Maybe you could download it during startup using something like https://github.com/cemerick/pomegranate

@nberger: indeed, that looks like what I’m looking for, I don’t know how to refer to my maven repo and deal with version numbers.

yeah, I don't know how to do that either... have you exhausted all the other options and are you really sure you need to do the manual copy?

nberger: I don’t know, I rather not do the manual copying, but my other options didn’t work: http://stackoverflow.com/questions/32350244/leiningen-java-agents-not-working-in-uberjar

@nberger: and lein-resources is not working either: https://github.com/m0smith/lein-resource/issues/15

@pupeno for the reference, Yeller has a few jars like this, and I just throw them in the filesystem in my deploy tool (which grabs em from s3)

on heroku you might be able to do something similar with a custom buildpack

(specifically: you want the multi buildpack, combining the default clojure buildpack and another custom buildpack that downloads the new relic jar)

tcrayford: I generally prefer more self-contained solutions. I’d like my app to be more portable. My biggest problem with having the jar in a build-pack is that lein ancient won’t upgrade that jar.

yep

as far as I know it's a limitation of the jvm that you can't uberjar an agent

That jar will be stuck in that version until someone happens to remember to check for an upgraded version.

but that's a guess 😉

tcrayford: oh, but I already did it. I put the jar in my source control and the passed -javaagent:newrelic/newrelic.jar and it worked. I don’t want the jar in my source control for the same reasons, it’s out of the build system and thus won’t get updated.

right - I meant "uberjar in the same jar as your app" 😉

I just need something to automatically copy it from $MAVEN_REPO/om/newrelic/agent/java/newrelic-agent/$LATEST_VERSION/newrelic-agent-$LATEST_VERSION.jar to $SOME_DIR/newrelic-agent.jar at uberjar build time.

yeah, so a buildpack or a lein plugin could do that for sure (pretty sure java can copy some files on the filesystem), but you'd have to write either of those yourself I think

I find it surprising that New Relic itself advocates having a jar on the source tree. In Ruby/Rails it’s a gem that gets updated with all the other gems, it’s part for the checks for security vulnerabilities, etc.

it's a java agent limitation thing - agents have to be seperate jars from the app

tcrayford: lein plug in seems to be the way to go. https://github.com/m0smith/lein-resource looked promising, but it’s not working.

(and if they didn't write it as an agent it couldn't get at certain information because of the jvm's security model)

fork it and fixit 😉

@tcrayford: of course. Two things are stopping me: I don’t understand the error yet (when the plug in is present, my code fails to compile) and I’m not sure it’s actually a viable solution as it’s not clear whether it can copy files from the maven repository.

Oh, Heroku is not picking it up from the uberjar, it’s picking it up from a copy of the source code 😞

Hey clojurians, anyone managed to get a web service running with ring that accepts gzipped request bodies? Wondering if it only works in our Dropwizard services because they implemented it themselves... (http://stackoverflow.com/questions/31555633/jetty-responds-with-400-bad-request-when-request-content-type-is-gzip)

Sorry I'm not following everything but perhaps :scope "provided" would help in the sense that it won't be able to "pick it from the source code"?

That's for @pupeno

nberger: in Heroku, in my app, I can see the source code of my app with a target directory with uberjar. When the jar is my source control, it’s naturally on my Heroku app as well, but I’m trying to avoid that.

nberger: does that make sense?

I almost got my jar-copying plug in.

any reason why you can’t pull the jar from maven?

Or push the jar to your own registry using something like artifactory or nexus?

Does this Clojure datatype flowchart still hold true? https://chasemerick.files.wordpress.com/2011/07/choosingtypeforms2.png

roberto: I could download it from maven, but maven already downloaded it to the local repo, why should I download it again and figure out how to talk to maven? About the second question, I’m not sure I follow, do you mean my own maven repository?

Behold! the jar-copier: https://github.com/carouselapps/jar-copier

yeah

http://www.jfrog.com/artifactory/

hmmm, I don’t think I understand what the issue is, if you are already using maven to download the jar

why are you copying it around?

roberto: I don’t want to have my own repository, I want to pick the jar from the central maven repository. One of the reasons why I’m doing all of this is to make sure I’m picking the latest jar, so that lein ancient upgrades it. Besides, having my own repository still leaves me figuring out everything else (doesn’t solve the problem of having the jar in a well known location at runtime)

roberto: I need the jar to be present in my production server in a well known location so I can pass -javaagent:path-to-the-library.jar so that the JVM knows where it is to pick it up and load it as an agent.

oh, ok

This code feels horrible: https://github.com/carouselapps/jar-copier/blob/master/src/leiningen/jar_copier.clj#L20-L28 It essentially checks that :jar-copier {:java-agents: true, :destination “blah”} exists in the project.clj. Any advice on how to make it more idiomatic, easier to read, etc?

I'm doing lein release for this library that I just wrote: https://github.com/carouselapps/jar-copier/blob/master/project.clj and I'm getting the error gpg: skipped "J. Pablo Fernández <[email protected]>": secret key not available, which is confusing, because that’s not the email address in my key which I specified in project.clj: http://keys.gnupg.net/pks/lookup?op=vindex&amp;search=pupeno%40carouselapps.com. Any ideas what’s going on?

suggestions how to generate PDFs in clojure from html, similar to https://github.com/mileszs/wicked_pdf ?

I was thinking of calling http://wkhtmltopdf.org/ myself

unless someone has a nicer solution

@pupeno: have you uploaded your key to clojars?

ragge: yes.

It’s failing well beyond that, as far as I know. It’s not finding the private key when signing because it’s looking for one that doesn’t exist.

@borkdude have you tried https://github.com/yogthos/clj-pdf?

@robert-stuttaford: that doesn't really use html. I need to render the same html/css as in the browser (sort of) but to pdf. I'm now trying https://github.com/gberenfield/ring-wicked-pdf

@borkdude I think some people also used enlive with clj-pdf

@yogthos: what is the advantage of this library compared to others?

I find using hiccup style syntax works really well for describing PDF docs

@yogthos: I can also write hiccup and render to pdf using html. Does your approach offer anything else that I'm missing?

@yogthos: I'll try and see how far I get with both libs

clj-pdf is explicit about what gets generated, but you can just look at the examples in the readme to see if it's a good fit or not :-)

@borkdude: I’ve used wkhtmltopdf by simply calling the executable. Has worked well so far.

@nathan.schile: thanks

I find myself writing this a lot:

(if (vector? y)
  (into [] (map #(assoc {} :x %) y))
  (assoc {} :x y))

Basically I want the apply the same operation to each element in y if y is a collection as to y itself if not.. Anyone got a better idea than using if?

(into base-map xform source-collection) is the idiomatic way to apply a transducer to a source-collection and, assuming that the transducer yields map entries, yields a new map based on the given base-map. This will overwrite any vals in base-map that share keys with any entries that the transducer outputs. But what if I want to merge vals in base-map with those in the entries that the transducer outputs, rather then overwriting them? I could write a custom reducing function (one that uses transients, like into does) and use transduce, but I suspect that there’s an easier, more idiomatic way.

@jstaffans: I haven't used it , but I wonder if Spectre would help. Can't wait to use it myself

@cigitia: maybe producing a map from transducer and then using merge-with?

but writing custom reducing function would be probably better optimized

@jstaffans: why don’t you make a helper function out of that snippet?

#(assoc {} :x %) can be an op parameter

@jstaffans: Looks like array programming — https://en.wikipedia.org/wiki/Array_programming

That may give you some ideas for abstracting it. @darwin's suggestion is good.

I could use some advice on converting from clj-time to SQL timestamps when using YeSQL. It's easy to convert to the timestamp format:

(let [timestamp (clj-time.core/now)]
  (db/create-user! {<other data>, :created_at (clj-time.coerce/to-sql-time timestamp)}))

@cigitia: something like? (merge base-map (into {} xform source-collection))

but you want it to merge during the transducing, not as a separate step

Yeah.

It’s fine; I wrote my own reducing function. Thanks for the help!

wait, you want to merge values when there are duplicate keys, right?

Yes, that’s right.

yeah, seems like a custom reducing function is the way to go

since what it means to "merge" the duplicate values needs to be determined somehow

merge-with?

I was thinking that update might be useful https://clojuredocs.org/clojure.core/update

Then you just supply update with a custom function that knows how to do the merging.

And since update was added to 1.7 that might be a good sign that it's useful as the reducing function in tranducers

@cigitia: Have you looked at update?

@exupero: I believe that update is the reducing function form that corresponds to merge-with which operates on the entire map.

But now I'm not so sure...

Just when you think you've got things all figured out... 😞

This really shouldn't be that hard to figure out since this is clearly the bailiwick of transducers.

I'm so used to just relying on conj as the reducing function.

Now I believe I'm wrong, and update is merely filling in the gap of needing a single-level version of update-in

But the question remains: how to get update/update-in functionality within a tranducable process without having to write a custom reducing function?

Which is what merge-with would do, and I'm just being stupid, right?

I just hope there is some entertainment value in all the stupid things I say or do...

where should one look if you get an error 500 from a ring application, but there is no exception printed in the console, nor do I get it back in html?

@borkdude, maybe this? https://github.com/nberger/ring-logger

I'll try it, thanks @akiva

I still haven't cracked this nut. Anyone got a better idea? This didn't work:

(transduce identity (partial merge-with +) {:a 1 :b 2} {:b 3 :c 4})
ClassCastException clojure.lang.Keyword cannot be cast to java.util.Map$Entry  clojure.core/key (core.clj:1496)

I found the exception by adding ring.middleware.stacktrace to the mix.

in lib-noir I got that one for free I think, never thought about it

borkdude: I use timbre logging to print to the console, stacktraces or any other messages I want printed

@ecelis: ring.middleware.stacktrace does that too

maybe I should take a look at that too

well, it's better to turn off ring.middleware.stacktrace in production, so your solution is probably better

ah, you can choose between log and web: https://ring-clojure.github.io/ring/ring.middleware.stacktrace.html

@meow: how about (transduce identity (partial merge-with +) {:a 1 :b 2} [{:b 3 :c 4}])

Recall that reduce and transduce take a list of elements to reduce onto the starting value

@aengelberg: aha! thank you!

(transduce identity conj {:a 1 :b 2} [{:b 3 :c 4}])
=> {:a 1, :b 3, :c 4}
(transduce identity (partial merge-with +) {:a 1 :b 2} [{:b 3 :c 4}])
=> {:a 1, :b 5, :c 4}

no prob

What threw me was this worked:

(transduce identity conj {:a 1 :b 2} {:b 3 :c 4})
=> {:a 1, :b 3, :c 4}

More fun with transducers. They keep getting easier.

One of the reasons I don't see why conj can operate on tuples or maps.

Rather, conj can operate on maps and tuples, or maps and maps.

i regularily ask myself that, find the answer and forget

all i remember is that there is a good reason for that

is it really the case that some and every?

?

or is there something I’m missing

tel: I didn’t follow your question

some and every? seem to be parallel functions

despite the difference in naming convention

whereas some? is different?

also some-fn and every-pred

some? is more recent

they are different

are you wondering about the question mark?

some stops as soon as one value is true

yeah

every? if all of them

and just the general naming convention

@tel some returns a value, every? returns a boolean

ah

that’s the logic

Looking for the best clojure finite state machine library, any advice?

Is anyone here using RabbitMQ with Clojure? Any library recommendations (or warmings)?