A question for the channel: What's the easiest way to call a side-effecting function only when a particular atom's value changes? I put one alternative on SO, but I think there must be a simpler way: http://stackoverflow.com/questions/35544113/call-a-side-effecting-function-only-when-atom-value-changes

Atoms must be free of side effects

In general Clojure's reference types might not be a good idea if you need side effects

If you could explain your use case, maybe we could come up with something that would be more Clojure-ish

I'm receiving messages from outside the process and need to sometimes update in memory state depending on the message. When I do, I then need to send a message on the network (side effect).

It would be an error to send the message when no update was actually performed, or to send it more than once when a single update was performed.

I like the answer by Timothy

hey @alexisgallagher for that use case, perhaps consider using an agent instead of an atom or ref. On message receipt, send-off a function that consumes the message, updates agent state, and determines whether to announce a change to other consumers. Agents ensure messages will be processed serially, saving you the trouble of managing subtle "did it change" semantics in a concurrent context.

@jonahbenton: agents are asynchronous. I need synchronous. I need code after the change to see the new state value immediately not for that new state value to be computed and realized at some point later. Or am I misunderstanding agents?

@alexisgallagher: certainly- the point was really just about architecture. questions around "previous state" and "did something change" come up often with atoms and refs and are not just tricky because of the seeming omission that functions that operate on refs and atoms don't return the previous value- that's the case because defining "previous value" can itself be complicated and expensive to determine in a concurrent environment. the fact that the previous value isn't returned is a signal that that atoms + refs may not be the right tool to solve that problem. if it's most important to know if a change occurred, the suggestion was to consider imposing a total order, e.g. by putting all messages on a queue (which is what happens behind the scenes with agents). then you know when a change occurred, and you save yourself some of the cost of the concurrency machinery, with the tradeoff of some potential latency/asynchrony letting another thread service the queue.

"the fact that the previous value isn't returned is a signal that that atoms + refs may not be the right tool to solve that problem" hmm, interesting. I'll have to think on that one...

Grimoire is down for a bit while I set up SSL, sorry for the interruption.

And we're back! g'night.

I just open sourced my action rpg game that I wrote a few years ago : https://github.com/damn/cdq

Try it out !

The one with buffer size 0 fails with (not (= 42 41))

@jarppe: That's intentional, you need a buffer when using transducers.

That's been been discussed a few times over at the core-async slack channel

http://dev.clojure.org/jira/browse/ASYNC-140

I noticed that chan has this assert: (when xform (assert buf-or-n "buffer must be supplied when transducer is"))

that should propably fail also if buf-or-n is zero then?

ah, thanks for the link, it seems that there is a ticket for that in here http://dev.clojure.org/jira/browse/ASYNC-143

A question for the channel, if you don't mind: I am creating a series of namespaces which will offer alternate implementations of a protocol. Is there any way I could replicate the tests among all of them? At the end of the day, the results should be equivalent. Ability to easily add future implementations would be good as well. Thanks in advance!

hi @cristobal.garcia were you planning on using clojure.test?

hi jonahbenton, yes

and you probably have constructor-like functions in your namespaces to allow a user to specify which implementation they wish to use?

@cristobal.garcia: you can see an example of such testing setup for a rather demanding use case (protocols + native libs + GPU computing) at http://github.com/uncomplicate/neanderthal

@jonahbenton, Thanks. I was planning to use a default implementation and the ability to select non-default ones with bindings. The non-default ones would be useful for mocking. Constructor-like functions will be there, yes.

@blueberry: I am having a look right now, thanks.

@blueberry: exactly what I was looking for, thanks . It might be good to switch to midje.

does anyone have the opinion that using let is a code smell? I've heard someone say it was but didn't understand their rationale.

let is certainly not a code smell

if it's 900 lines long, then maybe

@fenton: do you have an example, or this is more of a philosophical question?

I find code smells are usually not caused by Clojure forms of built in functions but rather by the wrong usage

@tolitius: both i can demo an example....gimme a sec.

@tolitius: https://gist.github.com/ftravers/ffbdda0d982b6212d0a2

not sure if i should aim to not have the temp-uri state somehow...

no problem with let in that snippet

there is however a problem in factoring

the uri is clearly an argument

Rather than generating temp-uri in the body, make a third function that sets up the arguments.

I made a comment on the gist

with an argument the main function is now somewhat reusable

@ghadi: ok. i'm using this snip to test out my *.edn files to load into datomic...just learing how to use datomic...so the URI really is temporary...but i do return a con, which can be used to connect to the db... but i understand your point that this isn't a reusable function

@ghadi: thanks for a having a look tho.

(np. i realize it's scratch code)

just reading a reddit on this question, and people suggest that let is sugar over a lambda.

others say the added readability that let documents the code is better than the snarl of function calls.

that is how let is implemented in some languages, but not in clojure

but yes in lambda calculus an immediately applied one-arg function is similar to a let

@ghadi: ok

@ghadi: thanks for the clarification!

hi all

do you know if jetty supports http pipelining?

according to google it does

dang I'm having problems with core.async then

doing a sample http server but requests are "sequential"

If I timeout on a go block it shouldnt block the curr thread right?

depends what you mean

I mean this: https://gist.github.com/mping/98bb8eb9faf3c51f9889

the timeout function from core.async returns a channel, which on its own does nothing

I'm trying to replicate an experiment my colleague did on nodejs

but my reqs are sequential

I would hope two immediate prints with "accpt" but they have a delay of 2 seconds

with nodejs I can pipeline two reqs and the "accpt" print is immediate

my guess would be either some kind of issue with the jetty async adapter, or some kind of issue with your client

oh, I take that back

of course, if you are pipelining you are not making sequential requests

err, concurrent requests

pipelined requests, are one after another on the same tcp socket

yes, but what should I do to make the "accept" immediate?

I should not need to wait for the 1st request to finish

the http requests are in order on the tcp socket, so you have to process them, at least to some degree, in order

no biggie with that, I hope that jetty takes care of it

http://www-archive.mozilla.org/projects/netlib/http/pipelining-faq.html

I just dont know why the 2nd accept "waits" for the first

because of pipelining you have a a single stream of requests and a single stream of responses, so in order to get to the "next" request in the stream, the first needs to be handled

Thats true, but i would expect not needing to close it,and having the ability of out-of-order writes. Só you are saying that until i respond to the first, the second is not handled right?

Maybe its jetty's fault :p

in order for responses to be sent back in order, either jetty has to buffer responses until the "previous" response are available, or process everything one at a time

like a pipeline

node is slightly diff because it has an internal queue

yep, Jetty is reusing the same HttpInput for the request body

assuming you posted this email to the mailing list about the same issue, if you look at the blog post you linked, it discusses the drawbacks of the buffering approach that nodejs takes

buffering the results uses more memory and is a possible ddos vector

jetty is entirely capable of handling actually concurrent http requests too, I think its default threadpool is limited to 50 threads, so if you want concurrently handled requests, you can just make them instead of trying to rely on how the server implements pipelining

speaking of jetty, has anyone ever found an easy way to get jetty to use HTTPS with client certificate authentication? Every time I try I bang my head into the wall for about an hour and then go back to nginx as a reverse proxy.

you need a newish version (or at least newish several years ago) it exposes an option you need to set

I can see two options on the jetty wrapper for a keystore and truststore. But the process of turning plain old certs into whatever file or object can be used to populate those seems quite painful, even with a helper library like https://github.com/aphyr/less-awful-ssl

oh, actually, I was using a custom version of the ring jetty adapter that exposed some more stuff, not sure if the vanilla one does it

it is painful

I find openssl hard enough to use. Trying to learn keytool as well -- too painful.

I wish there were some command line tool, like file, but which was super smart about the eight kajillion types of cryptographic assets one has to handle. Is this a key? a cert? In PEM? In DER? In PKCS12? PKCS8? armored or unarmored? It's very hard to keep straight unless you do it routinely.

I keep that stuff in gists

I wonder how many security breaches around the world every year are fundamentally results of how hard it is to use openssl? 1000? 10,000?

openssl is hard to use, but it can actually be used, it may take longer than it should, but it can be done. I suspect more breaches are the result of that kind of stuff getting shuffled to the bottom of the backlog

@hiredman tks I posted it. Im actually not going to use this stuff in prod, just wanted to see if I can implement the same behaviour

And ofc the devil is in the details

I also suspect pipelining is not worth it in terms of performance

the way http2 is multiplexed over a connection (no surprise) seems like it would avoid the issues with http 1.1 pipelining