jaredly: partition with the step arg is cool

also didnt know that

thanks! that works nicely

yeah, it comes in handy

pkobrien: yeah, makes sense

I have a problem that I cannot solve

I try to make a very simple accounting system

Now I try to take care that when a account increased with a amount , another account is decreased with the same amount

I thought I could do this with a map like this [ { :account cash withdraw 100} {:account bank desposit 100} } ]

and use a dosync to update both accounts at once

but now I wonder how can I descruct both maps in one time so the variable account1 will be cash here. amount1 will be 100. account2 will be bank and amount2 will also be 100

at this moment there are only 2 accounts but later on the list could contains 2 to maybe 10 entries

@roelof: You can do lots of computation inside of a dosync, if you wish. Looping over 2 to 10 entries describing a desired transaction isn't a lot of computation.

@roelof: No need to destructure every transaction in the sequence in a single expression. Loop through them and destructure each one, one at a time.

you could also just put the whole system in a single atom

@andy.fingerhut: so use a map for the looping

how can I then check the best if all entries together will be zero

@roelof: What you called a 'map' above is a vector of maps. Loop through the vector and process each map one at a time.

oke, but first I have to check if the whole vector amounts together will be zero

zo I need a filter and a map for that I think

zo = so

@roelof: You can check whether the transaction has 'total withdraws equals total deposits' before even starting the dosync, and avoid starting the dosync if that is not true.

oke, I think I need a filter and some maps for it

and maybe some variables holding some values

enought to test and experiment with this

so lot's of fun

@roelof: If v is equal to the vector of maps you gave above, try out (filter #(contains? % 'withdraw) v) in the REPL

and similarly replacing 'withdraw with 'deposit

@andy.fingerhut: I know that one and then I need to use I think sum to get the total of withdraw

the same for deposit

and the difference between those two must be zero

I hope only that clojure not thinmy numbers are floats otherwise I will not work

when the amounts are 11.50 for example

oke, then the outcome will be 0.0

@andy.fingerhut: thanks for the tips as I said enough to experiment with today and maybe tomorrow

Hmm, I see this output : => ({:account #object[clojure.lang.Ref 0x627f264f {:status :ready, :val {:amount 0}}], :withdraw 100}) I miss the name of the ref

How can I take care of this?

I get the output of this function : (filter #(contains? % :withdraw) v))

@roelof: you have to name your ref somehow, just like you have to name other values - (def something 12) or (let [something 12] ...) depending on what you want to achieve.

If that's your global state then you probably want to def it

can someone spot whats wrong with this? (defn thrower-catcher [f] (try (f) (catch ArithmeticException a "div b z") (finallY (print "done"))))

oops typo

pls ignore

Morning all, I have some pipe delimited files sitting in HDFS and I'd like to use Hadoop (ideally with Clojure) to load them into another system with via a http api. Any suggestions on the best approach?

oke, so I can make a let for the find and one for the deref ?

I'm not sure I understand the question, to be honest.

@jaen: do said earlier to me that I can use a let statement so :account has a value not a reference

right ?

ah what you mean is you want something like

(def account1 (ref ...))
(def account2 (ref ...))

(dosync
  (let [current-state-of-account1 @account1
        current-state-of-account2 @account2
        new-value1 (some computation)
        new-value2 (some computation)]
  (reset! account1 new-value1)
  (reset! account2 new-value2))

Or do you mean something else?

oke, current state is the "old" value ?

@jaen that is the next step

current-state is the current value

Generally

@something means "get the current value of something", when something is reference type.

So you just make a snapshot of how it looks at the moment you @ it.

Im at the point that I have this [ {:account cash :withdraw 100 } {account :deposit 100} ]

and I try to count all the items which has :withdraw in it

so I can check if all the withdraws minus the deposits are zero

Hmm

I suggest not having account inside the map, because when you iterate over the transactions, if you deref accounts for each map it's value can change in the mean time.

I think it would be better to structure your transactions like so

so I thought that filter bring me all the withdraws and later all the deposit and I could use a map to make the total of the withdraws or deposit

Yes, you could filter it out

Or just do it in one step with reduce, like so

(reduce (fn [balances {:keys [account withdraw deposit]}]
          (cond 
            withdraw (update balances account #(- (or % 0) withdraw))
            deposit  (update balances account #(+ (or % 0) deposit))))
        {} transactions)

You will then end up with something like

{account1 -100
 account2 100}

And in a transaction you could then do

(dosync
  (doseq [[account balance-change] balance-changes]    
    (swap! account (fn [account]
                     (let [new-balance (+ (:balance @account) balance-change)]
                       (if (< new-balance 0)
                         new-balance
                         (throw (Exception. "Insufficient funds"))))))))

I think

oke, and then i could use the outcome of the reduce to check if all the numbers added to each other will be zero

if that is not zero then the user has made a error and nothing has to happens

I will play and experiment with it

Yeah, that's what I did in the last snippet

At least if I understand what you want to do correctly

oke, then I misunderstood the code at first sight

one last question : are all the functions now pure ?

@roelof: give me a second, I've checked the functions in repl just now and I see I made some mistakes, let me fix that first.

(dosync
  (doseq [[account balance-change] (balance-changes transactions)]
    (alter account (fn [account]
                     (let [new-balance (+ (:balance account) balance-change)]
                       (if (>= new-balance 0)
                         (assoc account :balance new-balance)
                         (throw (Exception. "Insufficient funds"))))))))

This is now correct

I use mostly atoms so I forgot refs use alter not swap!

(and had the logic subtly wrong)

oke, but if atoms are easier, I can rewrite it to use atoms

Nah, refs are appropriate here

for me this is at this moment learning

You need to coordinate different things in a transaction

Only refs can do that.

and hopefully learn and experiment enough to make a real project later

Do I have to rewrite my calling vector with this code

this code I will need later in that project

If you wanted to change into atoms all accounts would have to be in one map, like so (atom {:account1 {:balance 100} :account2 {:balance 100}}). I think that would be more complicated for you right now, with having to use update-in and whatnot. Refs are probably simpler.

I don't know what a "calling vector" is, sorry.

I mean this with it [ { :account cash :deposit 100} { :account bank :withdraw 100}] with the calling vector

Here, this is complete, working code I just tested in repl:

(def account1 (ref {:balance 100}))
(def account2 (ref {:balance 100}))

(defn balance-changes [transactions]
  (reduce (fn [balances {:keys [account withdraw deposit]}]
            (cond 
              withdraw (update balances account #(- (or % 0) withdraw))
              deposit  (update balances account #(+ (or % 0) deposit))))
          {} transactions))

(defn run-transactions! [transactions]
  (dosync
    (doseq [[account balance-change] (balance-changes transactions)]
      (alter account (fn [account]
                       (let [new-balance (+ (:balance account) balance-change)]
                         (if (>= new-balance 0)
                           (assoc account :balance new-balance)
                           (throw (Exception. "Insufficient funds")))))))))

And then you can just call it like so (run-transactions! [{:account account1 :withdraw 100} {:account account2 :deposit 100}])

And it should work properly.

oke, thanks

@jaen: I see this as output : CompilerException java.lang.Exception: Insufficint funds

with looks no good to me

I dunno, that looks good to me

It just means you have insufficient funds

Check it

@account1

what does it say.

Found it. I think a old value was around. When I send first the def for both refs everything works well

That happens ; d

Now when you try to run-transactions! again

You should get the exception again

Which means it's working as it should

Not letting you go under 0

but I found another bug. When I do this one '(run-transactions! [{:account account1 :withdraw 100} {:account account2 :deposit 80}])

it schould also fail because 100 is not equal to 80

that is what I meant with all the withdraws minus all the deposit need to be zero

I will look myself how to solve this one

Yeah, you'd have to modify the code then

But I think a better solution

Is not to change this code

But write functions

That create proper deposits for the user

And make sure that is correct there

For example

(make-transfer-transaction [from to value]) should return [{:account from :withdraw value} {:account to :deposit value}]

That way you can make sure it is always correct

And don't have to complicate balance-changes or run-transactions! too much

Thanks, I will look at that

This should be a fairly simple, two line function and would solve your problem

@jaen: thanks

has anyone ever tried compojure-api/swagger with http-kit async channels?

The coercion mechanism doesn’t like it for obvious reasons, but not sure how to turn it off 😛

it attempt to coerce the channel to a json object, which obviously fails

@joelkuiper: Compojure-api default serializable? implementation should only serialize the response body if your endpoint has :return metadata or the body is a collection

So if you don't set :return metadata, I think channel as response should work

hmm doesn’t seem to be the case this is what I have now

(context*
   "/api" []

   (POST* "/annotateDocuments" req
          :components [matcher]
          :coercion (constantly nil)
          :middlewares [wrap-multipart-params]
          :swagger {:operationId "annotateDocuments"
                    :description "Provided a list of PDF files, will run the list of matched concepts and their annotations."}
          ;;:return dm/AnnotateResponse
          (process-documents req matcher))
….

where process documents returns the channel from http-kit (these are long running tasks, so don’t want to hog the request thread)

but gives

com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class org.httpkit.server.AsyncChannel: /0:0:0:0:0:0:0:1:3000<->/0:0:0:0:0:0:0:1:57095
    originalMessage: "Cannot JSON encode object of class: class org.httpkit.server.AsyncChannel: /0:0:0:0:0:0:0:1:3000<->/0:0:0:0:0:0:0:1:57095"

@joelkuiper: What does process-documents return?

the org.httpkit.server.AsyncChannel

where a json is pushed on when it’s done

@joelkuiper: https://github.com/metosin/compojure-api/blob/master/src/compojure/api/middleware.clj#L150

the gist of it

(defn async
  "Long-polling/Comet handler.
   Takes a core.async chan and sends the response when available"
  [request c]
  (http/with-channel request channel
    (async/go-loop [v (<! c)]
      (http/send! channel v)
      (close! c))
    (http/on-close channel (fn [_] (async/close! c)))))

hmm so that should return false, can I override it somehow?

or do you think it might be some other middleware?

full stack

java.lang.Thread.run              Thread.java:  745
           java.util.concurrent.ThreadPoolExecutor$Worker.run  ThreadPoolExecutor.java:  617
            java.util.concurrent.ThreadPoolExecutor.runWorker  ThreadPoolExecutor.java: 1142
                          java.util.concurrent.FutureTask.run          FutureTask.java:  266
          java.util.concurrent.Executors$RunnableAdapter.call           Executors.java:  511
                           org.httpkit.server.HttpHandler.run         RingHandler.java:   91
                  compojure.api.middleware/wrap-components/fn           middleware.clj:   79
                dre.datahub.middleware/wrap-internal-error/fn           middleware.clj:   40
                       dre.datahub.middleware/wrap-context/fn           middleware.clj:   35
       ring.middleware.x-headers/wrap-content-type-options/fn            x_headers.clj:   53
              ring.middleware.x-headers/wrap-frame-options/fn            x_headers.clj:   38
             ring.middleware.x-headers/wrap-xss-protection/fn            x_headers.clj:   71
            ring.middleware.not-modified/wrap-not-modified/fn         not_modified.clj:   52
      ring.middleware.default-charset/wrap-default-charset/fn      default_charset.clj:   26
            ring.middleware.content-type/wrap-content-type/fn         content_type.clj:   30
                    ring.middleware.resource/wrap-resource/fn             resource.clj:   28
ring.middleware.absolute-redirects/wrap-absolute-redirects/fn   absolute_redirects.clj:   36
                      ring.middleware.cookies/wrap-cookies/fn              cookies.clj:  161
                        ring.middleware.params/wrap-params/fn               params.clj:   64
    ring.middleware.multipart-params/wrap-multipart-params/fn     multipart_params.clj:  117
          ring.middleware.nested-params/wrap-nested-params/fn        nested_params.clj:   84
        ring.middleware.keyword-params/wrap-keyword-params/fn       keyword_params.clj:   35
                      ring.middleware.session/wrap-session/fn              session.clj:  102
                          ring.middleware.flash/wrap-flash/fn                flash.clj:   35
                      ring.middleware.webjars/wrap-webjars/fn              webjars.clj:   34
      ring.middleware.format-response/wrap-format-response/fn      format_response.clj:  185
                                cheshire.core/generate-string                 core.clj:   21
                                cheshire.core/generate-string                 core.clj:   32
                                   cheshire.generate/generate             generate.clj:  148
com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class org.httpkit.server.AsyncChannel: /0:0:0:0:0:0:0:1:3000<->/0:0:0:0:0:0:0:1:57095
    originalMessage: "Cannot JSON encode object of class: class org.httpkit.server.AsyncChannel: /0:0:0:0:0:0:0:1:3000<->/0:0:0:0:0:0:0:1:57095”

mostly from ring-defaults

Quite hard to see from stack trace is the r-m-f middleware that is causing problems is the one from compojure-api or not

looks like it might’ve been duplicate from the general middleware, let me check

bit unsure what the order of these things is

yep was a duplicate wrap-formats

Are there cases known, that a for not working as expected? I have a loop

(for [[reg-uri event-chan]  (:sub-on-call connection)
      :let [req-id (clj-wamp.core/new-rand-id)]]
  (println reg-uri req-id))

@juhoteperi: thanks seems like it was a bug on my side with the duplicate format; different question is it possible to add the schema for the return just as documentation, without coercion?

@joelkuiper: Currently, no

okay, just checking

@joelkuiper: should be. just add is under :swagger

as what key?

hmm conflict!

https://github.com/metosin/ring-swagger/blob/master/test/ring/swagger/swagger2_test.clj#L75-L80

i’m still on 0.2.13 btw

for compojure api, 2.1.3 for ring-swagger

not sure what version is that. but as the :swagger works, add :swagger {:responses {200 {:schema dm/AnnotateResponse}} and you should get the swagger-docs

thanks!

noprobs

so this is the working code thanks everyone! https://gist.github.com/joelkuiper/275897ec403752ec2e9a

I heard a lot about boot instead of leiningen. As a beginner can I better stick with leiningen or make the transition into boot

@roelof: boot is excellent, but leiningen is far from dead - I would advise you to plan to learn both

oke, then first I stick with lein so I can continue learning clojure as a beginner

and later look at boot

@meow: any tutorails you can recommend for a beginner in boot

@roelof: no, I would just google and see what comes up - the boot folks are extremely supportive and there is #C053K90BR here as well

Personally i'm switched off from boot because is too much slow 😞

@niwinz: any public projects I could look at where boot is to slow?

@niwinz: in case you have the project still setup I'd be very interested in seeing some numbers :)

@meow: thanks for the answers

I switched some my projects to boot and for example executing the uberjar is almost 3x slower than doing the same task in lein... But lamentably now, I don't have any project using boot..

@niwinz: uberjar perf has been improved dramatically with 2.5.0 (currently only snapshot releases)

Nice to hear that!

http://blog.cognitect.com/blog/2015-clojure-community-survey

It would be great to see a large response on the survey so we can take the pulse of the community! Thanks...

@alexmiller: So I filled out the first page, clicked Next, and it put me back on the first page with all my answers wiped out. 😞

doh!

I made one question change on pg 1 - maybe that messed up people in progress

if so, my apologies!!

@alexmiller: no problem, just wanted you to know

should I try again?

I hit the back button, it recovered my answers, then when I hit next it went to page two.

@alexmiller: missed page 2 as well

Oooh, nice, first state of Clojure I can actually fill in having finally done real Clojure work this year : D

@dnolen: I've been trying to apply core.logic to output of tools.analyzer, but the copying walk* in ground checking is really a big roadblock for the kind of massively interlinked data, the analyzer produces ...

would you consider that a use case worth supporting?

@alexmiller: how do you "store" the answers a page. When I ready to learn web development Im also thinking of using these wizards/multipage forms ?

I suppose "How long have you been using Clojure, ClojureScript, or ClojureCLR?" means seriously using the language not just "hey, I can write quicksort in that", right?

whatever makes sense for you

IM using clojure for some 2 - 3 weeks now

I guess, I'll go with 1 year then; I've been following Clojure since 1.3, but didn't do anything more than simple algorithms until this year.

I only did the clojure koans and I think the half of the easy exercises of 4clojure

So I have to develop a clojure and FP mind-set

Hi I am trying out schema...but, one thing that bugs me is that the names of the defs are so long like "defschema" and "required-keys" and so on. Is there a way to shorten them? I tried a simple (def ds s/defschema) but it fails with a compiler exception: CompilerException java.lang.RuntimeException: Can't take value of a macro: #'schema.core/defschema

@sveri: You'd have to do that as a macro: (defmacro ds [& body] `(s/defschema ~@body))

But I personally would favor legibility over abbreviation, i.e. use the real names

@alex.hall: Thanks, should have thought of them^^. They just take up so much space, especially defschema and required-key...they take most of the lines without providing anything useful

Fair enough. I just think that such abbreviations lead to unmaintainable code, especially in a team environment.

You don't have to use defschema thought. I do (def SchemaName {:key s/Int}) and it just works.

http://stuartsierra.com/2015/05/10/clojure-namespace-aliases gives an excellent overview

I agree. My plan is to keep them in one namespace, make them private, so they don't leak out of my schema definitions namespace

Similarly for required-keys - you don't have to use it for :keyword keys, they are required by default. Unless you have string keys, then tough luck.

@jaen: Well, thats better...awesome, thank you

Actually it's somewhat annoying to do the reverse - make optional keyword keys. For some reason schema doesn't accept any option to validate to flip the behaviour.

You have to postwalk them.

And not just any postwalk, since that won't understand most of schema's wrappers around values.

I asked about that a while ago here, let me dig that up.

Also, one downside to not using defschema, the schema name isn't assoc'd into metadata.

I agree...thats a disadvantage, have not thought of that

Though I'm unsure what that means in practice (maybe worse error messages?).

@sveri: here's the code for making all keys optional:

(ns scrabble.common.utils.schema
  (:require [schema-tools.core :as st]
            [schema-tools.walk :as st-walk]))

(defn with-optional-keys [schema]
  (st-walk/walk schema
                (fn [x]
                  (let [y (with-optional-keys x)]
                    (if (and (map? y) (not (record? y)))
                      (st/optional-keys y)
                      y)))
                identity))

@jaen thanks, I'll keep it and see when I need them

Schema tools in general seem pretty handy

Had a great time at ClojureX. Already looking forward to ClojureX 2016.

Were the talks recorded perchance? Though looking at ClojureTV I guess the answer is "no".

ClojureX is put on by Skills Matter and they usually host talks somewhere other than the ClojureTV channel

I think they usually are recorded though - you might want to ask on #C075TNSSC

@alexmiller: thanks for the channel pointer, someone linked to Skills Matter channel with the talks yesterday. it says I have to login, but hopefully an account is free.

About the Survey ... question 17 is a bit odd; it's asking us to evaluate the priority of improvements, but does not provide any idea of what those improvements might be. "Startup time" and "Runtime performance" are obvious ... but what improvements are being considered for "Namespaces" or "More compiler control"?

nothing specific, just general areas

Nicola suggested the latter one, really in regards to more control over how var linking, inlining, etc is done

we have no specific plans to do that, but that was the idea

re namespaces, there have been ideas floated about immutable namespaces and things of that nature in the past

Hah, I think that depends on how you interpret things. For namespaces I compared what Dunaj project did to current state of art in Clojure, which made me rate it low. And I took "more compiler control" to being able to tune optimisations for example.

https://skillsmatter.com/conferences/6861-clojure-exchange-2015#skillscasts

@jaen those both seem like reasonable interpretations

@jaen: accounts are free. I am watching one video right now.

@hlship: in the past there has been some free form question about adding features to Clojure. the intent here is to replace that with something that gives better and more actionable data (by both not listing things we have no intent of doing and listing specific priority areas)

Incidentally, Dunaj's modularisation and restructuring on API seemed a fairly interesting idea, what does core think of that?

@rcanepa: good to know it's free, thanks!

@jaen the stuff in Dunaj are interesting ideas but we tend to work starting from a specific problem

so the question is what problem needs to be fixed?

if the problem is startup time, then namespace reorganization might be one solution

but so might lazy vars or using invokedynamic for var invocation or parallelization or something else

if the problem is related to organization, that's a different set of possible solutions

of which restructuring the api is one answer

Was just looking at Clojure 1.8, letting IntelliJ decompile the compiled functions.

I noticed that the function classes still create references to Vars as static fields, even if those are Vars for functions that will be statically invoked.

Probably minor, but this is memory and execution time that could be shaved. Are these public fields being left around intentionally?

http://dev.clojure.org/jira/browse/CLJ-1861

Rich plans to work on cleaning that up in 1.9

@alexmiller: gotcha. I've read the Dunaj's docs and I think the arguments re: API organisation make sense. I do understand though there might be different solutions to the startup time part other than modularisation, I don't know enough of how Clojure works to have an opinion, since I've been doing it more seriously only for a year.

which channel should we ask general questions about Clojure web development? not clojurescript related, more on backend

@lumengxi: this one is fine to start with

Hi. using prismatic schema I have a map in a def like this: (def foo {:name "init-aval" :bar "bar"}). Now I want to define a schema for this. Can I define schemas for defs?

@sveri: sure, (def foo (s/validate yourschema {:name .... }))

Anyone have examples of client libraries (i.e. libraries that wrap a web API) that they love? I'm wondering about best practices.

@dmitrig01: Does it validate at runtime too?

@sveri: schema is a runtime-only tool

@sveri: Look at the docstring for s/validate.

Ah, I remember...you could turn it off and enable it only in dev...thanks everyone

Ok, another schema question. Can I define static values like 1?

So I have a map like this {:default 1}?

I found ther is (eq val). Is that the correct one to choose?

I have a list ({:employee/name Timmy}), why is it that, when I peek at this list I get an java.lang.ClassCastException exception?

Is it println that’s trying to cast it, but is not able to?

and I don’t see why println whould have a problem with that map

I enjoyed the Clojure community survey. I was surprised to see no mention of "debugging tools" in the survey though ("error messages" was the closest fit). I mentioned this in my final comments.