so I have the most recent version locally, is there a way I can work on it (via local checkouts) at the same time that other libs are coming in remotely from s3 (ie :repositories)?

@hiredman ^^

>checkouts don't replace a dependency I just ran lein install, and things work, I'll have to check to make sure that solves the "checkouts" workflow though

@akjetma I wouldn't have any clue about the right way to do that. I get a lot of mileage from creative use of what's at hand: Monitor cpu from a script and send it a KILL when nothing's been happening. Definitely not the right way, but if it works ...

Let’s say I have a set of 1,000 http requests I want to make, with up to 100 happening concurrently. My first thought would be that this is a great use case for core.async, but clj-http uses blocking io, and the default thread pool count for core.async is 8. I could increase this to 100, but that somehow feels wrong. Any advise?

Or, perhaps, there’s a library that works similarly to r0man/cljs-http, where network io is non blocking, and plays well with core.async

@josh.freckleton - I haven't seen lein checkouts before. I assume you've read [0] and [1]? If those don't answer your question, maybe try over in #leiningen. [0] https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#checkout-dependencies [1] http://jakemccrary.com/blog/2012/03/28/working-on-multiple-clojure-projects-at-once/

@brycecovert I’d reach for either http-kit or aleph; they’d both need some shim code to translate into core.async, but it’s not that hard

@csm using this? http://www.http-kit.org/client.html I’m presuming that if http-kit parks the thread for io, it will be available for core.async, is that correct? It sounds like the “shimming” might be as simple as (<! (go (http/get …))), right?

or, perhaps to be more specific, if parking a thread makes it available for other purposes, then simply putting responses onto a channel would be all that’s required

right, the “shim” would look more like (let [chan (async/promise-chan)] (http/get … (fn [r] (async/put! chan r))) chan)

i.e. since http-kit is callback-based, make the callback complete your channel

Ah, got it. I misread the documentation. Makes perfect sense.

I hadn’t noticed that it was callback based

👍 Very straightforward, thanks!

brycecovert the core.async thread pool size is (nprocs * 2)+42 - but they still shouldn't be used for blocking io

there's also core.async/thread for things that should happen outside the thread pool

@csm I'd argue for using >!! instead of put, since to get proper control of the task pool the http/get should be in a loop that reads a request from an in channel and writes the result to an out channel, and >!! can impose backpressure that prevents the request producers getting too far ahead of the request making

I'm picturing a loop (go-loop [] (let [r (<! in)] (http/get ... (fn [r] (>!! out r))) (>! consumer (<! out)) (recur)) - the extra indirection of a consumer channel and out channel is so that the go-loop has back-pressure, to have N parallel requests, start N loops

@noisesmith It used to be that size -- it got changed a while back I think?

oh, I checked fairly recently... one moment

you're right - wonder when it changed https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/exec/threadpool.clj#L22

it changed last june https://github.com/clojure/core.async/commit/a690c4f3b7bf9ae9e7bdc899c030955d5933042d#diff-2193dd8597437d6454bb74edd2f15e00

Early/mid-2016... I'm going back through commits now...

June 9th.

https://github.com/clojure/core.async/commit/a690c4f3b7bf9ae9e7bdc899c030955d5933042d

yeah, that's the same commit I linked

Oh, sorry I didn't see you'd posted two links while I was still looking for it in GitHub

no worries.

@brycecovert this may help out regarding core.async to generate http requests and guarantees a cap on concurrency http://stackoverflow.com/questions/42368332/easiest-way-to-use-a-i-o-callback-within-concurrent-http-kit-get-instances/

any preferable way to define spec keys' relationship?

e.g. I have a collection of seats, only one of them is active.

it will generate struct like {::seats [] ::active-seat 2}, which actually is not legal

Write a predict that is only true when the seats count matches active seat

And it to the coll of

You most likely will need to write a custom generator

Thanks for the pointers @joshjones @noisesmith @seancorfield

Predicate

@hiredman thanks. I modify it like s/def ::my-struct (s/and (s/keys [::seats ::active-seat]) (fn [{:keys [::seats ::active-seat]}] (< active-seats (count seats)))))

but do not know how to write the generator

The spec guide on the clojure website has a section on generators

user> (do (def arr (int-array 100)) (aset arr 0 (inc Integer/MAX_VALUE)))
-2147483648
user> (let [arr (int-array 100) (aset arr 0 (inc Integer/MAX_VALUE)))
IllegalArgumentException Value out of range for int: 2147483648  clojure.lang.RT.intCast (RT.java:1205)

wtf?

I had been opening a file with , then storing the resulting Writer in a map in an atom that I pass around. I write to the file in a different namespace, and then close it in the original namespace when a condition is met. This worked fine. I wondered if I could use with-open instead. Now I call wrap the writer call in with-open, but again store the writer object the atom. Inside with-open, I call a function that calls a function in the other namespace that eventually calls something that pulls the writer out of the map in the atom and then tries to write to the writer. While this is happening, I have not left the scope of with-open in the first namespace. However, the write fails with "java.io.IOException: Stream closed". I'm not 100% surprised--I know it's possible to try to do something outside the scope of with-open, and I wasn't sure whether this would work. But I don't understand why it doesn't work, given that with-open is still "running" when the write happens. Must all of the uses of the writer from with-open be visible inside the parentheses surrounding with-open? Am I confused about something?

are you sure the with-open form never exited?

what keeps it waiting?

https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L3802

I don't see how it could. run-sim is the last call in the with-open form, and that's what calls the functions that write. But now I'm thinking that there's another way for run-sim to be executed, outside of the with-open. I should make sure that's not happening by accident.

.close is called after the clojure form is evaluated

also, why use an atom, doesn't that just make it easier for something else to use the file handle after with-open exits? the usual pattern is to pass the file handle to a function inside the with-open call, so that with-open can't leave until the function call finishes

you can't store the writer

well you can, if you ensure it's never used after the form exits, but that seems like a lot of unnecessary complexity compared to just, you know, not storing the writer

also, be wary of any lazy ops inside with-open

what is it about js vs python that we have clj -> js but no clj -> python ?

(no, the answer is not Hy, it's too different from clj)

I could pass the writer rather than using an atom, but I'm storing a whole bunch of stuff that's needed in the other namespace anyway. It's the global config structure.

Good point about laziness. I don't think that's going to be relevant, but laziness bites when you're not looking.

if I had a dime for every call to map inside a with open that did something unexpected (especially when hidden by eg. a database api...)

OK, maybe I'll try passing the writer. It means adding a parameter to several functions, but I shoudl try it.

mars0i the thing is that adding an arg is much less complex than managing the validity of a temporary output resource manually

Thanks noisesmith and jr. There's a bit of work to setting things up to pass the writer explicitly. Will leave that for morning.

Yes, I see your point noisesmith. Seems like what I'm doing should be OK, but ... obviously it's not.

Anyway what you are telling me is that my understanding of with-open seems correct, so I have to look for other problems in my understanding of my code.

Or I could just go back to explicitly closing the file when I know I'm done with it. That worked. But sometday it might bite me.

now that I think abut it, what run-sim does is passes reifys to a scheduling function. this is from a java lib l didn't write. the write fns are called from the reify. this is starting to make sense. the scheduler probably runs the classes after with-open finishes.

with-open's not going to work if this correct.

@qqq js gives you a lot of reach. Everyone is compiling to js so that they can get into the browser. Also, with React Native, js can also get you into mobile devices.

clj -> python is possible, but is it worth the effort?

clj -> rpython may be worth it

pixie compiles to rpython but is not clj

clj -> llvm would be a killer (commandline tools!)

How do I spec a map, which contains string keys instead of keyword (i.e. {"foo" "bar"} instead of {:foo "bar"}) ?

(s/map-of string? ...) or you convert the keys to kw (:() if you want k->v validation

or a custom predicate/validator

:keywordize true 😛

thanks. didn't know about map-of, but it looks like every value would need to conform to the same spec.

ok so what I’m going to ask might fall a little bit in the realm of some crazy dark magic 🎩 but… has anyone ever thought about “re-using” their destructuring? sometimes I have a pretty big data structure that I de-structure, often in a let

(let [;destructure props
          {{product-options                              :db.product/product-options
            product-id                                   :db.product/id
            {c-uuid :uuid :as selected-options} :client/selected-options
            :as                                          product} :product} props)

and often I want to use that in multiple places, in different functions; any better way than simply copy/paste that everywhere?

i made an anaphoric macro that would destructure everything in the same way

(that’s an example from react/cljs, but it very well can happen on the server also)

it ensured that the signature was consistent and that i correctly got everything

A lot of the complexity there ^^ is due to nested maps. When I get to needing something like that i normally flatten the map then use :keys [...]

@tbaldridge interesting idea

very interesting indeed 🙂

what about… performance?

I guess if it works for your use-case… it works… just have to try it

a single level of map is probably going to be faster since you have 1 less lookup

right, I mean the transformation part

since I already have the nested map… but I see what you’re saying… pay the cost once, then access fast every time… you can cache that etc etc

because immutability rocks 🙂

ah, yeah, depends on where this information is from. Optimal method is to use namespaced keys and a single map, but you may not have control over that

@tbaldridge that's something that I've been slowly converting the edges of my APIs to use. the lack of nesting is refreshing in the parts of my codebase that have been adapted

@tbaldridge that’s a very interesting approach indeed… you suggested that in some other place as well in the google groups… I think I should use more of that

nesting is definitely a source of a lot of complexity… I’ve definitely cut the amount of it a lot, usually trying to stick to 3, max 4 levels, but in a way it’s infectious lol… once you have nesting, it only brings about more of itself haha

@tbaldridge so do you literally use a single level map everywhere, or you cut off at like, 2 levels maybe? 🙂

Unless my data is a tree, or I have a "many values" attribute, I use one level

yes… such a good tip, that should be a Clojure sticker 😜

USE SINGLE LEVEL {}

haha

+ namespaces… that really makes that happen

allows that to happen in a very pleasant and elegant way

Of course, if this were go, our next version of map wouldn't support nesting other maps.

You'd probably use structs anyway for anything "speced"

@bja haha is that a fact? 🙂 (I barely know any Golang)

sorry, my joke was out of place. backs away from the kindling

(don’t want to turn this into a golang discussion haha)

@not-raspberry the problem is that llvm makes you bring your own gc, and clojure demands a lot from its gc. You can already do fast launch cli with cljs via lumo or planck.

multiple “clojure on llvm” projects have started, none are usable yet

@gungfoos, in #(when (vowels %2) (inc %1)) %1 is the first param, %2 is the second param to the anon fn.

@gungfoos refer to https://clojuredocs.org/clojure.core/fn#example-542692c7c026201cdc326983

Why does the following occur?

(require '[clj-time.core :as t])

(with-redefs [t/today-at (constantly 11)]
  (t/today-at 0 0))
;; => Unhandled java.lang.ClassCastException
;;    clojure.core$constantly$fn__4614 cannot be cast to clojure.lang.IFn$LLO

(with-redefs [t/today-at (constantly 11)]
  (#'t/today-at 0 0))
;; => 11

heh, that's even more of a bizzare issue, it's not even timing related

So the problem is that today-at is type-hinted. So the compiler knows that it takes a long, and a long and returns an object. However, then you went and re-bound it to a function that takes an object an object and returns an object

you can't redef primitive type hinted functions

Ah, OK

The second one works because you're calling through a var.

@xiongtx Do you see the LL0 at the end? That means it's trying to invoke a primitive fn (optimization) because you pass in 0 0 (`L`ong, Long) which returns an object

so, a question for @xiongtx and anyone else, is there some source I'm missing where someone is recommending using with-redefs? There seem to be a lot of questions about it recently.

@rauh I see, I knew about type hints but not how exactly they’re optimized

whereas I've always considered it one of those things-I-wish-someone-would-delete-from-clojure-core.

@tbaldridge Well, with-redefs‘s own docstring says: > Useful for mocking out functions during testing. I guess we all thought it’s true (which it is, until you run into something like this)

eh, good point, maybe I'll go write a patch for that.

If you put > [WARNING]: Avoid like the plague in the docstring, there’d be fewer Q’s like these 😛

By the way, it’s LLO (letter O), not LL0

I have some defrecord values that seem to behave inconsistently wrt instance?. E.g. they’re all <#C4H71K9RQ|miraj>.co_dom.Element{…}, and both (type) and (class) return miraj.co_dom.Element. But some of them return true when I do (instance? miraj.co_dom.Element e) and some return false. What’s going on?

@xiongtx @bronsa @tbaldridge but if you redef it with a function that has the same type hints on its params, no problem

correct

this is driving me nuts. (type e) prints miraj.co_dom.Element, but (= (type e) miraj.co_dom.Element) is false.

so I can’t make a distinction between a plain ol map and a defrecord value. 😞

@mobileink Call getClassLoader on both: https://docs.oracle.com/javase/7/docs/api/java/lang/Class.html#getClassLoader()

Do you get the same classloader?

@mobileink if you are working on a REPL and reloaded the namespace that defines the record but still have remaining instances of the old one the instance? check will fail

@xiongtx: (log/debug "CL type" (type (.getClassLoader (class m)))) prints clojure.lang.DynamicClassLoader

@thheller: I’ll bet that it’s it. thanks!

hello fellow clojurans - had a newbie question

i am running a “lein run” command on my terminal

and want to connect to it via emacs+cider

for the life of me…i am not able to determine the default port to use for cider-connect command

appreciate any pointers in this direction ^^

@vadlamak when using lein run you won’t get an nrepl at all, unless your app starts nrepl manually

lein repl will start nrepl

@noisesmith thanks for pointing out my folly…use

lein repl

lein run

hey

can I circumvent the printing multimethod with pr-str somehow? i would like to use default record prints instead of the custom handlers that are active in one of the namespaces.

for reading you can do clojure.edn/read-string with a read-handlers map

@thheller that was indeed the problem, a remove-ns did the trick. thanks bigly.

hey guys, what is the best way to test this component ? http://dpaste.com/3FK1RJ0

(let [sys (component/start ....)] (try do http request (finally (component/stop sys))))

hey yall

so I'm getting this error: Uncaught Error: Could not find tag parser for error in ("inst" "uuid" "queue" "js" "datascript/Datom" "datascript/DB" "Agent")

so specifically, the #error tag is not being found in the tag-table

\tag-table\

I'm trying to shuttle code/data between webworkers and on the return trip, I'm getting an error which won't be read by the reader, because #error appears not to be registered

Is this intended behavior and, if so, does anyone know where the implementation of #error's reader function is, so I can replicate it?

#error doesn't have a reader function, if I recall

errors prn as data and it is up to you to interpret it

@hiredman I was noticing here: http://dev.clojure.org/jira/browse/CLJS-1831

the following: #error {:message "ERROR in file

sure

that is a prn'ed error

in mike's response

prn of a tagged literal does not imply a reader exists

I could be mistaken

see also #object...

the easiest thing to do would be to set identity as the handler

(for #error)

@hiredman so could it be that the error is being sent back over to the reader and the reader is mistakenly interpreting it as a tag?

you are doing something, prn'ing the result, the result is an error

#error .... is what you get when you call prn on an exception in clojure (not sure what kinds of objects do that on the clojurescript side)

understood

it isn't a mistake to interpret it as a tag, it is a tag

just one without a defined handler

you can either set a catch all handler (if I recall) or set up a handler for the error tag

if you want to read it

Should I add a handler for #error in a data_readers.cljc file or should I do so another way?

it depends how you are reading the data, I would look at the doc string on whatever read function you are calling

a number of them support passing in a map of handlers

oh it ate my comment - sometimes I have wished that object had a readable prn, though I understand the argument for not having one

@john I like using transit for round tripping clojure data between hosts, it makes it simple to set local handlers for tags / datatypes rather than needing to mess with clojure’s readers / printers

it’s also somewhat smart about structural sharing within data, which the printer /reader won’t be smart with at all

testing this out now: (cljs.reader/register-tag-parser! "#error" identity)

@noisesmith you know, I thought butler, the webworker library I'm using, used transit to marshal data between contexts, but maybe not. I'll look into transit to see if I can use it on top of butler

if so, you should be able to provide a map of data readers/writers, if it’s already using transit that’s the right way to handle it

it sure doesn't

So I'll look into that. thanks!

https://github.com/federkasten/butler/blob/master/src/butler/core.cljs#L2

yeah, they are using transit but aren’t letting you provide custom reader / writer maps - optional map arg would be a simple pr

@jr nice catch! thanks. I just looked at the project.clj

@jr @noisesmith okay, yeah, I see in the docs there

(def w (t/writer :json-verbose {:handlers {Point (PointHandler.)}})) (t/write w (Point. 1.5 2.5)) ;; => "{\"~#point\":[1.5,2.5]}"

good to go, thanks yall!

awesome