This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-01-27
Channels
- # aws-lambda (3)
- # beginners (3)
- # boot (327)
- # capetown (2)
- # cider (156)
- # cljs-dev (368)
- # cljsjs (13)
- # cljsrn (53)
- # clojure (403)
- # clojure-czech (5)
- # clojure-dev (4)
- # clojure-greece (2)
- # clojure-russia (72)
- # clojure-spec (12)
- # clojure-uk (129)
- # clojurescript (156)
- # core-async (1)
- # cursive (33)
- # datomic (35)
- # emacs (10)
- # events (1)
- # hoplon (4)
- # jobs-discuss (3)
- # klipse (1)
- # lein-figwheel (14)
- # leiningen (5)
- # luminus (5)
- # off-topic (15)
- # om (69)
- # om-next (2)
- # onyx (2)
- # parinfer (15)
- # perun (12)
- # re-frame (30)
- # reagent (7)
- # ring-swagger (8)
- # spacemacs (10)
- # specter (6)
- # untangled (69)
- # yada (13)
am I not understnading clojure apcakges -- or does ring, unlike other github/clojure projects, have "multiple porjects" inside a single "repo" ?
oh, i"m the idiot, I should have done [ring "1.5.1"] which would included all of the packages
hi everybody, is anyone aware of a lib that allows me to manage env variables and if they are not provided it would throw an error? im currently using Envinron and doing it by hand 😕
I'm AOT-ing clojure code. I need to generate a class which im[lements a certain Java interface.
I know how to AOT code in Boot. I know I need to use something like :gen-class ? I don't know anything else. WHat should I read up on?
@qqq the docs for gen-class are here https://clojuredocs.org/clojure.core/gen-class . just saying, my experience with it is that it's a bad idea. a small shim java class that calls your clojure code using Clojure.Var has always worked out much better for me
I need something which satisfies javax.servlet.Servlet -- I know that using ring.servlet/servlet will convert a ring-handler to a java-servlet
assuming you actually need a compiled class that is. if all you need is something that implements an interface proxy
is a pretty good winner
yeah, I need to output a *.class file implementing a Servlet. I'm also using boot, not lein ring
I assume you've seen this: https://github.com/cemerick/clojure-type-selection-flowchart
I tried definin a "HelloServlet" inside of foo/bar.clj , but it does not output a foo/bar/HelloServlet.class
clojure interop is very very good, especially at consuming java code. it gets complicated when you want java code to consume clojure code
(and I, personally, would replace the "use gen-class" box with "write a tiny bit of java")
@bfabry: the good news is that I now have a g/m/HelloServlet.class and am getting a different error 🙂
Hi! Quick q. why does
(let [port (cycle (range 1 200))] (dotimes [_ 10] (print (take 1 port))) )
return (1)(1)(1)(1)(1)(1)(1)(1)(1)(1)
instead of 1 2 3 4 5
etc. I guess it's lazy, but I'm not sure how to work around that laziness(take n) really means "create a new seq consisting of the first n elements, but don't modify the origal seq)
@bradford what is the goal? just '(1 2 3 4 5)
out of a cycled range?
(->> (cycle (range 1 200)) (take 5))
You're probably thinking imperatively you don't actually modify the port binding so you need to iterate over it somehow.
The whole context is creating a set number of proxies for webcrawlers, and consuming URLs for them to consume. Each proxy needs a port.
(defn start-async-consumers
[num-consumers ^core.CrawlService crawlservice]
(let [ports (cycle (range 1 200))]
(dotimes [_ num-consumers]
(async/thread
(while true
(let [url (async/<!! in-chan)]
(.start crawlservice (Integer. (take 1 ports)) url)
))))))
boot.user=> (defn create-crawler [port service] (println "creating a crawler" {:port port}))
#'boot.user/create-crawler
boot.user=> (map create-crawler (cycle (range 4)) (range 10))
creating a crawler {:port 0}
creating a crawler {:port 1}
creating a crawler {:port 2}
creating a crawler {:port 3}
creating a crawler {:port 0}
creating a crawler {:port 1}
creating a crawler {:port 2}
creating a crawler {:port 3}
creating a crawler {:port 0}
creating a crawler {:port 1}
it would depend on what functionality you want from extension, I would go back to the flowchart. gen-class probably though
it's not bad, it's just not what people mean when they say "extend an existing java class"
it might be me with a wider meaning of "extend an existing java class" 🙂 but I would say it extends java.util.Date
with a method to-ms
:
(defprotocol Dateable
(to-ms [t]))
(extend java.util.Date
Dateable
{:to-ms #(.getTime %)})
yeah, it allows you to extend a java type with a protocol. but that's not what qqq meant
@tolitius: @bfabry 's interpreattion -- I wanted to write clojure code that extends javax.servlet.http.HttpServlet
@bfabry : is https://github.com/tailrecursion/clojure-adapter-servlet/tree/master/src/tailrecursion/clojure_adapter_servlet what you meant by "easier to write a small bit of java" ?
yeah, that https://github.com/tailrecursion/clojure-adapter-servlet/blob/master/src/tailrecursion/ClojureAdapterServlet.java file is exactly what I'd write rather than trying to learn gen-class
or proxy
: https://github.com/ring-clojure/ring/blob/master/ring-servlet/src/ring/util/servlet.clj#L137-L145
lol, I think it's voodoo magic that I can have my code called by code written in another language at all
if your implementation needs to be referred to statically from java then proxy won't work
which is one of the points on chas's flowchart. you can probably all tell by now, I'm a big fan of the flowchart
if it was anything besides gae, i'd tell it to go f- itself, but since I want to use gae and datastore, I tolerate this bs
i can simplify taht flow chart: if what you're doing trivial? yes => do it no => use gen-calss
yes, proxy
and reify
create anonymous classes, so if you need a custom named class, you'd go with gen-class
did you import clojure.lang.Var;
? (Cursive is very helpful here as you just get automagic java file fixing in intellij)
clojure gen-class javax.servlet.Servlet <-- I kid you not, I think the one result that would be useful for this is cgrand from 2008, and apparently that page hangs
@U3E46Q1DG : ha! I got gen-class to work, the problem was not gen-class, the problem was I forgot to include clojure (via the (uber) task) for boot -- so the servlet container was loading my class, not finding the clojure support, then bitching about my calss (leading me to think I mis-used genclass), when I used it perfectly and the only problem was I did not include clojure as a dependency
Is there a neat way to update all hashmaps in a vector so that if a hashmap is missing a key then add it with certain value?
(def testing (repeat 1000000 {:name "NJ" :available? true}))
Will the map in the sequence testing
be interned or there would be copies of the map once testing
is realized?
@narendraj9 Yes: (let [[a b] (repeat {:a :b})] (identical? a b))
@rauh Thanks! 🙂
@rauh According to this article -- http://nyeggen.com/post/2012-04-09-clojure/ , the values of a hashmap aren't interned. Why is that the case?
@narendraj9 Strings and Keywords are both interned
Hello, I use schema to define my records, which works great. Now I have a question to understand the usage. Is it useful to validate the record in production or only within unittest? Actually I test it when I call (map->Record), but this throws an exception, if the input is wrong.
@rauh repeat
keeps the reference to the object around and hence all of them would be indentical. https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Repeat.java
Okay. Hashmaps in Clojure aren't interned it seems. Structural sharing happens because we start with a reference to the original map but two separate maps with the same contents would be two independent objects.
Hi, what is the most straightforward way to extend a PersistentVector instance to some protocols? or to create a type that behaves and looks exactly like PersistentVector but has extra capabilities (via implementing some extra protocols). Am I suppose to write a 200loc deftype to do that?!
What kind of problems does one avoid by heeding the warnings in lein deps :tree
about "possibly confusing dependencies"?
What is the idiomatic way of getting the namespace part of a keyword as a keyword, (keyword (name kw))?
@pbaille for a particular instance you can't do it with extend-proto/type, no easy way to creating a new type from another (potemkin does this for maps for instance)
yes i’ve almost do the metadata stuff but is there a particular reason for this to not being in core?
not necessarily, I am personally not against this kind of things. and could have used what you described a few times in the past, but that's how it is atm
Hey, I am evaluating bouncer for validation of forms and when using it, bouncer returns a vec like this above after validation. You can see that the errors are hidden in the vec at the second map inside :bouncer.core/errors. Is that how its intended to be? That looks pretty unstructred, almost randomly. Any ideas if I am doing something wrong?
Hello guys, another core.async
question for me 🙂
I am running this code to see with YourKit
which Threads are live or not
(let [ch (chan 1)]
(go (>! ch 42)
(println (<! ch))
(a/close! ch)))
when I capture thread usage with YourKit
, this is what happens =>the async-dispatch
Thread stays in waiting mode
I thought it should be GC’d or killed in some way
so it is supposed to stay in waiting mode until the end of the world ?
I thought go
point to ch
I am pretty sure it has a reference to ch
while it’s alive
and nothing has a reference to go
so it should be GC’d - I mean, that’s what I thought 🙂
from go
doc =>
...
Returns a channel which will receive the result of the body when
completed
in my case close!
returns nil
...US members are still sleeping ^^
@baptiste-from-paris IIRC it's 60s idle until the thread is then cleaned up
@rauh ; after 60s thread is still here
@baptiste-from-paris NVM, that's for the Cached Threadpools, for the async dispatch pool (newFixedThreadPool) they'll never get shut down unless done specifically
ok nice, thx
and if I do the test 100x times, it all sounds good =>
(dotimes [_ 100]
(let [ch (chan 1)]
(go (>! ch 42)
(println (<! ch))
(a/close! ch))))
10 threads which is my availableProcessors
+2
ah, ok - thx
yes indeed
and from the javadoc =>
newFixedThreadPool
...
The threads in the pool will exist until it is explicitly shutdown.
no I am wondering how it impacts me when I am using core.async
ok, thanks 🙂
by the way, why default to 8 ?
Well it used to be 2*processor + 42 but then people complained that this uses a lot of RAM by default, so it got lowered to 8. But I think people are now expected to choose their pool size (or even their own executor).
@baptiste-from-paris Good morning from the US 🙂. I did a (free) video awhile back about GC'd go blocks you may find interesting: https://www.youtube.com/watch?v=VrwVc-saWLw&index=7&list=PLhi8pL3xn1OTDGCyXnkZStox6yFjn2583
The gist: half the time channels hold a reference to go blocks (not the other way around), and yes that seems strange but there's a reason.
ahah good morning !, @tbaldridge that’s a great video, I think I really need to watch it another time
@mpenet I was wrong about go
-> ch
; it’s ch
-> go
video explain it greatly
and @tbaldridge , do you have any idea why we never shutdown
threads ?
I think that's the default for the threadpool. And we expect users to override the threadpool if they want something different.
I'm not aware of any design decisions in that area
coming back to clojure after a while, I am getting this error which I'm not sure about: >java.lang.ClassNotFoundException: clojure.java.shell looks like http://clojure.java.io is found, but not shell. anyone up for quick advice? 🙂
@matan did you require the namespace? 🙂
just "git" should work
@doglooksgood what do you want tree-map
to do differently than PersistentHashMap
?
@doglooksgood A sorted map is one IIRC
@pesterhazy I have not noticed in the docs that the namespace needs to be required... isn't it included automatically just like http://clojure.java.io?
what bronsa said
Wondering why http://clojure.java.io simply works without it
probably because it's used bei lein
is there a 'clojurelight' somewhere which is like clojure, but less features, and also less code / classes generated ?
that's a strange question, qqq!
@pesterhazy : yeah, on second thought, it's not clear to me what I would be wiling to "drop" to make clj 'lighter'
it has fewer features (no jvm libraries, you get node libraries instead), but the output size can be significantly smaller due to advanced mode from the google closure compiler
@doglooksgood array-map preserves insertion order, but if you conj
an element on an array-map, it will be transformed into an hash-map
and the ordering will be lost
also, you almost never want clojure.core/array-map due to it transparently promoting to a hash-map past a certain size
what @bronsa and @pesterhazy said
if you create an array-map using clojure.core/array-map
it will not be promoted regardless of size, but it will as soon as you conj
onto it
how should I do if I want to search for an item in a vector? just use (first (fliter ...))
?
Hey, I am getting Caused by: java.lang.IllegalArgumentException: No value supplied for key: {:handler project1.core/handler, :init project1.core/on-init, :destroy project1.core/on-destroy}
`:dependencies [[org.clojure/clojure "1.8.0"]] 6 [[ring "1.5.1"]] 7 :plugins [[lein-ring "0.10.0"]] 8 :ring {:handler project1.core/handler 9 :init project1.core/on-init 10 :destroy project1.core/on-destroy})`
@moxaj, I think parens are at right position... How do I install depedencies? doing lein deps
just gives the same error
@vs4vijay I usually recommend people start with http://www.luminusweb.net/docs until they form a strong opinion of their own on various libraries
I personally made almost none of the same choices as luminus for my latest project, but until you understand the choices, luminus will give you some sane defaults
Okay, and what about any book recommandations? I am reading Joy of Clojure... and have lined up SICP and Clojure for Brave... Is there any other resources worth checking out?
what's the proper way to type-hint multi-arity fn return types? Is it
(defn go (^Type []) (^Type [a]))
?@vs4vijay Clojure for the Brave and True is a good first book. I recommend to read it before Joy of Clojure, which is arguably a bit more advanced.
Kyle Kingsbury's "Clojure from the ground up" is a very nice series https://aphyr.com/tags/Clojure-from-the-ground-up (the cards are sorted by publication date desc)
beginner oriented though, sorry I didn't read carefully what you were searching for 😇
Didn’t know that cljs-ajax supports Clojure nowadays: https://github.com/JulianBirch/cljs-ajax
Aug 2015 it seems https://github.com/JulianBirch/cljs-ajax/commit/6dbcaaf336cf677627c081e034f0140cd8ef07d4 I didn't know either 🙂
when using generators created by s/gen, is there a way to parameterize the generator for a test - for example to fix the size of a collection ?
like, e.g. say i have a spec :qwe/coll (s/coll-of double?) which I don't want to specify the size of the collection, but then later when I exercise it I do want to specify a size
Is it possible to add multiple type hints for example when a thing could be a either a file or an input stream?
Any idea how to set up the working directory of a lein shell call?
iku00088: no. type hints set meta-data for the :tag key. The purpose of a type hint is to avoid reflection but if you had multiple options within a type hint, reflection would still be needed to determine type
@borkdude realy cool lib for client and server
(-> (clj-time.core/now)
(clj-time.coerce/to-string)
(clojure.instant/read-instant-timestamp))
Is is the "easiest" to do this? (I will compare with another timestamp)did with-ns
end up anywhere after the contribs were deprecated: https://clojure.github.io/clojure-contrib/with-ns-api.html
@souenzzo perhaps : > The functions equal?, after?, and before? determine the relative position of two DateTime instances:
@souenzzo what's your question?
an instant is simply a java.util.Date
you can use clj-time.coerce/to-date and from-date to translate back and forth to instants
If I understand correctly, he wants to compare a timestamp from datomic against current time
a datestamp from datomic is also just java.util.Date 🙂
it's the lingua franca of java time management
there's .before
Java's got your back
Not sure if this is the right channel for this question, please let me know if there's a better place! We just had a checksum mismatch error with a dependency from maven, and I'm wondering how this can happen? It's consistently the same incorrect shasum, so the artifact seems to have changed, just wondering how this can come about so I know who to talk to/where to look.
@akjetma yeah, it's a transient transit
dependency. I manually added the latest version and excluded the old one
anyone know what caused it to break?
whether this is an issue with the maven central repo or whether library authors are allowed to modify artifacts at the same version number
library authors are not allowed to modify artifacts at the same version, I believe. But I've seen artifact checksum mismatches happen in the past with maven central when sonatype's pushed updates to the servers that hasn't reached all of the mirrors yet (or there was a bug in their updates)
here's a github issue: https://github.com/jboss-javassist/javassist/issues/120
there's some discussion in #datomic
can anyone help me figure out how to create a rettag hinted fn without defn? trying to do it from a macro, but stripped down problem is:
(set! *warn-on-reflection* true)
(defn bam ^TestBean [] (TestBean.))
(.getFooField (bam))
=> nil
(.getFooField ((fn ^TestBean [] (TestBean.))))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> nil
; trying to replicate what defn seems to do internally doesn't work:
(.getFooField ((with-meta (fn [] (TestBean.)) {:rettag TestBean})))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> nil
i thought maybe it was on the var somehow, but that doesn’t seem to be the case:
(meta #'bam)
=>
{:arglists ([]),
:line 1,
:column 1,
:file "/private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj",
:name bam,
:ns #object[clojure.lang.Namespace 0x49661d5d "user”]}
@spieden the meta is actually on the var, but nested: (-> #'bam meta :arglists first meta)
can anyone explain the finer points of when to use volatiles vs. atoms? a typical pattern for me is having a core.async function consumed into a hash-map. values are being stored concurrently from multiple threads, but not updated concurrently under the same keys. i've always used atoms for this, but am wondering if that level of safety is necessary and i could speed it up by wrapping the hash-map in a volatile instead. thoughts?
@sophiago first question as to whether it will speed things up is knowing it’s the bottleneck i suppose
based on the pattern i described i'd sort of doubt it. unless it just adds overhead for each call to swap!
?
i’m actually not clear on where the atom comes in if you’re building up a map within a single async/go-loop style form(?)
i.e. multi threaded producing, single threaded consuming that’s building up a map inside a go block
the hash-map is a global i'm mutating with multiple threads that finish at different times
except in this case i'm not actually mutating each key, hence why i'm wondering whether atomicity is really necessary
is there any interaction between the values of the map? why not have, say, a hash-map of atoms?
@spieden that route really defeats the purpose of how i'm using core.async and would lead to a very significant speed decrease
@sophiago "That’s all there is to using transients, but they have another important constraint: Transients require thread isolation.” from https://clojure.org/reference/transients
rather than go too much down the design path I'll just say "volatiles won't work for your use case"
@bfabry ah ok, thanks. i was having trouble understanding exactly how they differ from atoms in terms of thread safety
@sophiago depends whether the “reduce” phase is the expensive one or the “map” phase. i.e. collecting all the values from expensive operations serially shouldn’t be a bottleneck i wouldn’t think. also you can buffer with channels
> volatiles - there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.
i'm not actually reducing anything at this point though. which also means there is thread isolation
i've used transients in other cases, but definitely don't think they make sense here
actually, since you brought it up and we're discussing mutation...i have an unrelated and more minor question about transients. i noticed you can't use nthnext
with them. is there something similar to that for when i need to access the part of a sequence past a certain index?
I think you'd have to write your own, my guess is it's just not implemented because it wasn't needed, and they want to keep that api relatively small
and @lewis that's the thing...the description makes them seem like they would be suitable for my case since the threads are isolated and i'm trying to optimize speed. but i've always seen atoms used for this pattern so am a bit confused
@bfabry ah ok, thanks. i'll just take a look at the source if i can't think of a way around it
nthnext
is a sequence operation. as @bfabry says the transient API is intentionally small
very vague thought: if I wanted fast thread coordination I wouldn't use an atom, I'd have all the threads pushing changes to a non-blocking queue of some kind and then a consumer thread consuming them. buuuut that's probably very specific to the "type" of concurrency problem I'm used to
@ghadi, but nth
works with transients so i would think nthnext
is ommitted for no reason other than simplicity
it seems that you want thread safety for the values, but not for the map itself. is that the case? if so, I really do think hash-map of atoms would be your best bet here
@bfabry do you say that because swap!
is inherently slow? my issue with that approach would be the complexity involved in labeling the output
swap! is not slow, but it is atomic, so if you have many threads trying to do it at once they will block each other
well, i'm using blocking threads at this point anyway because i think i was overloading the go
macro somehow... it was taking a ridiculous amount of memory and seemed almost as if the gc wasn't keeping up. although i'd have to do more tests to know whether that's actually a general issue with jdk on debian 😕
i think more likely the latter since my repl was slowing to a crawl today with just some simple sequential code and i kept having to call System/gc
directly, which is...bizarre
but otoh, my core.async code is a bit abusive in terms of the number of channels it generates with more complex input
i'm using more channels than actual threads in almost all cases, so i think having them block/park just makes sense regardless
@sophiago channels to threads shouldn’t matter as you’ll do any blocking stuff inside an async/thread
i presume
i've actually never explicitly called async/thread
. from looking at the threadcall
code it seems that might be the solution to my memory management issue? since it calls clojure.lang.Var/resetThreadBindingFrame
?
but i wasn't aware async/thread
did anything other than just using put!
and <!!
on channels. i guess i'll give it a try
hmm, if partial-diff
is the blocking thing you probably want to spin off a bunch of thread
s, collect the channels they return and do something like async/merge
with them(?)
i guess it doesn't make sense to call it inside a channel since it returns a channel...i'd likely have to rewrite this code to take advantage of it then.
Once again, something that took me 3+ days and failed in Java took me 30 minutes in Clojure. whyyy do I never learn
=> (bigint "1e8")
NumberFormatException For input string: "1e8" java.lang.NumberFormatException.forInputString (NumberFormatException.java:65)
=> (bigint 1e8)
100000000N
Is there a parse method for something like “1e8” ?Whats the idiomatic way to tell clojure.java.shel/sh to send the process to the background ?
@sophiago, at the cli the value comes in as “1e8” rather than 1e8. 1e8 is fine, but “1e8” is not. For an integer, a “123” can become 123. But what about a bigint?
if it's just some random script this is the point where I'd use the edn/read-string cannon
https://github.com/clojure/tools.reader/blob/master/src/main/clojure/clojure/tools/reader.clj
As far as I know, nothing will parse 1e8
as an integer. The closest you can get without losing precision is (bigint (BigDecimal. "1e8"))
http://developer.classpath.org/doc/java/math/BigDecimal-source.html Line number 377 is what does it I think for BigDecimal
or if you want it to throw an error if the string comes in with something that shouldn't be an integer: (.toBigIntegerExact (BigDecimal. "1e8"))
(i.e. (.toBigIntegerExact (BigDecimal. "18.5"))
throws an exception, .toBigInterger
will just ignore the error)
Note: java.math.BigInteger
is not the same as Clojure's bigint
.
^ that really confused me at first. it looks odd in the writer, but it's a good thing 🙂
@moxaj regarding my earlier, seems there may be more going on — this is when i set the metadata after the var’s initialized
(.getFooField (map->TestBean {:foo-field 1}))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> 1
(-> #'map->TestBean meta :arglists first meta)
=> {:tag bean_dip.TestBean}
(.getFooField (bam))
=> nil
(-> #'bam meta :arglists first meta)
=> {:tag bean_dip.TestBean}
@spieden afaik, for vars, it has to be on the symbol. For functions, it has to be on the args vector. Also, my map->X isn't type hinted for some reason
see https://github.com/clojure/clojure/blob/clojure-1.9.0-alpha14/src/clj/clojure/core_deftype.clj#L399