This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-11-09
Channels
- # bangalore-clj (1)
- # beginners (158)
- # boot (8)
- # cider (9)
- # cljsjs (9)
- # clojure (169)
- # clojure-austin (1)
- # clojure-denmark (1)
- # clojure-dusseldorf (5)
- # clojure-italy (9)
- # clojure-losangeles (2)
- # clojure-russia (31)
- # clojure-spec (53)
- # clojure-turkiye (1)
- # clojure-uk (56)
- # clojurescript (145)
- # cursive (72)
- # datascript (4)
- # datomic (3)
- # duct (121)
- # events (9)
- # figwheel (1)
- # fulcro (46)
- # graphql (4)
- # hoplon (16)
- # jobs (1)
- # jobs-discuss (4)
- # leiningen (16)
- # lumo (5)
- # off-topic (38)
- # om (1)
- # om-next (5)
- # onyx (104)
- # parinfer (5)
- # re-frame (106)
- # reagent (1)
- # ring-swagger (3)
- # rum (1)
- # shadow-cljs (235)
- # slack-help (4)
- # unrepl (25)
- # yada (9)
Anyone know of a good HTML extracting library? I want something that can use CSS or xpath selectors that can be represented in text so that I can store them in a config somewhere. Tried hickory and it works but it’s not geared toward text based parsing instructions.
this is what enlive does https://github.com/cgrand/enlive
or - at least - I have done this with enlive before
yeah, it templates too, but it can also break down html input and search it via selectors
I've used it alot but have never templated with it
A question about selecting Clojure libraries: Anyone know, offhand, of a site that will let you search a Clojure project by the other projects (with public dependency information eg on github) that have it as a dependency? I'd find this useful when understanding if and how certain libraries are being used.
I feel sure this thing / project does or did exist, but I've forgotten where to find it.
Hi Guys, I've tried to test pmap functionality and came across a weird (to me) behaviour. It would be nice if someone could tell me where am I wrong, or how does it make sence: For my understanding pmap goal is to run a function in parallel over a collection, limiting parallelism. From pmap code I underatand that the desired parallelism level is 2+num-of-cpus In practice i see a different behaviour:
(defn- f1 [delay-num]
(info "delay #" delay-num "started, sleeping")
(Thread/sleep (* 1000 5))
(info "delay #" delay-num "finished")
delay-num)
(defn run []
(let [delays (range 21)
results (pmap f1 delays)]
(info "num of cpus = " (.. Runtime getRuntime availableProcessors))
(info "total is" (reduce + results))))
This shows all 20 delays running in parallel
(I have 4 CPUs and running with clojure 1.8)
Trying to understand it I came up that applying the seq on fs is to "blame".
What do u say?@U7XBWABQT I guess this has something to do with chunked seqs. map
et al process sequences in chunks of 32 elements.
And this is indeed what I observe
(defn- f1 [delay-num]
(Thread/sleep (+ 100 (rand-int 1000)))
(println (java.util.Date.) "delay #" delay-num "started")
(Thread/sleep (* 1000 5))
(println (java.util.Date.) "delay #" delay-num "finished")
delay-num)
(defn run []
(let [delays (range 41)
results (pmap f1 delays)]
(println "num of cpus = " (.. Runtime getRuntime availableProcessors))
(println "total is" (reduce + results))))
(run)
#inst "2017-11-09T09:06:30.493-00:00" delay # 1 started
#inst "2017-11-09T09:06:30.498-00:00" delay # 2 started
#inst "2017-11-09T09:06:30.559-00:00" delay # 13 started
.....
cool - thanx! wasn't aware of that at all. so it seems that num-of-cpus calculation is redundant right? unless you have more than 30 cpus.
Not sure if it has been asked before - I can't find it googling for it - but with Java9 and the modularization of the JDK, I'm wondering if there's a kind of 'minimal JDK for clojure'? The reason for asking is: on small devices, RAM is still limited, and starting a few full blown JVM's takes quite some unnecessary baggage (CORBA, Swing, ...). It would be nice to have a JDK which contains the necessary modules, but not more, that clojure needs to run. I guess it could save up to maybe 30-40MB per JVM. Running 4-5 clojure apps on a 1-2 GB RAM device, well, it certainly makes a difference.
I've never actually used it: but have you looked into things like Wildfly? It's supposed to let you multiple applications in one JVM process. It might be able to help you now instead of waiting for the devs to do it.
Thanks! That's the one.
@kurt-o-sys the JVM already runtime loads classes into memory as you use them. If they end up in RAM that's because something accessed them at runtime. The main RAM killer with Clojure in particular is the assumption that RAM is cheap and throughput is more important, which informs the design of the collections and the way the core functions use those collections.
@cody_slack Yeah... I'm not particularly fond of application servers. I can't talk for all of them, but they still have a hard time if one of the applications is stopped unexpectedly (even in clojure, this happens 🙂 ). Many of them seem to have quite some issues with memory leaks as well. They're not real multi-tenant... I'm still hoping some day, there will be a real multi-tenant JDK. @noisesmith allright... I will check it. In my understanding - and seeing how much RAM 'hallo world' seems to use - the JDK loads the classes of the applications dynamically, but not the JDK itself. I may be completely wrong, though 🙂.
Why even use a JDK if not compiling Java code? Clojure doesn't need one. you can just use a JVM.
right... misphrased my question 🙂
Being able to modularize and customize your own JVM, is there minimal clojure-JVM? - oh, I seem to answer my own question: clojure doesn't need any special JVM, so just JDK9 base (and maybe some other modules), be be ok. Could work. However, without aot, would an an optimized JVM (which can compile clojure on the fly) make sense?
Clojure can't run without the compiler anyway can it?
right.
my hello.java has 21 megs resident (I threw a Thread.sleep so I would have enough time to find it in htop)
that's a lot more than I would see in C, but it's a lot less than clojure.core takes up
for reference
class Main {
public static void main (String[] argv)
throws InterruptedException {
Thread.sleep(100000);
}
}
Right... so, I'll check how much it would take for clojure - later.
my clojure 1.9.0-RC1 with no other deps is 83 megs resident before I do any require calls
pulling in core.async brings it up to 298 megs
this is without lein, boot, or any other tooling
OK, so it's pretty clear what the memory is necessary for
it's not the JVM 🙂. Meaning, no need to start fiddling with it 🙂
this is on
$ java -version
java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
yeah - java isn't the primary thing driving memory usage here - I'm sure java's design and optimization criteria play a part, but it's dwarfed by the memory used by clojure itself to define data structures and functions
FYI: you can get some info about memory usage with (bean (ManagementFactory/getMemoryMXBean))
IllegalAccessException class clojure.core$bean$fn__5975$fn__5976 cannot access a member of class sun.management.MemoryImpl (in module java.management) with modifiers "public" jdk.internal.reflect.Reflection.newIllegalAccessException (Reflection.java:361)
(i'm on java9)
ok, nice, thx a lot. This will do for now 🙂. I guess I'll just have to see if I can work it out to use clojure on small devices.
Clojure's design assumes RAM is cheap (Rich Hickey has talked about this when talking about Clojure's design assumptions)
yeah, I know... but I just seem to have a hard time to not using clojure these days.
there are other functional languages (some of them even in the lisp family) that don't make that assumption
in my experience OCaml is really good about runtime memory usage if you stick to the core language as much as possible
Or I'll may have to see for another lisp... or maybe Erlang or OCaml.
somewhat related: https://ferret-lang.org/
nice... looks pretty cool as well.
I use OCaml for a project on the raspberry pi, I could compile OCaml to machine code ~100 times then run it, in the time it takes Clojure to start up on the pi
(rough rough estimate)
@kurt-o-sys you could also look into clojerl or LFE
@sundbp you have experience with them (they seem to be experimental, whatever that really means)
np... again too many things to try out.
@kurt-o-sys lfe is quite solid/stable
ok... so I may give that a try.
but even tho clojerl is younger it looks more appealing, lfe inherits from a lot of common lisp idioms
yeah... well, I can bet it'll be stable and try clojerl. No harm done in the first months anyway 🙂
@len I prefer to only have a few keys in my mongo document (basically whatever needs indexing) plus an embedded transit string
because this allows using all the core clojure data types (plus whatever else I implement readers / writers for as needed)
monger will auto-convert all map keys to strings going in, and by default will turn those into keywords coming out, which leads to terrible things like {1 :a}
(key is a number) becoming {:1 "a"}
(key is a terrible, illegal keyword)
Thanks @noisesmith we are converting the keys to strings befor insertion and it seems to be working
right, my point directly to your question is that "namespaced keys" can't exist in mongo without you encoding
whether it's a simple assumption like "if there's a / that's namespaced and it should be a keyword" or a more advanced usage like embedding a full encoded clojure data payload (my choice has been the latter - it simplifies a lot of things)
1. In namespace foo.bar , I created a keyword ::apple 2. In namespace snip, I did (:require [foo.bar :as fb]) 3. I'm trying to use fb/:apple ... but am getting an 'no such var error 4. Is there a wa yot refer to a keyword in a namespace via the alias ?
@qqq use ::fb/apple
@seancorfield: works; thanks
The ::
auto-resolves the alias.
Or use spec...
This is from spec use. For the past few days, I've been thinking "getting errors with deep stacks is really annoying -- wouldn't it be great if, for each function, I had two functions: one to check if the input is correct format one to actually evaluate and the check-correct-format function is run before hand" -- then I realize I just rediscovered spec
@bronsa: good point, those all the 'pre-conditions' are checks of the form 'this var has the following shape' -- and the shapes are composable
My project depends on two libraries that both depend on respectively some-lib 1.1.1
and some-lib 1.1.2
. According to lein deps :tree
, some-lib 1.1.1
is pulled in. I thought newest version would win in this case?
Oh, btw, I get a warning when running lein deps :tree
that may explain it in the concrete case:
[bidi "2.1.2"] -> [prismatic/schema "1.1.3"]
overrides
[io.nervous/eulalie "0.6.11-SNAPSHOT"] -> [prismatic/plumbing "0.5.5" :exclusions [org.clojure/clojure]] -> [prismatic/schema "1.1.7"]
So how come bidi
overrides in this case?there is no “newest wins” logic - you can use :exclusions, or ensure that the the dep walking finds the version you want before the other one (including explicitly adding the one you want earlier in the list)
it uses the first one it finds, and it searches from the beginning of your deps vector to the end
@noisesmith I tried putting eulalie
on top without any luck. Maybe the dependency path length is taken into account?
hmm - that could be
Okay. Anyway, thanks for the pointer. I ended up adding an explicit dependency to prismatic/schema "1.1.7"
, and I’ll ask the bidi project to bump the version
Hi. I have a problem with HugSQL used with Postgres (`[com.layerware/hugsql "0.4.7"]`, [org.postgresql/postgresql "42.1.3"]
).
This code:
(try
(db/persist-job-invocation data)
(catch java.sql.SQLException e
(println "=================== caught java.sql.SQLException")
(println (type e))
(println (type (.getCause e)))
(println "==================="))
(catch Exception e
(println "=================== caught Exception")
(println (type e))
(println (type (.getCause e)))
(println "===================")))
throws a useful SQL exception wrapped in a useless java.lang.Exception.
=================== caught Exception
java.lang.Exception
java.sql.BatchUpdateException
===================
How to catch the real wrapped exception? Do I have to unwrap and raise the exception each time I call a HugSQL-generated DB function?@not-raspberry Take a look at the stacktrace and see what code is wrapping the exception. I thought that clojure.java.jdbc
made sure to throw bare java.sql.SQLExceptions
wherever it could, so maybe HugSQL is catching and rewrapping it?
HugSQL does this in the generated DB functions:
(catch Exception e
(adapter/on-exception a e))))))))
The default adapter method:
(on-exception [this exception]
(throw exception)))
Could this cause it?
(I'll reproduce it and get an exception in a sec)> Caused by: java.lang.Exception: Exception in :persist-job-invocation So it looks like that's hugsql.
Found the issue in conman. https://github.com/luminus-framework/conman/issues/44
Downgrading conman helped.
does anyone know if Juxt's Clojure radar is geting an update? [ 2016 versoin: https://juxt.pro/radar.html]
(defn len [x]
(let [x (vary-meta x assoc :tag String)]
(.length x)))
still produces a reflection warning.Either, really - in this case I’m trying to use the let binding to adjust the meta on the fn arg.
it needs to go on the left-hand-side...the way you have it written it is tagged at runtime, not at compile time
I was hoping to be able to calculate the tag value at runtime, but of course that affects compilation.
Lein 2.8.0 switched Aether to a new version which has moved from Sonatype to Eclipse
hi guys
which is better? (-> ll rest butlast butlast) or (subvec ll 1 (-> ll count dec dec))
or if there is something better than both please refer it?
@abdullahibra subvec should perform quite better, assuming II is a vector
thanks guys
another question
what is the best way to for ["hello" "world"] to swap first two characters in this list to get ["ehllo" "owrld"] ?
i can do it in ugly way
like:
(mapv (fn [w] (let [l (vec w) tmp (get l 0)] (-> l (assoc 0 (get l 1)) (assoc 1 tmp) )) ["hello" "world"])
i feel it's ugly way to do this
is there elegant way to this?
that's exactly i have
(->> ["hello" "world"] (mapv (fn [w] (let [l (vec w) tmp (get l 0)] (-> l (assoc 0 (get l 1)) (assoc 1 tmp))))) (mapv #(apply str %)))
(->> ["hello" "world" ""]
(map (fn [w]
(let [split-index (min (count w)
2)]
(str
(apply str (reverse (subs w 0 split-index)))
(subs w split-index))))))
@smith.adriane what makes your version more elegant ?
it depends on who you ask
i want your prespective
I prefer this way of writing it because I think it emphasizes what you’re trying to do
what makes my version is less elegant in your prespective ?
the fact that you’re swapping letters is a little harder to see
and the fact that you’re using assoc and manipulating vectors makes it harder to tell you’re doing string manipulation
another try is
(->> ["hello" "world" ""]
(map (fn [[first-letter second-letter & other-letters]]
(apply str second-letter first-letter other-letters))))
yeah last version is better for me
which matches your problem description a little more closely
the reverse
in my first version is a little closer to the “swap” part of “swap the first two characters”
> what is the best way to for ["hello" "world"] to swap first two characters in this list to get ["ehllo" "owrld"] ? also worth nothing that what you really care about is the string function. dealing with vectors is trivial and a separate concern by reducing problems to their essence, you get to think more clearly 🙂
the other thing I would consider is that in @abdullahibra’s example, the “swapping the first two letters” gets split up into two pieces
first the swap and then turning it back into a string
so even if you were going to do it that way, I would combine both pieces into one function since I think of swapping the first two letters of a string as one operation
like
(->> ["hello" "world"]
(mapv (fn [w]
(let [l (vec w)
tmp (get l 0)]
(-> l
(assoc 0 (get l 1))
(assoc 1 tmp)
(->> (apply str)))))))
the other point I would make is that tmp
isn’t very descriptive
just using good names can help readability a lot imo
(->> ["hello" "world"]
(mapv (fn [w]
(let [l (vec w)
first-letter (first l)
second-letter (second l)]
(-> l
(assoc 0 second-letter)
(assoc 1 first-letter)
(->> (apply str)))))))
@smith.adriane as a suggestion, you can replace that let block with (let [[first-letter second-letter] w] ...)
oh, you need it to be a vector never mind
(let [[first-letter second-letter & trailing] w] (apply str second-letter first-letter trailing))
one of my snippets above is very similar to that one
oh, I see that now
I was trying to show some other examples that were incremental improvements upon his original
another suggestion: (assoc 0 second-letter 1 first-letter)
- assoc is vararg
clojure is fun because you can often come up with a really concise readable version
(let [[first-letter second-letter :as l] (vec w)] ... )
would do what you need I think? (edited to use vec
instead of seq
so l
is a vector)
you don't need seq there
but as he said, he showed that above
Oh, I didn't scroll back to see -- it's a very long thread 😉
I think the main take aways are 1) name intermediate variables in a way that helps convey what you’re trying to do if possible 2) if possible try to write a solution that matches a declarative description of the problem. I think the other changes are less important.
(->> ["hello" "world"]
(mapv #(let [[first-letter second-letter :as word] (vec %)]
(clojure.string/join (assoc word 0 second-letter 1 first-letter)))))
(TIL recently: str/join
uses ""
if you omit the separator)
@seancorfield first time I see this « let in map destructuring » coding style... pretty cool
assoc
ing into a vector kinda bothers me... I think I'd do (into [second-letter first-letter] (nnext word))
(and then you don't need the call to vec
).