This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-02-06
Channels
- # announcements (2)
- # architecture (2)
- # aws (18)
- # babashka (7)
- # beginners (149)
- # bristol-clojurians (4)
- # calva (11)
- # chlorine-clover (1)
- # cider (8)
- # clj-kondo (2)
- # cljdoc (2)
- # cljsrn (2)
- # clojure (186)
- # clojure-canada (3)
- # clojure-europe (3)
- # clojure-gamedev (5)
- # clojure-italy (1)
- # clojure-nl (13)
- # clojure-norway (4)
- # clojure-spec (25)
- # clojure-uk (32)
- # clojurescript (75)
- # core-async (2)
- # cursive (16)
- # data-science (3)
- # datomic (20)
- # docker (1)
- # emacs (26)
- # fulcro (7)
- # graphql (1)
- # incanter (1)
- # leiningen (1)
- # luminus (7)
- # malli (7)
- # mount (11)
- # off-topic (19)
- # pathom (15)
- # re-frame (9)
- # reagent (9)
- # remote-jobs (4)
- # ring-swagger (4)
- # shadow-cljs (63)
- # spacemacs (11)
- # sql (2)
- # vscode (7)
Not really associated with Clojure, but why would docker be useful for a JVM project? You already have all the dependencies inside your jar file?
That might not be entirely true because JVM might assume certain os level resources/packages exist (such as graphics). And then all the usual arguments applicable to all ecosystems - unified development environment, better isolation and utilization OS resources, etc
We use dockerised JVM applications a lot in my place of work. We like them because they package everything up, are easy to scale up/down, and importantly are reproduceable. From an ops perspective, they love it as all they need to do is provision a swarm - no need to worry about upgrading application servers or having a JVM up-to-date (we have multple docker containers with different JVM versions running, due to both legacy and future stuff)
I too was in the camp of "You have a JAR, why on earth would you containerise stuff?" I then had my ah-ha! moment...
Indeed I write a lot of Go at work - and there the runtime is baked into resulting binary. If one compiles it statically it does not even use libc on the target host. It's still nice to use docker for deployment because it provides certain amount of isolation (e.g. no one will ssh in and delete files on you). Now this is applicable for larger organizations. Clojure or Go not sure why I would use docker for small things and for sure not for development.
For small things, yes, the JVM's container is sufficient. Sometimes you need something that fits deployment, like when deploying to kubernetes, and it likes docker images. Like all tools, use them when useful or required, and not a moment before.
Thanks a lot for all the insights!
What am I not getting when I often find it hard to add something to the end of a seq? So, I can use a vector
and conj
to it. But what if I do not want to use a vector?
Sequences are retained by their head. I am GUESSING the tail is not maintained and for good reason too, as sequences can be lazy. Adding to the tail of the sequence is thus really complex.
You have to hold onto the things you want added to the end and then wait for the sequence to be realized. Remember that a sequence is not a list. It is just a representation of a, for the lack of a better word, sequence. Think iterator in Java where hasNext and next are calculated on the fly.
A vector on the other hand is already fully realized. So you can potentially add to both end. But adding to the head is a costlier operation USUALLY. Not sure about the cost in Clojure's persistent collections.
You can also merge
things into a map or set
That doesn't put it at the end though. If I want to put :baz
at the end of (:foo :bar)
to get (:foo :bar :baz)
...
Oh, are you specifically wanting to put something on the end of a list?
(concat (list 1 2 3 4) (list 5))
=> (1 2 3 4 5)
does what you want, but if you're frequently putting things on the beginning of a sequence, it's cheaper to do so with conj and a vectorFYI
(ins)user=> (concat '(1 2 3 4) 5)
Error printing return value (IllegalArgumentException) at clojure.lang.RT/seqFrom (RT.java:557).
Don't know how to create ISeq from: java.lang.Long
(ins)(1 2 3 user=> (concat '(1 2 3 4) [5])
(1 2 3 4 5)
also in clojure we only use ` or ' to make lists if we are constructing code to be compiled (eg a macro or input to eval) otherwise just use []
Long ago irrelevant but I made those changes. I didn't know that about the backtick, there was no such stipulation in common lisp! 👍
you can use tick / backtick for making lists, it's just less likely to be useful in clojure as we have vector literals and vectors work as well or better than lists in most cases where we'd use literals
(conj [1 2 3 4] 5)
=> [1 2 3 4 5]
Thanks! I want to put it at the and of a seq, I think. Retaining the possibility to return a lazy seq. A vector will realize the whole thing, right?
I could be wrong but I think the laziness has less to do with list/vector and more to do with the function that returns your collection. Map is lazy, for instance
That's my concern. I often have a sequence in that I want to transform to a sequence out. But it is not always a straight mapping.
Be careful with concat
: it's lazy, so if you repeatedly call it you can effectively build up a stack of calls when it is finally realized.
With ”not a straight mapping” I mean that the process should not produce one thing for each thing in the input sequence. So I consume the input sequence from the start and step by step I build up a resulting sequence. I find myself often doing this.
I have never once had the need to add something to the end of a seq, so I have to question the question. What are you actually trying to do?
Neat. Yeah, so then I just need to understand @seancorfield’s warning. 😃
> the process should not produce one thing for each thing in the input sequence
user=> (map (fn [x] (range x)) [0 1 2])
(() (0) (0 1))
user=> (mapcat (fn [x] (range x)) [0 1 2])
(0 0 1)
Not sure I am following. But maybe I can be a bit clearer by sharing the concrete example I am struggling with right now. I'm on the run-length encode/decode step on http://exercism.io. The tests all pass and I want to submit my solution, but this vector
thing iches. Look at this decoding step:
(partition-by alphabet "12AB3CD4E")
;; => ((\1 \2) (\A) (\B) (\3) (\C) (\D) (\4) (\E))
(->> (partition-by alphabet "12AB3CD4E")
(parse-run-sequence))
;; => [[12 "A"] [1 "B"] [3 "C"] [1 "D"] [4 "E"]]
So it is in parse-run-sequence
where I build up that vector that I can then just (mapcat #(apply repeat %)
, and then (apply str)
and I get the uncompressed text. But inside there I build up the result as a vector, because I can't figure out a clean way to build it as a seq. In this particular case the input sequence builds one thing from either one or two things. Which to me is ”not a straight mapping”. I'd like to see if I can build a lazy solution before I submit it to my mentor. Maybe mapcat
is the way, but I don't see it, if so...One bit of food for thought: consider after the partition-by
call to do (partition 2 1 ...)
on the result, and then process that:
user=> (partition 2 1 '((\1 \2) (\A) (\B) (\3) (\C) (\D) (\4) (\E)))
(((\1 \2) (\A)) ((\A) (\B)) ((\B) (\3)) ((\3) (\C)) ((\C) (\D)) ((\D) (\4)) ((\4) (\E)))
The first element of processing that is ((\1 \2) (\A))
, which would become one thing in the output, e.g. [12 "A"]
. The second thing would become just [1 "B"]
, because the letter \A
is not a sequence of digits.
Ah, yes, boundary conditions. Doing (partition 2 1 ...)
on a sequence with a dummy element added at the beginning or end might help with that.
Wonderful!
(def alphabet
(let [upper (range (int \A)
(inc (int \Z)))
lower (range (int \a)
(inc (int \z)))]
(-> #{\space}
(into (map char upper))
(into (map char lower)))))
(defn run-length-decode
"decodes a run-length-encoded string"
[encoded-text]
(let [decode-char (fn [[head tail]]
(if (alphabet (first head))
[1 (str (first tail))]
[(Integer/parseInt (apply str head)) (apply str tail)]))]
;; "d" is for "dummy".
;; Prepending it at the start for `(partition 2 1)` to chew on
(->> (partition-by alphabet (str "d" encoded-text))
(partition 2 1)
(filter #(alphabet (first (second %))))
(map decode-char)
(mapcat #(apply repeat %))
(apply str))))
@U0ETXRFEW You can avoid a bunch of that work by just using regexps with capture groups. #"(\d*)([^\d])"
should do just fine. re-seq
returns capture groups BTW.
Hello dear mentor! 😍 I actually did it with a regexp as my first try. But didn't realize about re-seq so scrapped it. Can I make a lazy implementation that way?
@U0ETXRFEW absolutely! re-seq
is lazy.
The third element ((\B) (\3))
would become nothing in the output (e.g. using mapcat
with a function that returns an empty sequence) because it is a letter followed by a digit sequence.
(partition 2 1 ...)
can be a nice trick that lets you write later code that considers all consecutive pairs of a sequence, without having to explicitly remember state from one step to the next.
Not sure how much spamming is socially accepted here, but I'm trying to get accustomed to more tricks in Clojure (seems to have a wider range of built in functions for sequences than Python e.g. and so it's sometimes hard to know what you're looking for 🙂 ) I need a function that replaces the first 0 in a vector with a new value, which I implemented as follows:
(defn index-first-zero
[v]
(first
(first
(filter
(fn [[idx val]] (= 0.0 val))
(map-indexed vector v)))))
(defn replace-first-zero
[v new-value]
(let [idx-first-zero (index-first-zero v)]
(assoc v idx-first-zero new-value)))
Are there better ways for these kind of tasks than actually searching for the index first? I.e. update on the go somehow?a small thing - you can replace (first (first ...))
with (ffirst ...)
and always use (zero? x)
instead of (= 0.0 x)
- it catches some corner cases
Thanks! Changed it to:
(defn index-first-zero
[v]
(->> v
(map-indexed vector)
(filter (fn [[idx val]] (zero? val)))
(ffirst)))
this is how I'd write it
user=> (defn first-zero-index ([c] (first-zero-index c 0)) ([[h & t] i] (if (zero? h) i (recur t (inc i)))))
#'user/first-zero-index
user=> (first-zero-index [0 1 2 3])
0
user=> (first-zero-index [1.1 2 3 0.0 5.0])
3
that said - there's also interop
user=> (.indexOf [3 2 1 0.0 -1.0] 0.0)
3
Interesting! No Java experience, so interop isn't the first thing on my mind. It's a lot shorter here though. Also 6 times slower on the small vectors I'm using it. (not that 700ns vs 4.5us really matters)
I think interop might be faster than the clojure version if you type-hint the first arg to .indexOf
there's also a concise version with reduce / reduced, but you need extra logic to detect the case where the index isn't found
(ins)user=> (reduce (fn [i n] (if (zero? n) (reduced i) (inc i))) [1 2 3 0 4])
3
(cmd)user=> (reduce (fn [i n] (if (zero? n) (reduced i) (inc i))) [1 2 3 4])
4
though my loop version throws NPE for that case, so :D
@UTQEPUEH4 Didn't get to type-hinting yet, but good to know 🙂
like this: (defn rep0 [^clojure.lang.PersistentVector v new] (assoc v (.indexOf v 0) new)
you can use update
or update-in
on a vector
oh, right, sorry
yeah, this was mentioned in the thread under the initial question - funny enough without a type hint it's slower than naiive clojure
np, our discipline about threads under questions vs. main channel is definitely very loose around here :D
The dot in indexOf just means it's a java function, right?
it means it's a method, yes
java has a Function class which is a whole other unrelated can of worms :D
methods belong to objects, functions are objects that do something
Hahaha, nice. Does it generally operate like [4 5 6 0].indexOf(0)?
right, that's the java translation
awesome, thanks.
except [4 5 6 0] in java isn't a vector
Right, presumably it gets cast to an array?
indexOf gets implemented in one of the classes PersistentVector inherits https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java
I hope it doesn't do this via a type conversion, that would be pretty wasteful
vectors are java collections
it comes in via ApersistentVector https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/APersistentVector.java#L186
Cool. Lol, good to know since those methods come up once in a while
I am getting Not supported: class java.util.regex.Pattern
when using clj-transit and calling (transit/write writer edn)
I haven't found much googling around. So I am not sure if this is just a case of writing a protocol to support regex or if its much more complicated than that.
@mario.cordova.862 This means that the content you are trying to serialize contains a regular expression object
If you use transit to pass values between different language environments, e.g. JVM on one side, JavaScript run time on the other, then there is 0 guarantee that your regexes will behave the same in the other runtime where they were not written/tested. There are known common cases where they do have the same matching behavior, and known cases where they differ.
If you really, really know that you want to get regex pattern strings passed between such places, or you know the language runtimes are the same on all ends in your application, I would guess transit provides ways to enable their serialization? e.g. they could be turned into string plus tag on sender side, and reconstituted into a regex at the receiver.
transit accepts maps of custom serialize / deserialize functions. I actually like the fact it uses first class args for this instead of a global state
We can try to tell you 'here be dragons', but if you bring on the dragons, we can't stop you 🙂
also edn is a serialization format, and while it is fairly common practice to refer to clojure values as edn, it isn't unless it is serialized
hmm seems like its a bigger problem than its worth to look into. I pulled the Clograms https://github.com/jpmonettas/clograms project from github and tried to create a graph of my project but when I ran the program it spat out that error. I thought it could find a quick work around
Perhaps the fact that my project contains both clj/cljs source code is whats causing this issue
@mario.cordova.862 custom transit serializers/deserializers are not hard to write, it would be a good beginner PR if you felt up to it
either with regexes, or all the common literals that clojure / cljs support but transit doesn't as a stretch goal
assuming it's not just a bug in that project or combination of project + specific source code
it's here, they aren't adding or chaning any transit writers https://github.com/jpmonettas/clograms/blob/master/src/clj/clograms/core.clj#L39
so it's an easy fix
@mario.cordova.862 some simple examples of transit serializers / deserialiizers, https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot/handlers.cljc and how they get used https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot.cljc#L83 https://github.com/noisesmith/poirot/blob/master/src/org/noisesmith/poirot.cljc#L107
Does anyone know how to de-structure a nested vector into a map? For instance ["uno" ["dos" "tres" "cuatro" "cinco"]] --> {:uno {:dos "tres" :cuatro "cinco"}} The keys (uno, dos, and cuatro) will be different everytime and are unknown ahead of time.
destructuring is about creating bindings from a shape. it's not really a way to create new shapes of data
or another way of phrasing that comment is that the term "destructuring" has a particular technical meaning in the Clojure language, and it would be more clear if perhaps you asked how to transform the nested vector into the map you gave as examples.
@noisesmith Was able to get past the error using the examples you linked, :thumbsup:
Not sure what you mean by shape, but the 'format' of the vector will be the same, where as [key [key value key value.....key value]] if that helps?
Are you looking for Clojure code that would handle not just that example, but cases nested even 3 or more levels deep?
And it should handle arbitrary even numbers of elements in every vector?
I think for now, just that example... I haven't run into an instance (yet) where it goes more than 2 levels deep. And yes, it will always be an even number of elements (strings) in every vector.
user=> (clojure.walk/postwalk #(if (vector? %) (apply hash-map (map-indexed (fn [i k-or-v] (if (odd? i) k-or-v (keyword k-or-v))) %)) %) ["uno" ["dos" "tres" "cuatro" "cinco"]])
{:uno {:cuatro "cinco", :dos "tres"}}
These are the outputs of the carmine package for Redis streams using XRANGE. Everything is added to the stream as a map, but when you read it back, it's a nested vector.
You sir, are amazing... thank you! I'd been trying different things with walk for hours!
this problem made me wonder if there's a simple way to un-interleave, seeing as there is an interleave
. is there such a thing?
It seems hard to imagine (if not impossible) to write a lazy Clojure function that would be the inverse of interleave
but I may be lacking imagination/coffee right now. I am not aware of a single Clojure built in function that would undo interleave
, but certainly one could compose a few that would do it.
e.g. this might be close to what you are thinking: (defn uninterleave [n coll] (map (fn [i] (take-nth n (drop i coll))) (range n)))
It isn't written with optimal efficiency in mind, but brevity. e.g. it traverses coll
n times.
If the colls you interleave are not all the same length, then interleave
is not reversable, since it drops any elements that don’t have corresponding elements in all the other colls, i.e.
(interleave [1 2 3]
[4 6]
[7 8 9 10])
=> (1 4 7 2 6 8)
isn't it just partition then collect nth 0, nth 1, nth 2 etc from those?
more than one way to skin that cat, yes.
I am creating a same machine web app which uses clj for computation and cljs for web-interface/visualization stuff. I'd like to be able to send messages to the running server from the command line. I'd like it to be easy for users to do. The only way I can think of is: curl -H "Content-Type: application/json" -X POST -d '{"id":1}' localhost:3000/json
but this is silly verbose and ugly. What is the way to send commands to a running clojure program? This should be possible for non-programmers, so I'd like to avoid opening a REPL. I guess I could write a tool which converts simple commands into the curl
calls I need, but this is still likely not a good way to do it.
Let's just say I want to send arbitrary data to the running program. These will be interpreted as commands and their arguments 🙂
the simplest thing is to start a socket repl via java system property:
-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
that starts a server on port 555, you can use nc or curl or telnet or whatever to send it stringsI could be misinterpreting what you want - that literally gives you a repl
Okay, then I guess the best way to send data to a running program without entering a repl is to use curl. I will write a small cl program that converts my simple command-line calls $ re-make update f1.gz f2.gz
into ugly curl calls behind the scenesl and sends them to my running server.
curl is something most people can use and is a fairly standard way to interact with a web service from a command line (look at any swagger output) ... if you are already having them send commands. It sound like you want a user interface though (for "non programmers").
Hi 🙂 If I send-off
a long running action to an agent and my await-for
returns after my lengthy specified time-out, that action is still running happily in its thread. Is there a way for my thread (the one that did the send-off
and the await-for
) to reach out and forcibly kill the action or the action’s thread? Or is it the case that a rogue action with an infinite loop can claim a thread permanently for itself?
In general, I have read that cancelling/killing a JVM thread running arbitrary code can have all kinds of unwanted effects, e.g. if it was holding locks when you killed it, etc. A clean way to stop it is if somewhere in its inner loop(s) it checks for a request from outside for it to stop, but if you forget to make such checks in some loop that goes infinite, then it obviously won't be cooperating there.
certain methods (especially ones that are on an OS level polling for IO) can be "cancelled" (which is different from a forcible kill, it allows elegant shutdown)
which reminds me of a nice bit of poetry hidden in clojure.core
user=> (doc future-cancel)
-------------------------
clojure.core/future-cancel
([f])
Cancels the future, if possible.
nil