Fork me on GitHub
#beginners
<
2020-02-06
>
David Pham12:02:49

Not really associated with Clojure, but why would docker be useful for a JVM project? You already have all the dependencies inside your jar file?

jumar12:02:32

That might not be entirely true because JVM might assume certain os level resources/packages exist (such as graphics). And then all the usual arguments applicable to all ecosystems - unified development environment, better isolation and utilization OS resources, etc

Kamuela13:02:03

I’d say it just makes deployment all the more easy and standardized

dharrigan13:02:33

We use dockerised JVM applications a lot in my place of work. We like them because they package everything up, are easy to scale up/down, and importantly are reproduceable. From an ops perspective, they love it as all they need to do is provision a swarm - no need to worry about upgrading application servers or having a JVM up-to-date (we have multple docker containers with different JVM versions running, due to both legacy and future stuff)

dharrigan13:02:09

I too was in the camp of "You have a JAR, why on earth would you containerise stuff?" I then had my ah-ha! moment...

jakubl14:02:19

Indeed I write a lot of Go at work - and there the runtime is baked into resulting binary. If one compiles it statically it does not even use libc on the target host. It's still nice to use docker for deployment because it provides certain amount of isolation (e.g. no one will ssh in and delete files on you). Now this is applicable for larger organizations. Clojure or Go not sure why I would use docker for small things and for sure not for development.

nate14:02:06

For small things, yes, the JVM's container is sufficient. Sometimes you need something that fits deployment, like when deploying to kubernetes, and it likes docker images. Like all tools, use them when useful or required, and not a moment before.

David Pham17:02:55

Thanks a lot for all the insights!

pez16:02:34

What am I not getting when I often find it hard to add something to the end of a seq? So, I can use a vector and conj to it. But what if I do not want to use a vector?

hindol19:02:13

Sequences are retained by their head. I am GUESSING the tail is not maintained and for good reason too, as sequences can be lazy. Adding to the tail of the sequence is thus really complex.

hindol19:02:38

You have to hold onto the things you want added to the end and then wait for the sequence to be realized. Remember that a sequence is not a list. It is just a representation of a, for the lack of a better word, sequence. Think iterator in Java where hasNext and next are calculated on the fly.

hindol19:02:34

A vector on the other hand is already fully realized. So you can potentially add to both end. But adding to the head is a costlier operation USUALLY. Not sure about the cost in Clojure's persistent collections.

pez21:02:36

Thanks a million! I'll try to remember this.

Michael J Dorian16:02:41

You can also merge things into a map or set

pez16:02:34

That doesn't put it at the end though. If I want to put :baz at the end of (:foo :bar) to get (:foo :bar :baz) ...

Michael J Dorian16:02:09

Oh, are you specifically wanting to put something on the end of a list?

Michael J Dorian16:02:24

(conj [1 2 3 4] 5) => [1 2 3 4 5]

pez16:02:48

Thanks! I want to put it at the and of a seq, I think. Retaining the possibility to return a lazy seq. A vector will realize the whole thing, right?

pez16:02:00

(I am not quite rid of my pythonic ways, where I would just yield stuff.)

Michael J Dorian16:02:41

I could be wrong but I think the laziness has less to do with list/vector and more to do with the function that returns your collection. Map is lazy, for instance

bronsa16:02:23

well, converting a lazy seq to a vector will have to realize the entire lazy-seq

bronsa16:02:28

vectors can't be lazy

bronsa16:02:55

you can lazily map over a vector, but that's not the same thing

pez16:02:39

That's my concern. I often have a sequence in that I want to transform to a sequence out. But it is not always a straight mapping.

pez16:02:31

Would concat work for that purpuse?

bronsa16:02:59

sounds like you may want mapcat ?

seancorfield16:02:19

Be careful with concat: it's lazy, so if you repeatedly call it you can effectively build up a stack of calls when it is finally realized.

bronsa16:02:31

I'm not really sure I understand what you meant last

pez16:02:04

With ”not a straight mapping” I mean that the process should not produce one thing for each thing in the input sequence. So I consume the input sequence from the start and step by step I build up a resulting sequence. I find myself often doing this.

leonoel16:02:03

you can abuse templating syntax, e.g (@the-seq the-item)`

pez16:02:48

Hehe. What will that expand to?

leonoel17:02:18

a combination of seq, concat and list

ghadi17:02:52

I have never once had the need to add something to the end of a seq, so I have to question the question. What are you actually trying to do?

✔️ 5
pez17:02:11

Neat. Yeah, so then I just need to understand @seancorfield’s warning. 😃

bronsa17:02:35

> the process should not produce one thing for each thing in the input sequence

user=> (map (fn [x] (range x)) [0 1 2])
(() (0) (0 1))
user=> (mapcat (fn [x] (range x)) [0 1 2])
(0 0 1)

bronsa17:02:46

with mapcat you can produce 0, 1 or n for every input

pez17:02:57

Not sure I am following. But maybe I can be a bit clearer by sharing the concrete example I am struggling with right now. I'm on the run-length encode/decode step on http://exercism.io. The tests all pass and I want to submit my solution, but this vector thing iches. Look at this decoding step:

(partition-by alphabet "12AB3CD4E")
  ;; => ((\1 \2) (\A) (\B) (\3) (\C) (\D) (\4) (\E))
  (->> (partition-by alphabet "12AB3CD4E")
       (parse-run-sequence))
  ;; => [[12 "A"] [1 "B"] [3 "C"] [1 "D"] [4 "E"]]
So it is in parse-run-sequence where I build up that vector that I can then just (mapcat #(apply repeat %), and then (apply str) and I get the uncompressed text. But inside there I build up the result as a vector, because I can't figure out a clean way to build it as a seq. In this particular case the input sequence builds one thing from either one or two things. Which to me is ”not a straight mapping”. I'd like to see if I can build a lazy solution before I submit it to my mentor. Maybe mapcat is the way, but I don't see it, if so...

andy.fingerhut18:02:00

One bit of food for thought: consider after the partition-by call to do (partition 2 1 ...) on the result, and then process that:

andy.fingerhut18:02:07

user=> (partition 2 1 '((\1 \2) (\A) (\B) (\3) (\C) (\D) (\4) (\E)))
(((\1 \2) (\A)) ((\A) (\B)) ((\B) (\3)) ((\3) (\C)) ((\C) (\D)) ((\D) (\4)) ((\4) (\E)))

andy.fingerhut18:02:27

The first element of processing that is ((\1 \2) (\A)) , which would become one thing in the output, e.g. [12 "A"] . The second thing would become just [1 "B"] , because the letter \A is not a sequence of digits.

metal 1
pez21:02:19

Wow, yes. Very cool!

pez22:02:35

Almost. It fails for input like "XYZ" , => (((\X) (\Y)) ((\Y) (\Z)))

andy.fingerhut22:02:10

Ah, yes, boundary conditions. Doing (partition 2 1 ...) on a sequence with a dummy element added at the beginning or end might help with that.

pez22:02:27

Wonderful!

(def alphabet
  (let [upper (range (int \A)
                     (inc (int \Z)))
        lower (range (int \a)
                     (inc (int \z)))]
    (-> #{\space}
        (into (map char upper))
        (into (map char lower)))))

(defn run-length-decode
  "decodes a run-length-encoded string"
  [encoded-text]
  (let [decode-char (fn [[head tail]]
                      (if (alphabet (first head))
                        [1 (str (first tail))]
                        [(Integer/parseInt (apply str head)) (apply str tail)]))]
    ;; "d" is for "dummy".
    ;; Prepending it at the start for `(partition 2 1)` to chew on
    (->> (partition-by alphabet (str "d" encoded-text))
         (partition 2 1)
         (filter #(alphabet (first (second %))))
         (map decode-char)
         (mapcat #(apply repeat %))
         (apply str))))

jaihindhreddy15:02:13

@U0ETXRFEW You can avoid a bunch of that work by just using regexps with capture groups. #"(\d*)([^\d])" should do just fine. re-seq returns capture groups BTW.

pez16:02:50

Hello dear mentor! 😍 I actually did it with a regexp as my first try. But didn't realize about re-seq so scrapped it. Can I make a lazy implementation that way?

jaihindhreddy18:02:58

@U0ETXRFEW absolutely! re-seq is lazy.

andy.fingerhut18:02:31

The third element ((\B) (\3)) would become nothing in the output (e.g. using mapcat with a function that returns an empty sequence) because it is a letter followed by a digit sequence.

andy.fingerhut18:02:29

(partition 2 1 ...) can be a nice trick that lets you write later code that considers all consecutive pairs of a sequence, without having to explicitly remember state from one step to the next.

Frederik18:02:40

Not sure how much spamming is socially accepted here, but I'm trying to get accustomed to more tricks in Clojure (seems to have a wider range of built in functions for sequences than Python e.g. and so it's sometimes hard to know what you're looking for 🙂 ) I need a function that replaces the first 0 in a vector with a new value, which I implemented as follows:

(defn index-first-zero
  [v]
  (first 
   (first 
    (filter 
     (fn [[idx val]] (= 0.0 val)) 
     (map-indexed vector v)))))

(defn replace-first-zero
  [v new-value]
  (let [idx-first-zero (index-first-zero v)]
    (assoc v idx-first-zero new-value)))
Are there better ways for these kind of tasks than actually searching for the index first? I.e. update on the go somehow?

noisesmith18:02:04

a small thing - you can replace (first (first ...)) with (ffirst ...)

noisesmith18:02:16

and always use (zero? x) instead of (= 0.0 x) - it catches some corner cases

Frederik18:02:25

Thanks! Changed it to:

(defn index-first-zero
  [v]
  (->> v
       (map-indexed vector)
       (filter (fn [[idx val]] (zero? val)))
       (ffirst)))

noisesmith18:02:46

this is how I'd write it

user=> (defn first-zero-index ([c] (first-zero-index c 0)) ([[h & t] i] (if (zero? h) i (recur t (inc i)))))
#'user/first-zero-index
user=> (first-zero-index [0 1 2 3])
0
user=> (first-zero-index [1.1 2 3 0.0 5.0])
3

noisesmith18:02:18

that said - there's also interop

user=> (.indexOf [3 2 1 0.0 -1.0] 0.0)
3

Frederik18:02:18

Interesting! No Java experience, so interop isn't the first thing on my mind. It's a lot shorter here though. Also 6 times slower on the small vectors I'm using it. (not that 700ns vs 4.5us really matters)

Frederik18:02:53

Your implementation definitely looks a lot more clojuric.

jsn18:02:12

I think interop might be faster than the clojure version if you type-hint the first arg to .indexOf

noisesmith18:02:23

there's also a concise version with reduce / reduced, but you need extra logic to detect the case where the index isn't found

(ins)user=> (reduce (fn [i n] (if (zero? n) (reduced i) (inc i))) [1 2 3 0 4])
3
(cmd)user=> (reduce (fn [i n] (if (zero? n) (reduced i) (inc i))) [1 2 3 4])
4

noisesmith19:02:01

though my loop version throws NPE for that case, so :D

Frederik19:02:02

I prefer it crashing and burning tbf

Frederik19:02:26

@UTQEPUEH4 Didn't get to type-hinting yet, but good to know 🙂

jsn19:02:26

like this: (defn rep0 [^clojure.lang.PersistentVector v new] (assoc v (.indexOf v 0) new)

1
jsn19:02:48

(this also preserves your original exception-on-not-found semantics)

alexmiller18:02:30

you can use update or update-in on a vector

jsn18:02:06

no, I think they need to find the index of the first 0, not just update at 0 index

alexmiller18:02:16

oh, right, sorry

borkdude19:02:27

(.indexOf [4 5 6 0] 0) ;;=> 3

noisesmith19:02:33

yeah, this was mentioned in the thread under the initial question - funny enough without a type hint it's slower than naiive clojure

borkdude19:02:49

ah ok, sorry

noisesmith19:02:26

np, our discipline about threads under questions vs. main channel is definitely very loose around here :D

borkdude19:02:28

not sure if that's a public API

Michael J Dorian19:02:18

The dot in indexOf just means it's a java function, right?

noisesmith19:02:39

it means it's a method, yes

noisesmith19:02:02

java has a Function class which is a whole other unrelated can of worms :D

noisesmith19:02:21

methods belong to objects, functions are objects that do something

Michael J Dorian19:02:40

Hahaha, nice. Does it generally operate like [4 5 6 0].indexOf(0)?

noisesmith19:02:54

right, that's the java translation

Michael J Dorian19:02:02

awesome, thanks.

noisesmith19:02:15

except [4 5 6 0] in java isn't a vector

Michael J Dorian19:02:32

Right, presumably it gets cast to an array?

noisesmith19:02:18

indexOf gets implemented in one of the classes PersistentVector inherits https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java

noisesmith19:02:45

I hope it doesn't do this via a type conversion, that would be pretty wasteful

alexmiller19:02:13

vectors are java collections

Michael J Dorian19:02:15

Cool. Lol, good to know since those methods come up once in a while

Mario C.19:02:41

I am getting Not supported: class java.util.regex.Pattern when using clj-transit and calling (transit/write writer edn)

Mario C.19:02:33

I haven't found much googling around. So I am not sure if this is just a case of writing a protocol to support regex or if its much more complicated than that.

borkdude19:02:49

@mario.cordova.862 This means that the content you are trying to serialize contains a regular expression object

borkdude19:02:04

and there is no default implementation for that around

borkdude19:02:47

I'm assuming trying to serialize a regex to transit is a mistake

andy.fingerhut19:02:04

If you use transit to pass values between different language environments, e.g. JVM on one side, JavaScript run time on the other, then there is 0 guarantee that your regexes will behave the same in the other runtime where they were not written/tested. There are known common cases where they do have the same matching behavior, and known cases where they differ.

hiredman19:02:06

the edn spec, such that it is, doesn't include regexes

andy.fingerhut19:02:23

If you really, really know that you want to get regex pattern strings passed between such places, or you know the language runtimes are the same on all ends in your application, I would guess transit provides ways to enable their serialization? e.g. they could be turned into string plus tag on sender side, and reconstituted into a regex at the receiver.

noisesmith19:02:57

transit accepts maps of custom serialize / deserialize functions. I actually like the fact it uses first class args for this instead of a global state

andy.fingerhut19:02:04

We can try to tell you 'here be dragons', but if you bring on the dragons, we can't stop you 🙂

hiredman19:02:51

also edn is a serialization format, and while it is fairly common practice to refer to clojure values as edn, it isn't unless it is serialized

Mario C.19:02:39

hmm seems like its a bigger problem than its worth to look into. I pulled the Clograms https://github.com/jpmonettas/clograms project from github and tried to create a graph of my project but when I ran the program it spat out that error. I thought it could find a quick work around

Mario C.19:02:36

Perhaps the fact that my project contains both clj/cljs source code is whats causing this issue

borkdude19:02:19

maybe post an issue at that project with a minimal repro

Mario C.19:02:45

Yea I was thinking of doing that

noisesmith19:02:31

@mario.cordova.862 custom transit serializers/deserializers are not hard to write, it would be a good beginner PR if you felt up to it

noisesmith19:02:01

either with regexes, or all the common literals that clojure / cljs support but transit doesn't as a stretch goal

borkdude19:02:07

assuming it's not just a bug in that project or combination of project + specific source code

noisesmith19:02:09

so it's an easy fix

Mario C.19:02:15

I gotta read up on it a little bit and find some time

Andrew Brock20:02:32

Does anyone know how to de-structure a nested vector into a map? For instance ["uno" ["dos" "tres" "cuatro" "cinco"]] --> {:uno {:dos "tres" :cuatro "cinco"}} The keys (uno, dos, and cuatro) will be different everytime and are unknown ahead of time.

dpsutton20:02:01

destructuring is about creating bindings from a shape. it's not really a way to create new shapes of data

andy.fingerhut20:02:07

or another way of phrasing that comment is that the term "destructuring" has a particular technical meaning in the Clojure language, and it would be more clear if perhaps you asked how to transform the nested vector into the map you gave as examples.

Mario C.20:02:17

@noisesmith Was able to get past the error using the examples you linked, :thumbsup:

👍 1
Andrew Brock20:02:29

Not sure what you mean by shape, but the 'format' of the vector will be the same, where as [key [key value key value.....key value]] if that helps?

Mario C.20:02:37

Now just need to actually make it work lol

andy.fingerhut20:02:55

Are you looking for Clojure code that would handle not just that example, but cases nested even 3 or more levels deep?

andy.fingerhut20:02:19

And it should handle arbitrary even numbers of elements in every vector?

Andrew Brock20:02:15

I think for now, just that example... I haven't run into an instance (yet) where it goes more than 2 levels deep. And yes, it will always be an even number of elements (strings) in every vector.

bfabry20:02:27

user=> (clojure.walk/postwalk #(if (vector? %) (apply hash-map (map-indexed (fn [i k-or-v] (if (odd? i) k-or-v (keyword k-or-v))) %)) %) ["uno" ["dos" "tres" "cuatro" "cinco"]])
{:uno {:cuatro "cinco", :dos "tres"}}

Andrew Brock20:02:09

These are the outputs of the carmine package for Redis streams using XRANGE. Everything is added to the stream as a map, but when you read it back, it's a nested vector.

Andrew Brock20:02:08

You sir, are amazing... thank you! I'd been trying different things with walk for hours!

bfabry20:02:23

you're welcome

bfabry20:02:11

this problem made me wonder if there's a simple way to un-interleave, seeing as there is an interleave. is there such a thing?

andy.fingerhut20:02:41

It seems hard to imagine (if not impossible) to write a lazy Clojure function that would be the inverse of interleave

andy.fingerhut20:02:49

but I may be lacking imagination/coffee right now. I am not aware of a single Clojure built in function that would undo interleave , but certainly one could compose a few that would do it.

andy.fingerhut21:02:02

e.g. this might be close to what you are thinking: (defn uninterleave [n coll] (map (fn [i] (take-nth n (drop i coll))) (range n)))

andy.fingerhut21:02:12

It isn't written with optimal efficiency in mind, but brevity. e.g. it traverses coll n times.

manutter5121:02:13

If the colls you interleave are not all the same length, then interleave is not reversable, since it drops any elements that don’t have corresponding elements in all the other colls, i.e.

(interleave [1 2 3]
            [4 6]
            [7 8 9 10])
=> (1 4 7 2 6 8)

alexmiller21:02:54

isn't it just partition then collect nth 0, nth 1, nth 2 etc from those?

👍 1
andy.fingerhut21:02:00

more than one way to skin that cat, yes.

Endre Bakken Stovner22:02:42

I am creating a same machine web app which uses clj for computation and cljs for web-interface/visualization stuff. I'd like to be able to send messages to the running server from the command line. I'd like it to be easy for users to do. The only way I can think of is: curl -H "Content-Type: application/json" -X POST -d '{"id":1}' localhost:3000/json but this is silly verbose and ugly. What is the way to send commands to a running clojure program? This should be possible for non-programmers, so I'd like to avoid opening a REPL. I guess I could write a tool which converts simple commands into the curl calls I need, but this is still likely not a good way to do it.

bfabry22:02:51

like, any arbitrary command?

Endre Bakken Stovner22:02:21

Let's just say I want to send arbitrary data to the running program. These will be interpreted as commands and their arguments 🙂

noisesmith22:02:38

the simplest thing is to start a socket repl via java system property:

-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}"
that starts a server on port 555, you can use nc or curl or telnet or whatever to send it strings

noisesmith22:02:45

I could be misinterpreting what you want - that literally gives you a repl

Endre Bakken Stovner22:02:00

Okay, then I guess the best way to send data to a running program without entering a repl is to use curl. I will write a small cl program that converts my simple command-line calls $ re-make update f1.gz f2.gz into ugly curl calls behind the scenesl and sends them to my running server.

Chris Lester19:02:32

curl is something most people can use and is a fairly standard way to interact with a web service from a command line (look at any swagger output) ... if you are already having them send commands. It sound like you want a user interface though (for "non programmers").

fappy22:02:23

Hi 🙂 If I send-off a long running action to an agent and my await-for returns after my lengthy specified time-out, that action is still running happily in its thread. Is there a way for my thread (the one that did the send-off and the await-for) to reach out and forcibly kill the action or the action’s thread? Or is it the case that a rogue action with an infinite loop can claim a thread permanently for itself?

andy.fingerhut22:02:22

In general, I have read that cancelling/killing a JVM thread running arbitrary code can have all kinds of unwanted effects, e.g. if it was holding locks when you killed it, etc. A clean way to stop it is if somewhere in its inner loop(s) it checks for a request from outside for it to stop, but if you forget to make such checks in some loop that goes infinite, then it obviously won't be cooperating there.

noisesmith22:02:59

certain methods (especially ones that are on an OS level polling for IO) can be "cancelled" (which is different from a forcible kill, it allows elegant shutdown)

noisesmith22:02:44

which reminds me of a nice bit of poetry hidden in clojure.core

user=> (doc future-cancel)
-------------------------
clojure.core/future-cancel
([f])
  Cancels the future, if possible.
nil