Fork me on GitHub
#beginners
<
2020-02-26
>
theequalizer7301:02:28

Hi folks, I would like to convert something like this to a string: From this

({"Question1" "Text1"}
 {"Question2" "Text2"}
 {"Question3" "Text3"})
To this: “Question1- Text1  Question2 - Text2 Question3 - Text3” Thank you

Mario C.01:02:53

Perhaps a reduce. You join the k and the v with a "-" and join that with the accumulator which would be an empty string to start.

noisesmith01:02:00

one approach

(->> [{"Question1" "Text1"}
      {"Question2" "Text2"}
      {"Question3" "Text3"}]
     (map (comp (partial string/join " - ")
                (partial apply concat)))
     (string/join \newline))
 

noisesmith01:02:19

sending that through println gives

Question1 - Text1
Question2 - Text2
Question3 - Text3

theequalizer7301:02:49

The map is inside ( )

noisesmith01:02:02

map doesn't care

noisesmith01:02:22

it treats a vector and a list the same way, and vectors are more convenient for literals

noisesmith01:02:19

oh this also needs (require '[clojure.string :as string])can't assume too much in #beginners :D

😁 4
theequalizer7301:02:30

I will try that. Thank you

noisesmith01:02:31

one slightly odd thing is the usage of one key maps - since in clojure maps aren't ordered, so this would print more keys if present, but the order would not be guaranteed for the individual maps

👍 4
Mario C.01:02:10

(reduce #(let [[k v] (first %2)]
           (str %1 k " - " v "\n"))
        ""
        [{"question1" "text1"}
         {"question2" "text2"}
         {"question3" "text3"}])
Came up with this

Mario C.01:02:04

But I dont like the use of first here so

Mario C.01:02:10

doesn't feel right

noisesmith01:02:25

here's a version imitating string/join and using StringBuilder

(let [sb (StringBuilder.)]
  (doseq [m [{"Question1" "Text1"}
             {"Question2" "Text2"}
             {"Question3" "Text3"}]
          [k v] m]
    (.append sb k)
    (.append sb " - ")
    (.append sb v)
    (.append sb \newline))
  (str sb))

jumpnbrownweasel04:02:43

I like this solution because using StringBuilder is efficient. So I tried it with transducers, since I'm trying to learn how to use them.

(let [coll [{"Question1" "Text1"}
            {"Question2" "Text2"}
            {"Question3" "Text3"}]]
  (print
    (transduce
      (mapcat (fn [m]
                (let [[k v] (first m)]
                  [k " - " v \newline])))
      (fn
        ([sb] (str sb))
        ([sb x] (.append sb x)))
      (StringBuilder.)
      coll)))
I was surprised how well it worked out to create the StringBuilder and convert it to a string, all within the transduce.

jumpnbrownweasel04:02:10

Even better, the reducing fn can create the StringBuilder and be broken out as a reusable fn.

(defn build-str
  ([] (StringBuilder.))
  ([sb] (str sb))
  ([sb x] (.append sb x)))

(let [coll [{"Question1" "Text1"}
            {"Question2" "Text2"}
            {"Question3" "Text3"}]]
  (print
    (transduce
      (mapcat (fn [m]
                (let [[k v] (first m)]
                  [k " - " v \newline])))
      build-str
      coll)))
I think I'm in love with transducers...

noisesmith01:02:57

@orlandomr27 probably the more important thing is to explore clojure's various built in functions for iterating and consuming collections - they are weird if you haven't used fp before but they help make elegant code (and code with less places for bugs to hide)

noisesmith01:02:59

reduce, map, string/join, and doseq all specialize different tasks that consume an input one element at a time in order

theequalizer7301:02:47

The first solution worked just fine for me @noisesmith. I’m struggling to understand partial but I’ll get there cause I’m new to FP

noisesmith01:02:23

oh yeah I threw comp and partial in there haha

noisesmith01:02:55

this might help

user=> (def my-inc (partial + 2))
#'user/my-inc
user=> (my-inc 2)
4

noisesmith01:02:05

it's pretty intuitive in basic use

4
theequalizer7301:02:07

It helps! Thank you

noisesmith01:02:22

and (comp f g) returns a function that is effectively (fn [x] (f (g x))) - if g takes one arg

noisesmith01:02:38

user=> (def bump (partial + 10))
#'user/bump
user=> (def triple (partial * 3))
#'user/triple
user=> ((comp triple bump) 5)
45
user=> ((comp bump triple) 5)
25

Nopanun Laochunhanun03:02:31

Nice to know this solution, thanks!

noisesmith01:02:10

it's the same as the dot operator in algebra for function composition

noisesmith01:02:22

(except we don't do infix)

theequalizer7302:02:40

Nice explanation. I will save this messages as favorites. Everything is clear when you break it in parts and use the repl to evaluate each part

Luis C. Arbildo14:02:41

Hello, some link or books for learning more about Clojure? I read Clojure: Brave and True is an awesome resource

dharrigan14:02:40

I've been reading Programming Clojure, 3rd Edition - really good, also Elements of Clojure and Living Clojure.

dharrigan14:02:42

Lots of choices

Endre Bakken Stovner14:02:34

Can you refer to an atom you expect to be in the namespace in a macro? I have tried the following: (def rules (atom {})) (defmacro rule [rulename rulebody] (list swap! assoc rules rulename rulebody))

jsn15:02:16

by quoting it (list swap! 'rules assoc rulename rulebody) . you don't need macros at all for this particular case though, a function would suffice

Endre Bakken Stovner15:02:02

Thanks! I had tried that, but included other errors in my code which made me think the ' was mistaken. It turned out to be st else 🙂

ScArcher16:02:06

I was curious if anyone could point me to an example or help me understand how to setup retit-ring routes with ring middleware.

ScArcher16:02:50

(def router
  (ring/router
   [
    ["/ping" {:get ping-handler}]
    ["/echo" {:get echo-handler
              :middleware [[wrap-json-params] [wrap-keyword-params] [wrap-json-params]]}]]
   ))

ScArcher16:02:03

I was trying this, but I don't really understand what's going on.

shaun-mahood16:02:37

@scott.archer check out the example at https://github.com/metosin/reitit/blob/master/examples/ring-example/src/example/server.clj - there are a bunch of examples in that repository but I found it pretty hard to follow some of them when I was getting started with it. Are you trying to get middleware on only a specific route or just trying to get middleware working for the first time?

ScArcher16:02:52

Only on one specific route.

ScArcher16:02:00

I had a working "echo handler" in ring, but when I added retit, it's now broken. My ping / pong route works though.

ScArcher16:02:17

The only difference is I had some middleware wrapping the echo handler.

shaun-mahood17:02:29

@scott.archer Based on the documentation at https://metosin.github.io/reitit/ring/ring.html , your code looks correct to me - maybe the problem is somewhere else in your reitit configuration?

manutter5117:02:57

@scott.archer You have wrap-json-params in there twice, maybe they’re fighting each other?

Sy Borg17:02:45

is there an easy way to merge/convert a list of key-val pairs ('micromaps') to a big map? all keys are unique

bfabry17:02:41

(into {} your-list)

Sy Borg17:02:49

great, thanks

dharrigan17:02:21

You can also do (apply merge your-list) I believe.

4
bfabry17:02:24

if you had a list of actual (micro)maps that would work, but not for a list of keyval pairs

Sy Borg17:02:08

in my case it also works, thanks for the tip

Sy Borg17:02:33

i'm just trying to go through http://exercism.io , sometimes writing noob code and reinventing the wheel... this is the task btw - https://exercism.io/tracks/clojure/exercises/etl/solutions/3344fc53e309435284f45285b752da24

FiVo17:02:58

What is the idiomatic way for making a source file also available as resource?

FiVo17:02:13

For context I am starting a prepl in a docker container and would also like to pass it some init file, but I also have that file available in my application for a local dev testing.

bfabry17:02:27

clojure source files should be on the classpath, so should be available as resources

bfabry17:02:29

~/C/clj-scratch $ ls src/bfabry/                                                                                                                                                                                                      09:48:09
hello.clj   scratch.clj
~/C/clj-scratch $ clj                                                                                                                                                                                                                 09:48:15
Clojure 1.10.1
user=> (require '[ :as io])
nil
user=> (slurp (io/resource "src/bfabry/hello.clj"))
Execution error (IllegalArgumentException) at user/eval3 (REPL:1).
Cannot open <nil> as a Reader.
user=> (slurp (io/resource "bfabry/hello.clj"))
"(ns bfabry.hello)\n\n(defn -main [& args]\n  (println \"Hello, world\"))\n"
user=>

Scott Starkey18:02:42

I am trying to pull in a regex into a defn as a parameter, but having problems. Can someone please point me to the proper syntax? Right now I have something like this, but “no worky”:

(defn checker? [s re]
      (re-find #re s)
    )
(checker? "[a-m]" "b")
ClassCastException java.lang.String cannot be cast to java.util.regex.Pattern  clojure.core/re-matcher (core.clj:4667)
Is there any way to make that string I’m trying to pass in to be interpreted as a regex?

ghadi18:02:55

call re-pattern on the string

ghadi18:02:39

I'm surprised that #re s syntax got through the reader

ghadi18:02:06

(re-find (re-pattern some-unknown-string) ...)

Mario C.19:02:06

I have a question but I am really bad at wording things so hopefully this makes sense. Say I am running a web app that uses a Java object that is not thread safe (Doesn't support multithreading). The object is referenced by id, say :id => 123 and we can perform actions on that Object. If two requests come in and they both reference the same object 123 . Would this be the same thing as multithreading? Would they both be accessing the same object? Does each request run in isolation? Does my question make sense?

bfabry19:02:43

it depends on what framework/library you're using to serve requests, but in general yes a lot of them have multiple threads for serving

bfabry19:02:08

how are the requests getting a reference to this mutable object? is it a global?

Mario C.19:02:16

I am using Ring/Jetty

noisesmith19:02:29

with ring / jetty yes each request is in its own thread

Mario C.19:02:31

Yes it is a global. They are "cached" in an atom

noisesmith19:02:44

you need some other mechanism to prevent concurrent access

noisesmith19:02:58

never use mutable apis on objects inside atoms, this will break

noisesmith19:02:27

atoms don't lock your data, they retry if there's concurrent change, and if your operation happens to mutate an object, that mutation will be redone

Mario C.19:02:58

Sorry, didn't understand. So each requests will be accessing the same object?

noisesmith19:02:58

agents have the right semantics (only one thread can operate on an agent at a time), though they are weird in other ways

noisesmith19:02:44

you tell me - if the object is at namespace or global scope, yes, all the requests will use the same object

Mario C.19:02:15

Okay I thought so but I was told that each request was in isolation so I wasn't sure so wanted to ask.

didibus19:02:41

It isn't totally clear. What is the id? Is that the request id ?

didibus19:02:37

Two requests in ring/jetty are effectively each running on a different thread and thus yes, it is multi-threaded.

didibus19:02:10

So if two requests were to access the same instance of an object concurrently, where the object is not thread safe, this could cause bugs

didibus19:02:49

If each request get its own object instance though, then you are fine

noisesmith19:02:12

@UB0S1GF47 each request is in its own thread, but threads are not isolated

Mario C.19:02:24

We have something like (def workbooks (atom {}) and we load a workbook from a DB and store it within the atom with (swap! workbooks [wb-name wb-version] {:bit-stream wb-bitstream})

Mario C.19:02:58

After looking more into it, we are creating new object instances per request

Mario C.19:02:55

Thanks, was more curious about the multi threading part

didibus19:02:56

If its a new one per request then its fine. Each request execute within the same thread in a single threaded way (unless you yourself multi-thread it some more).

didibus19:02:37

What noisesmith was saying about atom is that, if you mutate the mutable objects as part of a call to swap! it can cause bugs.

didibus19:02:15

swap! should only make idempotent modifications to the state

didibus19:02:31

If you only modify the immutable map when you swap! that is fine

didibus19:02:35

In theory, if you make non idempotent modification in swap! in a single threaded context, that is also fine, but its kind of bad practice.

Mario C.19:02:57

Got it, will make a note of this! Thank you guys for the explanation

didibus19:02:59

And it can help to understand why. Basically, if two things try to swap! simultaneously the same atom (which can only happen in a concurrent scenario), one of them can execute the swap!, but fail to commit the result, and will thus retry the swap! a second time, and a third, etc., until it succeeds.

didibus19:02:50

Because the swap! can be retried, you need to make sure that whatever you do in swap! can be retried. Only idempotent operations are safe to retry.

didibus19:02:05

So say you read the next entry in the bitstream inside your swap!, you can imagine that this isn't safe to retry, since every time you read the next entry you get a different bit

Mario C.20:02:11

@U0K064KQV Thanks for this, explanation helps!

donaldball20:02:49

It's not commonly used in clojure, but the locking macro is an option for situations like these.

chepprey21:02:25

Can someone show a simple example of a "non idempotent modification in swap!"?

didibus21:02:18

(def a
  (atom
    (java.util.ArrayList.)))

(swap! a #(doto % (.add 10)))

didibus21:02:26

Its hard to accidentally do in pure Clojure, but if using mutable Java objects it can slip up

didibus21:02:14

In pure Clojure you might have:

(def a 
  (atom 
    (atom 0)))

(swap! a #(reset! % (inc @%)))

chepprey22:02:20

Ok, thanks. So the most common source of danger is Java interop / mutable datastructures.

chepprey22:02:14

In pure Clojure, is it idiomatic (or common) to store atoms in an atom?

noisesmith22:02:25

no, I would consider attempting to do so a bug

didibus22:02:37

Atoms in atom is definitly weird and a bit of an anti-pattern. Any mixing of mutable and immutable kind of is

didibus22:02:23

If you end up in a place like that, and you really really need it, I can only think of performance for why that would be, you might want to just go full mutable and replace your atom by a mutable Java or JS collection instead

chepprey22:02:10

Thanks all, this is what I wanted to hear, which is basically that non-idempotent atom swaps are really not a thing, so long as you're doing pure Clojure. Granted that Java interop isn't uncommon at all, but, anywhere I'm doing that, I already have my guard up, so to speak.

👍 4
chepprey22:02:30

Even thinking of putting Java arrays inside of an atom gives me a queasy feeling. I suppose I can imagine legit uses for it.

didibus22:02:31

That be an anti-pattern too mostly. Atom expects you to store immutable things inside it, and swap! expects your swap function to be idempotent.

didibus22:02:53

Any use of them that differs from that you're on your own 😋

👌 4
didibus22:02:12

If you go mutable, you don't need an atom anyways

noisesmith22:02:45

on the other hand, a mutable object inside an agent so that only on thread messes with it at a time is a useful pattern

Mario C.22:02:27

Does this mean that it will block the other threads? For example, say 10 requests try accessing an object inside the agent does this mean 9 req are on standby until one-by-one get access to the agent ?

noisesmith22:02:51

yes - for writes - reads still won't block

noisesmith22:02:19

also this requires using the agent correctly, it doesn't prevent doing something stupid like deref and modify - just makes it easier to do the right thing

didibus22:02:59

Agents are async, so not really, but the changes won't necessarily reflect immediatly. You can use await to block

noisesmith22:02:34

oh right - the thread won't be blocked, but it can return before the operation on the agent does

didibus22:02:26

And its possible to be awaiting forever

didibus22:02:56

If the queued modifications to the agent never empties

noisesmith22:02:30

await doesn't wait for the queue to be empty - it creates a function that flips a switch and returns identity to the agent, and waits for the switch to be flipped

noisesmith22:02:52

which is a nice clue for a way to block until your action has completed

chepprey22:02:10

So the await specifically waits for "your" particular modification to the agent?

noisesmith22:02:24

it isn't attached to a modification - it sets things up so your execution is resumed when the message await sends hits the agent

chepprey22:02:39

just read docs, I don't think I'm right about that...

Blocks the current thread (indefinitely!) until all actions
dispatched thus far, from this thread or agent, to the agent(s) have
occurred.  Will block on failed agents.  Will never return if
a failed agent is restarted with :clear-actions true or shutdown-agents was called.

noisesmith22:02:49

so if you call it after your modification, await returns after your mod, but maybe after some other as well

chepprey22:02:17

Ya. Waits until all "actions dispatched thus far" --- thus far meaning the time of awaiting, not the time of modifying.

noisesmith22:02:06

(cmd)user=> (def a (agent 0))
#'user/a
(cmd)user=> (do (future (dotimes [_ 1000] (Thread/sleep 100) (send a inc))) (await a) @a)
79
(ins)user=> (do (await a) @a)
167
(cmd)user=> (do (await a) @a)
388

noisesmith22:02:23

the future is repeatedly sleeping for 100 ms then incrementing a

didibus22:02:06

Hum... ok that bit is still confusing, docs arn't super clear

didibus22:02:39

My understanding is you can send to the agent from the agent, thus if awaiting, you might never be done, if the agent sends things to itself

noisesmith22:02:17

the way await works, if you read the source, is that it puts an action that counts down onto the agent, once that runs await returns

noisesmith22:02:41

when the agent sends to itself, that would add to the end of the queue (after the special payload from await)

didibus22:02:58

Hum.. interesting. So effectively it be, say the queue is at 130, and you await, it will wait until the next 130 modification run

didibus23:02:51

Why does it say from this thread though? Does it exclude modifications queued up by other threads?

noisesmith23:02:00

that's effectively it yeah - the source to await is easier to understand than its doc :P

didibus23:02:54

It does, hum

didibus23:02:08

Interesting

didibus23:02:21

So you can't await changes made by another thread?

noisesmith23:02:30

you can if you know they are already sent to the queue... but yeah you can only await "all things currently queued"

didibus23:02:06

Ok ya, that one make sense. Just the doc weirdly specifies from this thread which I thought was weird

didibus23:02:44

So its pretty good. Because they emphasise indefinitly! I assumed it was awaiting the queue to be empty