Fork me on GitHub
#beginners
<
2020-05-28
>
raspasov00:05:45

(into
  []
  (map (fn [x]
         (if
           (and (vector? x) (< 2 (count x)))
           ;if true, transform here 
           
           ;grab first and last
           (let [tag       (first x)
                 last-item (last x)]
             ;build the transformed vector
             [tag
              ;the new string
              (str
                last-item
                " of the "
                (->> x
                     (drop 1)
                     (drop-last)
                     (reverse)
                     (reduce
                       (fn [accum item]
                         (if (vector? item)
                           (conj accum (second item))
                           (conj accum item)))
                       [])
                     (interpose " of the ")
                     (vec)
                     (apply str)
                     ))])
           ;else return unchanged
           x))
       [:S
        [:ni-jar 'sensei]
        [:ga-jar
         [:no-jar 'gakkou]
         [:no-jar 'sensei]
         'hanashi]
        [:verb 'sabishii]]))

raspasov00:05:40

=> [:S [:ni-jar sensei] [:ga-jar “hanashi of the sensei of the gakkou”] [:verb sabishii]]

sova00:05:27

You are incredible! whaaat let me take a look at this

raspasov00:05:45

it is somewhat simplistic, and I’m not sure it covers all cases, but whatever we talked about it does… it is very simplistic in the sense that the way it checks whether to transform is via (and (vector? x) (< 2 (count x)))

raspasov00:05:04

But I went by your phrase “Essentially any vector that has more than two elements [:key and val] needs to be smashed to have just key and val.”

raspasov00:05:36

So for your data structure, at this limited level of nesting, that should cover the requirement…

sova00:05:30

that's very good. i'm trying to do it in cljs and it is complaining there is no rseq? ... hmmm. i very much appreciate your help & efforts!

raspasov00:05:55

sorry, reverse

raspasov00:05:10

(let [transformer
      (fn [x]
        (let [tag       (first x)
              last-item (last x)]
          [tag
           (str
             last-item
             " of the "
             (->> x
                  (drop 1)
                  (drop-last)
                  (reverse)
                  (reduce
                    (fn [accum item]
                      (if (vector? item)
                        (conj accum (second item))
                        (conj accum item)))
                    [])
                  (interpose " of the ")
                  (vec)
                  (apply str)
                  ))]))]

  ;run it
  (into
    []
    (map (fn [x]
           (if
             (and (vector? x) (< 2 (count x)))
             ;transform here
             (transformer x)
             x)))
    [:S
     [:ni-jar 'sensei]
     [:ga-jar
      [:no-jar 'gakkou]
      [:no-jar 'sensei]
      'hanashi]
     [:verb 'sabishii]]))

raspasov00:05:51

Second one is a bit cleaner since the big blob of code is separated in a function…

sova00:05:55

that is amazing

sova00:05:07

and quite clean. thank you. i gotta take a good look at it

sova00:05:34

There is one more thing

sova00:05:52

it can be :no-jars or :to-jars .. they work the same way, but don't mix.

raspasov00:05:12

Ahh… well 🙂

sova00:05:16

you got it though

sova00:05:22

i can probably add those conditionals

raspasov00:05:01

What do you mean by “don’t mix” ?

sova00:05:05

like if there is more than one :no-jar per squad, i can run the one transform, and if there is more than one :to-jar i can run a different transform, but all i need to do is make your one transformer into two

sova00:05:13

Let me give you an example

sova00:05:23

I guess it's ok if they mix, but the :to-jars should render to a different string: input: [:S [:ga-jar [:to-jar hakujo] [:to-jar yata] seuss] [:verb iru]] desired output: [:S [:ga-jar "hakujo and yata and seuss"] [:verb iru]] So if there were a way to distinguish transformers that would be cool

sova00:05:43

Thanks for your patience 🙂 my brain is mostly melty today

raspasov00:05:58

So like the difference is the concat word?

raspasov00:05:20

:no-jar -> ” of the ” :to-jar -> ” and ”

raspasov00:05:25

(let [transformer
      (fn [x]
        (let [tag        (first x)
              last-item  (last x)
              any-jar    (-> x (second) (first))
              concat-str (condp = any-jar
                           :no-jar " of the "
                           :to-jar " and "
                           " hmmm ... ")]
          [tag
           (str
             last-item
             concat-str
             (->> x
                  (drop 1)
                  (drop-last)
                  (reverse)
                  (reduce
                    (fn [accum item]
                      (if (vector? item)
                        (conj accum (second item))
                        (conj accum item)))
                    [])
                  (interpose concat-str)
                  (vec)
                  (apply str)
                  ))]))]

  (into
    []
    (map (fn [x]
           (if
             (and (vector? x) (< 2 (count x)))
             ;transform here
             (transformer x)
             x)))
    [:S
     [:ni-jar 'sensei]
     [:ga-jar
      [:to-jar 'gakkou]
      [:no-jar 'sensei]
      'hanashi]
     [:verb 'sabishii]]))

raspasov00:05:32

It starts to smell in the sense that now you’re expecting a certain shape of your data structure… if it’s guaranteed to be this shape, then OK… but if it’s not… but it does work for this specific case

raspasov00:05:58

Yea we can do it… the data structure here starts to smell to me… but you can do it

sova00:05:34

That's really wonderful, yeah it's pretty much guaranteed to be this way

sova00:05:45

the data shape i mean

raspasov00:05:47

Again, a lot of assumptions here for the data shape… if you get something in a weird shape, chances are this will throw an error or not work as expected

raspasov00:05:55

If that’s the case, then we’re good 🙂

sova00:05:09

the interface that generates the data is buttons you click, so it's very hard to evade the rules :2

sova00:05:24

thanks a lot man

raspasov00:05:45

Ideally, I would find a way to group things by :to-jar or :no-jar so that keyword appears once, rather than just grabbing the second item of a vector and then looking at the first item of that sub-vector

raspasov00:05:51

That’s the “smell” I’m talking about

raspasov00:05:09

any-jar    (-> x (second) (first))

raspasov00:05:16

This is the “smell” 🙂

endrebak8506:05:15

Are there patterns for threads (actually processes) updating an atom when they finish? After the update is finished it should trigger some function. Is the best way to let the process know about the data structure it should update? And then have a watcher on the data structure which triggers the function? I'm sure I could remove some complection with channels, but it makes everything non-deterministic.

endrebak8506:05:20

Yes, thanks 🙂 But I am trying to ask whether there are other and better ways of solving my problem.

tzafrirben07:05:06

Using channels maybe? Signal the channel (put) when thread is done and have a different thread park on the channel for output (take)

ben.sless08:05:18

if we think about it as a system of processes (going the CSP route), let's imagine a controller process C, and a worker process W. you can implement a sort of async RPC where the controller sends messages to a worker and the worker acts on them and returns a message via a response channel, i.e. C!cmd -> W?cmd; do-stuff; W!resp -> C?resp When you have several workers you can just index the response and control channels

ben.sless08:05:02

Diving a bit into the worker, it should itself contain a process that does the actual work and it will be the "interface" to it, which queries and reports if it's completed

noisesmith14:05:00

the future macro creates a new thread, and returns a promise that won't be ready for reading until the thread exits, and returns what the code in the future returned on deref

noisesmith14:05:31

CSP is cool but also overkill for the literal feature asked for here

ben.sless15:05:16

Maybe it is... I've been immersed quite deeply in it lately so I feel quite comfortable with the abstraction and semantics. Why do you think it's overkill?

noisesmith15:05:08

core.async (the official implementation of CSP for clojure) is quite "leaky", and imposes many assumptions / restrictions on your code reading more carefully, one could implement what you describe with thread safe queues, and that would avoid many of the problems (and impose the problem of maintaining your own channel infrastructure)

noisesmith15:05:16

the specific behavior asked for is easily built out of future and deref

ben.sless15:05:09

then you'd use watchers and callbacks to update the shared data which will be accessed by the "supervisor" thread?

noisesmith15:05:59

no, I'd use futures and deref:

(def state (atom {... ...}))

(def process-a (future (swap! state frob x y)))

(def process-b (future @process-a (swap! state flux z)))

noisesmith15:05:47

the deref of proces-a ensures process-b doesn't execute the rest of its body until process-a completes, and propagates any failure to interrupt it as well

noisesmith16:05:27

of course in real code I wouldn't put future calls in def - these would be let bindings

ben.sless16:05:46

wrt core.async being leaky, It's something I find with all the rich abstractions (promesa, manifold, callbacks), they all leak and infect the code and you have to write in them

noisesmith16:05:15

that's a separate problem: you need to lift other code into the abstraction to cooperate with it

noisesmith16:05:28

by leaky, I mean that with core.async you need to: • ensure that no CPU intensive or blocking calls happen in go blocks • ensure that a given block doesn't read and write the same channel in a loop • avoid certain macros inside go blocks that the callback transform compiler can't handle etc. things that aren't just the async paradigm, but limitations of how it's currently implemented

ben.sless16:05:49

Ah, I see what you mean. Yeah, there's a lot of discipline(?) required when working with core.async. With future/deref, how would you implement a DAG of tasks which you also want to be able to monitor? also you might want to kill, restart, etc.

noisesmith16:05:01

well, killing of processes inside the jvm simply isn't a thing, unless done cooperatively by checking for a cancellation flag

noisesmith16:05:05

regarding task graphs, I don't know of a good solution in clojure off hand, aside from plumatic/graph which I haven't used in anger and can't vouch for

noisesmith16:05:49

usually people go for task distribution DSLs (which then offer distribution across multiple servers etc.) but once again, I didn't infer the need for that kind of feature from the original problem description

noisesmith16:05:28

generally all of these things are methods of making your code less reliable and harder to debug, I wouldn't adopt any of them unless they were absolutely required for your domain

ben.sless16:05:44

The OP is the simplified problem description, I have prior knowledge from a post a few hours prior (that may be cheating 🙃 ) I've been thinking about similar problems lately, I'm leaning towards separating execution model from data model, then it's possible to try out different solutions and break them apart to isolate the fragile/unreliable parts as much as possible

noisesmith16:05:21

big picture, it's usually the breaking apart that makes things harder to code correctly

ben.sless16:05:55

Interesting. Why?

noisesmith16:05:32

when you turn procedural step-by-step code into a graph of callbacks, processes, and conditions, what would be trivial surface level bugs become non-obvious

noisesmith16:05:52

also, the call stack, which reveals the source of erroneous behavior, typically loses all meaning or utility

noisesmith16:05:44

then the desire to describe processes as config rather than code, further detaches the erroneous state from any of the language facilities that help us ensure things are correct - a typo on a keyword doesn't throw a compile time error the way a var typo does, etc.

noisesmith16:05:13

these tradeoffs can be worth it, but they have a huge upfront cost in code quality and maintainability that you need to repay

noisesmith16:05:17

which is why I suggest adopting as little distribution and async as you can get away with

noisesmith16:05:56

on top of everything else, these indirections make code slower, and the paralellism they enable doesn't always compensate

ben.sless17:05:44

I see. Thank you

ben.sless17:05:00

With what you said about a keyword error, that's a pertinent point, which I think is remedied a bit by the model separation. when you "compile" the data model you can overcome at least that issue

noisesmith17:05:54

that's true, and I don't mean to imply that these techniques are never appropriate, or that we don't have good tools to reduce the problems listed

noisesmith17:05:33

when I first learned how core.async worked I was excited about what it offered, and used it liberally in my codebase this led to very subtle and hard to debug intermittent problems, partially due to my incomplete understanding of core.async and partly due to the limits of that model definitely use it if you need async execution, and you need complex patterns of coordinating results, but it doesn't come for free

ben.sless17:05:57

I mostly do processing of infinite streams of data, so I use pipelines quite often

ben.sless17:05:14

not so much the more complicated coordination semantics

hindol.adhya08:05:39

Is it okay to create a dummy namespace just to be able to use namespace qualified keywords in that namespace? Are there other ways to achieve the same effect? I know :dummy.namespace/keyword works, but I want to avoid typing the full namespace every time.

alexmiller12:05:21

It’s ok. You can do it with create-ns and alias in the place where you need it

hindol.adhya14:05:33

Great, thank you for confirming.

danielostling10:05:10

What's the standard way of returning only the first regexp match? I realize re-find uses re-group to return results, but is it then safe to assume re-group always returns the input as first result?

danielostling10:05:27

> (re-find #".*[?&]some-param=([^#&]+)" "?some-param=some-val")
["?some-param=some-val" "some-val"]

andy.fingerhut10:05:35

If by 'first' you mean 'only one match, not multiple matches', then yes that is safe to assume.

andy.fingerhut10:05:10

If by 'first' you mean 'guaranteed to be the earliest match of the regex in the string', that depends on the regex and the string, perhaps. I don't have examples handy to demonstrate possible surprises there.

danielostling10:05:32

Yeah, I read on docs for re-groups that "

If there are
nested groups, returns a vector of the groups, the first element
being the entire match.

danielostling10:05:53

But for that example I had above, does that really count as a nested group?

andy.fingerhut10:05:56

In that regex you give, for example, does the initial ".*" part need to be there, really?

danielostling10:05:19

Well, I'm counting on input being possibly a full URI

danielostling10:05:39

And I'm looking to extract only specific query parameter values

danielostling10:05:57

I guess ^.* could have been better

andy.fingerhut10:05:57

In the regex, any parenthesized expression is a capture group, unless it begins with a special character sequence, IIRC something like "?:" just after the left paren

andy.fingerhut10:05:48

re-find looks for any match of a substring that matches your regex, so using re-find either ^. or . at the beginning are redundant, I believe.

danielostling10:05:48

Hm. I need to read the regexp docs again 🙂

andy.fingerhut10:05:58

re-matches restricts itself to only matching the entire string.

andy.fingerhut10:05:36

The Clojure doc strings for those functions refer to Java APIs for the full behavior.

andy.fingerhut10:05:46

and those Java docs are not short 🙂

danielostling10:05:19

Perhaps reasonably, regexp isn't a small topic 🙂

danielostling10:05:39

I will think on what you wrote, thanks 🙂

sova12:05:15

If anyone has a question, feel free to ask it 😃

dev-hartmann13:05:51

Hi fellow clojurians, I'm trying to read a file from the resources folder from the repl with `http://clojure.java.io/resource` . I'm using deps.edn and have added resources to the paths vector interestingly io/file works

dev-hartmann13:05:18

(defn load-config [resource profile]
  (-> resource
      ()
      (read-config)
      (get profile)))

dev-hartmann13:05:40

the call to io/resource always returns nil

danieltanfh9513:05:51

you should probably use the relative path from the resource folder

ghadi13:05:22

@dev-hartmann make sure the resources folder is on the classpath

ghadi13:05:41

:paths ["src" "resources"] in deps.edn

dev-hartmann13:05:27

@ghadi thx, that fixed it i had :path instead of :paths facepalm

mauricelecordier17:05:14

Hey all 👋 Does anyone know why 4clojure is screaming at me not to use count ? problem 22 http://www.4clojure.com/problem/22 I submitted and it says You tripped the alarm! count is bad! for the following answer:

(fn [x] (reduce + (for [position x]
                    (+ 0 1))))

noisesmith17:05:49

for must use count inside the expansion of that form

noisesmith17:05:31

yeah, for is pretty wild

user=> (macroexpand-1 '(for [position x] (+ 0 1)))
(clojure.core/let [iter__6305__auto__ (clojure.core/fn iter__140 [s__141] (clojure.core/lazy-seq (clojure.core/loop [s__141 s__141] (clojure.core/when-let [s__141 (clojure.core/seq s__141)] (if (clojure.core/chunked-seq? s__141) (clojure.core/let [c__6303__auto__ (clojure.core/chunk-first s__141) size__6304__auto__ (clojure.core/int (clojure.core/count c__6303__auto__)) b__143 (clojure.core/chunk-buffer size__6304__auto__)] (if (clojure.core/loop [i__142 (clojure.core/int 0)] (if (clojure.core/< i__142 size__6304__auto__) (clojure.core/let [position (.nth c__6303__auto__ i__142)] (do (clojure.core/chunk-append b__143 (+ 0 1)) (recur (clojure.core/unchecked-inc i__142)))) true)) (clojure.core/chunk-cons (clojure.core/chunk b__143) (iter__140 (clojure.core/chunk-rest s__141))) (clojure.core/chunk-cons (clojure.core/chunk b__143) nil))) (clojure.core/let [position (clojure.core/first s__141)] (clojure.core/cons (+ 0 1) (iter__140 (clojure.core/rest s__141)))))))))] (iter__6305__auto__ x))

mauricelecordier17:05:01

Oh wow I did not know you could do that

noisesmith17:05:19

with a for that basic, the map equivalent should be easy to write

noisesmith17:05:40

(I mean I'd write it now but this is supposed to be a learning exercise after all :D)

mauricelecordier17:05:19

Haha no worries…Thanks for pointing it out and macroexpand trick

mauricelecordier17:05:26

Will attempt another way

noisesmith17:05:34

another alternative is using a function other than + in reduce, since you are using reduce already

joshua.mcquistan18:05:07

I've found myself knee deep in a conformed spec data structure and I want to use match to couple destructuring + binding (Erlang style, I suppose). Should I be keeping it separate (the code is fairly trivial) or should I bring in something like core.match? https://github.com/clojure/core.match

didibus18:05:08

Destructuring already does binding?

didibus18:05:23

Do you mean you want to couple destructuring + control flow ?

didibus18:05:00

(let [{:keys [a b]} {:a 1 :b 2}]
  [a b])
;;=> [1 2]

ben.sless18:05:09

You might like this lib, it allows matching like multimethods with "wild cards"

joshua.mcquistan20:05:04

Yeah, destructuring + control flow was what I meant

joshua.mcquistan20:05:13

Thanks Ben, I'll take a look

noisesmith18:05:16

the main arguments I've heard against match: • it leads to brittleness that prevents late binding (RH advocates using multimethods so other code can add cases as needed) • it produces large bytecode, and since the JVM imposes a hard limit on bytecode size, combining it with eg. core.async or for or core.logic can lead to compilation errors

noisesmith18:05:50

all that said, it's a style choice, there are reasonable ways to avoid those issues, use match if it is the idiom you want for your codebase

joshua.mcquistan18:05:13

I think in the case I won't use it as it's just for style purposes on internal code I don't care too much over

joshua.mcquistan18:05:48

I'm surprised I've made it this far without reaching for it, actually

didibus18:05:23

I heard good things about Meander as well, wonder if it generates smaller bytecode.

hiredman18:05:56

I kind of doubt it, where core.match runs into trouble is it tries to an algorithm for compiling pattern matches that was created for compiling to machine code (or some lowlevel bytecode) that assumes the presence of things like 'goto', so core.match either has to duplicate code or emulate goto (in theory using a preallocated exception on the jvm does this well, which is how I think it does it)

hiredman18:05:31

meander looks like it similarly tries to be an optimal compiler for pattern matches (I am not 100% sure but a quick glance leads me to suspect it is preallocating exceptions for jumps as well), so I suspect it may hit the same issues as core.match

hiredman18:05:24

but you can write a (dumb, with no features) pattern matching macro in less than 200 lines of code, which likely will not have those issues

jimmy18:05:50

We don't actually compile to exceptions for jumps in meander. I don't know for sure if the problem that's bring talked about is a good fit for meander or not. But so far I've never seen a meander match that runs into the code too large problem. And we have some fairly complex and long matches.

hiredman18:05:23

what is all the cata stuff then?

jimmy18:05:08

Those turn I to recursive function calls. You can definitely blow stack if you aren't careful.

noprompt19:05:24

Its a crappy hack and yes it is, however, its only used for rewrite . The matching is compiled to letfn and we invoke those functions instead of throw to emulate goto.

jimmy19:05:57

That isn't a go-to in the sense you are talking about as far as I can tell. It is just a hack around the fact that if something fails we throw an exception. So when we cata we need to catch the exception and return a value. We aren't jumping to different cases.

noprompt19:05:32

Ah, yeah, duh.

jimmy18:05:01

But never been a problem in practice. We are even writing the next version of meander using the current version and have had no scaling issues.

jimmy18:05:51

But given that this is #beginners. Not recommending everyone go off and use meander right away. You can check out my talk or blog posts if you wanted to learn more. But it is something that takes a bit to learn.

joshua.mcquistan20:05:47

Thanks for your input. In my case just pulling a library for what I wanted was overkill but I can also see how it tightly couples the data. So the Clojure point of view is worthwhile 80% of Clojure seems to be learning the tools / libraries so I'll skim the docs 🙂

joshua.mcquistan21:05:46

Found a talk -- it was fun! I might have found a new hammer

jimmy22:05:38

Glad you liked the talk. :) Feel free to ask any questions you have on #meander.

mauricelecordier19:05:31

I am honestly struggling to convert this (reduce + (map (fn [message] (+ 0 1)) [[1 2] [3 4] [5 6]])) to the koan style of submission 😔 where [[1 2] [3 4] [5 6]] is meant to be any sequence

seancorfield19:05:55

@mauricelecordier as that stands it just counts how many elements are in the sequence -- is that what you intend?

user=> (reduce + (map (fn [message] (+ 0 1)) [[1 2] [3 4] [5 6]]))
3

mauricelecordier19:05:23

Needing to count the elements without using count an earlier submission I used, used for which uses count internally and got rejected

mauricelecordier19:05:19

Yip! Its problem 22 in 4clojure

dpsutton19:05:14

why (+ 0 1) and not just 1?

mauricelecordier19:05:22

That seems to error out

seancorfield19:05:25

Did you have (1) instead of just 1 perhaps?

seancorfield19:05:51

(1) means "call 1 as if it were a function" which with throw a class cast exception (because Long -- the type of 1 -- is not IFn -- the expected type of a function)

seancorfield19:05:03

(reduce + (map ,,, <some-sequence>)) => ( #(reduce + (map ,,, %)) <some-sequence> ) is the transformation from an expression containing some sequence to an anonymous function of one argument applied to that sequence.

seancorfield19:05:09

@mauricelecordier You might find constantly useful here...

mauricelecordier19:05:29

I see this has an example using constantly as a way to find the size of a collection…thanks! Thats interesting

seancorfield19:05:55

...but you don't need to map over the elements really since you don't care about their values: consider (reduce (fn [n _] ???) 0 <some-sequence>) for some function ??? as a possible solution shape (the _ is a convention for "I'm deliberately ignoring this argument").

seancorfield19:05:32

n will be 0 on the first call. What would you need ??? to be for that to count the elements?

seancorfield19:05:52

Or just (inc n) which is more idiomatic.

mauricelecordier19:05:27

True….I am still getting use to the language itself

seancorfield19:05:41

Don't worry -- becoming idiomatic takes a lot of practice: clojure.core has hundreds of functions that you slowly get used to combining.

mauricelecordier19:05:33

I am about to start chapter 4 of the Brave book which deals with clojure.core a bit more

mario.cordova.86222:05:07

When using the lein shell plugin I keep getting WARNING: You have $CLASSPATH set, probably by accident do I just have to live with this? I believe this is happening because the plugin is mostly likely executing a lein command in another session(?) and it inherits the classpath from the original lein process. But honestly have no idea

hiredman22:05:59

in your shell type "echo $CLASSPTH"

mario.cordova.86222:05:31

:aliases {"run-client-external" ["shell" "./scripts/run-client.sh" "external-local"]}

mario.cordova.86222:05:31

The scripts sets an env variable and then runs lein run-client

hiredman22:05:16

the number of jvms you must launch when you do a build must be pretty impressive

mario.cordova.86222:05:54

I am not sure to be honest

hiredman22:05:43

me neither, but since you are running a shell script anyway you can unset CLASSPATH in that shell script

mario.cordova.86222:05:37

Oh good idea, thanks!