Fork me on GitHub
#beginners
<
2020-01-09
>
vincent.cantin04:01:53

nowadays, what is the simplest way to build a project template for generating a deps project?

dpsutton05:01:42

I kinda think just clone something and then throw away the git directory

vincent.cantin07:01:45

I thought about that

seancorfield05:01:03

clojure -A:new app myname/myapp if you have :new as an alias for clj-new?

seancorfield05:01:23

Is that what you mean @vincent.cantin?

vincent.cantin05:01:31

I mean, I have a project which I want people to start their dev on.

seancorfield05:01:49

So you want make a clj-template from it...

vincent.cantin05:01:15

I found clj-new and I am reading it right now

seancorfield05:01:39

clojure -A:new template myname/mytmp will create a very basic clj-template that you can flesh out, and publish to http://clojars.org as mytmp/clj-template

seancorfield05:01:04

Then folks will be able to clojure -A:new mytmp yourname/yourapp

vincent.cantin05:01:55

thank, that’s what I needed

seancorfield05:01:18

DM me with any Qs. Happy to help.

seancorfield05:01:38

The template project is horribly minimal right now. I need to expand it.

seancorfield05:01:53

but the placeholder substitution is kind of a bear

seancorfield05:01:32

Essentially if you copy bits of the app template into your generated template project, you'll be close.

vincent.cantin17:01:48

I have a request for improving clj-new: do not force the user to have a multi-segment project-name.

vincent.cantin17:01:30

In many cases, the users don’t want it.

seancorfield19:01:27

clj -A:new app foo.core is the same as lein new app foo

seancorfield19:01:38

I will not drop the requirement for qualified or multi-segment names.

tajoddin.shima07:01:45

hi I want to use java executors to call a function in the java thread pool It seems that when I use let to define my pool as below I get this wired error that more than one matching method found: submit but when using def everything goes well why is that?

hiredman08:01:10

it has to do with type inference. in the let case when compiling the call to submit the compiler knows the type of pool and of the fn, so it tries to find an exact method to call, and it can't because fns are both Callable and Runnable. in the def case the compiler doesn't keep track of the type of pool so it compiles the call to a reflective call which just picks one of the methods even if multiple methods match

ban.istvan11:01:03

hi (clojure newbie here, obviously) a question regarding core.async: I’ve just read this: “Future directions - Networks channels and distribution are interesting areas for attention.” https://clojure.org/news/2013/06/28/clojure-clore-async-channels#_future_directions out of curiosity, were there anything happened in this field lately? or is that not really relevant/interesting any more?

alexmiller12:01:25

Some work was done on it. It's hard. :)

dennistel9013:01:36

Hi! Are there plugins for leiningen/maven that help with semantic versioning? I’m looking for something to generate a changelog and a new version number based on commit history

dennistel9013:01:54

Not sure if this is the right channel though

sova15:01:26

@dennistel90 awesome idea. not sure if something like this exists.

dennistel9016:01:41

@sova I got the idea from Lerna (JavaScript tool), might be interesting to see if it could be a plugin for clojure tooling

srijayanth16:01:54

Are there places where using with-test is recommended over a deftest ?

seancorfield17:01:09

@dennistel90 probably worth mentioning that Clojure folks tend to be a little "odd" as far as versioning goes. Some Clojure libraries follow SemVer pretty closely but a lot have a much more arbitrary approach (several libraries are still 0.0.x despite being stable and in heavy usage, others start at 0.1.x and never move to 0.2.x, several Contrib libs have rather arbitrary monotonically-increasing version numbers). Rich Hickey has talked about how broken SemVer is as a concept and why it's not worth following: https://www.youtube.com/watch?v=oyLBGkS5ICk -- transcript here https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md

alexmiller17:01:41

the contrib lib numbers are not arbitrary - they are # of git commits since the beginning of the repo

alexmiller17:01:03

so they actually are potentially sufficient information to find a particular commit (not that we expect anyone to do that)

seancorfield17:01:33

Yes, but it's not SemVer and the prefix to the number of commits changes over time in a way that isn't typical of version numbering "elsewhere"

alexmiller17:01:59

I think prefix changes in pretty much the same way people actually use semver

ramon.rios17:01:51

Hello guys, is there any function to give me all the elements of a list?

ben60617:01:59

Can you give an example input+output you’d want to see?

ben60617:01:43

Like

(f [1 2 3])
;;=> [1 2 3]
(in this case, f is identity)

ramon.rios17:01:19

I have a list with 3 maps on it, but i want to get those 3 values

ramon.rios17:01:34

I'll get some code to visualize better my request

ramon.rios17:01:57

i have this function (map #:( {name %}) (range 1 4)

ramon.rios17:01:11

And this returns what i want, but inside a list

ramon.rios17:01:29

The values that this function generates needs to be in a vector that i previously build

ramon.rios17:01:51

But i cannot be able to do it inside this list

ramon.rios17:01:43

So, what i'm trying to ask is: is there any function or strategy that give me this, but wihout being on a list?

seancorfield17:01:00

Is the issue that map produces a list but you want a vector?

seancorfield17:01:43

You could use mapv to produce a vector, or (into [] (map f coll)) or (vec (map f coll)) to produce vectors from functions operating on collections.

seancorfield17:01:20

into would let you append your new items into an existing vector:

(into my-vector (map #(hash-map :name %)) (range 1 4))
Perhaps like that @?

ramon.rios17:01:41

Let me test that

ramon.rios17:01:48

It really worked!! Thanks for the insights, friends!

noisesmith18:01:39

also - most of the time vector vs. list shouldn't matter, clojure thinks a vector and list with the same elements are =, so you only need a specific implementation if you are counting on insertion order or using indexed lookup

hiredman17:01:43

a list is a list of all the elements in a list

ramon.rios17:01:57

Sorry, i will reformulate the question

mehdihajjam22:01:11

Hi all! A question for Emacs + CIDER users. I am trying to add a deps.edn alias to the call to cider-jack-in, using C-u M-x cider-jack-in RET to access the command that will be run. I know I need to add say -A:dev just before the dependency injection, but for some reason, in my buffer, only this bit is visible and editable : "-m nrepl.cmdline --middleware '["cider.nrepl/cider-middleware"]'", while in the messages buffer, after the command is run, I do see the full command : /usr/local/bin/clojure -Sdeps '{:deps {nrepl {:mvn/version "0.6.0"} cider/cider-nrepl {:mvn/version "0.21.1"}}}' -m nrepl.cmdline --middleware '["cider.nrepl/cider-middleware"]' Would someone help me find why that is the case?

dpsutton22:01:27

sounds like an old cider version. It was updated to show the full command path instead of random hard to understand snippets. what does m-x cider-version report? also come to #cider for followup

mehdihajjam22:01:08

ok will do immediately, thanks!

fappy23:01:30

hi all 🙂 using swap! on an atom could result in the provided f being executed multiple times … in my example below, f is assoc … my question is, how many times might foo-fn be executed? (swap! a-map-atom assoc :key (foo-fn 1 2))

noisesmith23:01:22

@fappy it's limited by the number of threads attempting to swap foo-fn is evaluated before swap! is called, so it only gets called once

andy.fingerhut23:01:18

I don't think there is any kind of "first in first out queue" in the implementation between threads, so the same thread T1 could lose repeatedly to another thread T2 if T2 was repeatedly doing swap! calls, yes?

andy.fingerhut23:01:48

One could certainly imagine modifying the implementation to enforce a bound like the number of threads, but that isn't what Clojure has now.

noisesmith23:01:36

oh - that's a good point, in a pathological though unlikely case two threads could ping-pong indefinitely

andy.fingerhut00:01:11

not sure if this is what you mean by ping-pong, but the pathologically bad case would be T2 repeatedly succeeding in its swap! calls, causing T1 to keep retrying inside of one swap! call, and never returning from that call.

fappy00:01:27

My actual situation is I want to assoc in a new value at :key only if it isn’t already there inside the atom. But creating my new value has side effects / is expensive … so I only want to do it once.

fappy00:01:59

And I don’t want to create it ahead of time if there is something already at :key

noisesmith00:01:17

@ but after one iteration, you can't have a forced retry on one side without a success on the other, and that means you need at least one attempted swap by another thread per retry in the current thread - because one or the other will always win

noisesmith00:01:41

which makes me think the number of swap! / thread combos is a limiting factor, unless I'm missing something

andy.fingerhut00:01:15

If T2 keeps doing swap! on the same atom in a loop, changing the value inside on each call, and always "wins the race", then T1 can be starved. I agree it is pathological. Just not provably impossible.

andy.fingerhut00:01:44

Some kind of internal FIFO queue of competing threads that was auto-created when there was contention, or something like that, could guarantee all threads make progress. The pathological behavior is so unlikely that I doubt anyone has seen more than a few retries before success in practice.

noisesmith00:01:49

@fappy with your case, foo-fn is always called, before swap - you can use @ / deref to check for the key before attempting ot swap, but there's always the narrow chance of a race where foo-fn gets called twice

fappy00:01:16

no way to avoid that race condition?

noisesmith00:01:33

if you need strict ordering without retries, agent is the only clojure mutable that has that behavior

andy.fingerhut00:01:49

Well, and if. you check if :key is absent, then call foo-fn , then try to add the as the value of :key you might find :key is present after returning from foo-fn , even though it was absent before.

fappy00:01:30

So seems like I should use an agent to hold this instead.

andy.fingerhut00:01:56

You can use a plain old Java lock to make larger transactions. In this case, though, a Clojure agent might fit your needs, since the function you send to the agent can do arbitrary things inside of a change, with no starvation, and no interruption by other threads.

noisesmith00:01:14

agents never retry, but they do have different consequences - eg send! / send-off! return immediately and asynchronously take effect

andy.fingerhut00:01:32

well, not quite arbitrary, but it can do the "check :key is absent, only then call foo-fn, then add :key with that value", and guarantee foo-fn is never called twice.

andy.fingerhut00:01:19

await exists and can be useful if you need to know that a sent action is done, although if @ is correct (and I have no reason to doubt him, as I haven't dug into the agent code to check myself), the await might not return until not only after your thread's sent actions are complete, but also perhaps some other thread's actions, too, so await might wait a bit longer than you wish.

noisesmith00:01:52

clojure.core/await
([& agents])
  Blocks the current thread (indefinitely!) until all actions
  dispatched thus far, from this thread or agent, to the agent(s) have
  occurred.  Will block on failed agents.  Will never return if
  a failed agent is restarted with :clear-actions true or shutdown-agents was called.

fappy00:01:24

so it waits for only the actions dispatched from my thread

fappy00:01:06

from this thread or agent ?

andy.fingerhut00:01:35

It says that, but apparently it might mean "at least that long", not necesarily "exactly that long, but no longer"

noisesmith00:01:33

it's an inclusive or

fappy00:01:34

sure but it seems the part I quoted is there to say it’s not intentionally waiting for all actions from all theads sent to these agents finish before unblocking …

noisesmith00:01:40

so you wait for all pending operations on that agent

fappy00:01:18

I’m reading it as all pending operations on the agents sent from the thread who called await

noisesmith00:01:30

from all threads

andy.fingerhut00:01:33

Even if it is an inclusive or, if it meant "exactly that long, but no longer", then the actions sent by other threads would not matter, I would think.

fappy00:01:05

await explicitly cannot be nested

fappy00:01:15

the source throws if you attempt

andy.fingerhut00:01:42

I don't know how you could attempt to nest them, since it is a blocking call.

fappy00:01:21

actions you send could themselves contain a call to await

andy.fingerhut00:01:56

So by "cannot be nested" you mean "cannot be called inside of a function you send to an agent"?

noisesmith00:01:09

@ I might be misunderstanding you, but here I prove that regardless of which thread calls send, await waits for all actions to be fully processed if they were queued before await was called

(cmd)user=> (def a (agent nil))
#'user/a
user=> (future (send a (fn [_] (println 1 'start) (Thread/sleep 10000) (println 1 'done))))
#object[clojure.core$future_call$reify__8454 0x1bc715b8 {:status :pending, :val nil}]
user=> 1 start
(future (send a (fn [_] (println 2 'start) (Thread/sleep 10000) (println 2 'done))))
#object[clojure.core$future_call$reify__8454 0x3e8f7922 {:status :pending, :val nil}]
(cmd)user=> (do (await a) (println 'OK))
1 done
2 start
2 done
OK
nil

andy.fingerhut00:01:11

Thanks for the example -- I never really did doubt you -- more it made me wonder why the implementation might be that way, and yet be documented with its current doc string.

noisesmith00:01:49

yeah, I wonder what nuance "thread" is offering in "this thread or agent"

fappy00:01:53

(defn await
  "Blocks the current thread (indefinitely!) until all actions
  dispatched thus far, from this thread or agent, to the agent(s) have
  occurred.  Will block on failed agents.  Will never return if
  a failed agent is restarted with :clear-actions true or shutdown-agents was called."
  {:added "1.0"
   :static true}
  [& agents]
  (io! "await in transaction"
    (when *agent*
      (throw (new Exception "Can't await in agent action")))
    (let [latch (new java.util.concurrent.CountDownLatch (count agents))
          count-down (fn [agent] (. latch (countDown)) agent)]
      (doseq [agent agents]
        (send agent count-down))
      (. latch (await)))))

noisesmith00:01:58

oh - I did the three ops in the wrong order!

fappy00:01:13

nothing you send can call await

andy.fingerhut00:01:58

It isn't the first time reasonable people have wondered at the wording of Clojure doc strings. clojure.set/union simply doesn't say anything about what happens if you pass it non-sets, which is taken by the implementers as "promises nothing whatsoever if you pass it non-sets". This doc string might be "it will not return before time X, but we don't promise when after time X it might return"

andy.fingerhut00:01:48

("This doc string" == "doc string for await ")

fappy00:01:53

If actions a and b were sent from different threads to agent foo and I (await foo) and prior to my await, my thread sent b I think await will stop blocking some time after b is complete … either before or after a runs which is the nondeterministic part

andy.fingerhut00:01:51

Sure, some time after your functions sent to the agent, await will return, not before.

fappy00:01:01

meanwhile a and b will be run in exactly the order they were sent, which is unknown to me

fappy00:01:52

@ but my point is it could unblock before a if indeed a got sent after b by some thread other than me

andy.fingerhut00:01:01

The part the doc string isn't promising anything explicit about, but it is really easy to draw the wrong conclusion about, is that it returns as soon as the last action that the calling thread sent to the agent is complete, regardless of other thread actions sent to the same agent.

andy.fingerhut00:01:10

I think the discussion is that if your thread sends b , then await will block until at least b is complete, but experiments show that in some cases the implementation will block longer.

andy.fingerhut00:01:03

If that doesn't bother you, then great. If you somehow want to write code that needs await to unblock you the moment that your thread's sent actions are done, then don't use await

fappy00:01:04

what would unblock immediately? delivery on a promise?

andy.fingerhut00:01:24

I think so, but haven't necessarily thought through it as thoroughly as someone whose livelihood and sanity rests upon a correct answer 🙂

noisesmith00:01:48

my above example was sloppy - I think this demo is much better

(defn agent-action
  [n]
  (fn [_]
    (println n :start)
    (Thread/sleep 1000)
    (println n :end)))

(defn agent-test
  []
  (let [a (agent nil)]
    (future (send a (agent-action 1))
            (await a)
            (println "wait over"))
    (future (send a (agent-action 2)))
    nil))

noisesmith00:01:15

sometimes "wait over" prints after 1, sometimes in the middle of 2, sometimes after 2

noisesmith00:01:40

where I use futures in case the thread calling send had some effect

andy.fingerhut00:01:20

No guarantees, but any problems if I grab your demo code and publish it in an article some time? Happy to give you credit by name if you don't mind.

noisesmith00:01:40

no problem at all

fappy00:01:51

Brilliant! I wonder if the possible slow return of await has ever bothered anyone. If not, I shouldn’t let it bother me 😛

fappy00:01:17

Thanks for taking the time to discuss!

noisesmith00:01:53

it's much more interesting than my current JIRA ticket :D

andy.fingerhut23:01:28

In your example, foo-fn would only be called once, for the reasons mentioned in the previous comment.

andy.fingerhut23:01:22

assoc could be called an arbitrarily large number of times, but you would need at least two threads, and typically many more, calling swap! on the same atom very frequently for it to be called more than once.

andy.fingerhut23:01:29

well, more than once doesn't require "frequently" but does require multiple threads and both calling swap! very near the same time. Calling more than twice would be even more unusual timing, but possible.