This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-01-30
Channels
- # admin-announcements (1)
- # announcements (1)
- # babashka (8)
- # bristol-clojurians (1)
- # calva (36)
- # clojure (115)
- # clojure-europe (5)
- # clojure-italy (4)
- # clojure-nl (3)
- # clojure-norway (3)
- # clojure-uk (161)
- # clojuredesign-podcast (3)
- # clojurescript (71)
- # core-async (34)
- # cursive (26)
- # datomic (43)
- # docker (2)
- # emacs (24)
- # figwheel-main (1)
- # fulcro (36)
- # graalvm (7)
- # immutant (2)
- # jackdaw (1)
- # jobs (2)
- # leiningen (8)
- # luminus (5)
- # off-topic (29)
- # onyx (1)
- # other-languages (5)
- # pathom (6)
- # pedestal (3)
- # reagent (11)
- # ring (8)
- # shadow-cljs (42)
- # spacemacs (17)
- # specter (6)
- # tools-deps (80)
- # videos (1)
I'm running into some very strange behavior with manifold. I have two functions:
(defn example1
[]
(deferred/loop []
(deferred/let-flow
[]
"a")))
(defn example2
[]
(deferred/loop []
"a"))
If I run @(example1)
, the thread will be halted forever. If I call @(example2)
, I get a result back. Anyone familiar with manifold have any ideas here?@kenny I think this is a clue
(defmacro let-flow
"A version of `let` where deferred values that are let-bound or closed over can be treated
as if they are realized values. The body will only be executed once all of the let-bound
values, even ones only used for side effects, have been computed.
Returns a deferred value, representing the value returned by the body.
but intuitively with no bindings it should be a no-op
That's what I'd think too. The deferred documentation says "the combination of `loop` and `let-flow` can allow for the specification of highly intricate asynchronous workflows". That makes me think my code is valid.
this is what a dump of all stacks (via Ctrl-\
in the terminal) shows me when @(example2)
is blocking
"main" #1 prio=5 os_prio=31 tid=0x00007f9bb2802000 nid=0x2603 waiting on condition [0x0000700004bb7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000076cd6a238> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at manifold.deferred.Deferred.deref(deferred.clj:441)
at clojure.core$deref.invokeStatic(core.clj:2320)
at clojure.core$deref.invoke(core.clj:2306)
at user$eval1729.invokeStatic(NO_SOURCE_FILE:0)
what could it be waiting on? that seems like a bug to me, or a pathological usage of let-flow
Hmm, yeah. This does not block:
(defn example4
[]
(deferred/loop []
(deferred/let-flow
[a (deferred/future "a")]
a)))
I actually hit this case in a more complex example than the one I posted. I just worked it down to that simple example. I could probably come up with one closer to my issue.
so this is either a bug in deref-deferred, a bug in let-flow, or a pathological usage on your part that was never intended to work
I think
I have had lots of trouble getting let-flow
to work; I eventually gave up on it and wrote my own version that’s way simpler than what it tries to do
I see. In this case I don't care about running the calls in parallel. We do have several cases where running the calls in parallel is a big perf boost.
yeah, I didn’t dig into what the problem was; I just assumed let-flow was too brittle, or I was being stupid in some non-obvious way, and moved on
Ok, here's a boiled down case that I'm hitting. The below blocks forever
(defn- example10
[request-map]
(deferred/loop
[]
(deferred/let-flow
[resp (http2/request request-map)]
resp)))
@(example10
{:method :get
:url ""
:request-timeout 1000
:throw-exceptions? false})
Created an issue here https://github.com/ztellman/manifold/issues/187 on this specific issue. I really enjoy working with Manifold but these issues make it tough to justify 😩 Guessing it may be best to switch this code to core.async 🙂
core.async has its own issues (eg. all io should be in async/thread
and blocking in go blocks can lead to unexpected stalls or very low throughput under load)
not bugs per se but counterintuitive best practices
(defn simplify-declaration-xml
"Takes the verified XML specifying the node declarations for the program
and turns it into a more \"tame\" form"
[parsed-declarations]
(let [simplify-unique-field (fn [unique-field]
{:name (->> unique-field
(:content)
(filter #(= (:tag %) :name))
(first)
(:content)
(first))
:kind (->> unique-field
(:content)
(filter #(= (:tag %) :kind))
(first)
(:content)
(first))})
simplify-node (fn [node]
{:name (->> node
(:content)
(filter #(= (:tag %) :name))
(first)
(:content)
(first))
:unique-fields
(->> node
(:content)
(filter #(= (:tag %) :unique-fields))
(first)
(:content)
(mapv simplify-unique-field))})]
(->> parsed-declarations
(:content)
(first)
(:content)
(mapv simplify-node))))
but I can't seem to find a library that would simplify this that isnt marked as immature/experimental by the author
Question on nrepl
, The default mode seems to start a server that listens on a port number. However, network admin might not like this idea (in an cloud environment) for security reason. So, is it feasible to have the nrepl server to talk to an external host rather than opening a port
? (Basically, I mean because network admins are happier if traffic is outbound rather than inbound, can we still have a remote repl without opening a port?)
The long answer is nrepl (at least the last time I looked at it) is defined as an exchange of messages over any transport, so you can write your own transport to do whatever, but it can be tricky to get other tooling to work with custom transports
Yah, understand that it is tricky. But isn’t this a common use case? Don’t other people have similar need to be able to repl in cloud remote environments?
SSHing into the instance and launching a repl there (as opposed to connecting to nREPL from your computer via an exposed port) seems more secure
Anyway I try to keep production repl interaction at a minimum. e.g. for patching data, better to create a "task" to be deployed and run. It can be QAd, code-reviewed etc
At my workplace nrepl
doesn’t listen on 0.0.0.0
, so the port isn’t exposed. We connect to it via SSH tunnelling.
@U45T93RA6 I’ve done it before. Just the repl is not connected to the running Clojure application instance, right?
@U38004EG7 that’s cool, sounds more acceptable
> Just the repl is not connected to the running Clojure application instance, right? I've seen that recommended, yes
FWIW, we run a bare Socket REPL in a number of our production processes. We specify a JVM option when we start the process, which includes the port number, and it listens on localhost only, then we use SSH tunneling to enable connection from a local client (not nREPL).
That means no dependencies in production, no code needed in our processes -- the Socket REPL is built into Clojure itself, we control whether any given process starts a REPL via JVM options, and then we can use the exact same client setup that we use for local dev, since we use Socket REPLs locally for dev too.
That’s pretty cool, many thanks @U04V70XH6
We have a somewhat similar setup: each app runs local nreplserver, we SSH to the machine (no tunnels, requires going through a jump host) and launch an nrepl client via docker (more or less: docker run --rm -it --net host repl-client :connect 127.0.1:357381
)
may be the following might address this use case at some point: https://github.com/thheller/shadow-cljs/blob/master/doc/remote.md
is there something like cond->
but where I can use the value of the expression being threaded in the test expression (conditional)?
sounds like https://github.com/worldsingles/commons/blob/master/src/ws/clojure/extensions.clj#L20-L31
@roklenarcic isn't that already the case?
user=> (cond-> {:a 0} :a (assoc :b 1) :b (assoc :c 2))
{:a 0, :b 1, :c 2}
they are truthy values here
Do we have any channel to discuss about books?
I would like to know wich clojure books would it good to be.
@roklenarcic I did write some hybrid of as-> and some-> once: https://gist.github.com/borkdude/af4978cd9849357aed25144369fe007c
I guess it's a roll your own kinda thing. There's also a better-cond
library which may or may not do this.
Hello Team, Hope you are doing well. Here is the error I am getting during one of the builds in Jenkins pipeline
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class taoensso.timbre__init, compiling:(/tmp/form-init8075592954089703367.clj:1:73) Caused by: java.lang.NoClassDefFoundError: Could not initialize class taoensso.timbre__init 17:32:07 at clojure.lang.Compiler.load(Compiler.java:7514) 17:32:07 ... 12 more 17:32:07 Compilation failed: Subprocess failed
Here is my project.clj file. could you please help me here?
defproject XXXXX/XXXX "0.7.37" :description "Transaction Registration and Tracking" :url "https://github.com/XXXXX/RepoName" :plugins [[lein-sub "0.3.0"] [lein-tar "3.3.0"] [lein-set-version "0.4.1"] [lein-ancient "0.6.10"] [lein-cljfmt "0.5.6"] [test2junit "1.4.2"] [jonase/eastwood "0.3.5"] [lein-kibit "0.1.6"] [lein-ancient "0.6.15"] [lein-cloverage "1.0.10"] [lein-nvd "1.0.0"]] :sub ["types" "service" "client"] :dependencies [])
@balaji.sivaramgari I sometimes get similar error messages when I've AOT-ed something. Running lein clean
usually solves that for me
@borkdude I will try lein clean and the lein sub do cloverage. Is that your suggestion?
sure, I will try that and let you know my test results.
@borkdude, still I am getting the same error
sh "lein clean; lein sub do cloverage"
this is the command I am running
NP, thanks for your response. 🙂
I say this because I don't know for sure, but would be very surprised at any of those pulling in timbre
if it works locally you should check to see what is different about you local environment (often in ~/.lein/profiles.clj)
@hiredman so, do we need to add this "[com.taoensso/timbre "4.10.0"]" to ~/.lein/profiles.clj file?
you should look at your code, look at the stacktrace to try and figure out what is trying to load timbre, figure out why it is trying to load timbre, and if it is a good reason then you need a dependency on timbre (in the project.clj) and if it is for a bad reason you should figure out how to stop it
I think what may be going on is you have a lot of plugins, and not all of them interoperate well
so likely one subproject depends on timbre, but one of your plugins doesn't understand subprojects
so it is using only the dependencies in your toplevel project.clj while trying to load that subprojects code
@hiredman, I tried to add the [com.taoensso/timbre "4.10.0"] in dependencies section of project.clj as below. :dependencies [[com.taoensso/timbre "4.10.0"]]) but still getting the same error
@balaji.sivaramgari Each of your three subprojects have a project.clj
as well, yes? I think your error is coming from one of those...
I confirmed that none of those plugins listed brings in Timbre (`lein deps :plugin-tree`)
@U04V70XH6, so do we need to mention the ":dependencies [[com.taoensso/timbre "4.10.0"]])" in all project.clj files of sub projects?
yes, we have couple of project.clj files.
You need to share more information if you want anyone to help you debug this.
Folks, i got lost on reduce
sometimes
(defn add-attachments [attachments]
(when (some? attachments)
(reduce
(fn [attachments {:keys [filename url]}]
(conj {:type "attachment" :content (.File (get-template filename))}))
[]
attachments)))
My goal here is to add maps on a vector. Can soemone help me with it?You're calling conj
with a single argument. It should probably be (conj attachments {...})
.
(defn add-attachments [attachments]
(when (some? attachments)
(reduce
(fn [attachments {:keys [filename url]} new]
(conj new {:type "attachment" :content (.File (get-template filename))}))
attachments
[])))
[{:filename "name.pdf" :url
0m "url"}]
Still not what i wanted, but it's a advanceThank you
the first arg to the reducing function (here you call it "attachments" which is confusing) is the accumulator
The fn you pass to reduce has two params, but now you have three. Take a look at the doc for reduce where it describes that function.
(defn add-attachments [attachments]
(when (some? attachments)
(reduce
(fn [{:keys [filename url]} new]
(conj new {:type "attachment" :content (.File (get-template filename))}))
attachments
[])))
you are still trying to get "filename" from the first arg, which is wrong - the first arg is the accumulator
the second arg is the individual item
in fact your code should be fixed by just switching the order of the arg list (though the name "new" is confusing here)
(defn add-attachments [attachments]
(reduce (fn [acc attachment]
(conj acc {:type "attachment"
:content (-> attachment :filename get-template .File.)}))
[] attachments))
sorry
no need to apologize, just trying to help you understand
I admit the doc for reduce may be difficult to understand. It always helps to look at examples, see: https://clojuredocs.org/clojure.core/reduce
BTW you don't even need reduce
:
(defn add-attachments [attachments]
(mapv (fn [attachment]
{:type "attachment"
:content (-> attachment :filename get-template .File.)})
attachments))
my mind is blowing in a good way
hahah
Yeah, reduce it's a little bit tricky to understand at the beginning. I appreciate your help folks
@balaji.sivaramgari Each of your three subprojects have a project.clj
as well, yes? I think your error is coming from one of those...
Curious to know if anyone is using tap>
for anything besides debugging?
something like rebl is really kind of a repl explorer or something, not really a debugger
ah makes sense, haven’t seen much use of it i was just curious of where it’s being used
my impression is it isn't really suitable for use outside of tooling. it was introduced together with prepl which is a kind of special purpose repl for tools, but I don't know that I have seen specific guidance. I haven't used it much myself, but when I have my impression was it is best viewed as something similar to a logging statement
when you log something you want to record that some event happened usually as text for possible later viewing, with tap> you want to send some object to some tool for inspecting
yeah, when 1.10 came out I saw a few blog posts about using it to store intermediate values or log messages. putting values in it in the middle of some complicated code to inspect later for example.
if you haven't seen any of the rebl talks/demos they are pretty neat, but discussion of tap> to some degree takes a back seat to datafy/nav, but they really kind of work together
that makes sense
still need to checkout rebl
@christian.gonzalez It's worth noting that if you tap>
a lot of data, it may drop some -- it has a limit of how many items it can process at any time. I think it's 1,024.
I used it to pretty-print, that's all 😛