so a pretty simple problem here... I have a nested structure from which I need to produce some sequential data, however I also need to derive some accumulated information from the outer "loop" for the data. I can solve the problem with a loop, reduce or even with an atom but none feel very clojury. should I just go with a loop and two arguments and an inner for, or is there some other function that captures these aspects? something like a list comprehension with loop bindings

if you need updated bindings as you consume a nested structure, you could consider (reduce f {} (tree-seq coll? seq input)) where f can update whichever keys of the hash-map you might need to represent a composite state

that's the most general - it could be you could have a simpler value for an accumulator instead of a hash map, or a simpler form like (apply concat ...) instead of tree-seq depending on your task

but if you are consuming something from start to end, reduce beats loop because it avoids the boiler plate of navigation

for works for consuming one collection in order and generating a lazy collection from it, or working on all combinations of items from N inputs, but it's not a loop

I wouldn't advise an atom if you aren't coordinating mutable states from multiple threads, and it doesn't sound like that's called for here at all

thanks @noisesmith. I love tree-seq! just couldn't find any mention of it when I was googling for nested structures

it's kind of clumsy but it's great if you don't have control of a weirdly shaped data structure as long as you can make a predicate that detects and filters the leaves or subtrees you need

obligatory: even better is producing the best data to work with in the first place (if that's under your control)

sure, but seems much higher-order than the loop. the data is fully in my control and works for other things. I could easily solve the problem by producing cached values beforehand but trying not to

in the end, I went with reduce though. tree-seq is great but I need to produce the info during walking of the tree, not only at leafs, and at the end, I'm producing sequences of sequences no matter what, so might just as well apply concat the final result

is there a better way t owrite a clojure test that says: "this code does not throw an exception?" the best I have s ofar is: (ct/is (do ... true))

@qqq Using Expectations, I'd probably say (expect any? (some-code))

So with clojure.test I guess that would be (ct/is (any? (some-code)))...

If you wanted to make it kinda self-documenting, you could do something like (ct/is (:no-exception (try ... :no-exception (catch Exception e :exception))))

hey folks, how can i change the ::start-fn spec to validate the return value of the function against ::component spec. http://dpaste.com/09FZ438 (line 6)

::start-fn is a part of ::component spec, So it's like a recursive spec. The above code isn't working, basically spec can't exercise the ::start-fn

That is strange, when I call twitter.api.restful/account-verify-credentials during the course of my program, it won't exit even when I put (shutdown-agents) at the end

I would like to avoid calling (System/exit 0) in order to make it still runnable from the repl without killing it

is there some other threading system which needs to be shut down explicitly?

it doesn't halt at (shutdown-agents) or even (await) btw, so there don't seem to be any agents running

Anybody using clojure for high speed analytics?

yupe

anyone knows how I can interpret this from the thread dump?

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f173c14a800 nid=0x106a in Object.wait() [0x00007f17126a9000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000006c70422b0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
	- locked <0x00000006c70422b0> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

does that mean the thread is waiting on its own lock?

not super familiar with how java mutexes work, but sure looks like it’s deadlocked.

I think I haven't fully grasped how logging works in Clojure

I'm trying to use Timbre now with a ring server, and I add this simple config

(defn -main [& args]
  (tm/merge-config!
   {:level :info
    :ns-blacklist ["org.eclipse.jetty.*"]})

but it doesn't seem to care at all and I still see all these logs

17:31:44.964 [qtp1581040670-27] DEBUG org.eclipse.jetty.io.AbstractEndPoint - onClose [email protected]{/0:0:0:0:0:0:0:1:57244<->3001,CLOSED,ISHUT,OSHUT,-,-,27903/200000,HttpConnection}{io=0,kio=-1,kro=-1}

however if I log directly with timbre functions they get filtered out correctly

so maybe I need one of the various adapters to make the two logging systems talk together?

Does anyone have experience or know of prior art using core.logic for probabilistic logic (e.g. markov logic networks)?

No, but you might be interested in Anglican https://probprog.github.io/anglican/index.html

Very interesting, thanks @alexmiller 👍

is there a way to write a function/macro, which will qualify arg's symbol in an application ns and return a qualified keyword?

(ns user.utils)
(defn ->kw [x])

(ns user.foo
  (:require
   [[user.utils :as u]
    [clojure.string :as str]]))
(defn f []
  (u/->kw str/join))

(ns user.bar
  (:require [user.foo :as foo])
  (foo/f)) ;; => :clojure.string/join

I got it working for args which are functions, but another use case is a map. The global goal here is to avoid hardcoding fn names in Onyx jobs, which is prone to refactoring errors

All is good for task descriptions (where every interesting keyword is a function name), but for lifecycles, e.g. for :read-from-kafka task, you need to pass :onyx.kafka.plugin/write-messages-calls, which is a hashmap

I have a macro, able to do that, but it relies on application-site ns passed in as an arg (not even a *ns*), so this gets ugly af from the get go

the solution for functions is:

(defn fn->kw [f]
  {:pre [(fn? f)]}
  (->> (-> f               ;; str/ends-with?
         str               ;; "[email protected]"
         repl/demunge      ;; "clojure.string/[email protected]"
         (str/split #"@")) ;; ["clojure.string/ends-with?" "65576657"]
    (butlast)              ;; ["clojure.string/ends-with?"]
    (str/join "@")         ;; "clojure.string/ends-with?"
    keyword))              ;; :clojure.string/ends-with?

why would a macro need the application site value of *ns* to be passed in?

the value of ns from inside a macro is always the value of *ns* as bound in the application site

there are 3 distinct places: ns with defmacro ns where macro is applied ns where function-inside-which-macro-is-applied is called

and you need the last one?

the way I wrote macro 1st time, it resolved passed using aliases map in the *ns*, so the aliases map were not from the second place, but from 3rd

yes, I thought I got it, until I found out, that unqualified args get resolved not as "ns.2/arg", but as "ns.3/arg"

implementation is

(defn -impl [ns sym]
  (let [aliases (ns-aliases ns)
        ns* (or (some-> sym namespace symbol aliases)
              ns)]
    (keyword
      (.getName ^clojure.lang.Namespace ns*)
      (name sym))))

(defmacro ->kw [ns sym]
  `(-impl ~ns '~sym))

this works, I think, but requires

(def ns *ns*)
(defn f []
  (->kw ns x))

is there some introspection "trick" to see which ns fn or macro is applied in?

oh, I just comprehended what you said about *ns* and macro application side.

I just need to pass *ns* in defmacro to -impl

the trick is making it see any ns other than the one which is using the macro

(defmacro ->kw [sym]
  `(-impl ~*ns* '~sym))

that should work

it does 😄

sweet!

it looks weird, but you can replace that macro body to one that doesn’t use syntax-quote - (-impl *ns* sym) (edit - that first ns/ wasn’t needed because you aren’t trying to emit impl)

oh wait, never mind, you do want to emit a call to impl don’t you

yeah, everything is a :onyx.kafka.plugin/sym now

instead of :onyx.kafka.plugin/read-messages

oh - I edited to get rid of the ' which shouldn’t be there

(-impl *ns* sym) works. thank you

(I did not reload things in repl properly first time)

this means I can get rid of impl, and just put everything in macro, doesn't it?

if that clarifies things, I think it would make sense - I mean, it kind of looks like you can just replace defn -impl with defmacro ->kw in the form and that’s just the thing

though having the logic in a defn does improve things like being able to unit test