Hot damn the cognitect-aws API is good! Never before had done any AWS API work (I barely even know how AWS works), and I managed to cobble together a working Route53 utlity namespace in about an hour. Love it!

So, this:

labels-probs (sort-by :score >=
                              (map #(do {:score %2 :label %1})
                                   labels clean-probs))

java.lang.IllegalArgumentException: Comparison method violates its general contract!
        at java.util.TimSort.mergeHi(TimSort.java:899)
        at java.util.TimSort.mergeAt(TimSort.java:516)
        at java.util.TimSort.mergeCollapse(TimSort.java:439)
        at java.util.TimSort.sort(TimSort.java:245)
        at java.util.Arrays.sort(Arrays.java:1438)
        at clojure.core$sort.invokeStatic(core.clj:3115)
        at clojure.core$sort_by.invokeStatic(core.clj:3119)
        at clojure.core$sort_by.invoke(core.clj:3119)
        at closs.stages.classifiers$parse_results_threshold.invokeStatic(classifiers.clj:26)

@archibald.pontier_clo There must be a NaN somewhere in the values.

NaN's cannot be compared, so the sorting algorithm freaks out.

user=> [(= ##NaN ##NaN) (> ##NaN ##NaN) (< ##NaN ##NaN)]
[false false false]

Ok I will try using Double/isNaN to filter them out

thank you

@alexyakushev can an underflow error lead to this also ?

Because even if I filter the NaN out, it still crashes, I printed the vector I have no NaN in my vector, but 8.690312E-37 (among other)

I don't know if underflow can cause this, but this doesn't look like an underflow to me. This number is within float range it seems.

user=> Float/MIN_VALUE
1.4E-45

Could you post your vector?

(0.0 0.0 0.0 0.0 0.0 3.6712243E-31 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2708435E-36 0.0 0.0 0.0 0.0 1.6565916E-12 0.0 1.0 0.0 8.690312E-37 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)
(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2775614E-29 1.0 6.7907166E-7 0.0 0.0 0.0 0.0 3.689431E-32 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)

Can it be that println replaces values randomly ?

Could as well exclude that possibility, try pr instead of println

Try sorting (take 10 ...) of your sequence, will it still fail?

No

I would do a poor man's binary search here to try to figure out who's to blame. Something like (->> (partition-all 10 values) (map #(do (println "Sorting %") (sort %))).

I will try that, thank you for the help

You are welcome, will be interesting to know what was the problem in the end

Yeah I think I won't have a choice than diving in because the problem is not even determinist it seems

You may want to check for Float/isNaN, too. I think maybe Double/isNaN only checks for NaN's of type Double, so will not find any Float NaN values, if you have any in your data.

Never mind, some experiments I just ran at the REPL indicate that Double/isNaN returns true regardless of whether you give it a NaN of type Double or Float.

This StackOverflow answer mentions that someone encountered this exception when sorting float and/or double values, but that seems odd to me that the floating point values are not given a total order by the default comparator: https://stackoverflow.com/a/33971198

Do you perhaps have a mix of some Float and some Double values in your data?

Does anyone have experience logging to Stackdriver using clojure.tools/logging?

Using the standard appender does not do anything Dependencies:

[ch.qos.logback/logback-classic "1.2.3" :exclusions [org.slf4j/slf4j-api]]
                 [org.slf4j/slf4j-api "1.7.26"]
                 [org.slf4j/jul-to-slf4j "1.7.25"]
                 [org.slf4j/jcl-over-slf4j "1.7.25"]
                 [org.slf4j/log4j-over-slf4j "1.7.26"]
                 [org.clojure/tools.logging "0.4.1"]

                 ;; Stackdriver
                 [com.google.cloud/google-cloud-logging-logback "0.87.0-alpha"]]

<!--  -->
<appender name="STACKDRIVER" class="com.google.cloud.logging.logback.LoggingAppender">
  <!-- Optional : filter logs at or above a level -->
  <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
    <level>INFO</level>
  </filter>
  <!-- Batched logging requests get immediately flushed for logs at or above this level -->
  <flushLevel>WARN</flushLevel>
</appender>

Nevermind they do show up, but in another log

The default “java.log”

Thanks!

@andy.fingerhut so I tried something:

(let [clean-probs (into [] (aget result-array 0))
        labels-probs (try (sort-by
                           :score >=
                           (keep identity
                                 (map #(when (>= %2 0.0001)
                                         {:score %2 :label %1})
                                      labels clean-probs)))
                          (catch Exception e
                            (println e)))

Quick question, if I’m looking to build a somewhat simple event queue or pipeline for processing changes to app state, is it worth using core.async?

If it’s just a series of fns, probably not

If you want queuing and possibly stage concurrency, maybe so

cool, thanks!

(def nested-map {:a {:b 1}})
(defn my-func [m]
  ;; some magic
  )
(my-func nested-map)
;; => 1

how to achieve the ;; some magic part?

(:b (:a m)) is the simplest

right...ok

also (get-in m [:a :b])

Wondering if people have stories of using the STM in 'real world' code. I have used atoms a lot but have yet to see a use for the STM. I see some old Java code where it looks like it could be useful, if I could ever port it, but I'd need a way to coordinate the in-memory var changes with an external commit to a DB. Some online examples of STM show "bank accounts", but with no way to connect the transaction to external storage, I don't see how that would really work past a simple demo.

(comp :a :b)

I usually rely on a handler being called after the DB trx commits. Your memory changes could still fail though but the assumption is that the DB holds the Truth anyways

Ie. It’s not a proper 2 phase commit

@nbtheduke don't forget (-> m :a :b) too :)

ha I thought of it but didn't post! glad you did

macroexpands to (:b (:a m)) but reads left to right. prob faster than get-in.

oh wow, really?

I didn't know that!

not a difference that would likely matter, but last time I checked, yeah

you're basically adding another var deref/invoke so kind of has to be

I think either -> or get-in read the best, and I think that trumps other considerations

(:b (:a m)) has to be read inside-out, and comp is imo always the worst thing to read

A digression from -> but I saw this recently and was laughing at the names of the macros. The diamond wand, the trystero furcula, etc., ... https://github.com/rplevy/swiss-arrows

many have died on the hill of arrow fetishization - this is but one of many

as the greeks said, moderation in all things

less is more, etc

Haha. 🙂 Yup. I found the first few (similar to as->) useful at least.

I draw the line at understanding the word "furcula"

^^ The last arrow macro you'll need :rolling_on_the_floor_laughing: https://github.com/randomcorp/thread-first-thread-last-backwards-question-mark-as-arrow-cond-arrow-bang

now that one makes sense

we're going to add that to core

If you relax your eyes, it becomes a 3D diagram of the data flow, like those magic eye stereograms.

Hi! I need to merge a bunch of maps but the merge fn depends on the key. I couldn't find a shorter, simpler, easier to understand solution than copying merge-with to merge-with-kvv that invokes the merge fn as (f key val1 val2) instead of just with the two values. Or is there a better way?