Fork me on GitHub
#beginners
<
2021-04-13
>
Michael Stokley01:04:10

I'm trying to programmatically create a union type out of an existing spec and a new spec. i understand this is a complete hack, but i've tried many variations of the following:

(defonce spec-ledger (atom {}))

(defn add-to-ledger!
  [k spec]
  (swap! spec-ledger update k (fnil conj #{}) spec))

(defn union-spec
  [k spec]
  (let [existing (get @spec-ledger k)
        options  (->> spec
                      (conj existing)
                      (map-indexed (fn [i p] [(keyword (str i)) p]))
                      flatten)]
    ;; return quoted form instead of evaluating because this evaluates to an
    ;; object, which can't be "embedded in code"
    `(s/or [email protected])))

(alter-var-root
 #'clojure.spec.alpha/def-impl
 (fn alter-sdef-impl [sdef-impl]
   (fn register-union-spec-instead
     [k _form spec]
     (add-to-ledger! k spec)
     (let [union              (union-spec k spec)
           ;; can't seem to call `s/def` directly here
           ;; but in order to call `s/def-impl`, need to need to generate
           ;; updated `form` and `spec` (which `s/def` would otherwise
           ;; accomplish)
           [_ _ form' spec'] (macroexpand `(s/def ~k ~union))]
       (sdef-impl k (eval form') (eval spec'))))))
this ultimately fails when it sees the output of, for example, s/or presumably because it's an object. (`Can't embed object in code, maybe print-dup not defined: [email protected]`)

Michael Stokley01:04:39

folks are suggesting s/form to go from an object to a form

Michael Stokley01:04:32

are there other strategies (other than getting rid of my existing specs or getting rid of my non-conforming data) that jump out at folks?

Eric Ihli01:04:35

What's a good way to track down what line of my code is causing a stackoverflow? I'm only getting it when I work with a really big data set. So I don't think it's an infinite loop as part of some calculation. The biggest problem I'm having debugging it is that the lines in the error are all from clojure namespaces.

LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                   RT.java:  531  clojure.lang.RT/seq
                  core.clj:  137  clojure.core/seq
                  core.clj: 2927  clojure.core/drop/step
                  core.clj: 2932  clojure.core/drop/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
The function that I call that triggers it is about 100 lines long and kind of ugly but it's got a top-level loop/recur and I don't recur by calling the function. My best guess now is to keep placing Thread/sleeps and printlns and step through the code. But with a sleep of 10ms I just watched it run for about 5 minutes and all of the output looked fine. If I uncomment the commented out sexp below, that's when I get the stackoverflow.
(def tightly-packed-trie
  (let [tight-ready-trie
        (->> trie
             (map (fn [[k v]]
                    (let [k (map #(get trie-database %) k)]
                      [k v])))
             (into (trie/make-trie)))]
    #_(tpt/tightly-packed-trie
     tight-ready-trie
     encode-fn
     decode-fn)))
That function in it's current state is at https://github.com/eihli/clj-tightly-packed-trie/blob/main/src/com/owoga/tightly_packed_trie.clj#L225

Michael Stokley01:04:12

with lazy stuff like this, try mapv instead of map

Michael Stokley01:04:52

i've had good experiences getting better stack traces doing so

Eric Ihli01:04:03

Hm. I'm iterating over a custom datatype and I guess I'm not implementing what I need to in order to use mapv yet. Method com/owoga/trie/Trie.iterator()Ljava/util/Iterator; is abstract

Michael Stokley01:04:47

there are other ways to realize a lazy sequence

Eric Ihli01:04:57

Thanks for the idea. I think it's a good lead. I haven't made any progress with it yet though. I noticed http://clojure.github.io/clojure/clojure.stacktrace-api.html and thought I could get all traces rather than just what my editor was showing me. But that's still not showing anything other than core clojure code.

Eric Ihli01:04:25

Ah ha ha... -XX:MaxJavaStackTraceDepth=-1

Eric Ihli01:04:10

Bingo. You were right. Adding a doall to a concat fixed it I think. And that stacktrace pointed straight to the line.

🙏 1
Michael Stokley02:04:20

i've heard folks say never to use concat instead of into for that reason

Edward Ciafardini02:04:15

newb question: Just getting my feet wet with clojure....every time I run any lein commands, I get this warning: OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release. 1. Does this matter? && 2. How do I make it stop?

hiredman02:04:55

It doesn't matter a ton at the moment, upgrading your lein version might make it go away

hiredman02:04:12

Lein passes (passed?) those options to speed up jvm launching, lein because of the way it works usually ends up launching two

Sebastian Ohlsson07:04:25

Hi there! How do I run a python script with sh ?

sb07:04:17

I dont know what is the goal at your side, maybe wrong idea

Sebastian Ohlsson08:04:12

Thanks for the feedback! Due to only calling a python script one time I think its overkill to add another library and would rather try to learn to sh it. I tried the second suggestion

(use '[clojure.java.shell :only [sh]])
(sh "python3" "helloworld.py")
But I got the following result:
(sh "python3" "helloworld.py")
=>
{:exit 9009,
 :out "",
 :err "Python hittades inte. Kör utan argument för att installera från Microsoft Store eller inaktivera den här genvägen från Inställningar > Hantera alias för körning av program.\r
       "}
The error is essentially "python wasnt found."

sb08:04:49

I don’t have experience w/ Windows. On mac, works fine:

user=> (use '[clojure.java.shell :only [sh]])
nil
user=> (sh "python3" "hello.py")
{:exit 0, :out "hello world\n", :err ""}
user=> (sh "python" "hello.py")
{:exit 0, :out "hello world\n", :err ""}

zackteo08:04:33

Whoops, I realised you want to specifically call python are you on windows or linux?

Sebastian Ohlsson08:04:00

Hello! Yes I started looking at the docs but sadly I am no wiser. I am on windows

zackteo08:04:22

is your helloworld.py the right path? Perhaps you might want to run (sh "pwd" to check

sb08:04:25

Bad file location case: different the error message (exit 2). I think, he need to config python3 (maybe alias setup problem) on Windows. Or simply try with `

"python file.py"
python --version (could helpful)

zackteo08:04:10

Right python might not even be on the path ... :thinking_face:

👍 1
sb08:04:02

@U01MHDJSC76 I found an answer: error 9009 “The python.exe you are using isn’t a real Python executable. It’s just a link to Microsoft Store to download it from there.” maybe true?

Sebastian Ohlsson15:04:28

I will take a look into this, thank you

👍 1
Sebastian Ohlsson09:04:04

So I finally got back to this. Turns out that I had to type "python.exe" and not just "python3". Rookie misstakes were made 🙂 As you mentioned python3 just takes me to the windows store. Maybe there are different installation ways. Its solved, thanks for all the help.

zackteo10:04:43

Nice :D glad you managed to get it working!

👏 1
zackteo08:04:11

Hello! How do I do something like

(take 100 (cycle [{:type :invoke, :f :write, :value (rand-int 5)}
                         (repeat 10 {:f :read})]))
But make the rand-int run again

raspasov08:04:31

(sequence
 (comp
  (take 100)
  (mapcat identity))
 (repeatedly
  (fn []
   [{:type :invoke, :f :write, :value (rand-int 5)}
    (repeat 10 {:f :read})])))

zackteo08:04:49

Right right! I should use a function and repeatedly

clojure-spin 1
zackteo08:04:49

Thanks 🙂

👌 1
yuhan08:04:15

You could also use recursion~

((fn rec []
   (lazy-cat
     [{:type :invoke, :f :write, :value (rand-int 5)}
      (repeat 10 {:f :read})]
     (rec))))

raspasov08:04:18

As presented atm, the above is an infinite loop. Also, if you can solve your problem in straightforward way without recursion, you should.

yuhan08:04:23

yup, don't use recursion in normal circumstances - I think it's just an interesting thing for beginners to encounter

👍 1
raspasov08:04:55

Also, on the JVM direct recursion can blow up the stack, so be careful with recursion.

zackteo08:04:14

If I want to inc the value instead, would recursion be good for that then?

zackteo08:04:09

Like

(take 100 ((fn rec [x]
             (lazy-cat
               [{:type :invoke, :f :write, :value x}
                (repeat 10 {:f :read})]
               (rec (inc x)))) 0))

raspasov08:04:30

@UUSQHP535 Probably not:

(sequence
 (comp
  (take 100)
  (mapcat identity))
 (map
  (fn [n]
   [{:type :invoke, :f :write, :value n}
    (repeat 10 {:f :read})])
  (range)))

yuhan08:04:34

If each value depends on the previous ones, you can also look at iterate or reductions

zackteo08:04:57

right right :thinking_face:

raspasov09:04:36

(do
 ;bad - avoid
 #_(let [f-bad (fn inc-inc [x]
              (if (< 100000 x)
               x
               (inc-inc (inc x))))]
  (f-bad 0))

 ;OK to use
 (let [f-good (fn inc-inc [x]
                (if (< 100000 x)
                 x
                 #(inc-inc (inc x))))]
  (trampoline f-good 0)))

raspasov09:04:12

Notice that inc-inc returns a fn in the second case.

zackteo09:04:55

so the point is to use trampoline when doing recursion?

raspasov09:04:24

Yes… I believe in the example above, you might be able avoid the problem because the recursion is from within lazy-cat… But I am not 100% sure.

zackteo09:04:03

Will look into it 🙂 thanks

Stuart09:04:38

is their a way to do destructuring but give unique names to the keys, or should I just use a let? e..g

(defn is-left [{:keys [x y]} {:keys [x y]} {:keys [x y]}]
   ;; takes 3 parameters, all 3 have x and y key. BUt I'd like it to be like:
   ;; {px py} {x0 y0} {x1 y1}
)
Should I just do this?
(defn is-left [pxy xy0 xy1]
  (let [px (:x pxy)
        py (:y pxy)
        x0 (:x xy0)
        y0 (:y xy0)
     ... so on...]))

andy.fingerhut09:04:05

There is a way to do destructuring on maps, without using :keys, where you get to pick the local names for the value. See examples here: https://clojure.org/guides/destructuring#_associative_destructuring

andy.fingerhut09:04:29

:keys is unnecessary, but a nice shorthand when you want the local names to be the same as the keyword without the :

Stuart09:04:49

Thank you, that looks like that I want. TIL!

raspasov09:04:44

I recently wrote this REPL utility to solve this “problem” 🙂 It will print the destructure form for you (for one map): https://github.com/raspasov/alexandria-clj/blob/main/src/ss/auto_let/core.cljc#L112

(ss.auto-let.core/de
 {:user/id    42
  :message/id 52
  :data       {:comment/id 62}})
[{:user/keys [id], message-id :message/id, :keys [data]} your-map {comment-id :comment/id} data]

raspasov09:04:04

(it’s not a library atm, but you can copy the namespace…)

raspasov09:04:59

It should do the “right thing” and avoid shadowing of locals for that map.

Milan Munzar11:04:25

Hello, do you guys have any tips on how to debug the 'pending puts errror' in core.async for ClojureScript? It happens after several days running on our server on nodejs platform. I understand what is happening, but have trouble identifying the code which is throwing the assertion. Thank you.