Fork me on GitHub
#beginners
<
2021-07-09
>
Noah Moss02:07:27

Is there an idiomatic way to reduce over a string (or collection/whatever), but have access to the position of the current character, and have the ability to conditionally skip the next n characters? Basically, a reduce function that provides the same control over the iteration as for loops do in other languages? I know I could package the current index and number of chars to skip in the accumulator, and just write a bit extra of logic around it, but I feel like there could be a cleaner/more built-in way to do this.

noisesmith16:07:46

this feature does not exist, but writing a function with this behavior on top of reduce should not be hard

Cora (she/her)02:07:56

there are a lot of ways to solve that. you could loop/recur with a position and an accumulator, but you'd want to make sure what you're reducing over is cheap to index into with a numeric index. if you don't actually need the index and you have something expensive to index into you could loop/recur using an accumulator and use nthnext to skip positions

Cora (she/her)02:07:14

or use a reduce, like you said, to track number of skips and decrement every iteration until you reach zero

phronmophobic02:07:32

not sure what your use case is, but I would also consider interop with StringBuffer , BufferedReader, or some of the other java stream processing options

hiredman02:07:38

Just use loop/recur

Noah Moss02:07:48

thanks all, I didn’t think about recur, was too focused on modeling it as a reduction

quan xing03:07:18

I call (min [{:a 1 :b 3} {:a 5 😛 0}]) and return [{:a 1, 😛 3}{:a 5,:b 0}]. The https://www.braveclojure.com/organization/#To_Catch_a_Burglar show me the result is {:a 1 😛 0} How can I get a right result.

phronmophobic04:07:10

did you also run their snippets for comparator-over-maps, min, and max?

phronmophobic04:07:40

oh, it looks like you're supposed to make your own min function based on their example

quan xing04:07:50

no. I copy this code and run (min [{:a 1 :b 3} {:a 5 :b 0}])

quan xing04:07:11

Ok. I see. thank you!

bnstvn07:07:23

is it right that compilation with direct linking means like all non-`^:dynamic` and non-`^:redef` vars would have behave like they had ^:const set on them?

bnstvn07:07:03

(def a 1)
(defn ret-a [] a)
(ret-a) ;; => 1
(def a 2)
(ret-a) ;; => 2

(def ^:const b 1)
(defn ret-b [] b)
(ret-b) ;; => 1
(def b 2)
(ret-b) ;; => 1

alexmiller09:07:22

See https://clojure.org/reference/compilation for a description of direct linking

noisesmith16:07:56

@UGSM2S2CS the problem with "behaving like they had ^:const set on them", AFAIK, is that ^:const only makes sense for primitives, and direct linking works for any compile time calculated value

hiredman08:07:44

^:const is something totally different, I have never found a good use of ^:const yet, or someone using it who actually understands what it does (I am sure they are out there), so I would just forget it exists

bnstvn09:07:37

thanks, im pretty sure i will forget soon! i am just still not sure still what direct linking does and i saw ^:const which made me think that direct linkng is pretty much the same as if everyting (non-dynamic, non-redef) would behave like const, but parameter to the compiler

bronsa09:07:30

pretty much the only good use of ^:const is to define vars holding long values that clojure can inline unboxed

bnstvn09:07:15

(def a 1)
(decompiler/decompile a)
;; ...
;; public static void __init0() {
;;        const__0 = RT.var("user", "a");
;; }
;; ...

(def ^:const b 1)
(decompiler/decompile b)
;; ...
;; public static void __init0() {
;;        const__0 = 1L;
;; }
;; ...
this is not what direct linking does? from https://clojure.org/reference/compilation#directlinking > Direct linking can be used to replace this indirection with a direct static invocation of the function instead. This will result in faster var invocation. Additionally, the compiler can remove unused vars from class initialization and direct linking will make many more vars unused. Typically this results in smaller class sizes and faster startup times.

bronsa10:07:46

no, direct linking is orthogonal

bronsa10:07:55

and it's about removing var indirection from function calls

👍 3
bronsa10:07:07

^:const is about inlining the var value in the bytecode

bnstvn09:07:32

thank you, got it eventually i think but for me the idea still feels similar — as far as i saw both ^:const on a function and direct-linking removes the var lookup @U051SS2EU • no ^:const, no direct linking:

(aot-compile '[(defn a [] nil) (defn b [] (a))] false)
=>
"
 // Decompiling class: learn_clojure/aot_compiler$b
 package learn_clojure;
 
 import clojure.lang.*;
 
 public final class aot_compiler$b extends AFunction
 {
     public static final Var const__0;
     
     public static Object invokeStatic() {
         return ((IFn)aot_compiler$b.const__0.getRawRoot()).invoke();
     }
     
     @Override
     public Object invoke() {
         return invokeStatic();
     }
     
     static {
         const__0 = RT.var(\"learn-clojure.aot-compiler\", \"a\");
     }
 }
 
 "
• no ^:const, direct linking:
(aot-compile '[(defn a [] nil) (defn b [] (a))] true)
=>
"
 // Decompiling class: learn_clojure/aot_compiler$b
 package learn_clojure;
 
 import clojure.lang.*;
 
 public final class aot_compiler$b extends AFunction
 {
     public static Object invokeStatic() {
         return aot_compiler$a.invokeStatic();
     }
     
     @Override
     public Object invoke() {
         return invokeStatic();
     }
 }
 
 "
^:const, no direct linking:
(aot-compile '[(defn a {:const true} [] nil) (defn b [] (a))] false)
=>
"
 // Decompiling class: learn_clojure/aot_compiler$b
 package learn_clojure;
 
 import clojure.lang.*;
 
 public final class aot_compiler$b extends AFunction
 {
     public static final AFn const__1;
     
     public static Object invokeStatic() {
         return aot_compiler$b.const__1.invoke();
     }
     
     @Override
     public Object invoke() {
         return invokeStatic();
     }
     
     static {
         const__1 = (AFn)RT.readString(\"#=(learn_clojure.aot_compiler$a. )\");
     }
 }
 
 "

bronsa11:07:47

you shouldn't use ^:const on a function

bronsa11:07:55

it's a very bad idea

bronsa11:07:43

it's not supposed to work, and it won't

bnstvn13:07:56

got it, thanks. just hoped this helps me to understand the machinery a bit more

quan xing08:07:25

I define the function why I run code in REPL show Null error

quan xing08:07:16

How can I read this function for comparator-over-maps, where is call fn[maps] function and what's the value in maps arguemnt. It's too hard for read. 😭

indy09:07:35

It should be

(def min (comparator-over-maps clojure.core/min [:a :b]))
(def max (comparator-over-maps clojure.core/max [:a :b]))

indy09:07:20

Or make the REPL input

(min [{:lat 1 :lng 3} {:lat 5 :lng 0}])

indy09:07:42

the ks vector that is passed to comparator-over-maps is supposed to match the possible keys in your input

quan xing11:07:27

yeah! I can run it. thanks. but I understand too hard

adityaathalye15:07:13

Try to follow process of substitution in your mind (or a scratch buffer :)):

(comparator-over-maps a-custom-fn [:a :b :c]) ;; returns a function
Write the returned function with values substituted as:
(fn [maps]
  (zipmap [:a :b :c]
          (map (fn [k] (apply a-custom-fn (map k maps)))
               [:a :b :c])))
Now assume some maps are passed to the anonymous fn, and substitute those in the fn body:
(zipmap [:a :b :c]
        (map (fn [k] 
               (apply a-custom-fn
                      (map k [{:a 1} {:a 2 :b 3}])))
             [:a :b :c]))
... and so on...

adityaathalye15:07:06

@U025AG2H55F I think you are facing a very common hurdle we all face when we start to learn Clojure/Lisp. Try to think about all your code by doing substitutions like I showed above. That will help a lot. I teach a Clojure workshop that has https://github.com/adityaathalye/clojure-by-example/blob/master/src/clojure_by_example/ex01_fundamentally_functional.clj to become more familiar with how to read and mentally evaluate Clojure expressions.

quan xing16:07:25

thanks @U055NJ5CC @U051MHSEK. my english is poor. thanks brother! thank you very much!

adityaathalye17:07:57

Any time, Quan Xing. Happy to help :) > my english is poor No matter... May your Clojure be strong :spock-hand:simple_smile

zackteo11:07:14

I'm not sure why I keep forgetting this, but what's that clojure website that allows you type an expression with a missing function and your expected output. And do to spec(?) it can infer the missing function needed

zackteo11:07:32

Thank you! I can't quite figure out what terms I need to Google to find this 😅

dgb2311:07:57

remember that it’s from borkdude (clj-kondo, babashka…) and that it resembles re-find which is a clojure core function for regex

dgb2311:07:51

I also find it quite cool but forgot several times where to find it!

👍 4
Daniel Flores Garcia18:07:08

Hello, I’m trying to run propeller (https://github.com/lspector/propeller), a genetic programming system that normally runs on Clojure with Clojurescript. Since propeller is a computationally intensive system, I’ve been getting a persistent error that I am able to replicate with the following, simple but similarly intensive program:

(ns propeller.break)

(defn factorial [x]
    (loop [n x f 1]
        (if (= n 1)
            f
              (recur (dec n) (* f n)))))

(defn intense
  [x]
  (dotimes [n (factorial x)] (println "n is " n ", ")))
To run the program from a repl, I use the following commands :
$ yarn shadow-cljs compile app
$ yarn shadow-cljs cljs-repl app
following this, I open http://localhost:8080 on chrome (as specified in a shadow-cljs.edn file in my project). After connecting to the runtime, I switch to the propeller.break namespace and run the (intense) function:
> (ns propeller.break)
> (intense 9)
running this multiple times, sometimes I get the error on my terminal:
The previously used runtime disappeared. Will attempt to pick a new one when available but your state might be gone.
However, the js console continues the process until it’s done. Even though I get the repl prompt back after the error, if I try to run something else my terminal hangs, so I think I’m not able to reconnect to the js runtime. Is there a way I can get around this? Or is this just a computational limitation I should accept?

phronmophobic19:07:05

printing 350,000 strings can take quite a while and if the repl is also sending stdout back over the repl, that can also be pretty slow. It's hard to tell if the root cause is because writing to stdout is getting backed up or just due to some computation. Do you get the same issue with:

(defn intense
  [x]
  (binding [*out* (proxy [java.io.Writer] []
                    (write [_ ])
                    (flush []))]
    (dotimes [n (reduce * (range 1 x))]
      (println "n is " n ", "))))

Daniel Flores Garcia19:07:23

Hmm, I can’t get that code to run. Do i have to require another namespace in my file/set up another dependency?

phronmophobic19:07:13

oh, this is all cljs. not sure what the cljs equivalent is

phronmophobic19:07:32

maybe:

(defn intense
  [x]
  (binding [*out* (constantly nil)]
    (dotimes [n (reduce * (range 1 x))]
      (println "n is " n ", "))))

phronmophobic19:07:50

anyway, does your original problem print a ton of stuff to stdout?

phronmophobic19:07:20

because a version without printing, which doesn't do much, seems fine

(defn intense
  [x]
  (binding [cljs.core/*print-fn* (fn [x])]
    (dotimes [n (reduce * (range 1 x))]
      (println "n is " n ", "))))

Daniel Flores Garcia19:07:52

I ran that version of intense two times. The first time I got these errors:

ERROR: XNIO001007: A channel event listener threw an exception
java.lang.AssertionError: Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.
(< (.size puts) impl/MAX-QUEUE-SIZE)
alongisde:
Exception in thread "async-dispatch-4" [2021-07-09 15:53:22.866 - WARNING] :shadow.remote.relay.local/client-not-keeping-up - {:client {:client-info {:connection-info {:remote true, :websocket true}, :since #inst "2021-07-09T19:53:06.816-00:00"}, :client-id 4, :last-ping 1625860386816, :last-pong 1625860386816}, :msg {:op :client-not-found, :client-id 3}}
java.io.IOException: UT002027: Could not send data, as the underlying web socket connection has been broken
The second time I got the same error as the one in the original post

phronmophobic20:07:02

that's weird. It doesn't seem like it's even doing much in that version

Daniel Flores Garcia20:07:57

The original problem does not print that amount of stuff to stdout

Daniel Flores Garcia20:07:30

As in the one in propeller

phronmophobic20:07:11

well, I'm stumped 😕. maybe someone else will recognize what those error messages mean

hiredman20:07:23

it looks like you are pushing code to run in a browser

hiredman20:07:33

browsers are single threaded environments

hiredman20:07:53

so if you are running some intense thing, nothing else can run

hiredman20:07:07

so any keep alives, heart beats, whatever will timeout

Daniel Flores Garcia20:07:28

I see, I guess it’s too intensive to run in a browser then. Thanks @U0NCTKEV8 and @U7RJTCH6J

hiredman20:07:19

depending on whatever it is, you may be able to break it up in a such a way as to run it in chunks that periodically yield control of the main thread so other things can run

Cora (she/her)20:07:32

you could use web workers to not block the main thread

blak3mill3r20:07:46

Since using web workers means some additional complexity, I would recommend first trying to make it yield control as @U0NCTKEV8 said. If it's a reduce you could do that fairly simply by making it write a value to a channel inside the fn that you're passing to reduce.

blak3mill3r20:07:04

Then make another go-loop that takes values off of that channel (so the buffer does not fill up)

blak3mill3r20:07:40

Perhaps there is an even simpler way, but that would work

Daniel Flores Garcia20:07:51

I’ll try these things out, see what works in the original problem. Thanks @U8QBZBHGD and @UC681SR17

noisesmith15:07:25

I'm curious about the motivation to do this work in the browser

phronmophobic20:07:11
replied to a thread:Hello, I’m trying to run propeller (https://github.com/lspector/propeller), a genetic programming system that normally runs on Clojure with Clojurescript. Since propeller is a computationally intensive system, I’ve been getting a persistent error that I am able to replicate with the following, simple but similarly intensive program: (ns propeller.break) (defn factorial [x] (loop [n x f 1] (if (= n 1) f (recur (dec n) (* f n))))) (defn intense [x] (dotimes [n (factorial x)] (println "n is " n ", "))) To run the program from a repl, I use the following commands : $ yarn shadow-cljs compile app $ yarn shadow-cljs cljs-repl app following this, I open http://localhost:8080 on chrome (as specified in a shadow-cljs.edn file in my project). After connecting to the runtime, I switch to the propeller.break namespace and run the `(intense)` function: &gt; (ns propeller.break) &gt; (intense 9) running this multiple times, sometimes I get the error on my terminal: The previously used runtime disappeared. Will attempt to pick a new one when available but your state might be gone. However, the js console continues the process until it’s done. Even though I get the repl prompt back after the error, if I try to run something else my terminal hangs, so I think I’m not able to reconnect to the js runtime. Is there a way I can get around this? Or is this just a computational limitation I should accept?

well, I'm stumped 😕. maybe someone else will recognize what those error messages mean