Fork me on GitHub
#beginners
<
2022-02-28
>
Nathan Rogers00:02:45

(def b {false 0
        true  1})
(defn pow [n f a]
  "pow is an APL function that applies f to a n times" 
  (letfn [(stop [_ _ x] x)
          (id [x] x)
          (start [m t]
            (let [done? (b (> m 1))
                  again ([stop pow] done?)
                  doit  ([id f] done?)]
              (again (dec m) f (doit t))))]
    (start n a)))

(pow 3 #(* 3 %) 3)
;; => 27

(pow 0 #(* 3 %) 3)
;; => 3
Branchless repeated application, although this example has an integer overflow at low inputs of n Is there a way I might flatten the call stack?

dpsutton00:02:19

Branchless might be an overstatement here

Nathan Rogers00:02:34

I suppose what I meant is that this is pure function application with no "shortcutting" or control flow expressions, like cond, if, when, or such like

Nathan Rogers00:02:15

If there is branching happening under the covers, I wouldn't know, and I'd be interested to hear where that's happening

hiredman00:02:31

Jvm bytecode doesn't expose simd stuff, so branchless code just makes things weird looking for little benefit.

hiredman00:02:37

if you are trying to write branchless code to do simd stuff, at that level a jvm method call (which lookup on a vector is) is going to be a branch (I don't do too much below the level of jvm bytecode, but I am pretty sure that is the case)

hiredman00:02:06

Like, bytecode wise, the jvm doesn't give you any conditional operation that doesn't involve branching

Nathan Rogers00:02:38

But I'm not using any conditionals or control flow statements

Nathan Rogers01:02:19

you mean vector indexing isn't just indexing?

hiredman01:02:51

Clojure vectors are not arrays

hiredman01:02:33

So at the abstraction level of the jvm, there are branches being executed

hiredman01:02:15

But even if you were using arrays, below the surface level of the jvm, those array accesses are also going to get a bounds check (which the jit may or may not optimize away)

hiredman01:02:13

A lot of it depends on your purpose for writing "branchless" code

Nathan Rogers01:02:39

Well my purpose is just to monkey around with functional patterns

Nathan Rogers01:02:04

(pow 111111111 #(+ 1 % ) 5)

;; => 110101105

Nathan Rogers01:02:15

I can call on big inputs using what's in the paste

hiredman01:02:41

If it is to take advantage of processors simd features, the jvm is just not going to cut it (I believe there are some experimental vector apis in the lastest jvm release that you can turn on with a flag maybe)

Nathan Rogers01:02:56

I'm coming from APL and I was just wondering what it would look like to implement one of the primitives from APL called ⍣ power which (f⍣n) a applies f to a n times repeatedly applying the previous result

Nathan Rogers01:02:11

and I was trying to see if there was a strictly "data flow" way to implement it

hiredman01:02:12

Clojure has iterate

Nathan Rogers01:02:19

iterate accumulates results

Nathan Rogers01:02:34

yes, power doesn't

hiredman01:02:04

Sure, iterate is not pow

hiredman01:02:12

But clojure is not apl

hiredman01:02:56

I was not suggesting that iterate is pow, but that iterate is a similar thing on clojure to pow in apl

Nathan Rogers01:02:34

({1 + ⍵}⍣10) 0
10

Nathan Rogers01:02:56

this apply 1 + the argument 10 times

Nathan Rogers01:02:23

({2 × ⍵}⍣10) 1
1024

Nathan Rogers01:02:06

but accumulation is easy to add

Nathan Rogers01:02:24

({⍵, +/ ¯2↑ ⍵}⍣10) 0 1
0 1 1 2 3 5 8 13 21 34 55 89
This gives the first 10 iterations of fibbonacci for instance

Nathan Rogers01:02:07

(pow 10 #(apply list (apply + (take 2 %)) %) [1 0])
;; => (55 34 21 13 8 5 3 2 1 1 0)

Nathan Rogers01:02:17

or using the pow I just implemented

Nathan Rogers01:02:36

but maybe I don't want accumulated results, maybe I just want the last result

Nathan Rogers01:02:47

which iterate for large inputs would mean getting the last item in a linked list

hiredman01:02:49

And if you implemented pow in the standard recursive definition with if and a base case and a recursive case it wouldn't consume stack

Nathan Rogers01:02:48

of course, but then I wouldn't have discovered an interesting functional pattern 😛

hiredman01:02:29

(if is a branch, recur is a branch)

Nathan Rogers01:02:51

But I'm not using if or recur 😛

hiredman01:02:13

Function calls are branches

Nathan Rogers01:02:48

The point isn't the semantics, the point was just to see if this would be possible with pure function application

Nathan Rogers01:02:53

like church numbers aren't exactly performant

Nathan Rogers01:02:15

But they're fascinating anyway

hiredman01:02:23

Then just encode booleans as functions, then you can write less tortured looking code while still saying it is all functions

Nathan Rogers01:02:38

(pow (b (> 0 1)) someFunction input)

Nathan Rogers01:02:47

applies someFunction to input 0 times

hiredman01:02:02

Sure, that is the contract of pow

Nathan Rogers01:02:23

yes, so the point of encoding booleans as 0/1 is to conditionally apply a function without a branch

hiredman01:02:29

But with that same contract pow can be implemented internally any number of ways

Nathan Rogers01:02:39

Sure that's a fair point, and I might do that if I plan to use this in live code somewhere

Nathan Rogers01:02:24

I suppose that's a good lessoned learned from this experiment, internal representation doesn't have to reflect the API

Nathan Rogers01:02:44

how do I read from out?

hiredman01:02:47

*out* is output only, *in* is input

hiredman01:02:56

The behavior of reading from *in* can be pretty tooling dependent

Nathan Rogers01:02:20

this profiler says "the following is printed to out"

hiredman01:02:36

Yeah, so it is printed out

Nathan Rogers01:02:42

It isn't being printed out

hiredman01:02:14

It will depend on what *out* is bound to

hiredman01:02:35

And that will depend on tooling and how you are running the code

Nathan Rogers01:02:52

cider in emacs

hiredman01:02:18

So likely the messages are going to the stdout of the jvm process, not to the repl buffer

hiredman02:02:36

Because nrepl only runs in a client/server setup it is pretty much always the case that *out* is not the same thing as the server (jvm) process stdout

Nathan Rogers02:02:12

so how do I read from that?

hiredman02:02:59

Sorry, I am assuming you mean read from it using a program

Nathan Rogers02:02:18

Well I want to know what the output is somehow

hiredman02:02:27

If you mean you just want to see it, cider should dump it on to a buffer somewhere

hiredman02:02:56

Well, actually, that depends do you have cider launching the jvm?

Nathan Rogers02:02:47

I'm just using default installation of cider, I don't know anything much more than getting it installed

hiredman02:02:53

If cider is launching the jvm and managing it within emacs, then the jvms stdout should go to a buffer

Nathan Rogers02:02:14

Well I've looked through my buffers, I don't have anything extra

hiredman02:02:04

If you are lunching the jvm using lein then in that terminal

Nathan Rogers02:02:21

I don't know anything about any of that

Nathan Rogers02:02:42

I was told lein isn't the modern way of managing dependencies, so I don't know anything about lein

Nathan Rogers02:02:52

or who is doing what with the JVM

hiredman02:02:02

So, that is all assuming that the tuft code is actually running and printing stuff out somewhere

hiredman02:02:30

Maybe before diving into that you should verify that it is in fact running

Nathan Rogers02:02:35

It's running, or at least I've imported the references, and I am calling the macros it exports

Nathan Rogers02:02:04

(profile {}
         (p :pow (kata/pow 111111111 #(- %) 5)))

hiredman02:02:10

And if you put printlns next to it do they print?

Nathan Rogers02:02:12

This does stuff, there's just no expected output

Nathan Rogers02:02:58

println just before that p form does in fact print

hiredman02:02:07

Have you follewed the 10 second example from the readme?

Nathan Rogers02:02:22

Oh, I seem to have failed to run that println handler

Nathan Rogers02:02:31

I had copied that

Nathan Rogers02:02:51

I guess I mispasted ... Apologies

Nathan Rogers02:02:04

(profile {}
         (dotimes [_ 10]
           (p :last (last (take 111111111 (iterate #(- %) 5))))
           (p :iter (take 111111111 (iterate #(- %) 5)))
           (p :pow (pow 111111111 #(- %) 5))))
;; => nil
pId           nCalls        Min      50% ≤      90% ≤      95% ≤      99% ≤        Max       Mean   MAD      Clock  Total

:last             10    12.66s     13.06s     13.64s     13.71s     13.71s     13.71s     13.13s    ±2%     2.19m     64%
:pow              10     7.15s      7.35s      7.83s      8.13s      8.13s      8.13s      7.45s    ±3%     1.24m     36%
:iter             10     2.88μs     3.38μs     4.82μs    41.09μs    41.09μs    41.09μs     7.18μs  ±94%    71.83μs     0%

Accounted                                                                                                   3.43m    100%
Clock                                                                                                       3.43m    100%

Nathan Rogers02:02:38

How would you get the final value from the take approach? I tried using last and you can see the rules. I also tried the other naive approach: (first (reverse (take ...))) and that blows the heap so I can't even profile it

Nathan Rogers02:02:51

of course :iter is much faster, insignificant time compared to the others, but to actually get the :last result is nearly double the time of the inefficient :pow

Nathan Rogers03:02:04

Trying into [] I suppose I should have expected these results

(p :vec (last (into [] (take 111111111 (iterate #(- %) 5)))))

:vec   1    18.13s     18.13s     18.13s 18.13s 18.13s 18.13s  18.13s ±0%  18.13s     72%

Rambabu Patina10:02:30

Hi, What is the best course to get started with Clojure script? with functions too such as atom

jumar11:02:08

I think Learn Reagent by @U8A5NMMGD is a good one. Perhaps combined with a Clojure(Script) book and/or tutorial

1
Søren Sjørup17:02:26

Protocols are great for objects. But what is the best approach for static interfaces(when i never have instance members or use this)? Should I just use a map of anonymous functions?

dpsutton17:02:45

Are you familiar with reify? Not quite clear what your situation is

Søren Sjørup17:02:22

I am. I have several implementations of an API(the implementations are just collections of def). And I want to write a parameterized test suite that takes an implementation as an argument so I can reuse the same suite across the different implementations.

hiredman17:02:09

A collection of defs is almost always a mistake

hiredman17:02:33

I've made plugin kind of things in the past that tried to avoid a kind of this or self argument, so all static, and it is painful

hiredman17:02:32

How do you mock/stub for tests, and when business requirements changed and you need two implementations running at once, etc

Søren Sjørup17:02:29

To give a bit more context. What I’m trying to do is write tests of Datomic, datascript, datahike, asami and other Datomic like databases. So from a test I want to be able to create databases, transact on and query them.

Søren Sjørup17:02:18

If I do it with protocols I will have an implementation object that just has references to create-database connect transact q of each implementation. But that seems to not be what protocols are meant for.

dpsutton17:02:00

That actually seems to be what protocols are for in my opinion. But i question whether those underlying things all satisfy any one protocol. Can you program against an interface without caring about the underlying db? My gut is that you cannot

Søren Sjørup17:02:41

Ok, I’ll try both and report back. Thank you! 🙂

Alex Miller (Clojure team)17:02:22

I agree with @dpsutton that seems like a protocol (and I disagree that you won't have a this :)

Søren Sjørup17:02:26

Thank you both 🙂

Søren Sjørup18:02:09

The thing is I end up with this. And that doesn’t seem right to me:

(defprotocol Implementation
  (create-database [this db-name])
  (connect [this db-name])
  (transact [this connection data])
  (make-db [this connection])
  (-q [this query params]))

(defrecord DatomicImpl []
  Implementation
  (create-database [_ db-name]
    (datomic.api/create-database (str "datomic:mem://" db-name)))
  (connect [_ db-name]
    (datomic.api/connect (str "datomic:mem://" db-name)))
  (transact [_ connection tx]
    (datomic.api/transact connection tx))
  (make-db [_ connection]
    (datomic.api/db connection))
  (-q [_ query params]
    (apply datomic.api/q query params)))

ghadi18:02:20

where this falls apart is assuming tx and query is the same for all impls of this protocol

Søren Sjørup18:02:21

I ask myself if I’m never using this should I be making a record? Why not just a map of functions?

ghadi18:02:44

need some sort of argument that makes sense across all impls

ghadi18:02:19

each impl should be in charge of translating into local specifics, and issuing the query/transactions

Søren Sjørup18:02:02

Yes definitely. But that was also my purpose. To write tests that the implementations are doing the same. And find the cases where they’re not.

ghadi18:02:05

if the users of a protocol implementation know which specific implementation is being used, something is wrong

hiredman18:02:06

even if you never use it in the implementation, it will be used in the callers when selecting which implementation to use

hiredman18:02:02

you could do the same thing with a map of functions

Søren Sjørup18:02:57

Yes the equivalent map of functions looks like this

(def map-impl
  {:create-database (fn [db-name] (datomic.api/create-database (str "datomic:mem://" db-name)))
   :connect (fn [db-name] (datomic.api/connect (str "datomic:mem://" db-name)))
   :transact datomic.api/transact
   :db datomic.api/db
   :q datomic.api/q})
And in the caller I think it’s nicer because I can just use destructuring to bind the functions I need.

ghadi18:02:31

that doesn't really resonate with me: with protocols you don't have to destructure at all, the names are top-level

Alex Miller (Clojure team)18:02:43

and have call-site caching for better performance

ghadi18:02:44

(though there is often a place for a bag of functions)

ghadi18:02:01

(I don't think this is it, though)

Søren Sjørup18:02:51

Ok thanks. It’s just that here the impl parameter bothers me a bit as I know it’s not being used in the implementation object.

_ (create-database impl "test-map")
connection (connect impl "test-map")

ghadi18:02:11

you shouldn't care about knowing

ghadi18:02:19

that's the point of polymorphism

Søren Sjørup18:02:19

But thank you very much for your input! Really appreciate it!

1
dpsutton18:02:02

agree about “care about knowing”. If you have to know which implementation you are doing, probably better to just make regular functions that are specific in your test setup. I had feared a bit about this earlier with “But i question whether those underlying things all satisfy any one protocol.”

walterl18:02:22

In @soren's case above, what would the benefits/drawbacks be of instead using multimethods like this?

(defmulti create-database
  (fn [{:keys [db-type]} db-name] db-type))
Isn't that a more data-centric representation of a database?

Alex Miller (Clojure team)18:02:18

conceptually, I think these are the same - they're both opportunities for open polymorphism

Alex Miller (Clojure team)18:02:06

protocols are narrow in use in relying on the type of the first arg (but faster and encompass multiple functions), multimethods are more general, single-method oriented

👍 1
Alex Miller (Clojure team)18:02:41

in this case, the ability to bundle methods probably makes it a better choice, but you could just as easily do it with multimethods

👍 1
hiredman18:02:48

multimethods dont have a good story for local anonymous implementations (mostly useful when testing)

☝️ 1
walterl18:02:07

Wouldn't this be a sufficiently local story?

(defmethod create-database ::test-db-type
  [_ db-name]
  ;; ...
  )

hiredman18:02:59

it isn't local though, have you ever tried to get a pull request through code review that has a defmethod inside a deftest?

hiredman18:02:22

put a reify inside a deftest and no one cares

👍 1
Michael Stokley23:02:39

suppose i want to create a v2 ns of an existing ns. i would like the v2 to support the same api as the existing ns. should i just def all the vars from v1 to v2? eg (def foo v1/foo)

hiredman23:02:41

I would not do that

hiredman23:02:02

if foo can be def'ed unchanged from v1/foo, then leave it where it is and use v1/foo when needed

Michael Stokley23:02:35

although i do think it would be convenient to callers to just have one ns to worry about. one api.

Michael Stokley23:02:34

or (ns my ns (:use …))