Fork me on GitHub
#clojure
<
2017-08-06
>
leira05:08:10

I'm reading Joy of Clojure 2nd edition, I'm confused by one of the sample code:

boot.user=> (def add-and-get
       #_=>   (let [ai (java.util.concurrent.atomic.AtomicInteger.)]
       #_=>     (fn [y] (.addAndGet ai y))))
#'boot.user/add-and-get
boot.user=> (add-and-get 2)
2
boot.user=> (add-and-get 2)
4
boot.user=> (add-and-get 7)
11
boot.user=>
I don't understand, shouldn't there be multiple ai instance every time add-and-get is called?

leira05:08:33

OK, I get it now, add-and-get is not a function, it's a variable~

tdantas10:08:53

hey guys, I was trolled by my ignorance, I was trying to prove myself that binding is using threadLocal underhood and the code above I was expecting one result and got one completely different

(defn ^:dynamic my-max [a b] (max a b))

(defn find-max [x y]
  (my-max x y))

(async/thread (binding [my-max min]
                  (do
                    (Thread/sleep 3000)
                    (async/thread 
                      (println "Expecting: 20, Actual:" (find-max 10 20))))))

tdantas10:08:14

the result was: Expecting: 20, Actual: 10

tdantas10:08:38

looks like the threadlocal was inherited

rauh10:08:47

@oliv Use (.start (Thread. (fn [] ...)))

tdantas10:08:40

The async thread is using one thread pool underhood, right ? So the threadlocal is shared among all threads inside pool

leonoel11:08:21

@oliv dynamic vars are not threadlocals (althought they happen to use threadlocal under the hood). I used to think so and got bitten by the same kind of experience you just made. Then I read the source code for vars and binding conveyor functions, and now I just don't use dynamic vars anymore, because they're so overcomplicated and counterintuitive. If what you need is a thread local variable, just use java.lang.ThreadLocal.

tdantas11:08:52

Hey thanks @leonoel , my experience was to see how binding function works. Because inside the binding function body we could remap the var , but only inside binding body . Outside remains the same

tdantas11:08:56

And I went to source code and discover is using threadlocal , but how different threads share the same value is what I didn't understand yet

noisesmith14:08:15

if you look at core.async/thread most of what it does is set up a copy of all the parent thread bindings to propagate them - it's intentional - future does the same thing

noisesmith14:08:25

(the source, that is)

tdantas14:08:24

❤️ lovely , yeah just saw that @noisesmith

leonoel11:08:17

it's done by the frame mechanism, basically when you ˋfutureˋ or ˋsendˋ or ˋasync/threadˋ you capture the bindings of the current thread, and the new thread restores the bindings before doing its job

leonoel11:08:05

obviously the result is that you have shared mutable state between threads, which is definitely not what thread locality is

ttx12:08:51

Hi! What library do you guys prefer to manage your app configuration?

ttx12:08:50

I've tried juxt/aero. But it feels like somethings are not doable with it. Like include an optional config file, only if it exists. Any suggestions?

henrik12:08:53

@ttx You could do it by using merge in the app. Maybe something like this would help:

(defn load-config
  "Load configuration. Merge default values with
  loaded configuration."
  []
  (merge {:db {:host "localhost"
               :user "temp"
               :dbname "postgres"
               :password ""
               :port 5432}}
         (when (file-exists? "config.edn")
           (aero/read-config "config.edn"))))

henrik12:08:55

Where file-exists? would be something like

(defn file-exists?
  "Check if file exists"
  [file-name]
  (.exists ( file-name)))

drowsy13:08:26

@ttx shouldn't it be easy enough to define an own conditional include tag?

drowsy13:08:08

by extending the reader multimethod in aero

joelkuiper13:08:42

with the main config as edn, and overwriting production vars with ENV or JVM flags

ttx13:08:16

@henrik Thanks! Your solution definitely works. But I was looking into something which wouldn't make me change the code much. Tried hacking a custom 'resolver' which wraps around the default resolvers, but no luck; as resolvers emit file path, not content of the file. Poor design choice IMO.

ttx13:08:12

@drowsy Good point. I'm fairly new in clojure and didn't yet dive into macros and protocols. Looks like it's the perfect opportunity. Thanks for the suggestion.

deg13:08:11

Where can I find docs of the new'ish syntax for destructuring namespaced keywords? In particular, I'm looking for the cleanest way to define a function that takes many (several dozen) un-namespaced keyword parameters as well as a few namespaced parameters (all in the same namespace, if that helps).

deg13:08:11

... found the answer to my immediate question: (defn f [{:keys [:a/x y z]}] [x y z]). But, I did not find a doc that gives the current full grammar.

alex15:08:08

Hello, maybe someone can help me with a piece of code. I have something like this: ["Name" "Price"] and `[ ["Foo" 100] ["Bar" 200] ] and I want to merge them together to this: `[ ["Name" "Price] ["Foo" 100] ["Bar" 200] ]. How can I do that?

alpox15:08:33

Hi all 🙂 i already have a question about using clojure with emacs. Is there an easy keybinding to move whole key-value pairs up/down in the listing?

deg15:08:48

@a_duerrhauer - (into (vector ["Name" "Price"]) [["Foo" 100] ["Bar" 200]])

deg15:08:37

And, note that [foo] is the same as (vector foo), and often easier to read.

deg15:08:01

@mfikes - Thanks. I did not find that page. That, eventually augmented with the 1.9 namespaced keyword syntax, would be exactly what I needed.

alex15:08:19

@deg thanks a lot!

dpsutton15:08:46

also, if you want to overwrite based on keys, you can easily turn these into maps, merge with map-like semantics, and then return back to the sequence of vectors

dpsutton15:08:48

(merge (into {} '(["key1" 2] ["key2" 3])) (into {} '(["key1" 5])))
{"key1" 5, "key2" 3}

noisesmith15:08:53

this might be simpler as

(into {} (into [["key1" 2] ["key2" 3]] [["key1" 5]]))

dpsutton18:08:01

wow. didn't know it would do that

deg15:08:25

@mfikes Hmm, so it is. Not sure why my google searches missed these. Mostly, I guess, the usual problem that it's hard to search for non-alphabetic syntax. Thanks!

noisesmith15:08:56

(vector ["Name" "Price"]) is weird - why not just ["Name" "Price"] ?

tdantas17:08:57

is there any lein command to see the final project map ?

tdantas17:08:55

I’m using lein print plugin

noisesmith19:08:54

there's :eval-in pprint iirc

noisesmith19:08:48

@oliv yeah if you add :eval-in pprint to the top level of the project map, then run lein repl or lein run it will print everything, fully filled out and expanded and pprinted

bcbradley21:08:36

why is this giving me a CompilerException? (unable to resolve symbol: f in this context):

(defn graphify [structure]
  (cond
    (set? structure) (mapcat graphify structure)
    (vector? structure)
    (mapcat
      (fn [[a b]]
        (concat
          (let [f (fn [[a b]]
                    (let [A (cond (set? a) :set (vector? a) :vector (map? a) :map)
                          B (cond (set? b) :set (vector? b) :vector (map? b) :map)]
                      (case [A B]
                        [:set :set] (mapcat f (for [i a j b] [i j]))
                        [:set :vector] (mapcat (fn [i] (f [i (nth b 0)])) a)
                        [:set :map] (mapcat (fn [i] (f [i b])) a)
                        [:vector :set] (mapcat (fn [i] (f [(peek a) i])) b)
                        [:vector :vector] (f [(peek a) (nth b 0)])
                        [:vector :map] (f [(peek a) b])
                        [:map :set] (mapcat (fn [i] (f [a i])) b)
                        [:map :vector] (f [a (nth b 0)])
                        [:map :map] [[a b]])))]
            (f [a b]))
          (concat
            (if (or (set? a) (vector? a))
              (graphify a)
              [])
            (if (or (set? b) (vector? b))
              (graphify b)
              []))))
      (partition 2 1 structure))
    :default [structure]))

noisesmith21:08:33

@bcbradley functions don't see their own let binding, but it will work if you change (fn [[a b]] ...) to (fn f [[a b]] ...)

bcbradley21:08:06

thanks i didn't know that!

noisesmith21:08:41

also, the (fn f [[a b]] ...) version will have a name with f in it in a stack trace, which can make debugging a lot easier

noisesmith21:08:26

so my code is full of (let [foo (fn this-does-foo [...] ...) ...]) just so that if foo blows up, I can easily see in the stack trace which function that is

bcbradley21:08:36

that is super helpful

bcbradley21:08:09

i may be asking too much: but do you think you could suggest a different approach to this problem?

bcbradley21:08:23

i'm just using a bunch of recursion, but the code is kind of hard to follow

bcbradley21:08:56

i'm trying to output a bunch of ordered pairs (as vectors) which form the edges of a graph

bcbradley21:08:26

its a dependency graph, and the rules for interpreting the input as as follows:

noisesmith21:08:27

that case could be a multimethod

bcbradley21:08:51

a vector like [X Y Z] implies that Z depends on Y and Y depends on X

bcbradley21:08:05

a set implies no dependency between elements in the set

bcbradley21:08:36

but the issue is that you could compose these (vectors and sets) with no bound

noisesmith21:08:55

(defmulti f (fn [a b] [(type a) (type b)])) (also, come up with a better name than "f")

bcbradley21:08:28

so a couple of sets in a vector like [#{A B} #{C D}] would need to become [{A C] [A D] [B C] [B D]] for instance

bcbradley21:08:50

Basically vectors capture the idea of "sequentially" and sets capture the idea of "concurrently"

bcbradley21:08:05

(in the dependency graph)

noisesmith21:08:04

@bcbradley the way I would usually represent that is an adjacency list, which tends to be more usable in code - so your [[A C] [A D] [B C] [B D]] would instead be {A #{C D} B #{C D}}

noisesmith21:08:28

it contains all the same info, but associatively

bcbradley21:08:43

thats not a terrible approach but i'm concerned about how it scales when you get lots of depth

noisesmith21:08:56

adjacency lists are always shallow

bcbradley21:08:09

i'm making a graphics engine for clojure, and at times its necessary to define a highly complicated order of draw calls

noisesmith21:08:19

you might need to loop more times to get to your bottom, but the data stays shallow

bcbradley21:08:26

where you "don't care" about the order of a set of draw calls, but do care about the order of sets of sets

bcbradley21:08:57

and i can't really know what the shape will be for any given application

noisesmith21:08:00

right, that works with vals being sets (everything A depends on is "equal" at that level)

noisesmith21:08:21

adjacency lists can represent arbitrary graphs, even if they have loops etc.

bcbradley21:08:31

oh wait i think i see what you are saying

bcbradley21:08:43

yeah that is a lot easier to work with

bcbradley21:08:02

i do have to repeat myself a bit

noisesmith21:08:07

walking the graph becomes a bunch of cheap hash lookups

bcbradley21:08:11

but that shouldn't be an impossible issue to handle

noisesmith21:08:17

instead of linear walks across inputs

bcbradley21:08:45

oh well i spent a couple hours making this complicated rats nest of a function for nothing lol

misha21:08:28

@noisesmith offtopic, did you release "record-as-a-function" lib/snippet yet? hi

bcbradley21:08:46

thankyou for your help

noisesmith21:08:50

@misha I got to a point that wasn't satisfying, I have code on github but it's far from ready

bcbradley21:08:51

i really do appreciate it

noisesmith21:08:59

np - glad I can help

misha21:08:07

@noisesmith did it turned out to be harder than you thought? or is it "presentation" issue?

noisesmith21:08:35

I got to a point where I realized I needed to make some hard design decisions, or I would end up with a mess

noisesmith21:08:52

so I'm doing the design work (which doesn't always look directly like work, haha, ....)

noisesmith21:08:02

hammock driven development for the win

bcbradley21:08:09

i know that feel

noisesmith21:08:53

but it is in progress and will end up here as it progresses - the current code is likely to be replaced entirely https://github.com/noisesmith/fixing-to

misha21:08:04

@noisesmith I want to merge FSM with an emit! function, and it reminded me of your idea

misha21:08:43

simply to avoid passing machine around everywhere where I need to emit events. But, on the second thought, I'd need to pass around that emit-containing-machine function anyway, so it might not be worth it opieop

misha21:08:31

(otherwise it'd end up being global var)

bcbradley21:08:02

hrm on second thought i'm not sure using an adjacency graph is ideal-- it does make lookups faster, but my use case involves converting a large amount of data into a dependency graph, which is then converted into a sequence that can be conveyed into opengl commands-- in other words its just one big batch and I don't really need to use it as a look up map

bcbradley21:08:32

the repetition inherent in an adjacency map is irritating

bcbradley21:08:45

idk though, they both have tradeoffs

bcbradley21:08:49

software engineering is hard 😞

noisesmith21:08:58

what repetition? what's redundant there?

misha21:08:20

cycle in the graph?

noisesmith21:08:00

I thought your example indicated both A and B had direct dependences to both C and D

noisesmith21:08:24

so that info isn't redundant (if that's the case) and if that's not the case, that wasn't the right translation

bcbradley21:08:34

basically if i have something like [#{A0 A1 A2 A3 .. AN} #{B0 B1 B2 B3 ... BM}], the adjacency graph will have N repeated instances of #{B0 B1 B2 B3 ... BM}

bcbradley21:08:11

i mean the adjacency graph is precise, but it isn't very compact

dpsutton21:08:40

adjacency is considered way more compact than a dependency matrix. if you have sparse data then adjacency is for sure the way to go

bcbradley21:08:40

because this is the top level abstraction, i'd like to be able to just change a given draw call in one place rather than in N places

noisesmith21:08:44

if the concern is size, you can give #{BO ... BM} a name, or even index it as a key to another map

bcbradley21:08:50

yeah the name idea was my first thought but it doesn't really jive well with what i had planned

bcbradley21:08:09

i wanted to be able to literally define a draw pipeline as a data structure, with no let or special forms

bcbradley21:08:13

just plain old .edn

bcbradley21:08:55

i'm just afraid that if I do an adjacency map it will blow up the .edn file (not a terribly big issue) and that it would be a pain to change any specific draw call (because it is repeated a bunch of times in a bunch of structures)

bcbradley21:08:30

thats the reason I had to think up a way to communicate what is basically a DAG without repeating any node

bcbradley21:08:01

and the best i could think up was attaching semantics to vectors and sets (sequentially and concurrently respectively), with arbitrary nesting

noisesmith21:08:27

you could introduce "pseudo-nodes" representing shared dependencies but not carrying any domain meaning

noisesmith21:08:39

it should be straightforward to write code that parses them out when using the data

noisesmith21:08:43

eg. you start with a->c a->d b->c b->d and end up with a->A b->A A->c A->d

noisesmith21:08:51

maybe that's weird, but it preserves dep order and reduces link count if used properly

bcbradley21:08:50

that isn't a bad idea but if i'm not mistaken writing an algorithm to optimally reduce the link count to its minimum would be tantamount to the algorithm i had already written

bcbradley21:08:09

so its just a matter of presentation

bcbradley21:08:42

in fact if you think about it, i suppose there may be many ways to present the interconnectedness of a DAG while not repeating yourself-- you could either simply not repeat yourself, or you could repeat yourself but make it less painful by only repeating small names like 'A' in the example you just gave with "pseudo nodes"

bcbradley21:08:28

idk maybe i'm overengineering all of this

bcbradley21:08:24

i guess what i'm trying to do really is super recursive so my function is ugly because it has to be ugly

bcbradley21:08:51

i just had hopes that maybe the duck didn't have to be black

noisesmith21:08:24

well, like I initially said, f would be a lot nicer as a multimethod

bcbradley21:08:48

i'll try that, do you have any other tips on how to make it a little less nasty?

bcbradley21:08:01

all these concats in particular

noisesmith21:08:36

why are you nesting concat directly inside concat anyway?

noisesmith21:08:56

(concat a (concat b c)) can always be replaced with (concat a b c)

bcbradley21:08:31

i think at some point i had a cond in between them but didn't think about simplifying when i removed it

bcbradley21:08:46

but thats a good observation, i should definitely get rid of that

noisesmith21:08:32

also (if p? a []) inside a concat context can be safely replaced with (when p? a)

noisesmith21:08:46

since concat acts on [] and nil identically

bcbradley21:08:07

i didn't think of that!

noisesmith21:08:38

also, mapcat inside concat is suspicous - not that it's never needed, but it makes me wonder

bcbradley22:08:11

recall the case for [#{...} #{...}]

noisesmith22:08:19

err, concat inside mapcat this is, of course

bcbradley22:08:32

what if you had [#{#{...}...} #{...}]

bcbradley22:08:53

i had to pick some meaning for that

bcbradley22:08:09

another example would be [[...] [...]], which is redundant with [...]

bcbradley22:08:46

basically the rules for "do these concurrently" and "do these sequentially" don't imply that "these" have to be draw calls

bcbradley22:08:58

"these" could be "do theseses"

bcbradley22:08:40

which ultimately (hopefully) are draw calls, if you keep crawling down for it

bcbradley22:08:25

i guess i've probably spammed my problem enough though, cheers and thank you for helping me

schmee22:08:02

hmm… how can I get readline functionality in a REPL launched with clojure.main/repl?

schmee22:08:29

Ideally, I would like to start a REPL that has the same settings as the one it’s launched from :thinking_face:

noisesmith22:08:49

via jreadline? maybe what you want is to make another nrepl connection to your existing one

schmee22:08:09

I want something like https://github.com/razum2um/clj-debugger, i.e. a way to start a REPL at a breakpoint with all local variables etc

schmee22:08:34

but unfortunately that lib has the aforementioned problem, as well as some others

schmee22:08:56

I’ll check out jreadline, thanks for the suggestion

schmee22:08:30

is there a way to reload a JAR you’ve modified with lein install in a REPL, without closing the REPL?

schmee22:08:53

I guess I would need to reload the classpath somehow…?

noisesmith22:08:59

you can reload the files by file path

noisesmith23:08:03

that's what editors do

noisesmith23:08:18

you can't reload the jar itself if the deps are already provided

schmee23:08:18

why not? is it a limitation in java or in clojure?

noisesmith23:08:24

there's no good way to replace some class that is already in the classpath in clojure - (except built in things like defprotocol and defrecord that are actually making a new class with the same name that replaces the old one on future lookups)

noisesmith23:08:44

I'm not sure where the limit is, but people have a lot of reason to want that to work and it doesn't so far

schmee23:08:10

I see, thanks for the heads up

noisesmith23:08:35

but you can re-load the source files without loading the new jar, and that works

noisesmith23:08:56

but it's easier to do that from the file path rather than the jar, using load-file

noisesmith23:08:34

that's the typical repl friendly workflow (whether done explicitly, or under the hood when you tell an editor to reload an ns)

noisesmith23:08:30

I have a macro that lets me minimally describe the file name, and it finds the file based on the project naming and location conventions in my directories

noisesmith23:08:07

so that (le foo bar) turns into (load-file "/home/justin/clojure/foo/src/foo/bar.clj")

schmee23:08:53

that works perfectly! 😄

schmee23:08:59

I thought I would have to reload all the dependencies as well if I did that