Fork me on GitHub

Is this about the scalable language?

😂 6
🔥 3

i'm willing to flip a coin on whether this is about scala or python


The original author is involved with PyPy afaik


are there any good examples of projects that effectively utilize refs?


@smith.adriane I do know that clojure.test uses a ref, but I think it might as well have been an atom:

👀 3

In similar spirit to the ant sim, , I could imagine a todo app with the following model:

(def todo-app-state
  (ref {:todos
         {:work-todos (ref [(ref {:complete? false
                                  :description "first"})
                            (ref {:complete? false
                                  :description "second"})
                            (ref {:complete? true
                                  :description "third"})])
          :home-todos (ref [(ref {:complete? false
                                  :description "first"})
                            (ref {:complete? false
                                  :description "second"})
                            (ref {:complete? true
                                  :description "third"})])})
        :filter-fn (ref :all)
        :next-todo-text (ref "")}))
This "feels" wrong, but I'm not convinced either way. I'm looking for other examples that might help think about the design tradeoffs here.


i've never actually used refs before. but i'm assuming nesting them is no good


I can't tell if that intuition has a real basis, or it comes from the fact that nesting atoms doesn't really make sense. the key difference between atoms and refs in this circumstance is that you can deref/update nested refs within a transaction to maintain consistency.


I didn't know you could nest refs and get consistency.. that sounds really hard to reason about


I would like to write something more complete than the following belief at some point, but I am pretty sure that having atoms/refs/agents nested within the values 'contained within' other things of those kinds, can lead to incorrect behavior of updates to their contents.


@U0CMVHBL2, would love to see the more complete follow up! I assumed that nested refs should be technically possible, but would love to have more details how that would or wouldn't work either way


I suspect it is simpler to create cases where atom-within-atom has update bugs, and then see if that generalizes to single-ref within single-ref examples.


well, atoms within atoms definitely fails for trying to obtains a consistent snapshot


I'll definitely ping you if I do write up anything there, but isn't among the hottest 10 things on my plate right now.

👍 3

I realize I'm definitely off in the weeds, but it's fun to explore!


I wouldn't answer if they weren't weeds I would like to understand in more detail myself 🙂

bananadance 3
😁 3

@U0CMVHBL2, one last question if you ever get time: In the ants simulation, the if the model did allow the world to shrink and expand, how would that be modeled? My assumption was that you would simply use a ref for the world rather than a var, but I would love to know if there is a better approach or if using a ref which contains other refs is fundamentally broken.


If you stop the simulation in some suitably clean way, then you could re-bind the var to a larger world, I would think. Are you asking about how one might model the world to change size in a concurrency-safe way?


I am not sure if this makes sense, but if you access every ref in the entire world in a way that would update the contents of every ref (i.e. every cell), then it seems like it might be safe to change the size of the world during such a transaction, but that is just me guessing out loud, so far.


Adding new cells seems like it might be easier in some sense, since you know the new cells cannot be involved in any transactions until after they have been added to the world. Removing cells/refs seems trickier to get right.


I've made a version of the original ants that allows the world to grow and shrink that seems to work : Overview of changes: • world is now a ref:

(def world
    (apply vector 
           (map (fn [_] 
                  (apply vector (map (fn [_] (ref (struct cell 0 0))) 
                                     (range initial-dim)))) 
                (range initial-dim)))}))
behave is now fully enclosed in a dosync with (ensure world) at the top, now checks if the ant is still within the world bounds. if not, the ant "dies", • added a function to resize the world:
(defn set-world-dim! [new-dim]
   (alter world
          update :places
          (fn [places]
            (apply vector 
                   (map (fn [i] 
                          (apply vector (map (fn [j]
                                               (get-in places [i j]
                                                       (ref (struct cell 0 0)))) 
                                             (range new-dim)))) 
                        (range new-dim))))
resizing the world while the simulation is running seems to work just fine


Nah, that's gross


I don't think @smith.adriane meant nesting refs led to consistency, but rather than transactions could ensure the consistency despite nesting


but regardless, agreed, don't put mutable things inside refs / atoms


(which includes other refs / atoms)


I understand not nesting atoms, what's the reasoning for not nesting refs?


because it means more mutable things, and you want immutable things


the generic problem is that that consistency functionality about atoms and refs relies on retries, and is totally undermined by putting mutable objects inside them


I realize this code is ugly, but I can't see how this would cause consistency issues:

 (let [work-todos (-> @todo-app-state
       a-todo (-> work-todos
   (alter a-todo update :complete? not)
   (alter work-todos (fn [work-todos]
                        (drop-last work-todos))))))


I understand putting most mutable objects within a ref would have issues, but are there issues specifically with nesting refs?


refs are subject to write skew, so if you are going to nest them you should at least use ensure

👍 3

that's good to know!


but like, if your interest is in guis, you should look at elm, and thing about the state being passed around as an immutable database, with each phase (or whatever elm calls them) returning a possibly updated database


I have indeed spent a bunch of time looking at Om, Om next, elm, react, reagent, fulcro, hoplon, cljfx, re-frame, svelte, and others


it is a perennial thing that people keep trying to build these dataflowy mutable cell things for doing uis


the rock they all founder on is starting from mutable stuff, regardless of how much you put around the mutability (stm, etc) to try and make it "safe"


the references for this would be at the edges


instead of trying to bind a ui component to a mutable cell, instead bind a ui component to a query on an immutable database and an update message to change the state of that database


ui components aren't bound to mutable cells. they're still pure functions. the idea would be to use refs as a model and produce a single immutable value from the model before handing it to the ui component


@smith.adriane What if your todo app wants undo? How could you implement that using this model?


It becomes looooots easier with immutable values


if you look at the ant sim, I believe it doesn't actually nest any reference types


The entire Clojure APIs are optimized for working with (nested) immutable data structures, update-in, etc. Diverging from that is asking for more work and more bugs.


wanting multiple layers of reference types usually means you've out grown using a map as your database and need something more powerful


so use clojure.set/index or datascript, etc


@smith.adriane Are you familiar with, reagent, re-frame etc?


I was going to mention datomic as something using refs, but of course that's not open source for the most part


(speaking of datascript)


yes, none of the cljs libs can use refs.


The tendency in CLJS front-end has been more towards one single mutable atom for the entire app state and using some kind of event or query system around that

Michael J Dorian18:06:35

Just my two cents, my game simulation has at atom that serves as a collection of agents, the agents being the actual moving parts in the game, and I've found it immensely streamlined versus a single atom that contains the universe directly


@doby162 Why do those agents have to live in an atom though?

Michael J Dorian18:06:01

Because the number of agents is every changing


i think using refs in user interfaces is an interesting design direction. I know there are other explored design spaces, but I'm curious if there is anything to learn from trying to use refs to represent models for user interfaces

Michael J Dorian18:06:16

every client connection or NPC is a new agent


@doby162 I do something similar with websocket connections in an atom in a webserver.


I realize this code is ugly, but I can't see how this would cause consistency issues:

 (let [work-todos (-> @todo-app-state
       a-todo (-> work-todos
   (alter a-todo update :complete? not)
   (alter work-todos (fn [work-todos]
                        (drop-last work-todos))))))


I would be both surprised and not surprised if that worked, but either way very confused and burnt out trying to understand what happens 😉


I am fairly certain that one can easily write straightforward-looking examples of nesting refs/agents/atoms within other instances of the same kind of thing, or other kinds of mutable references, that lead to subtle and occasional bugs when updating their values. I would not be surprised if it is possible to write such a thing that was provably correct, but would expect it would require pretty careful reasoning about the implementation of refs/atoms/agents, and/or the user-provided update functions, to prove that.


Whereas if you limit the values contained within refs/agents/atoms to immutable values only, you are in a much-easier-to-reason-about situation.

💯 3

fwiw, the idea would be have the "hard parts" done by a library


Such a library seems possible, but if I were recommending that someone use such a library, I would probably want text reasoning why it was correct, with examples of when it was correct, and when it was clearly not correct (because of update function violating assumptions of the library), where that text was 3 to 4 times longer than the implementation, at least, and demonstrated in-depth knowledge of Clojure's implementation.


currently, the UI components don't really care what reference types the model uses. the components themselves are pure functions that return immutable values. I currently have a use case where I'm trying to build a user interface for a model that does use refs and I'm thinking about what that might look like


the big issue with nesting refs is it means the only way to read them consistently is in a transaction. which if you've seen rich talk about clojure's epochal time model, he very much wanted to avoid readers impeding writers


readers still wouldn't impede writers due to mvcc


they may or may not


I guess I should clarify that the the ui components are all pure functions and don't have mutable parts "inside" them. it's still : model -> pure data -> view-fn -> pure data


the question is concerned with only the model part. currently everything is setup to work with a single atom as the model. however, I'm interested in supporting: model -> pure data where the model uses refs


hence "are there any good examples of projects that effectively utilize `ref`s?"


So Im in some weird sht with Java. As in the comment, if I create static ExecutorService consumers doesn’t run, but if its not static or if I use CompletableFuture.runAsync (common pool) it runs. Also I created WorkStealingPool since I have no control on consumer functions. (probably cpu bound tho) (it makes sense?)


what is the value of



I'd also suggest that is a the universe telling you not to use create threadpools using static initializers

😂 6

available=12 (ryzen power)


not using statics means drilling it down from the first Singleton parent object to make sure its created only once


Maybe I should just stick to common pool. No one should go crazy with their callbacks anyway, thx


I would maybe replace that while loop and whatever consumers is with a loop polling a linkedblockingqueue


collapsing all the weird conditions into a single x = q.poll(); if(x == null) ... else ...


I wouldn't be surprised it changing it to a static initializer altered the timing enough that it exposed some races/deadlocks


It's 2020 and emacs is still the only text editor with an undo tree


vim / nvim has had undo-tree for ages


(as a plugin for ui, but the underlying tree structure was always there IIRC)


yeah, confirmed, the tree is built in, the UI is a plugin


Do you actually use the tree structure of saved-in-memory Emacs undo state, then? Even with a graphical tool for exploring such a thing, I would personally feel a lot safer having those states in separate files, or at least separate revision-controlled branches in git or similar.


for the vim version, the tree introduces a new node for each recovered state from the tree, no action removes an item from that tree


I don't use it for long term differences, I use it when I realize something I deleted five minutes ago might be useful to restore


and yeah I git commit often, so that is my "checkpoint"


IntelliJ also tracks history, not sure I would want it in a tree. Even removed files are easily set back from the history of the folder.


Yep, eclipse does the same and I cannot remember not having been able to go back to an older state because the history was linear and not tree like.


I wonder what the advantages of a tree like undo would be?


a tree like undo means that I can undo, try changes, undo again, try other changes, undo again, try more changes, and all states since opening the file are still accessible


I would never want to create that many git branches


and I don't always know until later "oh, the fourth thing I tried is what I want here"


finally, this means I never leave weird code in a file "because I might want to try it again later"


I get it, thank you for explaining the advantages 🙂


I guess a three structure would help in that case. You can still do the same with IntelliJ. It's linear, but append only, so going back creates a 'revert' change.


oh, interesting - so it's like event-sourcing editor changes :D


(and thus the content of the editor is a materialized view...)