This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-06-05
Channels
- # aleph (4)
- # announcements (2)
- # aws (37)
- # babashka (20)
- # beginners (105)
- # calva (30)
- # cider (6)
- # clerk (8)
- # cljs-dev (2)
- # clojars (8)
- # clojure (18)
- # clojure-brasil (1)
- # clojure-europe (25)
- # clojure-nl (1)
- # clojure-norway (21)
- # clojure-sweden (7)
- # clojure-uk (5)
- # clr (2)
- # cursive (37)
- # datalevin (2)
- # emacs (6)
- # events (2)
- # graalvm (35)
- # graphql (1)
- # hyperfiddle (3)
- # lsp (4)
- # malli (1)
- # off-topic (7)
- # pedestal (7)
- # practicalli (8)
- # reitit (5)
- # releases (4)
- # remote-jobs (1)
- # shadow-cljs (15)
- # xtdb (19)
Is there prior art for a system to capture effects as data (as re-frame does) but for backend, or more general usecases? I have a sketch written out here, that has a state-modifier, and a log fx handler: https://gist.github.com/escherize/58d09deb1fde49805052b56cc65de27c
• verbs are analogous to re-frame events
• fx-handlers do the side effecting
• the script is a (potentially nested, but here just a sequential) data structure that gets walked by execute
, and ends up doing stuff.
The thing I want to explore and enable is allowing some higher level programming by letting functions return a series of operations, something like:
[:transaction
[:insert! :user {:name "bill"}]
[:update! :last-user (fn [state] {:name "bill" :id (bills-id state})]
[:query :last-user]]
;;=> [1 1 {:last-user {:name "bill" :id 7}}] ;; or similar
I think https://github.com/exoscale/interceptor are the generalized version of of what re-frame is doing. However, I've been researching algebraic effects and I think they might be the future for this type of thing. https://youtu.be/hrBq8R_kxI0?si=p1KJy1uMcDUsXk2x is a good intro video. I think algebraic effects are in a really great sweet spots because algebraic effects: • have a strong theoretical foundation for providing guarantees and reasoning about complex systems • are much easier to grok than typical FP theory like monads • are generally intuitive • have existing implementations that are highly performant The one major downside is that algebraic effects generally require support from the runtime, but some languages are already support them (eg. OCaml, Haskell) and I think some more popular languages might add support which would allow them to go mainstream (eg. rust, javascript, more?). You can also look into the free monad for prior art.
Sort of orthogonal things, having a data representation vs how the data representation is built
Effect systems can be annoyingly opaque, because basically you get the effect you want to "call" and a callback that takes the result (very much like a free monad)
The pattern I end up following on the backend is I extract complicated logic that does lots of reading of databases and calling http apis into a function that is just all the control logic and takes callbacks that do the database and http calls. It ends up being similar to what you would get from a free monad or effects system, but without having to introduce new concepts into the code base.
As for having a data representation of database operations, CAS style read-modify-write loops are key to how I usually approach concurrency control with the database, and a data representation doesn't seem like it can capture the modify and loop parts of that
You could regard log-based systems in this way... each requested operation is serialized (as a 'command' as it were), put on a log or queue, which is saved for auditing and also 'replayed' in real time (and can be replayed any other time for testing)
https://github.com/matterandvoid-space/subscriptions
https://github.com/lilactown/flex
Could have sworn there was another one - some re-
named thing, but I can't find it
Ah here it is. Maybe that works on nodejs, but it's cljs only https://github.com/lynaghk/reflex
the other thing is on the backend manipulating a database over the network has failure conditions that just don't exist when manipulating an in process store
Wondering if there’s an easy way to fix an issue i’m having with spec. Stripped down to the basics, I want to ensure that a collection is ordered. And if it’s not, ideally i’d like the ones that are out of order. But the spec is actually a property of the collection, so it prints the entire collection. ie the collection [3 2 4]
of course is unordered. But I’d love if i could get an error from spec that 3 2
is out of order rather than the entire collection [3 2 4]
. Imagine the collection is actually ~400 maps with migration ids. Printing the whole thing is murderous.
Judging by the source, you'd have to rely on the impl details and create your own spec type by extending the Spec
protocol and implementing the explain*
method in the way you want.
An alternative is to avoid relying on the explanation data and instead prepare it in some way before passing it further. You'd be able to find if the ordered spec, whatever it might be, is violated and preprocess the reported value accordingly.
Also, maybe the spec-tools
library has something useful for this, but dunno.
yeah i was hoping to run a related spec on map vector coll (rest coll)
and let that propagate back up as the error.
I honestly don't know how robust it is, but seems that you might be able to check all the :problems
one by one and see if the last spec in :via
is the order one, and then you can find unordered items from the :val
.
No clue how it all works with the multitude of spec features though.
identifying that the order spec is the failure and then post process the collection to find it? Seems workable. Just not quite sure how to return that. I’ll have to read up on some docs again. Thanks @U2FRKM4TW
A brief example:
(s/def ::x (s/map-of keyword? int?))
=> :dev/x
(s/def ::y (s/coll-of ::x))
=> :dev/y
(s/def ::z (s/coll-of ::y))
=> :dev/z
(s/explain-data ::z [[{:x 1}]])
=> nil
(s/explain-data ::z [[{:x ""}]])
=>
#:clojure.spec.alpha{:problems ({:path [1],
:pred clojure.core/int?,
:val "",
:via [:dev/z :dev/y :dev/x],
:in [0 0 :x 1]}),
:spec :dev/z,
:value [[{:x ""}]]}
Those are the :via
and :val
I was talking about.
Correction: seems that :pred
should be used, not :via
.