This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-12
Channels
- # announcements (1)
- # aws (1)
- # babashka (63)
- # beginners (108)
- # calva (12)
- # cider (6)
- # cljdoc (2)
- # cljsrn (33)
- # clojure (150)
- # clojure-europe (28)
- # clojure-nl (13)
- # clojure-spain (1)
- # clojure-spec (8)
- # clojure-uk (25)
- # clojurescript (16)
- # conjure (7)
- # cursive (7)
- # datomic (15)
- # duct (2)
- # eastwood (2)
- # figwheel (1)
- # figwheel-main (1)
- # fulcro (6)
- # graalvm (1)
- # graalvm-mobile (1)
- # helix (6)
- # honeysql (23)
- # integrant (6)
- # introduce-yourself (4)
- # jobs (10)
- # lsp (132)
- # malli (4)
- # meander (1)
- # membrane (1)
- # off-topic (223)
- # pathom (23)
- # pedestal (3)
- # re-frame (18)
- # reagent (13)
- # releases (1)
- # remote-jobs (2)
- # shadow-cljs (68)
- # tools-deps (217)
- # vim (19)
- # xtdb (79)
Does anyone here have a good book recommendation on introductory finance (or accounting)? Think programmer/engineer perspective if applicable. I recently wrote a bunch of interactive GUIs that render and display financial data (structured products) for a client and while everything went well, I feel like I would benefit from learning more about the field. I had to ask quite a few basic questions about jargon and how things relate to each other. Ty!
Perhaps https://www.amazon.com/-/es/Peter-D-Easton/dp/1618533592. I think understanding balance sheets, income statements, and cash flow statements are important when producing financial reports. See also gaap https://en.m.wikipedia.org/wiki/Generally_Accepted_Accounting_Principles_(United_States)
So the recommended approach would be to start with reading about accounting, such as double entry bookkeeping and then maybe look at financial mathematics?
I’m going off of “display financial data.” I don’t know what you need to show exactly, but there’s a strict format to financial reports, specifically balance sheets, income statements, and cash flow statements. I don’t recommend rolling your own. Other things like double entry bookkeeping probably aren’t relevant
This is almost certainly not the kind of book you're looking for if you want to learn about it the way your users learned it, but small chance it's exactly the kind of book you're looking for: https://www.amazon.com/dp/B00IEY7QJM/ it's a very mathematical treatment of accounting systems and various operations as vectors, matrices, graphs, automata, etc that might be useful for building related applications.
this is a good one for finance: https://www.amazon.com/dp/0199666598/
@U0P7ZBZCK luckily my responsibility was on the functionality side of things, the “what” was being answered by the experts, and it wasn’t that accounting related but rather a dynamic/interactive interface for structured financial products. I feel like I lack domain knowledge there that could have been useful. Certainly need to brush up my basics as well. @U015879P2F8 thank you for these suggestions. I think these go into the direction I like (and maybe a bit too far as well, but that is always interesting!)
for what it's worth, I bought the accounting one with no particular knowledge of accounting just because I saw it mentioned in a thread somewhere and it sounded interesting. I thought it was well organized and very clearly written and the authors do a good job of explaining the underlying accounting and math concepts they introduce before using them. Definitely still parts of it that went over my head, but if you're comfortable being uncomfortable there's a lot of good stuff in there.
@U015879P2F8 that sounds really cool and just about impractical and complicated enough to keep my attention! I can generally relate to the experience you’re describing! Thank you again for the recommendation!
Not really from a programmer perspective, but I found the following book a great introduction to how the various aspects of a financial report relate to each other. Highly recommended to get an understanding of the domain. https://www.amazon.com/gp/product/0470405309
Double entry accounting is how Rockefeller kept his books and impressed everyone around him from a relatively young age (18+)
so whatever you do I recommend including that somehow. Maybe there’s an interactive demo online someplace that shows it quickly.
Thanks to all the book indications everyone (added a couple to my personal library). If you do end up interested in double entry accounting @U01EFUL1A8M, I believe you should at least take a cursory look at https://plaintextaccounting.org/ and the plain text accounting tools like Ledger. It's a deep rabbit hole full of wonders if you like the hard way of doing things. Here be dragons!
You know, now that I think about it, the project Nano (originally called RaiBlocks) uses something like a double-entry ledger to keep track of balances. Makes the “blockchain” very compact, rather than making everyone keep copies of every single receipt that corresponds to fund movement, so in a practical way it’s also more amendable (fault resistant, lightweight) for keeping a distributed record of (current) balances.
This one by Martin Kleppmann is a great introduction to accounting for engineers https://martin.kleppmann.com/2011/03/07/accounting-for-computer-scientists.html
I’ve read this one and it gives a nice overview and solid data structure. I really liked it!
Don't want to add noise in the main channel as this is a very broad q: anyone here ever used state macgines, and in particular in Clojure? What have you used them for?
I’ve been using different types of state machines, mostly FSMs in GUI code (Javascript). Also recently (but slowly) I’m learning to use Fulcro (Clojure fullstack framework) where state machines are a key concept. Whether you use Clojure or not, they are a very useful tool for GUI programming, animations, video games, embedded, parsing, and likely other areas as well. The cool thing is that they can be drawn on a piece of paper very quickly and thought/talked about. They also give you a robust structure for handling state and events in code.
In Clojure we have some convenient ways to impement them. For example you can put your transitions into a map of tuples [from, to] to effects.
if you don't need your state machine to be reified as data,`letfn` and trampoline
work nicely as well - you can design your machine as a series of functions using letfn, and have each return the next function
Early in my career, I worked on a product that implemented various telephony signalling protocols, notably a few different flavours of ISDN. These were implemented as state machines. Our code was all C / C++ and had to be for performance reasons - even Java was much too slow back then. I recall state machines as a concept mapping quite nicely into the protocols that we had to implement and I imagine you could implement stuff like that very cleanly and simply in a language like Clojure nowadays.
@U053B97JX that makes tons of sense, there are several protocols that talk back and forth to signal some kind of state. TCP/IP comes to mind too!
dumb letfn / trampoline example:
(let [source (java.io.StringReader. "hello, world!")]
(letfn [(input []
(char (.read source)))
(foo []
(let [c (input)]
(println "foo consumed" c)
(condp = c
\h bar
\l baz
quux)))
(bar []
(let [c (input)]
(println "bar consumed" c)
(condp = c
\e foo
\o baz
quux)))
(baz []
(let [c (input)]
(println "baz consumed" c)
(condp = c
\l bar
quux)))
(quux []
(let [c (input)]
(println "quux consumed" c)
true))]
(trampoline foo)))
output:
foo consumed h
bar consumed e
foo consumed l
baz consumed l
bar consumed o
baz consumed ,
quux consumed
true
@U0NCTKEV8 Do you want to talk a bit about our billing state machine?
most programmers when they hear state machine, they think something like the above, basically a loop around a dispatch
clojure has all you need to create a mathematical state machine: vectors, sets, and maps
then you have a nice rich data representation, you can query, or generate your loops around dispatches, etc
there are also slightly nicer models to work with, like a finite state transducer, which maps state and input to new state and output (fsm's technically just map state and input to a new state) which I think better match what systems built around fsms do
once you have your machine defined as data, you can generate a nice face lookup table for transitions, and even things like custom case analysis macros that do exhaustiveness checking (making sure all cases are handled) at macro expansion time
the reason to have state machines in a billing system (or really any distributed system) is a state machine ends up being a way to serialize basically the program counter
there are places where we want to kind of check point the currently running program, if you define you program as a state machine it is easy to do, and often states have a nice label you can write into a database that is the state of a customer
and once you are writing that to a database you can start atomically changing it (compare-and-set! style) which allows you to ensure a linear progression of states, which is helpful to keep things correct in say a billing system
(we have a custom atom implementation backed by database records for doing a cas on this kind of thing)
the issue I've run into with state machines is that I tend to think of the current state as a derived value rather than a base value (state that's stored directly) and most of the state machine implementations treat the current state as a base value.
I don't follow
having state being derived makes it much more likely to have your system end up in a state that doesn't match something in your machine
I think what is described above is to separate the state machine into data and an interpreter right?
> having state being derived makes it much more likely to have your system end up in a state that doesn't match something in your machine right, which is why I've avoided state machines since I feel like they don't model the problem well. I can't tell if it's because I'm not applying it to the right problems or I don't understand how to use state machines properly
I'm genuinely curious since they're often recommended
more like define the state machine as data, a set of tuples, just like the mathematical definition (we actually use maps instead of sets) and then use that data to generate other stuff, a like a fast lookup table for state transitions
The utility is very obvious there @U7RJTCH6J
> Have you ever written a video game or something like that? That's most of my professional experience
I guess I'm just dense :rolling_on_the_floor_laughing:
I don't think so. Maybe I've been using the wrong definition in my head this whole time
I have only written hobby stuff, but there, it seemed a good model for simple mechanics. Think of plattformers and the like.
as an example for a simple enemy AI. You would have the base values (directly stored values) that have the positions of all the entities, the attributes of all the entities, etc. The enemies "state" wouldn't be a base value (although, it might be stored for efficiency) but logically, whether the enemy is in "attack" mode or "retreat" mode is derived from all of the base values rather than the other way around.
I was really in to dinstributed systems for a bit, and that is all state machines and/or state transition systems, for consensus, for replication, for write ahead logs
you are thinking of state broadly, in terms of some data associated with some entity
but there is other state associated with that entity, the execution state of your program
Ive noticed that they are smeared all over gui code implicitly. Think of web forms, validation and so on. We often write them ad hoc without being explicit about them.
a state machine is a way to transform that opaque function pointer or whatever bits and bytes make your cpu go into some data, and describe that relationship between that data, and some other data that represents some other execution state
right, but I've always thought of state machines as enumerating how to transition to other states. I've always found it easier and more flexible to keep track of all the raw data and decide what to do based on all data available
I would describe the approach I like as similar to the one described in http://curtclifton.net/papers/MoseleyMarks06a.pdf. I think there's something I'm not understanding about state machines. Would you say state machines are 1. compatible with Out of the Tar Pit style FRP, 2. incompatible with Out of the Tar Pit style FRP 3. orthogonal to Out of the Tar Pit style FRP My understanding of state machines is that they are incompatible
I haven't read out of the tar pit in a long time, and even when I did I am not sure I paid much attention to their proposed solution
It’s definitely compatible. @U0NCTKEV8 explained above how they store user state in a database for example.
The proposed solution is to use relational logic where possible and functional programming where not, as well as dividing modules in the broader categories of essential and accidental. At least that’s what I remember.
ootp talks about State and Control, and in that paradigm state machines are not about state, they are about control
Yes, it’s pure control quasi as opposed to relational logic. But it’s explicit and clear.
A good heuristic is whether you can draw it on a single sheet of paper and the model is still clear.
that diagram is great for understanding two well-behaved programs communicating over tcp in an idealized environment, but I'm not sure how well it represents an actual tcp program. Every step along the way has timeouts, you're not always communicating with a well-behaved program on the other end, and the environment might add further complications.
state machines are critical to the analysis of distributed systems with exactly these problems
I am not huge in to TLA+ (a language for specifying and formally verifying algorithms) but from what I have seen typically the first thing you do is describe things as a state machine, and this is usually easy to do because whatever paper you are looking at describes the algorithm as a state machine
for state machine libraries, it was my impression that the states and the transitions are provided explicitly by the programmer. for analyzing distributed systems (eg. TLA+), it was my impression that the states and transitions were derived from the raw data and how the data can change
https://github.com/ongardie/raft.tla/blob/master/raft.tla#L167 so like here is a raft (consensus algorithm) spec, where every server explicitly has a nameded state that starts as "Follower" and the spec defines how that field can change, and checking the spec comes down to asserting that with those rules, no two servers are ever leader at the same time (that is like the first part of raft, then you have to do log replication etc)
https://github.com/hiredman/raft is a "fully functional" raft impl, it does no io, and you sort of manually pass it in the current time and process it's input and output queues to run it
the raft tla does what I was saying, which is to derive the state (eg. "Restart") from the raw data (eg. votersResponded, votersGranted, etc.)
which is different from the state machine libraries I've seen, where the states are provided https://github.com/statelyai/xstate#promise-example rather than derived
Is there a state machine library where the current "state" is derived from the raw data stored in the system?
that sounds a lot like the hidden markov modeling some ad warehouses have done for click tracking
e.g. you assume there are some unknown states (the hidden part) and do some analysis of log events to try and determine structure
something like http://www.columbia.edu/~on2110/Papers/Hidden_markov_models_in_marketing_chapter_final.pdf
there are lots of architectures that do it and it's the more or less how I structure most of my programs, but I haven't described it as a state machine approach since I've never seen a "state machine" library that works like that
https://github.com/juliangamble/clojure-ants-simulation/blob/master/src/ants.clj#L170
it is a state machine, but without a defined state machine you lose all the nicer things, like checking, generating, etc
if you know all the states to start, you can check they are all handled, if you know all the outputs(for a finite state transducer), etc
you can do checking, generation, etc without the defined state machine since you can just derive the "state labels" from the raw data
I guess we'll have to agree to disagree on that
because the state machine is like a spec, if you derive it from the program, you cannot use it to check the programs correctness
you don't derive it from the program. you derive the state label from the raw data
yea, it might just be the state machine libraries that I've looked at
would love to see a better example if someone has it
our state machine code at work starts with:
(defrecord MealyMachine
[current-state state-transition state-output]
clojure.lang.IFn
(invoke [_ input]
(assert (contains? (get state-transition current-state) input)
(pr-str [current-state input]))
(let [new-state (get-in state-transition [current-state input])
output (get-in state-output [current-state input])]
[output (->MealyMachine new-state state-transition state-output)])))
which follows pretty well from the math in the wikipedia article, I don't think we ever actually use that invoke definitionthe rest of the file is a bunch of helpers for adding states to a state machine and querying properties of the machine (given a start state what all states are reachable, etc)
like, because the machine starts at :init, we assert all states loop back to :init eventually, so if a member stops their subscription, we can sell them a new one if they come back
do all of the states have human provided names?
or are there programmatically generated states?
my intuition says that 1) it would be hard to manage if there were too many states 2) that it would be hard to represent any kind of useful system without a ton of states. without any real experience, not sure if either of those things are true.
one of the helpers is something I called "graft" which grafts one state machine into another and can do some state renaming
I have seen some stuff about recursive state machines
which would help with 1&2
which allows for defining machines that represent some common sets of transitions, and then graft them onto the main machine multiple times
so the mealy machine is created semi-programatically?
if that's the case, I'm not sure there's much of a difference between what I was talking about and semi-programmatically creating state machines
a big arrow (-> empty-machine (add-transition ....) (add-transition ...) (graft ...) (for-all-states ...) ...)
every example I've seen is that all the transitions and states are enumerated explicitly by the programmer
the sort of sub machines being grafted on is really the only place where it is non-trivially programmatic
and the renaming there is, if I recall, only adding a prefix to states to keep them unique when required
yeah, graft takes a machine to graft on to, the machine being grafted, a function to changing state names, and a set of states not to change the name of
so initial billing and rebills share a lot of logic, but initial rebills have some extra bits, so the common stuff is factored out and grafted on twice once
well, I think the state machine library way to deal with that would be something like state charts
I have some code to dump the whole thing out to dot, but I haven't used it in a long time, pretty much incomprehensible that way
inspiring discussion! I really liked “a state machine turns control into data” especially. I never thought of it this way and it gets my juices flowing!
if I want to know something about the machine, I query it like a database, I think of it that way as a table where each row defines a transition
in my head, each state would encode things like: • last successful payment • current subscription • subscriptoin start date • etc. hopefully, it's not too dumb of a question, but why not just have the raw data and make the current state a pure function of the data? Or is that just another way of doing the same thing?
we have all that, but it mostly doesn't effect the control state of the machine, so it sits in the database in some other table
if you have data that you might do case analysis, or if or branch in some way on, that is control state
so there are a set of states for when auto-renew is on, and a set when auto renew is off, all those states accept a :rebill
event, but the states when auto-renew is off trigger expiration of the subscription when they get the rebill
and the way rebilling works is some code asks the billing machine what states can receive rebill, then queries the database for all the customers in one of those states (each customer has a record that records their state) and sends that customers machine the rebill event and records the new state and executes the output, because it is a finite state transducer every transition has output event along with a new state
if there's a bug or a policy update that means users should be in a different state, how do get users into the right state?
I recently asked a very vague question in this channel. It was something along the lines of “what is the name of a relational model that maps (user) events to actions/queries”
> if there’s a bug or a policy update that means users should be in a different state, how do get users into the right state? sounds like something you’d hesitate to do retrospectively, i mean the transitions happened in the old policy
depends, I have sometimes had to manually poke at the state, which involves setting both the state machine state (the control state) and the actual data state sometimes
I don't recall (this has been in production for a few years) ever removing a state, or needing to mass migrate members from one state to another
well, actually, I take that back, it grows new states in spurts and now is kind of one of those
it seems like when you add a new state, that are potentially customers who should be immediately transitioned to that state as a one time occurence
by analogy, you can imagine if you had like a billion arduinos each running a little loop of code for handling the billing for each member, if you want to manually change things you need to change the memory and registers (data) but may also need to change the location the instruction pointer is pointing to to keep execution consistent
right, that doesn't sound appealing
yeah, I have thought about it a little, and things like, well, given the init state is the start "blank" state, and is created as needed for customers, we could do something like delete the state record on transition back to init
it doesn't sound appealing for a real micro architecture, but your state machine is domain specific and not as granular as instructions
my other big project here has been the messaging (sort of text/instant messaging) service, which is a fair bit of core.async code, so state machines in there too, but not as grandiose as the billing machine
I would describe it as machines with state rather than state machines
yeah, that is what I meant, the go macro expands into the classic programmers state machine implementation
a key difference is that the state machines are programmatically created and programmatically manipulated. it seems like the go
macro is an example of making it so that you can avoid explicitly naming and working with states and transitions
each case is a state, control flow defines the transitions between the states, I don't entirely recall, but I think it was while I was digging through the macroexpansion of a go macro when I first made the connection between state machines and control state
it works in core.async, to generate the names, because everything lives on one computer
actually, maybe aws step functions, they define a state machine, but the state machine is not exposed
https://gist.github.com/hiredman/60e5e6f0025cb38c8a7739d21f5a8ce6 is another state machine thing, define a (communication) protocol as a state machine and then check that clients and servers are obeying that protocol
the states in this example are active and start?
Yes, the link to the ubf page at the top has an example of specifying an irc like chat service and the intent was to replicate that example
As an alternative to Yet Another State Management System This also jives well with the Mealy machine model, the state machine is like the CPU and IO/state can be handled by another system entirely
What do you think about the idea if reifying side effects as data and passing them off to another system? Too / unnecessarily complex or a good idea?
Aws step functions basically do this, the state language doesn't let you express any kind of io you have to aws lambdas
And that isn't dissimilar to how I use a finite state transducer instead of a finite state machine, for every input event I get a state transition and an output event
The events and states are keywords, and the output event is handed off to some other code to determine which actions are required for that output event
What do you do with the state machine associated with the specific event? Do you send it on with the event or does the result come back via some input channel and has metadata on it which associates it back to that particular event
During actual execution the state machine isn't really used, there is a macro that bases on the state machine generates a function which given a state and an input event returns a pair of new state and output event
There is nothing to send along or return, the current state for a customer is kept in the database associated with their account
We don't actually send output events on a queue to another system, we have a system where output events can have a chance to run some code pre and post commit of the state change
You want to do X to a customer, you get their current state, send the input event, get the new state and the output event, run the pre commit part of the output event, compare and set the old state in the database to the new state, and if that succeeds run the commit part of the output event
Your use case has state living across a very long time and space. I'm thinking about something which is localized to one process and very transients. Even relatively short lived events in an event-driven system can be handled by a state machine and not spaghetti
We also use fst to handle entity reconciliation, ex to make converge hardware/entity state with db state (ex desired VM state vs actual/current state of the vm)
It's just the bottom layer of some complicated stuff to make that happen with signals competing/coming from various sources, with various priority levels.
some nice things we get for free: compile time validation of transition tables & nice graphs for documentation, plus it's data in/out so easy to test/dev against.
I like this. If you turn control into data, then you can test control more directly instead of doing blackbox testing and mocking.
This also lets you use interesting algorithms if performance becomes an issue. If your signals are strings you can compile tries. you can use a library like core.match if you need to dispatch on multiple keys in a map, etc.
https://fabricate-site.github.io/fabricate/finite-schema-machines.html
If anyone in this thread is interested in a FSM implementation where the states are defined purely by the schemas matched by data being passed through the system, I just wrote up my own experience of using malli
to build a very simple version of that concept.
To be honest, I don't know enough about automata theory to determine whether this implementation is a Mealy machine, Moore machine, or something else entirely.
I just tried having .edn files with plain hiccup instead of HTML files. The idea was just to have a better editing experience with plain HTML. I found myself wanting to slurp HTML, split an expression in two. Or just have less brackes. So far, I'm liking it. Semi-live preview with `find . | entr -c clojure -X:build`. I'm curious about how others prefer to write HTML.
For babashka's book I made a watch script which spins up a browser and refreshes it every time I make an edit: https://github.com/babashka/book/blob/master/script/watch.clj