Clojurians
#clojure
<
2017-07-21
>

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

lwhorton00:07:06

i feel like i’m missing 2 things about protocols; 1) if a record implements a protocol, and that record has fields defined in defrecord MyRecord [foo], what’s the difference between using foo and (:foo this) in a protocol function’s implementation? 2) how does a protocol call one of the other functions defined in its own protocol interface? I’m getting compiler warnings about my-protocol-func not defined if attempting to do something like

IMyProtocol
(foo [_ a] ...)
(bar [_ a] (foo _ a)

noisesmith00:07:06

protocol functions need to be used the same way you would any other function

hiredman00:07:07

you look like you might be using the interface generated by the protocol instead of the protocol

noisesmith00:07:11

they belong to the other namespace

noisesmith00:07:53

you could also use the method call syntax .foo without namespacing of course, but it’s better to use the protocol function

lwhorton00:07:13

ahh.. they are defined in the protocols.cljs so you have to protocols/foo in the above example

noisesmith00:07:49

very common beginning mistake with protocols - to think their functions are like methods

noisesmith00:07:05

(and they are - but not in this aspect)

lwhorton00:07:06

so the this context is kind of the method context? “invoke my protocol func on this

noisesmith00:07:38

it’s the object that the method was called on - and don’t call it _ if you are going to call a function on it

bfabry00:07:44

I would say the answer to your first question is "there is no difference use whichever you prefer"

noisesmith00:07:58

_ is a naming convention that means “I plan on ignoring this, on purpose”

hiredman00:07:15

the behavior of this is likely slightly different in clojurescript vs. clojure

bfabry00:07:28

although honestly I haven't used a defrecord in 2 years so I may be misremembering something

hiredman00:07:52

(this meaning the first argument to something defined inline on a defrecord, deftype, or reify)

noisesmith00:07:04

I suspect that using the name of the field directly leads to a faster lookup, but if that difference means anything in your code you probably want something more performant than a record anyway

lwhorton00:07:24

so even if I invoke (protocols/foo this), it still doesnt matter if I refer to the record’s field or the (:foo this) because at that point the context has already changed

hiredman00:07:05

the first argument is always going to be the thing you invoked the protocol function

lwhorton00:07:15

(defrecord MyRecord [field]
  IMyProto
  (foo [this a] (do-something field)) <- here shouldnt matter if I use field or (:field this)?
  (bar [this a] (protocols/foo this a)

lwhorton00:07:59

I just want to make sure I dont get this very subtle bug by implementing a protocol that’s using a field versus a lookup on this, when self-referencing(?) a protocol

bfabry00:07:00

field and (:field this) will always be the same thing

lwhorton00:07:19

i’ve been stabbed in the foot by javascript so many times I’m very paranoid

lwhorton00:07:48

okay, good to know the context stays the same no matter how it’s invoked

lwhorton00:07:14

thanks @hiredman @bfabry @noisesmith :thumbsup:

lwhorton00:07:57

let my_obj = {
  lolGoodLuck(args) {
    let that = this
    function() {
      that.call(this, args)
    }
  }
}
my_obj.lolGoodLuck.apply(youWillNeverGuess)

josh_tackett03:07:14

Got a tough problem if anyone has an answer. Basically I have a large data structure (vector of hashmaps) about 50,000 hashmaps. And I need to pass the data to a Java function, but the hashmap has keyword keywords and those keywords need to be transferred to strings for the java function. Is there a fast way to convert keyword keywords to strings? right now the conversion takes about 2 seconds

potetm03:07:58

Use strings from the start instead of keywords?

potetm03:07:00

You're talking linear time + string manipulation. Probably not a fast operation no matter how you slice it.

noisesmith03:07:45

name shouldn't be manipulating strings - it should just be getting a field from the keyword

potetm03:07:59

Turning string data into keywords is a bad habit that clojure developers have. Benefit is really only concision, downsides are slow and buggy translation.

noisesmith03:07:06

(I mean, if you are converting by manipulating strings, use name instead)

potetm03:07:19

Yeah right. Fair enough.

joshjones04:07:16

@josh_tackett I just generated 50,000 keywords, at 50 random characters each (using c.s.gen), and both str and name take about <10ms to convert the whole vector from keywords to strings

(defn trunc [s n]
  (subs s 0 (min (count s) n)))

(def kws
  (doall
    (gen/sample
      (gen/fmap
        #(keyword (trunc (name %) 50))
        (gen/keyword))
      50000))

(time (do (doall (map name kws)) nil))
"Elapsed time: 5.684874 msecs"
=> nil
(time (do (doall (map str kws)) nil))
"Elapsed time: 5.781799 msecs"
=> nil

joshjones04:07:59

(where gen is clojure.spec.gen.alpha)

josh_tackett04:07:57

@joshjones ya your test doesn’t compare as I am talking about a series of hashmaps with multiple keys in each one

josh_tackett04:07:53

but @potetm your suggestion is what I landed on, so I’ll just change my json decoding to get string keys

joshjones04:07:01

oh, so there are many such keywords in each hashmap?

josh_tackett04:07:16

@joshjones correct

joshjones04:07:17

how many total?

noisesmith04:07:25

I bet the map construction when converting is the more expensive part, not the keyword/string part

noisesmith04:07:32

but avoiding converting avoids both

seancorfield04:07:17

@lwhorton re: field vs (:field this) -- I asked a while back and Alex said to use field as it will be faster and it's the "correct" way to deal with records /cc @bfabry @hiredman @noisesmith

solve-calc.com06:07:33

I'm using boot for my build system. Is there a library to help with the following task. I have a function TRANSLATE, which does .foo -&gt; .clj. I also have a bunch of .foo files. Whenever any of them changes, I want TRANSLATE to be called to generate the corresponding .clj file. Is there a library to handle this for me?

madstap06:07:35

@solve-calc.com This should probably do the trick https://github.com/wkf/hawk

madstap06:07:01

Or, actually, I think boot has that as a built in task

solve-calc.com06:07:23

I see now. The core of my problem is just a "watch + notify." Right, there's a boot task. Boot can do this already for me.

isankhairul08:07:14

hi everyone, i want ask, how create multiple connection database or host in sqlkorma ?

matan08:07:39

How well would a clojure dev environment work on a Raspberry Pi?

noisesmith12:07:22

The arm jvm is not so great last I checked, and startup times go into the multiple minutes if using nrepl and lein

noisesmith12:07:13

But it can run things once it gets going.

ibarrick16:07:16

I have two REPL clients connected to the same nrepl and I want them to share a stdout/stderr, is this possible without unix hackery?

noisesmith16:07:53

nrepl is a network library, it doesn’t share stdio between processes

noisesmith16:07:01

you’d need some other infrastructure entirely

noisesmith16:07:07

you could set something up for multicast output to clients, but it would require new middleware on both the client and server sides

noisesmith16:07:23

hm.. maybe only server side

ibarrick16:07:52

Thanks! @noisesmith I guess nirvana is not for this world. I'll just do the unix hackery

misha16:07:28

can anyone explain the difference between event and trigger in FSMs? In particular in context of UML state charts specification. Is it just a java-esque OOP overhead, or is there key semantic difference?

noisesmith17:07:01

“trigger” is more general, “event” assumes a specific interpretation of how the state machine interacts with an unknown outside world

noisesmith17:07:14

by unknown I mean “something not specified by the machine itself”

noisesmith17:07:39

you can of course describe all things as events, but that’s a more specific term, and some semantics prefer to draw a distinction between eg. an event and a start of a timer

misha17:07:42

can you give an example, where having an event and a trigger would not be "redundant"?

noisesmith17:07:51

since that can be triggered internally by another part of the machine

noisesmith17:07:35

maybe I did that already as you were typing? if a change in one part of the machine activates another part of the same machine, that’s a trigger, but not an event because it came from inside the machine itself

noisesmith17:07:26

so let’s say a parser: if your language rules say “you are now in a closure” that’s a trigger but not an event, if you read a semicolon and that means you end the closure, that is an event that is also a trigger

misha17:07:36

I have a feeling, that having both events and triggers (which are followed with guards (predicates)) - is somewhat excessive, at least in the clojure/edn/library context

noisesmith17:07:08

they are different levels of abstraction - an event is a special kind of trigger and most users of fsms differentiate them

noisesmith17:07:33

definitely implement in terms of triggers first, if you do it right you can build events on top if you need them?

noisesmith17:07:50

but calling a trigger that is totally internal to the state of the machine an event seems weird (unless you also reify the trigger as some output and re-consume it I guess? I’ve done that for coordination)

misha17:07:32

I get that there are less/more specific events. what I cannot pin point, is why some would be triggers, not events (or vise versa)

misha17:07:24

so basically: internal=trigger, external=event? (interna/external to machine, not particular state)

misha17:07:45

or even: event/type is a trigger, but event is an entire "object"? sort of

noisesmith17:07:23

no, event is not the general thing, a trigger is the general thing

noisesmith17:07:38

all things that cause transitions are triggers, period

noisesmith17:07:51

if you only want one abstraction, it’s a trigger

noisesmith17:07:57

an event is a special kind of trigger

misha17:07:08

thank you for the input

noisesmith17:07:02

@misha it could be that what you really want is event sourcing. That works very nicely with clojure / immutable data.

noisesmith17:07:23

while a state machine is kind of designed for c or assembly, it’s all about when to do what mutation

noisesmith17:07:06

if so, yeah, everything should be an event, you have a log of events to consume, and you project various views from the events via a projection (aka fold, aka clojure.core/reductions).

noisesmith17:07:21

I’m currently building a system around that, and it’s very nice (especially the fact that when I hit a bug I can just dump an excerpt of the log, and replaying that log builds my regression test)

misha17:07:55

@noisesmith I am trying to reconcile my understanding of the components of: rule based systems, FSMs, and event sourcing, to "solve" single page apps state management for myself. I tried basic event sourcing, but when you have: few audio/video html tags with their own states, some i/o, a bunch of buttons depending on the above, and custom performance-sensitive progress bar – it kind of is not enough, and coordinating all of this is not much easier, than just a bunch of atoms with a page of add-watches connecting them. So i went looking for a better way, or just to improve my understanding of the building blocks.

noisesmith17:07:47

@misha OK - I suggest checking out petri nets btw because they are async and have a formal provable semantics

misha17:07:17

those are on my reading list, yes

noisesmith17:07:26

@misha also the very high level response to that problem is “when event sourcing leads to local state changes, reify the state change by sending it out as a new event and then consuming it”

noisesmith17:07:51

that way you still have the events as the canonical source of all truth

noisesmith17:07:18

when you have local state changes that will guide behavior that are not visible in the events, you’ve undermined the event sourcing system because you now have an irreproducible state

noisesmith17:07:26

it’s actually not event sourced any more

misha17:07:54

I sort of did that. But having few "a pure state {} depending on side-effectful io/video/audio and vice versa" loops messes things up. So I went for specs to get some vocabulary for "internal/external/pure/io events", etc.

misha17:07:19

the challenge is: from event sourcing pov: there are like 100 states, but from ui components' pov: 10, and I am looking for a way to manage those 2 "projections"

noisesmith17:07:22

my solution was to base things on a petri net - which have two kinds of things, places and transitions, a transition consumes tokens from 1 or more places and puts tokens into 0 or more places. All canonical state is the content of places (keys in a hash-map, obviously) and transitions are described by predicates on the contents under those keys and an update that removes tokens from N places and places tokens in N places

noisesmith17:07:17

but yes, I have dealt with that problem you describe in my reagent app

misha17:07:21

what do tokens represent? "should I run this predicate?"?

noisesmith17:07:29

tokens are immutable data

noisesmith17:07:47

the predicate says “given these tokens that currently exist, do I fire?”

noisesmith17:07:00

and firing consumes some tokens and maybe produces some in other places

noisesmith17:07:14

it’s like actors without the data hiding

noisesmith17:07:52

instead of mailboxes they have some places they watch (and anyone else can watch), and instead of sending messages they remove data from one place and maybe put new immutable data in places

misha17:07:30

so the places are basically like queues?

noisesmith17:07:36

they aren’t ordered

misha17:07:06

like an (atom {}), which you take values from or put into?

noisesmith17:07:37

and you might say “I only fire if I can consume 3 from :a plus one from :b” - eg. that is what assigning work to a worker ends up looking like - you have a token for a worker and a token for the job, you consume both and create a new token representing a pending result

misha17:07:46

so 10 places (nodes) = 10 atoms?

noisesmith17:07:57

no, you don’t even need an atom

noisesmith17:07:30

I put it all in one hash map, but you could eg. represent the whole thing with reductions which would mean each state is another item in the lazy-seq that reductions outputs

noisesmith17:07:55

(free history!)

noisesmith17:07:06

or reduce on an atom

samcf17:07:44

thats how i designed my deterministic game simulation

noisesmith17:07:08

@samcf hehe my event / petri based library is called “ludic” and the primary metaphor is playing chess by mail

noisesmith17:07:28

but really it’s doing a bunch of data transitions based on a petri net model from an event source :smile:

noisesmith17:07:45

(ludic is a super intellectual word describing playfulness)

misha17:07:00

so you have a list of "workers", and just (while true) apply them on the same {}? and each of those modify it, if "there is enough of particular tokens"

noisesmith17:07:23

@misha one thing it does is manage workers and tasks, but that’s just because I’m running a computation cluster

noisesmith17:07:46

if you are in the browser, you don’t need workers - make places / tokens that model your domain

noisesmith17:07:09

eg. a place could be a dom node and the tokens represent the data driving it… (maybe?)

noisesmith17:07:31

or maybe better would be a place being a UI state, tokens describing it, and transitions deciding where it should go next

hmaurer17:07:36

Hi! I have a set containing vectors, e.g. #{[1 2 3]}, and I want to get the first element of the first member of the set. Is there a more idiomatic way to do this than (first (first set)) ?

misha17:07:42

I'll read your lib's readme first. Did not quite get the analogy yet : )

noisesmith17:07:56

@hmaurer sets don’t have “first” anything, they are unordered

noisesmith17:07:28

if you want an implementation-derived item that will change when you change contents in odd ways, sure, you can call ffirst

hmaurer17:07:34

@noisesmith right; in my case I don’t care about which element of the set gets picked; hopefully it should be a singleton set, or empty set

noisesmith17:07:35

but don’t count on any particular item coming out

seancorfield17:07:48

There's a ffirst function.

hmaurer17:07:16

ffirst is exactly what I was looking for; thank you!

noisesmith17:07:40

@misha I almost regret mentioning my lib - it’s still super young and I don’t think the examples are good yet and it’s still changing fast

noisesmith17:07:16

I only brought it up because of the game thing

misha17:07:53

@noisesmith I just need lots of input and analogies to straighten my thoughts at this point :)

noisesmith17:07:10

@misha on that account, this book is very well written, has beautiful illustrations and typography https://www.amazon.com/gp/product/3642332773/ref=oh_aui_detailpage_o04_s00?ie=UTF8&amp;psc=1

noisesmith17:07:22

(it’s a book about petri nets, since the url doesn’t reveal that)

noisesmith17:07:51

definitely the most beautiful CS book I own

misha17:07:29

no kindle edition :harold:

dpsutton17:07:39

just a reminder that the first element of a set is not a well defined concept. it may be consistent but this would only be due to implementation details

misha17:07:22

thank you. everything was really helpful

hmaurer17:07:21

@dpsutton thanks! I’m aware of that however, and in my case it doesn’t matter. The set should always be either empty or a singleton

noisesmith17:07:24

np - and keep in mind you might not need the petri net model - you might just want eg. CSP instead (since core.async is right there) - but do think about how well the foo you connect your events to is something that works nicely in the clojure ecosystem

noisesmith17:07:44

events -> datalog works great too

dpsutton17:07:45

sounds good. just wanted to remind just incase

hmaurer17:07:22

thanks :slightly_smiling_face: it could have introduced a nasty bug if I relied on the first element being stable..

misha17:07:40

about that :kappa:

misha18:07:28

the bigger picture I keep in mind, is some kind of format, which can help describe system in concise way, so you could see it all at once, to some level of details

misha18:07:28

channels are cool, but when all the plumbing is spread across entire project - it becomes challenging to see the "project mini map". And anything I encountered so far – are either one way graph wiz pngs, or completely new visual-first language.

noisesmith18:07:33

it was pretty straightforward to make a function that consumed my rules and based on the places they consume from and produce to (fields on the rule record) generate a .dot file for graphviz showing a petri net.

noisesmith18:07:03

not the precise petri net my code implements (since my code can easily break that abstraction…) but at least a diagram that is helpful

misha18:07:55

yeah, I saw the .dot support

noisesmith18:07:24

@mobileink oh nice thanks for reminding me of that

mobileink18:07:03

np, thanks for the petri book ref, always wanted to find sth like that!

wei19:07:59

is there a parallel transduce? say, something like (pmap send-messages!)

hmaurer19:07:57

Is https://github.com/stuartsierra/component “state of the art” to structure a system in Clojure? Or does anyone have better advice?

kurt-o-sys19:07:10

I prefer https://github.com/tolitius/mount by far. Simpler, less opinionated, less viral.

hiredman19:07:54

mount is just terrible though

kurt-o-sys19:07:10

could be. I rather don't like component.

hmaurer19:07:52

:smile: well, let me turn my question around then: what would be your preferred way to structure a system to as to make dependency injection / mocking easy?

hmaurer19:07:07

I am new to clojure and about to build my first app

kurt-o-sys19:07:07

mount :slightly_smiling_face:

weavejester19:07:24

Integrant :wink:

hiredman19:07:24

mount is a formalized system of global state, component gets rid of global state

hmaurer19:07:29

I guess I’ll try all three then

weavejester19:07:30

But I might be a touch biased

hmaurer19:07:45

oh, integrant is your project @weavejester

misha19:07:48

@hiredman why is mount terrible?

weavejester19:07:54

Component is the most used. Integrant is a little new, but YMMV.

kurt-o-sys19:07:57

yeah, I guess that's the best thing to do: try all :slightly_smiling_face:

hmaurer19:07:01

@weavejester what does integrant do better than mount and component?

hiredman19:07:09

mount is based on global state in global atoms

hiredman19:07:16

I've done that, it sucked

hiredman19:07:55

component turns your system in to another first class value you can pass around, have multiple copies of, etc

kurt-o-sys19:07:56

@hiredman we've all done all kinds of stuff, I never liked component. No need to argue with : I like it - I don't.

hmaurer19:07:02

I fear I’ve asked a more contentious question than “should I use VIM or Emacs?”

weavejester19:07:03

@hmaurer Compared to Mount, Integrant doesn’t have global state. Compared to Component, Integrant can have dependencies between things that aren’t maps/records.

weavejester19:07:45

I feel like Vim and Emacs users can be collectively smug toward users of lesser editors :wink:

kurt-o-sys19:07:24

That discussion actually drew my attention to mount, and I like it since. Thanks to @hiredman :slightly_smiling_face:

hiredman19:07:50

huh, I didn't know there was acopy of that floating around

misha19:07:54

seems bad, like sticking (def state (atom {})) in every namespace would be
isn't this the way I am supposed to do things? :kappa:

hiredman19:07:36

singleton global state is bad and sprinkling clojure on it doesn't make it good

kurt-o-sys19:07:57

still, thanks for pointing me to mount :wink:

misha19:07:06

actually, I'd love to see good example of not doing global atoms, asking for a friend™

misha19:07:11

I guess good component example would suffice as well

hiredman19:07:11

all the main component projects I have worked on have been closed source apps

hmaurer19:07:26

Oh, I didn’t realise you’re the maintainer of Ring and Compojure!

spangler19:07:33

Here is the problem with global state: you don’t know beforehand how you will want to use your code later. So, inevitably the system grows to the point where your previous “application” is subsumed in some larger structure, and you have to refactor all the global state out of it. Which is a much bigger pain later.

weavejester19:07:59

Indeed. Feel free to hit me up with any questions when/if you happen to have any.

kurt-o-sys19:07:05

Read the discussion... it's enlightening

hiredman19:07:09

https://github.com/hiredman/songs-of-future-past/blob/master/src/com/manigfeald/sofp.clj is an example of something using component, I don't know that it is good

misha19:07:35

thank you

hiredman19:07:36

it doesn't have any tests, which is where the lack of global state would really shine

hmaurer19:07:38

@weavejester thanks, will do! Watching your talk on Integrant now

spangler19:07:53

I only say this because I went through this before. Component solves a problem I was having, and has been great ever since.

spangler19:07:21

I feel like you can go through this yourself, maybe that is the only way to see it

mobileink19:07:37

hmaurer: in that case, build it yourself. once you've got that experience under your belt you'll be able to evaluate mount, component, etc. walk before you run.

hiredman19:07:01

I would say it is a classic easy/simple split, component is way simpler (https://gist.github.com/hiredman/075b45eaeb01e4b526ce6f8854685487 implements all the functionality of component in 30 lines, but lacks refinement), but mount is going to be easier because using globals is always easier for a few months

hmaurer19:07:23

@mobileink good point. that has been my general philosophy so far

spangler19:07:53

I think of it as an “easier now”/“easier later” divide

spangler19:07:33

Global state is easy now. You don’t have to learn anything and it doesn’t take work to set up

kurt-o-sys19:07:43

Going the 'simpler' and 'easier' way, are we?

mobileink19:07:33

hmaurer: fwiw all that stuff is interesting and useful, i suppose, but i've not needed to use any of them. mastering clojure is already a tall order, those things are for ninjas (imho). good luck!

kurt-o-sys19:07:43

Right, it may be easier, and if it solves the problem, why make it more difficult? - What is the problem you want to solve? This may be the first question to @hmaurer

spangler19:07:01

But there will come a day when you are screwed

kurt-o-sys19:07:12

Having a viral framework, I never liked that. Component is more a framework - I will be screwed as well

hmaurer19:07:16

@mobileink thanks! any tip on mastering clojure itself?

spangler19:07:25

@kurt-o-sys Agreed. @hmaurer If you are trying to learn Clojure, just use a global atom

kurt-o-sys19:07:30

Have used enough frameworks, they always hit you in the face as well... and hard :slightly_smiling_face:

hmaurer19:07:10

I don’t really want to use a global state atom. While I am new to clojure I’ve been interested in functional programming for quite a while; if there are ways to avoid global state I rather do so

hmaurer19:07:15

But I’ll try out mount either way

spangler19:07:18

@kurt-o-sys We disagree in general, it is okay

hmaurer19:07:20

I might learn some valuable lessons

spangler19:07:32

@hmaurer That is good context. In that case I think you will appreciate component

misha19:07:36

fwiw, there is a yurt for mount https://github.com/tolitius/yurt

Multiple brand new local Yurts with components can be created and passed down to the application / REPL to be used simultaneously in the same Clojure runtime for fun and profit.

kurt-o-sys19:07:37

Exactly - check what works out for you.

hiredman19:07:40

integrant seems ok, I haven't looked at in depth, but it looks like it uses multimethods for dispatch, which I've done with component before, but is annoying if you want to create an anonymous sort of mock of something in a test

spangler19:07:17

@hmaurer Once you understand it, it is not as bad as people make it out to be

hmaurer19:07:18

A global state atom also sounds a bit messy for mocking

spangler19:07:21

It is a conceptual hurdle

spangler19:07:25

not a technical one

hmaurer19:07:31

I mean, you could set the global state atom before runing your tests with the mock dependency

hmaurer19:07:33

but it sounds a bit dirty

kurt-o-sys19:07:04

@spangler right... one prefers frameworks, the other doesn't. Fine with me :slightly_smiling_face:.

kurt-o-sys19:07:37

Global state atom can be bad, of course. That's why mount manages it, so you don't have to.

spangler19:07:59

I would call component more of a pattern. It is not doing much for you, most of it is explicit

kurt-o-sys19:07:24

Anyway, we'll never agree on that part - that discussion was pretty clear about how it's not that different from component (passing in state everywhere)

spangler19:07:34

It is a pattern of state access, treating state as a value

kurt-o-sys19:07:56

@spangler right... it does not do much... I like that :slightly_smiling_face:

spangler19:07:16

Basically, it is the functional way to do things: you have to pass everything in

kurt-o-sys19:07:48

yes, that's a dogmatic reason. I prefer pragmatism in many cases.

mobileink19:07:10

write code, heh. to me the key is compositionality. it was not obvious to me just what that means when i was starting. i guess i would recommend focussing on the small stuff first - sequuences, laziness, core.async, etc. explore the api - what does juxt do? how to use protocols? etc.

hmaurer19:07:26

Is there a good blog/book/resource somewhere on clojure ways of doing things? (i am already familiar with “programming clojure” and “the joy of clojure”)

mobileink19:07:14

better: forget the app for now, just spend a lot of time in the repl experimenting.

dm319:07:22

heh, I went to Mount after a couple years of Component straitjacket. In a bigger team I’d use Component though :slightly_smiling_face:

kurt-o-sys19:07:25

clojure does have state management constructs because you don't pass everything in. There's a reason why clojure isn't dogmatic on FP.

weavejester19:07:08

https://leanpub.com/elementsofclojure isn’t finished but has some interesting ideas. It’s not a general purpose programming book, though. I think the first chapter on naming is free.

seancorfield19:07:12

As someone who initially went down a Mount-like path when building a system and has since switched to Component, I'll "vote" for @hiredman's position here that global state is a terrible idea.

kurt-o-sys19:07:37

It is a terrible idea in general, right. The dogmatic reason. I can understand it.

kurt-o-sys19:07:55

But having a viral framework has hit me many times, including component.

seancorfield20:07:20

We're still feeling the pain of the global state approach as we piecemeal migrate away from it (by using Component instead but having its start/`stop` functions keep the legacy global state in sync until we can replace it).

kurt-o-sys20:07:42

That's why I prefer mount - pragmatism, and as long as you manage state, it's ok.

samcf20:07:52

single global atom is the rallying cry for frontend projects lately

kurt-o-sys20:07:03

:slightly_smiling_face:

mobileink20:07:15

@hmaurer as you can tell from this thread, there is always More Than One Way to Do It. Master the basics first, then you'll see just how true that is!

seancorfield20:07:33

Component is not a "viral framework". Functions need to be based stuff as arguments, not reach out to global state for it.

hmaurer20:07:48

Haha, I am glad it sparked a discussion though. It would have been quite boring if someone linked me a lib and 5 people thumbed up

kurt-o-sys20:07:51

Once you start using component, it's hard not to use it everywhere in your code base.

kurt-o-sys20:07:13

So well, I call that viral (inside the code base).

hiredman20:07:19

@samcf I wouldn't be surprised if that is the primary split front end / back end on mount / component

kurt-o-sys20:07:39

You do it 'the component way' or not. There's nothing in between... using component.

spangler20:07:39

I think it is more about providing each part of your system the minimal amount of information it needs to do its job. That way you can decompose and rearrange it later. Which at some point will become very important.

spangler20:07:59

global state creates interdependence, which is weakness

ghadi20:07:05

Hate to make an argument by authority, but it's really not insignificant that some of the most veteran clojure developers (@hiredman / @seancorfield ) favor component. Personally I think mount teaches new clojure users bad habits.

kurt-o-sys20:07:13

Same with mount, honestly. Check that discussion I posted above, it's really all not that different.

ghadi20:07:19

s/mount/global state patterns/

seancorfield20:07:29

@kurt-o-sys any given function just needs to be passed what it needs -- not the whole system -- and that's just functional cleanliness. You initialize all your resources at application startup, put them in a map, pass the relevant bits down the call chain. Component just formalizes that and manages dependencies for you.

hmaurer20:07:39

can you do that with component? don’t you pass the whole context around usually? or do you intentionally strip some keys from the context?

kurt-o-sys20:07:57

Same with mount, really, if you like so.

dm320:07:12

@ghadi - even though I prefer Mount - I agree :slightly_smiling_face: Mount should only be allowed once you spent a year with Component

kurt-o-sys20:07:28

@dm3 :slightly_smiling_face:

spangler20:07:51

You only give each component access to the components it asks for (basically you declare a dependency). Your component only ever sees the state that it asks for

seancorfield20:07:30

Perhaps the pain level with Mount doesn't show up until you have a large enough system? Just like global state only causes you pain after a certain point?

spangler20:07:54

which makes it really nice, because your systems are modular by default

dm320:07:33

@seancorfield it’s probably true. In my case I’ve avoided building large applications (modules). I just have many of them.

seancorfield20:07:57

We have about 60,000 lines of Clojure right now...

dm320:07:11

I have half that, but I’m the only dev

spangler20:07:38

@dm3 I kind of fear your codebase ; )

dm320:07:59

you should :slightly_smiling_face:

spangler20:07:13

Sweet, your glyphs and wards are working

kurt-o-sys20:07:16

anyway, as said before, one prefers dogmatic, other prefer pragmatism. One prefers frameworks, other prefer libs/patterns. One likes pink, the other blue. Whatever :slightly_smiling_face:

spangler20:07:57

If you want to do things functionally, component is what you want

kurt-o-sys20:07:41

(some ran away from component, others ran away from mount, so maybe we all should try integrant and decide it's not a silver bullet either :slightly_smiling_face: )

hmaurer20:07:01

can you elaborate on the potential issues with multimethods please?

kurt-o-sys20:07:12

lol - because someone has an opinion? Nice.

seancorfield20:07:26

Like I say, hard lessons learned building a system the "easy" way with Mount-like managed state, instead of the "simple" way with Component-like managed dependencies.

mobileink20:07:19

zzzzz. clojurians having another vigorous debate/ disagreement without calling each other names. wtf is wrong with you people?! :wink:

hiredman20:07:41

@mobileink I dunno, can't we just hate each other's software without hating each other?

hiredman20:07:24

(hating your own software is of course also acceptable)

mobileink20:07:26

hate the code, not the coder!

hmaurer20:07:31

@hiredman ^

mobileink20:07:23

anyway that was a lame attempt at irony - the clojure community is extraordinarily irenic. (look, i made a pun! whee!)

hiredman20:07:35

generally multi methods are also global, created with defmulti (technically you could create an anonymous one, but I have never actually seen anyone do that, and it would kind of defeat the purpose)

hmaurer20:07:06

@weavejester quick question on Integrant (I’m half way through the talk): how would I proceed to mock the implementation of one of the services?

hiredman20:07:08

so if you want to create a mock, you have to use defmethod which effects the global multimethod

hmaurer20:07:12

I am not sure how to do it with multimethods

hiredman20:07:33

if you use protocols you can generally reify the protocol inline in the test, and outside of the test there is no effect

hmaurer20:07:34

ah, so the latest defmethod will be used, overriding a previous definition?

hmaurer20:07:53

that’s a bit nasty

misha20:07:54

"hate a la carte" joke

spangler20:07:59

@mobileink Online peace is a noble goal

weavejester20:07:26

For that, I tend to use protocols. In Duct (which builds on Integrant), I call them “boundary protocols”, as they define the I/O boundary of the system.

hmaurer20:07:29

ah nevermind, got an answer by @hiredman :slightly_smiling_face:

hmaurer20:07:39

oh, I’ll check Duct

hiredman20:07:45

uh, yes, but that is not what I meant

weavejester20:07:09

Instead of just returning the database spec, I wrap it in a record.

weavejester20:07:38

This allows me to write a protocol against it, and I can mock out the protocol for tests using a tool like Shrubbery.

hiredman20:07:57

each multi method has a dispatch table for behaviors, so say you want stick in a :my-special-mock, well that goes in the table associated with the multimethod, less likely to cause harm than actually replacing the dispatch, but still kind of a drag

weavejester20:07:00

It’s also useful for having a “local” service that fakes out cloud services or external APIs etc.

hmaurer20:07:36

Yes that’s exactly the sort of application I’m thinking. Two more projects to look at on my list :stuck_out_tongue:

weavejester20:07:43

Duct builds on top of the ideas of Integrant. It’s the vocabulary to Integrant’s grammar. https://github.com/duct-framework/duct

hiredman20:07:47

(following the thread with @weavejester) it looks like he uses protocols for some stuff, so maybe that is less of a problem

hmaurer20:07:41

@weavejester the last web framework I looked into was Pedestal. Do you have any opinions on it?

hiredman20:07:56

at my last job we used component but had a team member who hated defrecords, so we more or less retro-fitted component to use multimethods instead (which is pretty easy to do) which is where my experience with the pain of mocking that kind of thing comes from

weavejester20:07:47

I haven’t use Pedestal in anger. Some ideas it has are good, but it feels more complex than what I need, and it’s web-focused.

weavejester20:07:03

Duct/Integrant are more generic

mobileink20:07:38

did i just kill a discussion? ouch.

erwinrooijakkers20:07:31

Interesting discussion. At my work we use mount (mount.lite actually), but I always liked the IoC-pattern in OO, which I miss doing mount. It is nice that you do not have to pass state as parameters everywhere, and if you want to mock a state you can substitute state for a mock implementation. BUT I miss having a composition root responsible for composing modules, and seeing that you have to refactor because a module takes too many constructor parameters.

danboykis20:07:11

@hmaurer when you looking at mount, besides the docs, these little example apps may help you to see how it could be used: https://github.com/tolitius/stater

hmaurer20:07:12

@weavejester sorry, watching the rest of your talk and reading a bit of Duct’s doc I realise this was a silly question. Duct seems more like an approach to structure an application around Integrant

erwinrooijakkers20:07:36

I find initializing state in one file like below (where a timbre appender is added) quite nice. I don’t know what the equivalent would be for a Component application. Logging is a cross-cutting concern, so perhaps not really something you inject.

(ns blabla.logging
  "Logging setup for blabla."
  (:require [blabla.config :refer [config]]
            [clojurewerkz.elastisch.native :as es]
            [mount.lite :refer [defstate]]
            [taoensso.timbre :as log]
            [blabla.logging.elastic :as log-es]))


(log/handle-uncaught-jvm-exceptions!)

(defstate log-level
  :start (let [current-level (:level log/*config*)]
           (log/set-level! (:log-level config))
           current-level))

(defstate blacklist
  :start (when-not (:development config)
           (log/swap-config! update :ns-blacklist conj "io.pedestal.http.impl.*"))
  :stop (log/swap-config! update :ns-blacklist #(vec (remove #{"io.pedestal.http.impl.*"} %))))

(defstate elastic-client
  :start (let [{:keys [elastic-native-pair elastic-cluster-name]} config]
           (when elastic-native-pair
             (es/connect [elastic-native-pair] {"cluster.name" elastic-cluster-name})))
  :stop (when elastic-client
          (.close elastic-client)))

(defstate elastic-appender
  :start (when elastic-client
           (let [options {:base-doc {:app "blabla"
                                     :k8s {:namespace (:kubernetes-namespace config)}}
                          :mapping  log-es/mapping}]
             (log-es/add-elastic-appender! elastic-client "hd-logging" options)))
  :stop (log-es/remove-elastic-appender!))

hmaurer20:07:37

Duct is a backbone, and all the web/else related logic is handled by other librairies, e.g. Compojure

weavejester20:07:48

Right. It provides a template, a bunch of integrant-compatible libraries, and “modules” which are essentially pure functions that transform the Integrant configuration.

weavejester20:07:32

Modules provide a way of automating gluing together libraries

hmaurer20:07:30

@weavejester do you have an example of how a module would work?

weavejester20:07:08

An example, or an explanation?

hmaurer20:07:39

explanation, or an explanatory example :slightly_smiling_face:

weavejester20:07:12

Okay, let me find something quick…

weavejester20:07:31

Actually, I wrote a blog post about this, which might be more useful: https://www.booleanknot.com/blog/2017/05/09/advancing-duct.html

hmaurer20:07:18

@weavejester ah that’s perfect, thank you!

weavejester20:07:27

One module is :duct.module/web. That sets up a handler and middleware, and looks through the configuration for a web server. If it finds one, it connects the handler to the existing web server. If none exist, then it creates a new one.

weavejester20:07:18

The explanation I find myself often using for modules are web sessions that are stored in a SQL database.

hmaurer20:07:25

how does it “find the web server” in the config? based on the name of popular web servers?

hmaurer20:07:31

or a special key?

weavejester20:07:55

Keyword inheritance. The :duct.server.http/jetty key is derived from :duct.server/http.

weavejester20:07:21

So to find all the web servers in a Duct config, you just run (ig/find-derived config :duct.server/http)

weavejester20:07:31

The idea is to use derive a little like a dictionary definition or a thesaurus. It allows us to describe what a keyword is, to give it some meaning.

hmaurer20:07:35

keyword inheritance is just based on the name + namespace of the keywords? e.g. a keyword X inherits from a keyword Y if X’s namespace is Y’s namespace plus its name?

weavejester20:07:02

No, it’s its own thing. Any namespaced keyword can derive from any other.

weavejester20:07:13

Namespaced keywords have multiple inheritance in Clojure.

hmaurer20:07:06

ah, I wasn’t familiar with Clojure’s clojure.core/derive function. I’ll look into it before bothering you further

weavejester20:07:26

It’s not often used, but I feel like it has some important niches

hmaurer21:07:29

mmh, so derive is effectful from what I gather

hmaurer21:07:00

Where does it hold the relationship data between keywords? A global registry? I can’t see it setting metadata on the keywords themselves

weavejester21:07:58

A global registry. Derive can be used functionally if you supply a hierarchy, but mostly it’s used globally.

weavejester21:07:12

To my mind it’s like vars, or specs.

hmaurer21:07:22

Oh, I think I get what you meant by a “grammar” then. You use keywords + derive to build a sort of ontology?

weavejester21:07:29

Exactly :slightly_smiling_face:

weavejester21:07:44

Jetty, for instance, is a HTTP server that supports synchronous and asynchronous handlers. So we could write:

(derive :duct.server.http/jetty :duct.server/http)
(derive :duct.server.http/jetty :duct.server.http/async)
(derive :duct.server.http/jetty :duct.server.http/sync)

hmaurer21:07:59

that’s brilliant

weavejester21:07:18

If Integrant is the “grammar”, Duct is attempting to build a “vocabulary”.

hmaurer21:07:44

So duct is an ontology for configuration + tools that understand and work with this ontology

weavejester21:07:13

Right. That, plus a way of transforming configurations via a preprocessing “prep” stage.

weavejester21:07:20

Basically query+transformation

hmaurer21:07:58

I’ll check out Duct’s code if it’s not too complex

hmaurer21:07:08

That’s super exciting

weavejester21:07:12

Start with duct/core

weavejester21:07:19

Essentially Duct tries to solve the problem of wiring up libraries. Even something seemingly trivial, like storing sessions in a SQL database, requires a lot of wiring.

weavejester21:07:33

You need a database migration to create the SQL table to store the sessions

weavejester21:07:38

You need a DB connection

weavejester21:07:46

Session storage protocol

weavejester21:07:58

And a worker process to clean up old sessions

weavejester21:07:52

Very hard to do with a normal library, but a module could ask questions like “Where do I register a new periodic cleanup job?” “Where do I add new middleware?” “Where can I add a database migration?”

hmaurer21:07:15

I guess it’s a form of convention over configuration

hmaurer21:07:42

But in the sense that with Duct you establish a language to talk about your configuration, deriving Duct concepts

hmaurer21:07:56

then modules can use this language to understand your configuration and act on it

weavejester21:07:05

Right. It borrows a lot from tools like Rails, but whereas Rails used the filesystem as the basis for its convention, Duct uses an immutable data structure.

hmaurer21:07:06

Instead of following some predefined key structure for your configuration, you just need to follow an ontology/grammar

weavejester21:07:24

Exactly :slightly_smiling_face:

weavejester21:07:09

It’s still early days, but that’s ultimately where Duct is going.

hmaurer21:07:13

Are there any other projects that you know of which take this approach? Obviously ontologies are quite talked about in the context of the semantic web / knowledge bases / etc, but I think this is the first time I’ve seen it applied to a dev tool

weavejester21:07:38

Arachne is another of Duct’s inspirations

weavejester21:07:13

It takes a similar approach, but I don’t think it emphasises the semantic ontology angle

weavejester21:07:56

And rather than a data structure, it uses a bunch of scripting functions to create an in-memory datomic database, which is then translated into a Component system.

weavejester21:07:04

Duct IMO is a little simpler.

hmaurer21:07:03

@weavejester thanks for taking the time to explain this out! I’ll read up on Duct/Integrant. The theory sounds great; let’s see if it holds up in practice :stuck_out_tongue:

weavejester21:07:25

No problem - let me know what you think

weavejester21:07:12

The two blog posts I linked, especially the later API-focused one, should give you a good idea of how to start.

michaelblume21:07:56

If you want to evaluate mount/component/integrant, I’d recommend starting by just reading them

michaelblume21:07:09

they’re all in the 300-500 LoC range

hmaurer21:07:42

@weavejester I think I am going to have some fun with clojure.core/derive itself too. Such a neat little function

weavejester21:07:37

I wouldn’t say it’s a function that should commonly be used; it’s got niche functionality. But I also think it’s underused. Not many people seem to have heard of it.

michaelblume21:07:11

I mean there are things you can only learn by using them in a large project for a long time, or talking to someone who has, but I think reading them is a reasonable starting point

hmaurer21:07:54

Yep that’s what I intend to do now that I’ve seen they’re pretty concise. Thanks for the advice!

bfabry22:07:57

if I'm understanding your example correctly, component also wires everything together in one file like you prefer

bfabry22:07:47

generally you have a 'system' or whatever ns with a big map that looks like

(defn example-system [config-options]
  (let [{:keys [host port]} config-options]
    (component/system-map
      :db (new-database host port)
      :scheduler (new-scheduler)
      :app (component/using
             (example-component config-options)
             {:database  :db
              :scheduler :scheduler}))))

dotemacs22:07:03

Aaand don’t forget the new-new kid on the block: deferst: https://github.com/employeerepublic/deferst

erwinrooijakkers22:07:46

Yes that is what I am missing from Mount :slightly_smiling_face:

erwinrooijakkers22:07:57

Your example-system is a Composition Root

hmaurer23:07:10

@weavejester Out of curiosity, why did you make the choice of using multimethods for Integrant?

hmaurer23:07:03

It seems to me (as a clojure beginner) that it introduces a form of global that makes stubbing a bit harder

hmaurer23:07:34

e.g. wouldn’t it be nicer to pass a second argument to ig/init: a map from keys to implementations?

hmaurer23:07:53

I am sure you had a good reason for using multimethods and not the approach I’m suggesting, but I would like to understand it

hmaurer23:07:31

@weavejester is it related to hot-reloading?

dpsutton23:07:57

do clojure files have to have a namespace?

dpsutton23:07:28

and if so, should tooling not assume that there will be a namespace form at the top?

hiredman23:07:23

and you can put other things before the namespace

hiredman23:07:28

(the ns form)

dpsutton23:07:56

never really know where to go to find the hard requirements on these things so thanks for the info

hiredman23:07:02

it is super common to have the ns form be the first thing, so some tooling does assume it

dpsutton23:07:17

ok. i was not positive if it was convention or spec

hiredman23:07:36

so some tooling requires it, but clojure does not

dpsutton23:07:01

well, if possible, i'd like to conform to clojure requirements and not some subset

dpsutton23:07:48

there were some namespace cache changes in CIDER that seems to affect files without a ns form so wanted to see what the requirements were. thanks a bunch