Well, for spec, the whole process is defining a spec for a keyword

Can’t speak to reframe. I’m not sure what they do with them or why.

Really it’s massively use-case dependent. I don’t think there are any general rules. Do you have a use case you’re curious about?

It's more a general question I was wondering about - I saw this "keyword registry" pattern in a few other libraries like Malli

Spec's different for sure, just referring to the idea of using a global registry to map keywords to some domain-specific data

I wonder if it's simply a matter of ergonomics (being more DSL-ey and not having to explicitly require and refer namespaces ), but you end up having to implicitly make sure that the ns registering those keywords has been loaded

At a certain point once your DSL gets too complicated and bloated, you end up essentially reimplementing Clojure itself badly

People are uncomfortable with serializing code as a rule sometimes, but DSL's with enough expressivity and the intent-to-run are code

My tendency is to default to clojure vars until there is a concrete reason to move to keyword references.

I think most people drift towards keywords first because they are more familiar.

But when you end up with a bunch of keywords that end up invoking functions elsewhere, you’ve kinda clearly gone too far.

That makes sense - so if I have an extensible domain-specific list of 'actions' that are meant to be interpreted in several places and invoke different functions, like

[{:type ::action/START
  :attr ::color/red}
 {:type ::action/PAUSE}
 ...]

(require '[foo.actions :refer [START PAUSE]])

[{:type START
  :attr ::color/red}
 {:type PAUSE}
 ...]

or actually just

[(START {:attr ::color/red}})
 (PAUSE)
 ...]

Yeah in cases like these I always start with the data, then maybe move to a function for concision.

So say you had this:

[(START {:attr ::color/red
         :state ::state/wait
         :name "hello!"})
 (PAUSE)]

I would make that more concise by doing this:

[(START (init-start ::color/red
                    ::state/wait
                    "hello!"))
 (PAUSE)]

where init-start returns:

{:attr ::color/red
 :state ::state/wait
 :name "hello!"}

hmm, so each action takes the same arglist format (for composability?) and you define preprocessor functions to pipe into them

ie. why not

(START ::color/red
       ::state/wait
       "hello!")

I guess there's also the tension between "expanding" the DSL early vs keeping it user-readable through intermediate stages

;; Hiccup DSL
[:div {} args]

;; vs

(div {} args)

=> {:type ::dom-nodes/div
    :render-fn #function[0xabc3d]
    :html-name "div"
    :lots-of :noise
    ...}

Yeah I don’t know what the START fn does/returns.

I was just riffing on your initial suggestion of:

(START {:attr ::color/red})

But yeah, I also lean toward “expanding early.” Removes a lot of rigamarole.

You can always remove things for readability. Can’t add things back in when you need them.

ymmv — that’s just my tendency

Thanks a lot for your perspective - I'm not experienced in this sort of library design and wasn't able to find any best practice guidelines

I’m reminded of the difference between Compojure and Reitit

One interesting design point is that afaik, spec only provides functions that work with the global registry. if you call (s/valid? ::my-spec m) and m includes an attribute, that attribute will be validated even if that attribute is never mentioned by ::my-spec. This is in contrast to the hierarchies for multi methods where both using a global registry or a local registry is supported. re-frame also has a global registry, but I think it's mostly a side-effect of the event system provided by the DOM (see https://github.com/day8/re-frame/issues/137). It's also probably partially influenced by clojurescript's reduced var support. If you're designing a system where the goal is for names and constraints to be enforced globally (like spec), then keywords support that more directly. By global, I don't mean just mean across a whole program, but across multiple programs or systems. For example, you could store a spec keyword in a file or send it across the network which is less direct if you were trying to use vars as names across programs and systems. This video touches on some of the reasons spec uses a global keyword registry rather than vars, https://vimeo.com/195711510

Thanks, that's quite enlightening! In our case the names will likely never leave the program, so that's another point in favour of using vars :)

A mutable global registry is obviously bad because it is mutable and global. Malli also gives you other options. I don't want to debate the Spec design but I think the global registries are definitely unnecessary in re-frame and when I will end up making a frontend framework it will not have mutable globals 😜

> A mutable global registry is obviously bad because it is mutable and global. This is one of the more Developer Brain sentences I’ve ever heard uttered.

Nothin’ wrong with a global, mutable registry so long as it doesn’t change at any time you’d particularly care. Half-kidding.

This might be the series of blog posts for you https://crypto.stanford.edu/~blynn/compiler/

That would be Racket https://racket-lang.org/ The one Naughty Dog used more recently

There have been attempts to make a non-hosted Clojure or similar like https://github.com/pixie-lang/pixie but it seems that people did not actually want one as much as they wanted to complain about Java And now you can do scripting in cljs or babashka

For some other uses of Common Lisp, Reddit used to be implemented in it, and Google uses it for airline search (via a company they acquired, ITA)

I mean, Clojure and Scala are the only two mainstream functional languages that are being used at scale across different use cases That cannot be said of Haskell. Haskell is nice, but Haskell being more "active" is more of a surface phenomenon that arises out of its academic usage than its professional usage. You don't see people harp about c++ all the time and yet its still in the top popular languages

Not sure why the question cannot include clojure because clojure is a lisp. If you're referring to CL implementations vs Haskell, CL is still being used in production, and the tooling is still under active development in Japan. It's just that Haskell has alot more english speakers.

But Haskell in production? Heh I recently got my hands on a Haskell Library at work for evaluation but ultimately rejected it because you can't find anyone with related experience needed to juggle the clusterfuck that is Haskell in production

I’m not exactly going to rush to defend Haskell’s ecosystem but I’ve talked to quite a few people over the years who are using Haskell in production — and often in seriously complex, critical usage situations — so I think that’s a bit disingenuous.

*around my region that is

I wish the Haskell folks would bite the bullet and treat the JVM as a first-class target so we could actually use it via interop and mix’n’match more. There’s Frege, which is actually really nice — I’ve done some Clojure/Frege interop — but it’s basically a one-man passion project so I would be concerned about its viability in production. There’s Eta, which came in with a fanfare (and, frankly, the project maintainers dumped all over Frege as not being “Haskell enough”) and has faded out as abandonware.

Took me a while to get that… Miranda… but that’s a simplification: Haskell has its roots in a lot of languages that UK university researchers had been creating for many years before the committee got together to create Haskell.

NPL, Hope, KRC, SASL, ML, SURE, … many, many FP languages were created in the ’70s and ’80s before Haskell appeared at the beginning of the ’90s.

right, Haskell was the common lisp project of the type safe / immutable world

I tried to learn Clean at one point (much too early in my programming career to make sense of it)

I did my Ph.D. in the 80s and state-of-the-art then were Standard ML (eager) and KRC/Miranda (lazy, combinator-based as I recall). This was about the time that Glasgow were building combinator reduction hardware. For SML we were happy with our Sun SPARC workstations.

@U6BUSDFMX My PhD was also in the ’80s “The Design and Implementation of Functional Programming Languages” — interesting times in the FP world back then.

I think we had almost the same title.

(I never completed my write-up — Prof Turner from Kent was going to be my external examiner, according to my supervisor at Surrey)

Well, as it happens he was my external examiner…

Hey! Maybe you’re me and neither of us ever knew it? :rolling_on_the_floor_laughing:

You mean we’ve been accidentally aliased?

If you create a custom atom-like type, yes. Reagent does exactly that with its ratoms and reactions.

ah exactly, thanks 🙂

not natively but it’s possible to extend IAtom

the behaviour i am looking for is basically same as a TTL cache so i use that

(def my-deref (let [a (atom 1)
                    deref-fn (fn [] (prn :foo))]
                (reify
                  clojure.lang.IDeref
                  (deref [this] (deref-fn) @a)
                  clojure.lang.IAtom
                  (swap [this f]
                    (swap! a f)))))

@my-deref ;; prints :foo
(swap! my-deref inc)

nice!

genius

might just use that instead of ttl cache

any copyright? 🙂

I hereby permit the right to copy

haha thanks

You will have to elaborate further on the symptoms you are experiencing. There is the #kaocha channel too.

I think Ill go there! Thanks a lot!

Nice. This makes sense. Then it’s a convenience for a keyword being internal to a namespace, with zero risk of colliding with any user keys that happen to flow through. Just have to be really careful that they never “escape”.

it's not escape per se but no one else cares. and in that sense it's quite easy to do. Think of a middleware for a handler. if it wants to annotate the request with a start time and then an end time afterwards, it can add ::start-ms or something, and this is a sign that only this namespace is aware of, and cares about it. If it's introducing something that other things should care about, say user details it should use :cached/user or :context/user or whatever you think makes the most sense. Then it's clear this is introduced for others. In both cases the keyword "escapes" but the use is clear if its intended for others to access or not

Interesting. How would this work with something like datomic schema, where my keywords are scoped to my domain? Like :org.company.model.user/id? It’s not really private to any one namespace, but still deserves the fully qualified name.

Isn't the approach there simply to always refer to those with the full name :org.company.model.user/id ?

You can do that, regardless of whether a particular project has a namespace org.company.model.user or not, I thought.

in that case you've already gone down the road of tying keywords to namespaces. I would ignore the fact that there happens to be a ns that currently shares the same name and always use the :org.company.model.user/id

> Isn’t the approach there simply to always refer to those with the full name That’s what I do yea. I was more curious whether it was an exception to the above guideline. Another example, pathom places a lot of fully domain qualified keywords into the parsing environment, with the intent that you can modify them and analyze them. So they’re not really “private” per se (or maybe they are and I shouldn’t be poking them lol).

interesting, thanks for mentioning this if I’m correctly understanding what this does, the binary could be used in multiple repositories for searching all occurrences of some symbol, which is quite helpful for finding deprecated functions usages

sounds correct! a classpath https://github.com/borkdude/grasp/tree/464f86af5866a134374af9ef7ed152882846b531#grasp-a-classpath can easily span multiple repos

There is no need to create a namespace that matches the qualifier part of a qualified keyword, right? i.e. you can use the keyword :foo.bar/baz regardless of whether there is a namespace foo.bar or not.

as long as you are willing to copy/paste (or type, or use IDE auto-completion) its full name

yes, but 90% of my ns-qualified kw usage is spec-related. The spec semantics imply an underlying implementation, which increases the chances for having an actual circular dep

That won’t work tho. Because then I can’t validate my transaction map that uses temp-ids. I have to use the keyword as it exists in my database schema.

no you don't

(d/transact conn {:tx-data [{:order/line-items ["tempid-1" "tempid-2"]}]})

I can’t use :order/line-item-ids in that tx-map

a transaction may be different than an entity

Exactly, yet they use the same keyword. That’s my point. The same keyword can take different values depending on the context.

at the point you're transacting you're past the point of using spec, though

I think that’s specific to that one example. Take the case of using idents. {:order/line-items [:li/id "123"]}. This could very well be pre-transact.

You could use s/or in some way, but I don’t think that really covers it, because now it’s too admissive. There might be contexts in which idents really aren’t valid.

I believe the main point of spec is to have a stable definition within your program of what a key might be. e.g.

(defn order-total
  [{:order/keys [line-items]}]
  ,,,)

the things you're talking about are necessary because at the edges of your program, it might not be in a useful shape yet to use throughout your program

between the database and the meat of your program there's typically code that will transform the shape of data into what it should look like.

Yea. I suppose the lines of “your program” can get a bit blurry in web applications too, because you’re managing so many things at once. Imo, a global registry is just wrong. There’s simply no such thing as a “stable definition” at that level. Reductio ad absurdum: scope “your program” to be just a single function. That’s really the only level of granularity that allows a keyword to be stable.

e.g. you might transform {:order/line-items [:li/id "123"]} to {:order/line-items [{:li/id "123"}]}

I think if the novelty of your program is shuffling data to and from a database then the majority of your code will be handling your data in its un-conforming state

which does make spec less attractice

I've worked in webapps basically my whole career and that hasn't usually been the case; eventually there's a bunch of interesting business logic or some other process that needs to occur, e.g. taking data from one place, transforming it and annotating it and doing some logic, and then putting it somewhere else, which having a stable definition of what a key should look like or what an entity looks like is quite useful in that in-between step

and separating the I/O and the data shape required to effect changes with those two different systems is useful

and you're right, spec isn't going to ensure that my datomic transaction is correct, but it could check to see that my data is correct before I create the transaction

so it's not useful in all contexts

That’s really the core of my struggle with it. It’s such a useful thing to have that specification to point at in those in-between steps like you mention. But I really only want to make that specification at that location. As implemented, spec forces you to make one grand statement about a keyword, and in my experience I always end up getting stuck. Basically, what if you didn’t use s/def at all, and only whipped up specs “on demand”? Like what https://github.com/metosin/malli does (specs as data).

I do think that having a global definition is a feature, not a bug. but I agree that it would be useful to opt in to - or override locally - that global definition

It’s something that takes practice I think. I’ve attempted it several times and usually end up scratching my head. With a global registry, you have to be very intentional ahead of time what exact “level” you’re going to be running specs at. From your examples that would be at the validation layer of the stack.

With a web stack tho, there’s just so many layers…and several of them might benefit from specs “a la carte”

Nice, that’s pretty nifty

@borkdude playing with #:foo{:bar …} syntax in vs code. It seems to confuse #lsp when using “Find references” on the fully qualified keyword. The line numbers are way off. Where would I file this issue?

@cjsauer filed: https://github.com/clj-kondo/clj-kondo/issues/1251

@cjsauer Fix is on clj-kondo master. I compiled a local lsp version with it and it now works.

So it's only a matter of time before it ends up in the downstream tools

Very nice! Eager to try it out. Thanks for this.

@cjsauer do you use macos perhaps?

then I can just give you my locally compiled one ;)

You can also compile locally: https://github.com/borkdude/clojure-lsp and then run make prod-native

Yep I’m on Mac. Where should I place the binary once compiled?

Are you using Calva?

Then it's under Calva settings

the location is configurable there

With emacs etc it's different. Best to ask in #lsp

Calva yea. I found the setting. I might wait for the bot to catch up with releases. I think I botched something in my rushed attempt haha 😅 Looks like lsp will auto-bump kondo on the next cut.

Here's my local version: https://www.dropbox.com/s/j6prls3g2dl9u6s/clojure-lsp?dl=0

Thanks, got it working. Find references on keywords is working as expected now!

Yea it’s a weird one. The schema is what allows for the disambiguation.

I tend to want this. :)

don't it is silly

it can be seen as an enforcement of decoupling, which is the opposite of having knots

you can see it that way, but the fact that it ties your code base in knots is evidence that seeing it that way is not correct

I prefer to have (and solve) an explicit knot to an implicit one else the circular dep is there, it may just not manifest itself until much later

it isn't

a keyword is a value, like the number 5, so when you using :foo/bar you don't implicitly have a dependency on the namespace foo, and you don't have a dependency on everywhere that keyword is used

just like using the value 5 doesn't mean you have some kind of dependency on wherever else 5 is used

With spec in play though, consuming :foo/bar, where :foo/bar is backed by a spec, implies an implicit dependency on whatever ns is performing s/def :foo/bar

only if you are using spec features (s/valid? what have you)

and spec dependencies are much more lenient than clojure namespace/var dependencies

> only if you are using spec features using a spec-backed defn is common enough yeah well I don't particularly appreciate that leniency for one thing it can result in spec checks not being performed because some ns didn't happen to be required by the consumer also can result in circular deps iirc

I dunno, I basically never spec fns, defn or otherwise

at work we have 34 s/fdefs and 1011 s/defs

yeah varies widly per team. I tend to use s/fdef (or a variation) for most 'public api' defns. Accordingly I try to keep APIs small

Yea I’ve never actually written out #:foo{:bar …}, but I see it printed quite a bit.

It's useful when you have a large static map where many keywords have the same ns.

But then you have the one odd-ball keyword that’s in a completely different namespace, and now you have to rewrite the whole map, or resort to merge haha.

user=> #:aaa {:b 1, :c/d 2}
{:aaa/b 1, :c/d 2}

Oh wow. I didn’t know that was possible.

:D Did I just convert you?

opens repl to test…

whelp, I had no idea you could opt out of the sugar like that!

I might have just been converted

you can also write

#:foo{:_/a 1}
{:a 1}

Now it's my turn to use 🤯

This actually makes typing out long namespaces (and avoiding ::sugar) that much easier.

#:_{:a 1 :_/b 2 #_:c}
;; => {:_/a 1, :b 2}

I wonder what a Obfuscated Clojure contest would look like