Fork me on GitHub
#clojure
<
2020-02-22
>
jasonjckn00:02:36

whats canonical when you have defmethods spread across several files if I require the defmulti, but not the defmethods, the multimethod won't be implemented correctly (?) so I created a file all.clj that requires all the other namespaces with defmethods, is there a canonical pattern here

noprompt00:02:40

The only other options to requiring them all is require either some or none of them. 🙂

noprompt00:02:38

But in my experience I mostly see people in this situation requiring them all in whatever their “root” namespace is.

jasonjckn00:02:52

is there a programatic way to crawl the root namespace

jasonjckn00:02:58

and require it all

noisesmith00:02:07

one elegant construct is that the ns that defines the defmethod also defines the data that you want the method to handle, but that's not always how defmulti / defmethod are used of course

noisesmith00:02:20

namespaces don't have a tree structure

jasonjckn00:02:35

hm okay file-seq seems too much of a hack to me

noisesmith00:02:40

when you package into a jar, you can't use file-seq as it's all in an archive

noprompt00:02:10

If I’m being totally honest I think this is a smell.

noisesmith00:02:19

one option would be to scan for clj files available via classpath whose full path matches some pattern...

jasonjckn00:02:39

if I go down this route, i'd query clojures namespace meta data

jasonjckn00:02:45

but it doesn't seem very canonical

noprompt00:02:52

I might take moment to rethink the design.

noisesmith00:02:57

that doesn't tell you about clojure files that haven't been required

noisesmith00:02:05

and if they have been required, the multimethod is there :D

noisesmith00:02:54

I can picture an idiom of saying "all clj files with foo.bar in path will be loaded because it is assumed they provide implementations of the bar multimethod"

noisesmith00:02:23

I'm not sure if that's the best solution, but it can be done, by scanning for classpath resources via a regex

noprompt00:02:36

Or you can just do the simplest thing and maintain a namespace.

jasonjckn00:02:55

yah i'm going to stick with that

noisesmith00:02:10

clojure.core/load exists, it finds a resource and loads it

noisesmith00:02:50

the reason to do the resource scanning is if you are effectively doing a plugin system - promising to load third-party extensions if available

noisesmith00:02:01

but agreed, just requiring the files is a lot simpler

noisesmith00:02:20

most code I've seen that defines a multimethod defines a bunch of methods in the same file immediately afterward - just because it operates on a different kind of input doesn't mean it needs its own file

noprompt00:02:24

Yah, I tend to stick with keeping the method definitions in the same file too because spreading them across files is a little bit like spreading mutable state around. Not saying its bad or anything but it does have implications.

hiredman00:02:34

the correct structure is a diamond

hiredman01:02:04

defmulti/defprotocol in a ns(spi), that namespace required by every namespace with an implementation of that multimethod/protocol, then a namespace(interface) that requires the defining namespace, and the implementations, and invokes the multimethod or protocol, doing any argument massaging, etc, as needed

hiredman01:02:03

that is when you have this stuff all in the same project, of course protocols and multimethods are an open set

noprompt01:02:11

I would s/the correct/a because a line would be fine too.

defmethod -> impl -> api

noprompt01:02:39

And agree diamond is nice too. 🙂

hiredman01:02:41

the diamond is clear, easy to understand, easy to use, and has no footguns

hiredman01:02:00

tell your friends

hiredman01:02:40

(it is kind of less of a thing with protocols for various reasons)

noprompt01:02:47

I think a strait line has the same properties and advantages, I haven’t been in a situation of my own design where a diamond would have provided more benefit.

noprompt01:02:22

At the same time though, over the years I’ve gotten more conservative with how I use multimethods/protocols.

noprompt01:02:07

@U0NCTKEV8 do you have any resources that discuss the relevance of the diamond in connection to open sets? Diamonds come up in other spaces (Chuch-Rosser) so I’m interested.

jeroenvandijk08:02:10

@U0J3J79FE For a big project I have several multimethods spread over a dozen of files. I use seperate files to keep it manageable in my mind (and for my editors). I have a main ns that requires all implementation namespaces and a protocol namespace [where (defmulti ...) lives]. I have all the requires in main at the top, but actually I have forgotten to add requires and I would re-eavaluate and that extension wouldn't be picked up. Definitely annoying. I can imagine a helper macro (require-* ) that requires all namespaces in a subdir can be of use here. Depending on the type of project this could or could not be a code smell :)

Wes Hall20:02:22

Hopefully this is more diagonal than orthogonal but I think trying to drop the habit of having many small namespaces is a good path out of this kind of problem. You don't need to put everything in the same namespace, but if you are finding yourself in a situation where there is a namespace that you need to require for the sole reason that it has a defmethod you need then it might be a sign you are going a little too fine-grained. It's subjective of course, but I personally find namespaces that include some general architectural concept (impl, spi, protocols, etc), a bit of a smell in themselves. You quickly run into the "2 dimensional" problem. Do you want .users.spi or .spi.users or just .spi and .users and then agonise over what goes where? clojure.core is pretty big and it's fine.

4
jeroenvandijk22:02:42

I don't think there is a right or wrong here. This is also not a thing of habit, It's a moment where one approach feels better than the other. For the record, after splitting up the namespaces these namespaces are still big. It might be a matter of taste or maybe I have less brain capacity or editor skills :) I like the fact that protocols and multimethods give you another way to organize your code. Even clojure.core is defined in multiple files e.g. https://github.com/clojure/clojure/blob/28efe345d5e995dc152a0286fb0be81443a0d9ac/src/clj/clojure/core_proxy.clj or https://github.com/clojure/clojure/blob/e78519c174fb506afa70e236af509e73160f022a/src/clj/clojure/gvec.clj . Using multiple files also helps during reviews as it is easier to see what parts of your application are being touched.

✔️ 8
Wes Hall23:02:29

@U0FT7SRLP All entirely fair. There are some practical reasons why you might want to split up your code into some smaller namespaces, you list some of them here, but I think there are also reasons why you might consider merging them. One of them being problems like the OP has, or having circular dependency issues. I do think, for people who come from, say, a Java background (as I did), where classes are the most fine grained level of "namespace", and the file division, there can be a tendency to perceive clojure namespaces as being "like classes". The clojure approach to requiring vs Java imports and things like :gen-class can serve to support this idea. Having some of clojure.core spread across different files also serves to demonstrate that this is an option available to us if we need it. The primary clojure.core file is still pretty big though. When I have had problems like the OP in the past, I have often started to merge some of my namespaces, solved the problem and been OK with the result. Often the result has felt better since I suddenly find I am not sitting with 4 or 5 emacs buffers open while trying to work on a single feature. It's hard to give concreate advice without the full context, but I just wanted to throw it out there as an option worth considering.

jeroenvandijk23:02:45

Thanks for explaining. I agree with all the different views here (all depends on the context indeed)

Chris Lester03:02:01

Has anyone had problems using specs as generative tests, where spec's stest/check cannot generate (gives up) but s/exercise and s/exercise have no problems with the respective parameter specs and fn spec? Also tried stest/check-fn after reading the source for stest/check, but it is complaining about the result not being a map ... and since no one ever seems to have used that assume I'm using it incorrectly.

seancorfield03:02:03

@activeghost That can happen, yes. Generative testing pushes the generators a lot further than exercise so you get bigger / crazier values going through the specs.

seancorfield03:02:51

If you run (s/exercise ::the-spec 100) or maybe 1000 you may well get the "such-that" error.

Chris Lester04:02:02

Yep, that's what is going on -- so some grovelling over huge amounts of data is needed ... or is there a means to limit it (looks like I'd have to override the generator?)?

seancorfield04:02:20

There's no easy way. You have to exercise each component spec from the bottom up to figure out at what point the constraints are too tight for the built in generators...

seancorfield04:02:35

...or if you have s/and constraints, you might start there.

Chris Lester04:02:47

That's what I was afraid of .. which I've done to get to this point lol .. .but not enough apparently.

seancorfield04:02:38

The such that failure is generally at a point where two constraints/specs are combined -- and then you have to add a custom generator somewhere, or rethink the constraints.

seancorfield04:02:51

I just went through exactly this problem about two weeks ago.

Chris Lester04:02:19

I have several custom generators, and a bunch of s/and specs. Will go back through them starting at the leafs again with the size set to 100 then 1000.

seancorfield04:02:53

I was adding specs to a big piece of legacy code that I didn't write and it took me... three or four days I think? I finally got to a point where everything worked and I learned a huge amount about that code and exactly what its data structures really were.

Chris Lester04:02:02

Nice. I'm specing this for a legacy monolith (these maps are cable channels essentially) and as a materialized view for clients while they move to a new data model. It does teach you a lot about the system.

seancorfield04:02:42

This was a big data processing pipeline that took a bunch of metadata in, transformed it based on what's in a database, then used that new metadata to drive queries against a bunch of database tables, and then transformed that into data suitable to drive a bunch of graphs being displayed.

seancorfield04:02:37

I did actually find a couple of bugs in it while I was spec'ing it all -- bugs we'd never noticed while using the system for maybe five years? -- and then I was able to refactor it to make it easier to test some new processes we were building on top of the graphing engine.

Chris Lester04:02:37

That sounds like a fun project, haven't done any graph ux work outside of R.

Chris Lester04:02:34

Found the problem, and there was only one 🙂 ... apparently it can't go beyond 50 with the collection in the snippet, even though the underlying specs generate at 100 (but not at 1000). However switching to what I should have used - pos-int?, fixed that problem for the entire map and stest/check now passes. ... Thx!

seancorfield06:02:54

@activeghost Oh cool! Glad you found it. Yeah, learning all of the available predicates that generate is a big job.

maxt10:02:40

One thing I seem to do fairly often is to optionally run a function on a value based on the result of a predicate of the same value, and otherwise return the original value unmodified. Is there a nice way to do this, without having to repeat the value? Expressed as an if, this is an example of logic I want

(if (string? x)
  (read-string x)
  x)
I can avoid repeating x once with cond-> , but basically I want a version of cond-> that also threads the value through the predicate. I can write that, but maybe there's another way to do it in clojure.core?
(cond-> x
  (string? x) read-string)

p-himik11:02:31

Had similar thoughts before. I don't think there's another way to do it with the built-in constructs. But it's a fairly simple macro if you want to implement one.

Crispin12:02:22

I say leave it the if expression. There's nothing wrong with it. Totally clear. Unmagical.

👍 4
Crispin12:02:54

but if you do want to obfuscate it I can think of two ways

Crispin12:02:56

(-> x string?
    (and (read-string x))
    (or x))

😱 8
😂 4
Crispin12:02:40

(defmulti processor type)
(defmethod processor String [x] (read-string x))
(defmethod processor :default [x] x)
(processor my-x)

👍 4
Crispin12:02:59

also... nothing wrong with putting a simple if like that on a single line, I say

p-himik12:02:42

I think cond-> is perfect among all other built-ins. As terse as possible, without losing any legibility.

yuhan13:02:57

There's also the condp-> macro from https://github.com/worldsingles/commons

👍 4
maxt13:02:23

Thank you for your thoughts! I hadn't made the connection with defmulti, that was interesting, though obviously too verbose for the simple case. Also thank you for the mentioning of condp-> which is indeed what I was thinking about although not in core. At least it shows I'm not the only one thinking about it.

caio18:02:36

Maybe a simpler function would be better, like (modify-if val pred fn & args), then you'd be able to use it with thread macros

👍 4
seancorfield20:02:11

Thanks for mentioning our library from work (`condp->` etc).

borkdude13:02:49

If I were to re-publish an article about the state of Clojure with a Java magazine, that has to be updated from 2015 -> 2020, what would be good to add, focus on? spec, deps.edn, ..? Feel free to respond in a thread.

borkdude13:02:08

I think one thing to add is the stability of the language (which was already true, but still is)

4
emccue16:02:17

Clojure: Now with error messages

4
💯 4
jaihindhreddy16:02:08

+9000 for Stability! Spec, tools.deps, the graal story, datafy, nav are all valuable to mention IMHO. We'd have much more to say about spec though, once it comes out of alpha, where the new syntax for fn specs, the schema-select stuff and programmatic spec manipulation will make it that much more valuable. We'll have more to talk about interop when 1.11 lands because of the integration with new functional APIs of Java that Alex Miller was hinting at.

johnj18:02:02

slower start-up times

seancorfield20:02:02

@U04V15CAJ Did the 2015 article cover 1.7 with transducers? (just figuring out what part of the history is "recent" compared to the article)

borkdude21:02:44

yeah, it did

borkdude21:02:04

not that we explicitly mentioned them, but we covered that point in time let's say

seancorfield21:02:10

OK, so direct linking (performance), socket server/repl, Spec, CLI/`deps.edn`, Java 8 as a minimum (1.10), datafy/nav, tap, error messages.

💯 4
seancorfield21:02:51

It's really interesting to read through https://github.com/clojure/clojure/blob/master/changes.md and see just how many tickets get addressed in each release but how sparse the "new features" tend to be -- but how impactful they often are.

4
sogaiu23:02:41

will you mention rebl?

seancorfield00:02:29

Ooh, yeah, a +1 for REBL!

4
Guillermo Ithier21:02:48

Hello all, I'm currently working on building micro-services for a client on a real-time application and would like to know if anyone is interested in working with me at a part-time bases. My services will be communicating with a MONGO database and the front end of the app will be in React. There is compensation for this project so if you are interested I would prefer to workout the logistics in private. Thank you

p-himik22:02:29

Just FYI, there's also #remote-jobs

Guillermo Ithier22:02:40

Thank you 😊 I'll give that a try; new to Slack