Fork me on GitHub
#clojure
<
2020-05-29
>
Nolan04:05:30

Hey everyone! Hope everyone's night is going along nicely. Trying to make sure I'm not building a footcannon and I'm having trouble coming up with a concise search phrase for this. Essentially I'm trying to expose another namespace's defprotocol functions in a separate "top-level" namespace. The setup is something like:

(ns some.ns.proto)

(defprotocol Proto1
  (f1 [x]))
with
(ns some.ns.api
  (:require [some.ns.proto :as proto]))

(def f1 proto/f1)       ; Won't work as expected when `extend-protocol` is called "after" this `def`, presumably because of the `alter-var-root` call in `clojure.core/extend`
(def f1 (var proto/f1)) ; Works!
I'm pretty happy with the namespace organization, but I'm trying to figure out if I'm asking for bad by redef'ing a var like that... Is there a "right" way to do this? Any input would be vastly appreciated.

seancorfield04:05:45

My instant reaction is "don't do that!" ...

šŸ˜‚ 8
šŸ‘ 8
parrot 8
seancorfield04:05:56

There's a well-known library called Potemkin that has an import-vars macro to do this sort of thing but the consensus is usually expressed to avoid it šŸ™‚

didibus09:05:01

I wouldn't say its a consensus, feel its more like a "hotly debated topic"

seancorfield04:05:11

Why do you want to pull the protocol into another namespace like that?

Nolan04:05:00

Well, the idea was to have the protocol available essentially in isolation, a given implementation available in another namespace (also in isolation), and then have a "batteries included" namespace at the top, that requires the protocol AND the "default" implementation, without consumers needing to require two different namespaces to work with /all/ extended objects.

Nolan04:05:09

that's a mouthful, but hopefully the point comes across

seancorfield04:05:17

If you look at most libraries that have protocols, they tend to have their protocols in a single namespace and the expectation is that clients should explicitly require that namespace in order to access those protocols.

Nolan04:05:27

Right, right. Well it's certainly easy enough to require the protocol namespace in consuming namespaces. I sure am glad I asked šŸ˜‚

seancorfield04:05:44

The other thing to consider is that it is common to write wrappers for protocols -- and expose those in the top-level API namespace.

seancorfield04:05:54

Look at next.jdbc, for example.

seancorfield04:05:22

The primary API is wrappers in next.jdbc. The protocols are in next.jdbc.protocols and most clients do not need to require that.

seancorfield04:05:44

Then next.jdbc.result-set etc have standard implementations of those protocols (which are required into the top-level API namespace, but can be directly required into client code when access is needed to something specialized).

seancorfield04:05:11

The other benefit for writing wrappers is that you cannot instrument protocols but you can instrument wrappers šŸ™‚

šŸ‘ 4
Nolan04:05:34

Out of curiosity, if someone were to extend those protocols "downstream", would the wrappers work just as well on /those/ extensions? It seems that is the case I was running into

seancorfield04:05:51

If you need to extend the protocols you must require the namespace that defines the protocols -- and so you should: you are explicitly opening up the protocol namespace and adding things.

seancorfield04:05:43

That's fairly typically of Clojure's explicitness.

Nolan04:05:22

Right. Well excellent! Ok this has become fascinatingā€”last question, I promise (thank you so much for your time tonight Sean!). The following does exactly what I was hoping it wouldā€”does this approach trigger the same knee jerk, or is this a better "wrapper":

(ns some.ns.api ,,,)

(def f1 (fn [x] (proto/f1 x))

;; or any equiv, e.g. (defn ,,,), (partial ,,,) etc.

seancorfield04:05:59

Why not

(defn f1 [x] (proto/f1 x))

seancorfield04:05:08

Oh, I see the comment...

seancorfield04:05:19

Guess I don't understand what you're asking?

Nolan04:05:20

yeah, sorry. that was just quick repl bad style

seancorfield04:05:30

I mean that's exactly the wrapper I'm talking about.

šŸ‘ 4
seancorfield04:05:02

(mostly it's clearer to use a -name for the protocol functions so that folks cannot confuse the wrapper for the protocol)

parrot 4
wizard 4
šŸ™ 4
seancorfield04:05:36

That's particularly important when you write multiple arity wrappers (like next.jdbc does)

Nolan04:05:35

Oh man, this is so helpful. Makes perfect sense. That fully settles it, then. Apologies for the poor framing there toward the end, but your answer totally covered it.

Nolan04:05:50

Thanks again Sean.

seancorfield04:05:00

next.jdbc has direct (same-named) wrappers for get-connection, get-datasource, prepare and then has multi-arity wrappers for -execute, -execute-one, and -execute-all. And -transact. The latter has a macro wrapper as well.

seancorfield04:05:18

(and probably also worth noting that the default implementations are split across three namespaces: next.jdbc.connection, next.jdbc.result-set, and next.jdbc.transaction šŸ™‚ )

Nolan04:05:05

Is there any consideration with naming protocol functions with a leading - in java interop? I feel like I've seen blog posts where some java interop expected/required -xyz, or something. But maybe that was just the conventional/context-specific rather than a real syntactical requirement

seancorfield04:05:42

For gen-class, the - is the default prefix for (Clojure) functions that are exposed as Java methods.

wizard 4
šŸ™ 4
clj 4
seancorfield04:05:55

That tends to spill over into protocols where folks will often use --prefixed names for the protocol functions and unprefixed names for the wrappers -- but that's purely a "common convention". I've seen other conventions... and some fairly random mismatches between wrappers and protocol functions.

Nolan04:05:19

gen-class was totally what it was. Got it. You have tremendously impacted my course through protocolspace tonight Sean. Thanks again!!!

seancorfield04:05:01

Happy to help any time -- I'm online nearly all the time šŸ™‚

parrot 4
šŸ™ 4
metal 4
Nolan04:05:08

Ah, yes. I think I'm seeing where this all went astray. I think I can pretty easily work around this without the dirty var redef while keeping everything relatively clean. Can't tell you how appreciative I am of the help @seancorfield!!!

4
David Pham04:05:39

Why is import vars a bad thing from potemkin?

seancorfield04:05:08

This thread has some good discussion about it @neo2551 [thread link removed] this message https://groups.google.com/d/msg/clojure/Ng_GXYSrKmI/MJsKDKhJCQAJ in particular

šŸ‘€ 4
seancorfield04:05:49

Also from that first thread https://groups.google.com/d/msg/clojure/Ng_GXYSrKmI/ULKD5cN9BAAJ (Alex Miller cautioning against import-vars)

seancorfield04:05:43

To me, use of Potemkin indicates either poor or lazy API design (for several of the reason mentions in those threads).

vemv04:05:54

Great reads! Didn't know those aspects. From intuition, I never used Potemkin i-v as a usual weapon, but it seemed like a good occasional resource. Will correct that conception myself. As an additional data point, using import-vars will create an annoying https://github.com/jonase/eastwood linter complaint (which is correct in itself, but tedious to address)

seancorfield05:05:17

I have used a similar technique in a very specialized, limited context: creating expectations.clojure.test which is a clojure.test-compatible version of the classic Expectations library and, for convenience, you can just require expectations.clojure.test instead of clojure.test and still have access to a handful of API functions that are identical -- and "imported" from clojure.test: https://github.com/clojure-expectations/clojure-test/blob/master/src/expectations/clojure/test.clj#L437-L453

seancorfield05:05:18

But I definitely do not advise it as a general way to organize/build APIs. And this is very specific in how it deals with metadata, in order to work better in editors/IDEs.

āœ… 4
Black09:05:09

What were the reasons you switched to clojure? I am trying to convince my OOP colleagues to switch into FP programming but even when they see how cool clojure is using repl they still stay with OOP. So how would you summarize benefits to using clojure instead of other OOP languages.

Johannes F. Knauf10:05:41

ā€¢ Interactive Development style ā€¢ REPL interaction with other JVM processes (in fact, I established that idea using Groovy before) ā€¢ Simplified DSL development ā€¢ Compactness of code ā€¢ Superior immutable data structures ā€¢ Absolutely reproducable and testable behaviour (for all pure parts) ā€¢ Simple code generation ā€¢ multimethod dispatch magic ā€¢ getting rid of superfluous bloaty framework magic Just to name a few of the very pragmatic advantages... Summary: Iterate faster, improve maintainability, have more fun.

šŸ’Æ 8
markus10:05:44

Yeah, faster iterations are very good for morale.

didibus10:05:04

Benefits are the same as having a freshly sharpened chef knife over a dull one. You can do all the same things, but it's way more satisfying to do so when you're knife is really sharp and slices through things easy, painfully and quickly

šŸ˜„ 4
p-himik10:05:32

"Painfully"? :)

4
šŸ˜‚ 4
Oliver Smit10:05:38

that got dark real quick Didibus šŸ˜‚

p-himik10:05:42

To add to what others have said: - Clojure has a really nice polymorphism model. It's much more robust and extensible than others I've seen. Although it might not be apparent at first sight, especially to someone coming from the dreaded OOP world - "Code is data" goes a long way - It's much easier (and simpler) to compose functions than classes

šŸ’Æ 4
p-himik10:05:23

Also, the ecosystem is really nice. Tools, libraries, community. Libraries is my pet peeve with other languages. So much garbage around that managed to become popular, it's mind boggling.

pmooser11:05:48

I'm not sure how easily you can rationally convince most people to change. The most compelling reason for me personally is that it made my job less painful, less boring, and more fun. But a lot of people are really reluctant to take the temporary hit and feel less capable by switching away from languages and paradigms in which they've become comfortable.

šŸ’Æ 4
Johannes F. Knauf15:05:35

@U07VBK5CJ Very good point! These decisions are often less about rational reasons and more about excitement and joy. So inspiring curiosity and sharing Rich Hickey talks or other cultural community artifacts (live coding overtone?) with your people might be a better approach for getting buy-in of developers. Convincing Management is a different issue, though.

seancorfield15:05:09

@UC0SN2H0E Folks who are deeply rooted in OOP thinking, especially if it's the only paradigm they've ever used, often find it really hard to get into FP. It forces them to "unlearn" a lot of what has become second nature to them. This can make it really hard to convince them that FP is "better" because, right now, they're competent and productive and any attempts they make at FP will be alien to them and even the simplest code will take them much longer to write.

seancorfield15:05:06

In addition, if they're used to a statically typed OOP language, they'll probably find Clojure's "lack of types" to be very disorienting.

seancorfield15:05:19

At work, we cross-trained two developers from OOP backgrounds to Clojure. After a year or so, one left to return to the same OOP language he was used to because, although he enjoyed Clojure, he never really felt comfortable with it. The other developer eventually left for personal reasons but has gone on to other full-time Clojure roles at other companies. In other words, not everyone will stick with Clojure even if they try it (and seem to like it).

seancorfield15:05:17

I would suggest writing your own code in a more FP style within the "home" language for your company and see how your colleagues react to that. If they push back and want your code to remain in a more OO style, that says you're very unlikely to get them to switch. If they like what they from your change of style, you'd probably have a better chance of introducing Clojure.

seancorfield15:05:56

Going back to your first question: I switched to Clojure because a) I already had a background in FP (from the '80s, before OOP exploded) and b) I'd been slowly moving that direction (my overall arc was C, C++, Java, ColdFusion/CFML, Flex/ActionScript, Groovy, Scala -- and finally to Clojure a decade ago).

Black15:05:27

@seancorfield "I would suggest writing your own code in a more FP style within the "home"..." Actually this is the path I am trying, slowly introdicung map, filter, reduce, etc.. And your point about lack of types is great too, in my last job where I build FP app (using ramda in js) when I left they switched to typescript and said they could not wrap their heads around ramda style (probably mostly my fault because I was not as good in FP as I should be at that time)

Jakub HolĆ½ (HolyJak)17:05:22

IMO https://www.manning.com/books/grokking-simplicity is great for introducing FP thinking to people new to it. In my experience, JavaScript is a much better stepping step to Clojure than Java - it is dynamically typed, uses mostly pure data, and can use just function (every time I have to touch our OOP JS I want to cry). My personal #1 for Clojure is immediate feedback thanks to REPL-driven development - but this is a skill of its own and the value of this people often fail to appreciate at start. I have written a few posts comparing OOP (java) and Clojure, though not sure whether it is helpful here - https://blog.jakubholy.net/tags/clojure-vs-java/

didibus08:05:44

"made my job less painful, less boring, and more fun" - this is it to me. I just find Clojure more fun.

Daniel Tan09:05:14

with clojure you can care a lot less about breaking someone elseā€™s class due to some logic change

markus10:05:45

This! Safer to make local changes is the killer feature.

pmooser11:05:50

I must be missing something, but I wish that update-in did something useful with an empty path, like executing f against the map.

šŸ‘ 4
souenzzo20:05:09

Hello I'm trying to invoke clojure process from another clojure process. ATM I'm using java.util.Process stuff with ["java" "-co" "a clojure cp with clojure jar" "clojure.main"] It start nicelly but when I try to (.write p (.getBytes "(prn :hello)")) it writes, evaluates and exit, like $ echo "(prn :hello)" | clojure How do I avoid this exit behavior? I'm sure that i'm (almost) not sending a end-of-file

noisesmith20:05:45

an end-of-file isn't a special message, it's a condition that's hit when input closes

noisesmith20:05:56

and the default main behavior is to exit when stdin closes

noisesmith20:05:27

you likely want something like the socket server that lets you evaluate chunks of text but keep running

noisesmith20:05:45

either that, or you are accidentally closing p

Alex Miller (Clojure team)20:05:27

I assume you meant -cp up there

āœ”ļø 4
Alex Miller (Clojure team)20:05:17

also I'm assuming that you are writing to an input stream, not the process itself

Alex Miller (Clojure team)20:05:18

also I assume you mean java.lang.Process, not java.util.Process

Alex Miller (Clojure team)20:05:50

also the number of discrepancies makes me think it would probably be better to post more of the actual code :)

šŸ’Æ 4
souenzzo20:05:43

I will play a bit more and ask again in #off-topic

Ben Grabow21:05:49

Are there any good resources for applying Rich Hickey/Clojure wisdom to writing regular Java code? I'm looking at some java that uses classes to represent domain information, with synchronized methods for fine-grained mutation of individual data fields, and I'm searching for a path towards a more clojurey solution but I don't quite know where to begin.

Ben Grabow21:05:49

What I'm looking for is something akin to a book written by Rich called "Java, The Good Parts"

hiredman21:05:44

the first thing is to get rid of mutable fields

andy.fingerhut21:05:51

He uses final fields an immutable instances of Java objects all over the place in the implementation of Clojure, and Java Concurrency in Practice often mentions immutable objects as a way of not having to think harder about whether you have gotten the concurrency aspects correct. That is atypical Java code, of course.

andy.fingerhut21:05:27

Prefer creating APIs based on interfaces, often very focused ones with perhaps only one method signature in them, or perhaps a few closely related ones that really always belong together, versus implementation inheritance.

hindol21:05:36

Google's Java guides strongly advocate for immutable classes. I have begun to think this is common wisdom now and not atypical Java.

andy.fingerhut21:05:57

Could be. Rich was strongly recommending reading Java Concurrency in Practice in 2008 (perhaps earlier before he was giving talks that are available on YouTube). I read it last year, and it still seems to me like a good source for that kind of info.

šŸ‘ 4
hindol21:05:25

JCIP is an amazing book. "You think you know concurrency? Think again."

andy.fingerhut21:05:50

It walks an amazing line of not telling you: "You should always use immutable objects", giving plenty of advice on how to make things safe when you have mutable objects, but the rules for safety are certainly harder to follow, and keep following them over the maintenance lifetime of a program.

raspasov21:05:42

@U0CMVHBL2 haha I guess the enlightenment of immutability is stronger when you reach the conclusion yourself šŸ˜ƒ

hindol21:05:55

(I have not read it cover to cover yet.) Maybe when it was written, immutable everything still was not practical (too slow).

raspasov21:05:05

@UJRDALZA5 also without persistent immutable data structures which Clojure has, yes, it can be a problem, esp with large collections

hindol21:05:01

Yeah, I also think Clojure would not be the way it is if persistent collections were not invented. They form the backbone of Clojure itself.

Alex Miller (Clojure team)21:05:27

persistent with structural sharing is the hard part

šŸ‘€ 4
Alex Miller (Clojure team)21:05:54

If there were enough good parts, he wouldn't have written Clojure :)

raspasov21:05:30

@ben.grabow I would sayā€¦ if you really need to write Java, write it as if youā€™re writing Clojure, wonder if thereā€™s an ā€œeasyā€ way to pull the data structures out of Clojure, and use them from Java

Alex Miller (Clojure team)21:05:55

you can't easily do that, but there are other options in Java

raspasov21:05:11

something akin to immutable-js but for Java

raspasov21:05:20

I donā€™t know on top of my head, maybe somebody else does

raspasov21:05:37

I tell people ā€œif you have to write JavaScript, you MUST use immutable-jsā€

raspasov21:05:44

or youā€™ll suffer

raspasov21:05:03

and really use it, everywhereā€¦

raspasov21:05:36

@ben.grabow thatā€™s your thing, I believeā€¦

andy.fingerhut21:05:19

Caveat emptor: bifurcan is cool. I'm not sure it is as battle-tested as Clojure's immutable collections are, yet.

hindol21:05:38

@raspasov What I used to do was annotate every class with @Immutable and then use a static analysis tool like FindBugs.

Alex Miller (Clojure team)21:05:00

bifurcan has been extensively tested with generative testing via collection-check so I would expect it to be pretty solid

andy.fingerhut21:05:05

I might be advising more caution than necessary there. My use of it was only testing its equivalent of core.rrb-vector, and finding neither it nor any other implementation of RRB vectors that I couldn't find bugs in within a day or three of trying.

Ben Grabow21:05:01

Once mutation has been moved out of domain objects, is it just a matter of using java.util.concurrent.atomic.AtomicReference compareAndSet to do coarse-grained mutation?

hindol21:05:46

What do you mean by coarse grained mutation?

Ben Grabow23:05:32

Instead of atomically setting a person's first name, then atomically setting the person's last name (fine grained mutation) you construct a new map with the new values of the first and last name, then atomically swap the reference of the old map for the new map. This concept can be extrapolated until any mutation in your system is an atomic swap of a single, top-level atom. See cljs reagent as an example.

noisesmith21:05:47

that should happen only where you'd use an atom in clojure (in other words almost never), most of the time you'd just accept and return immutable values from each method

noisesmith21:05:46

another thing to note is that building an algorithm to use mutation inside a small scope is often fine, as long as the interface is pure and the scope of the mutable values is narrow enough

noisesmith21:05:23

eg. - don't mutate things that are passed in / pass mutables to other code, but mutating for intermediate values is often just fine

Ben Grabow21:05:50

There are other knock-on issues to tackle as well, which is why I'm looking for something like a comprehensive reference. How do you manage deep equality on nested data structures? How do you represent domain information in a composable way?

andy.fingerhut22:05:36

Nested immutable values do compose quite well, at least certainly they do in Clojure, and I cannot think of why that would not be the case in any programming language with immutable values, including immutable collections that nest.

andy.fingerhut22:05:41

Equality is deep equality in such a scenario. There are implementation optimizations you can play, as Clojure takes advantage of, like "if two immutable values being compared are pointer/reference equal, then they are equal, without having to recurse into their components"

andy.fingerhut22:05:53

but that is an optimization, not a definition of the meaning of equality.

andy.fingerhut22:05:46

Clojure programmers often use maps to represent domain information, nested if that is useful, with vectors in places where order is important.

Ben Grabow23:05:39

I believe in and agree with all these things in theory. I suppose what I'm looking for are some examples of this ethos put into practice in Java specifically, in hopes of learning from someone else's hard work to manage connecting the pure, immutable parts with the impure, mutable, idiomatic Java parts.

andy.fingerhut23:05:53

I don't have examples of Java applications to give you off hand, if that is what you are looking for. Most Clojure applications that use Java libraries are of this kind, with various approaches used at the interface between Clojure code and mutation-heavy Java library.

noisesmith21:05:31

isn't "standard immutable collections" the answer to both those questions?

Ben Grabow21:05:34

The answers to these questions generally have to play well together, so it's hard to build a picture of a working system with isolated answers to each question.

Ben Grabow21:05:47

I think it might be. @noisesmith

seancorfield22:05:10

I think this discussion just highlights how difficult it is to follow Clojure semantics in a language that doesn't have immutable data structures built-in: there's so much in Java-the-language that keeps leading you to mutation-based code. It's not expression-based, so you're constantly fighting against statements, trying to produce values and avoid the "lure" of side-effects. Basic language structures for iteration all assume mutability: you have to work hard to write computations that avoid that (using streams everywhere, which is kind of ugly at scale).

šŸ’Æ 24
ā˜ļø 12