Fork me on GitHub
#clojure
<
2021-04-15
>
hipster coder00:04:53

I also found @bruno.bonacci repo. it adds a bin/sh wrapper to the jar... it also works... but it would be nice to use @borkdude interpreter... so I can start using Clojure for a lot of my devops scripts.

yuhan00:04:49

Generally if i have abstractions X and Y defined in separate namespaces, where should the function X->Y belong? ie. does X require Y and define to-Y , or does Y require X and define from-X

didibus02:04:50

I'd put it all into one namespace and call it x->y

didibus02:04:14

Otherwise it weirdly feels like you're using namespaces like classes, which is wrong

yuhan02:04:04

That's actually the current situation in the code I'm trying to refactor- the issue is that Y is conceptually in a different domain, and there will be other implementations such that x->w, x->z

didibus02:04:22

Hum... What I normally do is define my entire domain in a single namespace I call domain.clj. What I put in there is only the data schemas, so it would be data specs or records. Then the rest of my application would be defining operations over this domain data. If I want conversions between my entities for example, I could define a conversion.clj namespace. Now honestly, I wouldn't use protocols for this, I'd just have normal functions called X->Y, X->Z, Y->Z, etc. Its much clearer to me that way. Do you have a reason to want a protocol? Like do you need to have a generic conversion where you don't know what you are getting out of multiple possible type and need it converted to a Z ? That would be the question I think that can answer your question. If you want "one of many possible types"->Z then I'd just create a protocol with a ->Z definition or a make-z-from kind of thing. I'd put this protocol in the conversion.clj namespace and I'd extend all the types to implement it in that namespace as well, but the records would all be in the same domain.clj namespace.

💯 3
didibus02:04:42

If you needed the other way, like a Z->"one of many possible types", then I'd have another protocol which defines a to-z function.

didibus02:04:14

And if you needed it both ways, like a "one of many types"->"one-of-many-types", then I'd make it a multi-method.

didibus02:04:34

So what happens is the namespaces just group conceptually similar kind of functionality. They don't group all functionality related to a particular record. The latter would be what OO does, OO will say, ok group all functions that operates on a given type together. This just seems wrong to me in Clojure. So I don't put convertions from/to Z in the Z namespace. Instead I put all conversions from any entity to any other in a conversion namespace.

💯 3
didibus02:04:26

So if I take your juice example, I'd do:

(ns domain)

(defrecord Juice [color])
(defrecord Apple [name])
(ns conversions
  (:require [domain :as d]))

(defprotocol Juiceable
  (make-juice [juiceable]))

(defprotocol Tarteable
  (make-tart [tarteable]))

(extend-protocol Juiceable
  domain.Apple
  (make-juice [_apple] (d/->Juice "red")))
(ns app.core
  (:require [domain :as d]
            [conversions :as c]))

(c/make-juice (d/->Apple "Hello"))
;;=> #domain.Juice{:color "red"}

yuhan02:04:25

Thanks! I think I've seen the "Namespaces group functionality, not abstractions" advice elsewhere, but this made it finally click 🙂

🎉 3
hipster coder00:04:42

@qythium where can I find the namespace docs on defining to-Y and from-X?

hipster coder00:04:38

or, is that a convention? not a built in function

yuhan00:04:07

oh just a convention

hipster coder00:04:52

it's an interesting question. you are asking how to setup the dependency... compose the 2 functions

hipster coder00:04:29

my first thought is, how to gain maximum flexibility... so x->y or y->x

yuhan00:04:43

there's conceptually only one function from X to Y, I can't define it in both namespaces otherwise it becomes a circular dependency

hipster coder00:04:45

so you can call them either way

yuhan00:04:52

assume the transformation is one-way only

hipster coder00:04:49

still, the 1 way transformation isn't the actual dependency

hipster coder00:04:01

for example, let's say this

hipster coder00:04:14

x multplies by 2, y adds by 3

yuhan00:04:14

apple.clj define to-juice vs. juice.clj define from-apples

yuhan00:04:51

now I think about it, probably juice.clj defines Juicable protocol and apple.clj extends it

hipster coder00:04:01

ok... you could have apples, oranges and pears... but 1 juice

hiredman00:04:46

I usually have a single namespace where all defprotocols live

3
hipster coder00:04:47

I would use "from"

hipster coder00:04:52

it makes your code more extendable

hipster coder00:04:05

I think of it likes this...

hipster coder00:04:15

juice.from(any object)

hipster coder00:04:34

versus... explicit http://object.to(juice)

hipster coder00:04:00

then, you can define how th juicing is done, on the object

hipster coder00:04:16

it's losely coupled

yuhan00:04:04

What it the situation is the other way and you have a single X that needs to be transformed into multiple different "output format"s

hipster coder00:04:33

like apples to juice, smashed, chredded ?

hiredman00:04:36

Once you start mixing code written against a protocol with the definition you have a pretty good chance of introducing cyclic dependencies between namespaces

hipster coder00:04:06

@hiredman do you mean, there's nothing enforcing the 1 way transformation... so the two could call each other, circular ?

hipster coder00:04:26

does protocal mean... interface

✔️ 3
yuhan00:04:50

Yeah, that's the issue I'm facing now, refactoring a gigantic namespace into different parts and realising there were all these implicit cyclic dependencies

hipster coder00:04:07

the protocal is the 1 way transformation? the enforced rule

hipster coder00:04:29

that apples must go to juice... juice can't go to apples

hiredman00:04:11

I assume the juice thing is a for example and not an exacting specification

hipster coder00:04:29

easier, than x or y

hipster coder00:04:46

I still think the juice.from(objectType) is better

hipster coder00:04:14

is there some design pattern for this? protocals? interfaces?

hiredman00:04:34

Protocols are not java interfaces though

hiredman00:04:48

They are functions that live in a namespace

hiredman00:04:57

Not methods that live on an object

hipster coder00:04:24

k, and, they are the outward facing functions that need to be avalable?

hipster coder00:04:42

like... apple would need a "protocal" function named "juice" ?

hiredman00:04:46

To invoke the protocol functions you have to depend on the namespace where they are defined

yuhan00:04:02

Yeah the apple juice was just an example, my actual domain is related to compilers

yuhan00:04:12

but also I'm interested in general architectural patterns separate from domains

hiredman00:04:26

The protocols are all defined in one file, which only has protocol definitions, which keeps it separate from implementations, which avoids circular dependencies

hipster coder00:04:39

this is a good idea...

hiredman00:04:49

The same pattern and issues applies to defmulti based interfaces as well

hipster coder00:04:53

it's an extra layer, between the object and behavior

hipster coder00:04:14

yes, defmulti, polymorphics, based on types

hipster coder00:04:58

@qythium I think @hiredman is right... that's the best way

hipster coder01:04:29

well, that was very intesting... it's like the good parts of inheritance... interfaces... but for functional paradigm

hipster coder01:04:49

Golang took the good parts too. replace inheritance based polymoprhism with interfaces

yuhan01:04:58

let me try and extend my example:

(ns fruit.protocols)

(defprotocol FruitConvert
  (make-juice [fruit])
  (make-tart [fruit]))
(ns fruit.juice)

(defrecord Juice [color])
Won't it still have to declare a dependency on the output type?
(ns fruit.apple
  (:require [fruit.protocols :as p]
            [fruit.juice :as j]))

(defrecord Apple [name]
  p/FruitConvert
  (make-juice [_]
    (j/->Juice "red")))

hipster coder01:04:40

@qythium can you push a small demo of that code to your github?

yuhan01:04:52

Maybe I'm still thinking of this wrong - trying to understand the core.async code

hipster coder01:04:27

I'd like to actually see how protocals makes the code maintainable... when you add in more types

hipster coder01:04:57

I was just studying this stuff, yesterday. functional style, types, monads

yuhan01:04:15

I think the Expression Problem refers to something like this

hipster coder01:04:54

@qythium I am reading it... my first thought is... following domain driven design... Juicing is the most domain specific part of this problem

hipster coder01:04:11

this is important.... because Juicing is the dependency... on the outside world

hipster coder01:04:51

and, in the real world, when I juice veggies, fruits... I use the same behavior. I use my OmegaVert juicer

hipster coder01:04:15

everything else should be deterministic

hipster coder01:04:00

that's why I like the juice.from(apple) approach

hipster coder01:04:25

but a protocal is used, to tie 2 APIs together... so juice is the protocal... and each type is repsonsible for defining how they are juiced

hipster coder01:04:19

I am going to code up something, now... you have my mind going

yuhan01:04:08

Another annoying consideration is that my current data structures are maps with mostly qualified keys - if I have to turn them into defrecords to implement a protocol against, record fields can't be namespaced..

yuhan01:04:44

Multimethods might make more sense there

jjttjj01:04:58

You can consider :extend-via-metadata on your protocols to use them with maps https://clojure.org/reference/protocols#_extend_via_metadata

hipster coder01:04:35

I am trying to figure out... is the main advantage of the protocal... that you can use namespaces?

hipster coder01:04:05

opposed to... "AD HOC" polymorphic with multi methods.

yuhan01:04:27

Protocols implement ad-hoc polymorphism, because the functions dispatch only on the type of the first arg. Multimethods have more flexibility

yuhan01:04:26

I never quite trusted metadata 😅 You have to take extra care everywhere to not accidenatally drop it

didibus02:04:26

So if I take your juice example, I'd do:

(ns domain)

(defrecord Juice [color])
(defrecord Apple [name])
(ns conversions
  (:require [domain :as d]))

(defprotocol Juiceable
  (make-juice [juiceable]))

(defprotocol Tarteable
  (make-tart [tarteable]))

(extend-protocol Juiceable
  domain.Apple
  (make-juice [_apple] (d/->Juice "red")))
(ns app.core
  (:require [domain :as d]
            [conversions :as c]))

(c/make-juice (d/->Apple "Hello"))
;;=> #domain.Juice{:color "red"}

hipster coder05:04:22

when I am running tests... the most useful part of the error is at the top... then the stack trace... is there a way to reverse the stack trace... so I can see the error message at the bottom...

hipster coder05:04:33

the actual line #, with error, is at the top, past the screen

apt15:04:37

Not a fix to your problem, but I usually do is to search for ERROR or FAIL (in terminal, using cmd+f)

hipster coder05:04:47

if I scroll up inside my tmux... I can see it... Are all of you using your monitors in vertical position?

seancorfield06:04:07

I consider this a pretty bad usability problem with clojure.test but until it is spun out as a separate Contrib project, I don’t think it ranks very highly on the list of things to “fix”. There’s a solid argument against not hiding or munging the stacktrace but in terms of reporting errors to the user, it’s pretty awful output. It could take a leaf from the CLI’s book and write stacktraces to temporary EDN files by default and just report the message/cause to the screen (and have an option to report the full stacktrace to the screen, as now).

seancorfield06:04:34

(I have a lot of feelings about clojure.test 🙂 )

chrisblom07:04:26

out of curiosity, which test framework do you prefer?

hipster coder13:04:25

do you use the property testing library? I saw that one... looked interesting.

seancorfield14:04:51

@U0P1MGUSX This one: https://github.com/clojure-expectations/clojure-test — I initially switched from clojure.test to the “classic” version of Expectations after seeing it presented at Clojure/West one year but the custom tooling/lack of broad support in tooling made it quite frustrating to use, so I created a clojure.test-compatible version of Expectations and we probably have about 80% of our tests based on the expect-style tests.

seancorfield14:04:35

And, yes, we also use Clojure Spec for some tests — both to generate random, conforming example data and to do actual generative aka property-based tests.

didibus21:04:29

I think you can hook to clojure.test reporting, and report wtv you want.

didibus21:04:54

I normally run my tests inside Cider, so this isn't a problem, since cider displays things in the editor

didibus21:04:16

But I remember a library that gives you nice commmand line reporting for clojure.test

seancorfield22:04:58

Yeah, humane-test-output does a pretty decent job of providing better diffs but I can’t remember how it handles exception reporting. Several IDEs also hook into clojure.test reporting and they are typically all incompatible (because they all override the same multimethod). See https://github.com/pjstadig/humane-test-output#ides

seancorfield22:04:57

Ultra can be extremely problematic because it messes with so many things — I would never recommend Ultra to anyone who isn’t fully aware of how all this stuff hangs together.

seancorfield22:04:23

(Ultra is probably the #1 plugin problem I’ve seen people trip over in #beginners)

didibus23:04:15

I never used it, just remembered it.

hipster coder06:04:14

@seancorfield let me see if I can pipe the test responses, reverse it, but I will need to play with the n number of lines

yuhan06:04:49

If you use Emacs just run the test with Cider - it does all this filtering for you and more

hipster coder06:04:25

I have cider running in Spacemacs... checking now for that ability

hipster coder06:04:48

I am new to Spacemacs... not sure what the , comma key is

yuhan07:04:14

are you using Evil? , should just be the comma key in normal state - there's a #cider or #emacs channel if you want to bring it there

hipster coder13:04:46

I found it... they just mixed up the commands... it's t T

chrisblom07:04:26

out of curiosity, which test framework do you prefer?

roklenarcic14:04:44

Does AOT compiling the code affect alter-var-root ? I keep forgetting if there’s any interaction there.

borkdude14:04:36

@roklenarcic Direct linking (`clojure.compiler.direct-linking=true`) will affect this.

borkdude14:04:54

since many of the indirections will already be gone by the time you execute alter-var-root

borkdude14:04:22

but AOT in general does not affect this since the indirection through the var will still be there

roklenarcic14:04:08

direct linking doesn’t touch dynamic vars though, right?

borkdude14:04:51

Direct linking only affects 1) non-dynamic vars or vars that aren't marked with ^:redef, 2) if those vars are used in call position, like (foo x y) but not if used like (apply foo x y)

roklenarcic14:04:17

Ok so for my caching library that means that those that want direct linking will have to set the cache when creating var, not later 🙂

Alex Miller (Clojure team)14:04:11

if you have a lib where something should not be direct linked, then that's what ^:redef is for

borkdude14:04:09

@roklenarcic direct linking only affects vars that are called directly as functions. if your cache value is created and passed around as an argument and not used as a function, direct linking won't affect that. but if f is some kind of caching function that should be altered, then yes

borkdude14:04:47

An example of your API would be better here

roklenarcic14:04:30

I think I already have both covered. You can write:

(memo #'a-var cache-config)
and it will wrap existing function at a-var into a cache and change the var root. Or you can write:
(memo (fn [] ...) cache-config) which just returns a cached version of the fn

Marcus Pemer15:04:58

Greetings Clojurians. I managed to get myself confused over a syntactic detail today, and I am hoping that someone here might help. If you look at the enclosed screen shot, there is an implementation of the str function. Within that implementation there are some type hints in the signatures, for performance improvements. I am absolutely on board with the ^String hint, But what does the ^some hint do in the line (loop [^some xs xs] ... ??? I have spent some time searching for documentation that explains this particular annotation, but so far I have found no mention of it. I suppose it does not help that search engines seem to ignore the special character ^ . Either way, any information would be appreciated - thank you!

yuhan15:04:21

Did you write this yourself or get it from somewhere? I'm almost certain the ^some doesn't do anything meaningful

djblue15:04:06

So it might be meaningful in dart

yuhan15:04:56

whoa, never heard of ClojureDart before

yuhan15:04:34

so it's not standard JVM clojure syntax in any case

Alex Miller (Clojure team)15:04:42

that's not a JVM clojure thing, must be specific to the Dart stuff

Marcus Pemer15:04:26

ah - ok well that helps

dpsutton15:04:36

i wonder what it's intent is. My first though was it was annotating that as not-null to perhaps elide a null check but then there's a null check right after

Marcus Pemer15:04:58

yes, I took the screen shot from the above - I was reading something else, came across this code example, and got mentally stuck on the ^some part.

Marcus Pemer15:04:23

if it isn't standard Clojure syntax, then I can stop obsessing over it. Still curious how it may be used in dart, but won't lose sleep over it. Thank you for confirming.

hiredman16:04:50

the tweet has the generate dart code, but the generated dart code doesn't look like it runs correctly (the loop unconditionally breaks?) so a work in progress

Marcus Pemer16:04:53

Interesting - thank you!

cgrand22:04:37

@hiredman it’s a wip but this code happens to work (there’s a continue which skips the break)

hiredman22:04:38

ah of course, been a long time since I've used continue and break

Lennart Buit16:04:10

I have a bit of an odd issue I can’t really pinpoint. I have an app that depends on slf4j-timbre (https://github.com/fzakaria/slf4j-timbre). When I uberjar this app, and start it, I get an exception:

Exception in thread "main" java.lang.NoSuchFieldError: __thunk__0__
(... stacktrace elided ...)
When I remove slf4j-timbre, all is well again and I can start my jar. Now, I found https://clojure.atlassian.net/browse/CLJ-1886, which has the same exception, and the same library but on a different version. The suggestion there is that it has to do with AOT compilation, but I don’t fully understand what I should watch out for in my app that could cause this issue.

Lennart Buit16:04:51

Not the most sharp … problem description, sorry about that 😞

hiredman16:04:17

the problem is slf4j-timbre is aot compiled

Lennart Buit20:04:21

Maybe for my understanding, as I’m still arguing with this :’), what should slf4j-timbre have done? They seem to need to AOT compile to generate classes found by SLF4j… (Genuinely curious ^^)

hiredman21:04:38

what should they have done? hard to say, the most flexible thing would have been to write a shim in java that loads and executes their clojure code as needed, and not aot the clojure code

Lennart Buit21:04:03

Yeah alright, that makes sense 🙂.

didibus21:04:26

What you have to do in cases like this is only compile the java stuff, not the clojure

didibus22:04:25

So like, you AOT, but then include only the .class files you need in your Jar. That's the correct thing to do if you want to remain all Clojure and use gen-class/gen-interface over a Java shim.

didibus22:04:39

Lein doesn't have support for this though, you need your own build step that packages a Jar this way

Lennart Buit16:04:36

Right, but I have other apps that also have slf4j-timbre, same version, in which the jars do start. So, my assumption is that there is something special about this particular app that doesn’t play nice with slf4j-timbre being AOT compiled. How could I (or should I?) diagnose what is causing issues in this particular app.

❤️ 3
hiredman16:04:49

it will be something like an overlap of depedencies between the rest of your app and sl4j-timbre

Lennart Buit16:04:46

I’ll take a look whether I can find some misaligned deps then, thanks

zalky18:04:41

Hi all, let's say I'm working in my REPL namespace, and have some other namespace required and aliased. Now that other namespace gets reloaded. It appears that this leaves my REPL aliases referring to stale namespace objects, none of the new interns that I expect to be in the reloaded namespace resolve. However, if I use the the fully qualified namespace prefix, I see those new interns as expected. Is this correct and expected behaviour? Does maintaining REPL aliases require some kind of specific intervention?

seancorfield19:04:33

@zalky can you be a bit more specific about exactly what you’re doing and what behavior you’re seeing? This doesn’t sound like how things work for me — I have REPLs running for weeks without running into that sort of thing…

zalky19:04:37

@seancorfield, I think this might be one of those c.t.n. "hoops" you alluded to in our previous thread. 😛 My best understanding from looking at the c.t.n. code is that reloading changed namespace and their dependents results in a new set of namespaces objects. For those namespaces that are in source, this is fine because the dependency graph keeps everything consistent. However, I don't think c.t.n. touches the REPL namespace, and so any aliases in the REPL namespace will continue to refer to old namespace objects. If you reference the namespace fully qualified without an alias, you'll get the new one. At least that's what I think is happening.

seancorfield20:04:32

Yup, this is one of those “don’t use a ‘reloaded’ workflow” things. It breaks in mysterious ways.

seancorfield20:04:59

Given that nearly half the readme is given over to https://github.com/clojure/tools.namespace#warnings-and-potential-problems I’m amazed it’s still such a popular approach 😐

nilern20:04:15

I don't recall having any problems like that. YMMV I guess.

nilern20:04:19

And some of those caveats apply to any namespace reloading

hipster coder20:04:38

@seancorfield would using an auto test runner... also be considered "don't use a workflow that reloads" ?

seancorfield21:04:56

I don’t like auto/watch test runners for different reasons 🙂

apt23:04:34

Just a remark on tools.namespace: I find it useful when something defined with a macro like schema is changed and you need to reload all namespaces that use it (https://github.com/plumatic/schema), But yeah, I don’t like the idea of using this reloading stuff as a core part of my workflow either.