Fork me on GitHub
#clojure
<
2021-03-07
>
quadron01:03:25

would it make sense for packages to include the version in their namespace? would it not be more in accord with the idea of immutability and robustness? why should my project break if some implicit dependencies lose consistency over a simple update?

quadron01:03:02

i would understand if this limitation is due to the legacy way of doing things in production

andy.fingerhut02:03:40

Rich Hickey didn't recommend keeping immutability of code -- he recommended that updates to code should not break the API provided by previous versions, i.e. you can add new functions or other kinds of Vars, or extend their capabilities to require less in terms of inputs, or provide more as return values, but not vice versa. https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md

andy.fingerhut02:03:09

If you want to break the current API, then you should consider creating a new name for the library and its namespaces, so it is a new thing, not a broken version of the old thing.

quadron02:03:47

@andy.fingerhut are there any ecosystems that consider the version to be part of the namespace?

quadron02:03:01

@andy.fingerhut does it make sense to have such a thing?

andy.fingerhut02:03:07

One way to provide a new library that is similar to an old one is to put a number in its name, e.g. the first version was mygreatlib, and when you want to break the API of mygreatlib, you instead publish mygreatlib2. I believe the talk I linked above mentioned a few cases of Windows APIs where this was done, but it was pretty rare.

quadron02:03:02

@andy.fingerhut but say different packages dependencies are pulling different versions of the "same" library, would there not be a conflict?

andy.fingerhut02:03:02

Depending upon what mygreatlib and mygreatlib2 do, they might conflict with each other, e.g. they both open a network socket listening on the same TCP port, or might write to a config file with the same name, etc., but in many cases there would be ways to prevent such conflicts. I could imagine some library functionality where it doesn't make sense for them both to be loaded at the same time in the same program, and some where it would make sense.

andy.fingerhut02:03:34

If they have different names like mygreatlib and mygreatlib2, most package management systems will treat them as different libraries that can both be depended upon. Whether they can both be used from the same program is a separate question. Most package management systems I am aware of that have library names and version numbers do not let you depend on both mygreatlib version 1.1 and mygreatlib version 1.2 at the same time -- you must pick one.

em02:03:36

Slightly off topic but immutability in code might be one of those things that needs a lot more of the language built up around it, like how you wouldn't be doing as much functional/immutable programming in Java. Interesting experiment on this very idea is Unison lang https://www.unisonweb.org/

quadron02:03:14

hmm, but isn't it bad practice to hide such stateful actions in the behaviour of a namespace? i suspect that in any case the namespace shouldn't open files or ports implicitly without knowledge of the caller

quadron02:03:16

what you described sounds like a bug to me, it does not justify the expectations of the package manager

quadron02:03:32

come to think of it... how do linux package managers treat this?

quadron02:03:40

@eagonmeng unison seems like a hardcore implementation of what i had in mind

andy.fingerhut02:03:43

If the library you are using is intended to help serve web requests, then it is supposed to listen on TCP sockets. In that particular case it is likely that merely loading the library doesn't open up sockets -- you would have to make an explicit call, and in most cases you could probably provide an optional, or required, parameter for the TCP port number to listen on, so even then two differently named libraries that do that might not conflict with each other.

andy.fingerhut02:03:22

Linux package managers and Clojure Leiningen/deps.edn/boot are doing related, but still quite different things.

andy.fingerhut02:03:51

There are Linux packages that conflict with each other, even if they have completely different names. Sometimes this is documented, sometimes not.

quadron02:03:28

@eagonmeng i am not sure if the namespace/version issue is addressed there

em02:03:03

Unison's pretty early on but as all code is immutable and more importantly, all names are transient, you get that extra layer of indirection that symbols/vars give in Clojure. See https://www.unisonweb.org/docs/codebase-organization#upgrading-libraries-to-the-latest-version

quadron02:03:19

so if different versions of the "same" lib are present, does the importer include the version explicitly where the import is declared?

quadron02:03:15

how does a namespace know which version it is importing?

em03:03:45

Code is always referenced by hash, so since it's completely immutable, nothing is ever updated. A new "same" version of a lib needs to be explicitly updated by a "patch". Think of it exactly like Git - when someone "updates" the dev branch with code, or when the update the library version, you have to merge the changes. If those merges are like cases in git where you have strictly additive changes (like Rich talked about in non-breaking changes) then you have nothing to do. If any definitions are at all changed, you need to merge those locally. Unison has an interesting mechanism for always making such "patches" or diffs visible, so you always have the granular power to update functions you want to. In fact there's not really a concept of a library, just a bunch of functions that might depend on one another as part of a big group.

em03:03:59

It's an interesting idea overall, but I don't think it applies to Clojure very well, as I agree with Rich in general that code doesn't need the same immutability properties. I think you get a lot of leverage out of the idea if you build an entire system around it like Unison (free distributed routines, zero compile ecosystems, no "broken" code ever), but a lot of what makes that work is the Haskell-level type system and needing to rebuild the entire thing from the ground. I'm curious how many of the ideas could be somewhat imported into Clojure though

1
emccue03:03:19

Is it correct to say that in transit, if a json map is produced as output then it must be produced via json-verbose and there won't be any caching

emccue03:03:11

i'm trying to write a transit library for elm (just doing the decoding side for now)

Ben Sless07:03:12

Is there a way to define a "mutually defined" :inline implementation for a function? Something like

(defn foo
  {:inline
   (fn [code]
     (if (pred code)
       `(inline-form [email protected])
       `(foo [email protected])))}
  [& args] (,,,))

pez07:03:08

I just noticed that when constructing a string from a keyword the string is not identical to its literal form. Anyone knows why this is so?

(= "foo" "foo")                ; => true
  (identical? "foo" "foo")       ; => true
  (= (str 'foo) "foo")           ; => true
  (identical? (str 'foo) "foo")  ; => true
  (= (str :foo) ":foo")          ; => true
  (identical? (str :foo) ":foo") ; => false

Ben Sless07:03:24

Keyword's toString() instances a new string, while ":foo" is statically initialized in the context of your code

Ben Sless07:03:59

Unless you use some JVM option to force it to remove duplicate strings they won't be identical

pez07:03:46

Ah, thanks. But for a symbol it is not like that, I take it?

Ben Sless07:03:53

Beware of context

(def sym 'foo)
(def s "foo")
(identical? (str sym) s);; => false

❤️ 1
pez07:03:19

Also, can I safely instruct others that keywords are always identical, no matter how they come to be?

Ben Sless07:03:56

because keywords are interned

pez07:03:29

I thought strings where too, and was writing in a beginners guide that they are, then tested it some…

pez07:03:59

While I have your attention then, the reason this throws is that map lookup is based on equality?

{(str :foo) :bar
 ":foo" :baz}

Ben Sless07:03:16

Yes, but it probably throws on the hash collision even before testing for equality

pez07:03:50

duh! Thanks!

Ben Sless07:03:03

Strings are usually interned but you have no guarantee they'll be the same object if not initialized statically. Especially with dynamically created strings you wouldn't want to intern them. Keywords are always interned and guarantee identity, which is why they make excellent map keys

Ben Sless07:03:16

When in doubt I just read the source

pez07:03:57

It was this suitability for map keys that I was writing about.

pez07:03:03

Then I went on a tangent with how strings can be lookup efficient too, but I have scrapped that now, since it was a bit deeper than suites the context.

Ben Sless08:03:38

If you're writing about map keys consider adding the following piece of data: collections as map keys are nice but their performance is hot garbage because lookups always entail an equality check

Ben Sless08:03:37

With less hyperbole

pez08:03:09

Haha, I was about to ask about that, actually. Thanks!

Ben Sless08:03:17

Nested maps or synthetic keys have better performance

andy.fingerhut15:03:49

Even collections as keys can be fast, if your application is set up so that it happens to always do lookups with identical collections, since then clojure.core/= finishes quickly after checking identical?

andy.fingerhut15:03:12

But that is not necessarily a typical case, I agree. I have seen it sometimes be the case, for some applications.

andy.fingerhut15:03:06

e.g. a library given some set of values (that might be collections) that it then internally uses as keys in a map to associate data with each of those values. The library will never construct a fresh value of those types -- it will always use the values provided by the caller.

Ben Sless07:03:42

they were probably only identical because you initialized (symbol "foo") and "foo" in the same context

pez10:03:27

The line between special forms and macros just became blurry to me. I see that the defmacro of let has metadata saying :special-form true . What is the implication of this? let is listed as a special form here as well: https://clojure.org/reference/special_forms#let

yuhan13:03:35

I believe this is an implementation specific thing - let can be thought of as a special form, but in practice it's defined as a macro in terms of the actual special form let*

p-himik14:03:33

Makes sense to me, but it doesn't sound like an entry-level comment. let is a special form, that's it. The fact that it's implemented as a macro is an implementation detail that regular users should not care about, let alone beginners.

pez14:03:58

True, true. But it might be confusing to read about it in a guide where you are instructed to examine all things and find that defmacro. I actually had let among the macros in the guide first, because that’s what I thought it was.

pez14:03:45

Tricky to strike the balance. The guide promises to be basic in both the conflicting meanings of the word (short, yet foundational).

pez14:03:14

I solved it by not leading with that non-beginner material, and by not explaining it so much when I do mention it: https://github.com/BetterThanTomorrow/dram/blob/56d259ab6af50ca12aa7f3aff5ceca252b61ab52/drams/hello_clojure.clj#L468

p-himik14:03:55

I think it looks better. But I'd still mentioned that it's a special form (right now it says "is a form"). Also, a typo at https://github.com/BetterThanTomorrow/dram/blob/56d259ab6af50ca12aa7f3aff5ceca252b61ab52/drams/hello_clojure.clj#L505

pez14:03:20

Thanks. It is part of a section labeled special forms, but I changed it anyway. Staring at where I have a typo without seeing it.

pez14:03:48

Now I see it! I should rest from this a bit now…

😄 1
kwrooijen10:03:12

Do we have a channel to ask about Clojure internal implementation?

kwrooijen10:03:27

For example how namespaces work on the compiler level

kwrooijen10:03:00

I've been reading clojure core but I still have questions

schmee11:03:19

@kevin.van.rooijen yes, #clojure-dev is the place

borkdude11:03:55

I think #clojure-dev is more for development of clojure itself

borkdude11:03:25

so this #clojure channel would probably be appropriate for asking questions about clojure internals as well

👍 2
Ben Sless11:03:36

Anyone here ever extended a function via metadata with Component's lifecycle protocol?

vemv13:03:47

Probably you mean implementing Component's lifecycle protocol via metadata? Yes, it works well

Ben Sless14:03:07

I mean implementing it for a function. I want to return a function which will depend on a closed over stateful component. I ended up going the other way and implemented IFn on the record

vemv14:03:54

What should the function do?

Ben Sless14:03:48

It's the handler passed to a web server

Ben Sless14:03:40

somewhere in the handler I have an interceptor with a kafka producer

vemv14:03:34

I have a pretty hard time understanding what you're trying to do. It would help to state your original problem, plus your current attempt and its perceived drawbacks

Ben Sless14:03:49

I have a web server where I want to produce incoming messages to Kafka. I wanted to experiment with interceptors a bit so I used Reitit and Sieppari. I handled the production to kafka by instantiating an interceptor which takes the producer as an argument. The interceptor is instantiated when creating the router, so the function which instances a router needs to take the producer as an argument The implementation which emerges from this is two components, one for the web server, one for the producer, where inside the web server I call (http/start-server (routes producer) options) I wanted to create an implementation where the server doesn't directly depend on the producer, only on the handler

👍 1
vemv14:03:01

> The interceptor is instantiated when creating the router, so the function which instances a router needs to take the producer as an argument This part might be the odd one. Creating routes shouldn't have to pull much detailed knowledge about Kafka concerns. Wouldn't it be possible to create an interceptor in the producer component definition, so that the web component simply depends on producer and pulls the already-done work of creating an interceptor?

Ben Sless15:03:30

And then the function which instances the routes will take the interceptor as an argument?

Ben Sless15:03:12

Still seems a bit arbitrary. Why just one interceptor as an argument? what about when the API expands in the future?

vemv16:03:15

The interceptor wouldn't be an argument but an extra k-v in this Component systems inject component dependencies into one another via using. See ExampleComponent in https://github.com/stuartsierra/component/blob/9f9653d1d95644e3c30beadf8c8811f86758ea23/README.md , where database is used / depended-on but never passed explicitly. Translating that defrecord pattern to vanilla defns, the result would be:

(defn start-routes [{{producer-interceptor :interceptor} :producer
                     :as this}]
  (assoc this :routes (make-routes producer-interceptor ...)))

vemv16:03:18

So, adding more 'external' interceptors means simply pulling more data from the destructuring part, which avoids having to add more positional arguments.

borkdude13:03:36

This just crossed my mind: since clojure.spec.alpha is a transitive dependency, anyone using it should probably declare this as an explicit dependency.

alexmiller13:03:35

That seems wrong

alexmiller13:03:11

You should consider it part of Clojure

alexmiller13:03:32

Even though it’s broken into a library from an impl point of view

borkdude13:03:54

Can we rely on clojure.spec.alpha being there always from now on? If not, then declaring the dependency is probably a good idea for future proofing?

alexmiller14:03:27

The intent was that spec is part of clojure and you don't need to depend on it explicitly

alexmiller14:03:47

whatever migration we do for spec 2 will depend on that assumption

borkdude14:03:19

sounds good, thanks

borkdude11:03:36

Btw, are you suggesting that there will be a migration thing in clojure 1.next that will allow you to use clojure.spec.alpha as is, and it will stay this way in future versions?

Yehonathan Sharvit15:03:54

Is there a way to make byte arrays comparable by value, in the context of of a unit test. For example, I’d like to write a midje prerequesite with a byte array argument, something like this:

(fact … (provided (foo (.getBytes "abc")) => nil)
The problem is that two byte arrays generated from the same string are not equal in Java (for good reasons, because a byte array is mutable)
(.equals (.getBytes "aaa")
         (.getBytes "aaa")) ; => false

borkdude15:03:23

(java.util.Arrays/equals
 (.getBytes "aaa")
 (.getBytes "aaa"))
=> true

borkdude16:03:24

(= (vec (.getBytes "aaa"))
   (vec (.getBytes "aaa")))
=> true

Yehonathan Sharvit16:03:29

My question is: how do I tell = to use java.util.Arrays/equals instead of .equals ?

borkdude16:03:13

you cannot "tell" =, it's not a protocol (and even if it was, you should not override it for types you don't own)

Yehonathan Sharvit16:03:38

So what can I do in the context of unit tests?

noisesmith14:03:57

you can write unit tests that don't use =, I do it frequently

Yehonathan Sharvit14:03:14

I need a way to convince a midje prerequesite

borkdude16:03:44

write your own =

Yehonathan Sharvit16:03:28

The problem is that I don’t know how to progragate my definition of = to midje

borkdude16:03:39

Then don't use midje =)

2
💯 1
Yehonathan Sharvit16:03:52

I was hoping that = would call to a method from a protocol that I could override in the context of the unit tests

borkdude16:03:54

I'm half kidding, but you can't state a "fact" using some wrapper function?

Yehonathan Sharvit16:03:11

A fact, yet. But not a prerequisite

borkdude16:03:21

Ask in #midje

rgm17:03:58

Can anyone point me to a connection pool library like https://github.com/mperham/connection_pool in Ruby? Seems a very Clojure-y concept to have a lib that splits this concern out from eg. hikari. Or does everyone just roll their own?

flowthing17:03:38

Isn't HikariCP specifically a connection pool library?

flowthing17:03:32

That's what I've always used, and it's always Just Worked™.

rgm17:03:43

Maybe I’m reading wrong … isn’t it a JDBC connection pool lib?

flowthing17:03:58

Oh, yes it is.

rgm17:03:14

My use case is for keeping 10-20 TCP connections to Apple’s notification service going.

flowthing17:03:15

I didn't realize you were talking about connection pooling in a more generic sense.

rgm17:03:12

er, yeah, meaning generic connection pooling (eg. TCP connections) not just connections to a JDBC data source.

p-himik18:03:53

So many JDBC connection pools around - perhaps it would be easier to create a JDBC-compliant interface for TCP connections. :D

seancorfield18:03:06

A quick search does not turn up any TCP connection pooling libraries for Java...

rgm18:03:08

haha, maybe. I’m just a little surprised since it’s almost difficult to avoid using the word “decomplected” to describe what I’m after 🙂

schmee18:03:13

there are generic object pools like https://commons.apache.org/proper/commons-pool/, I haven’t used them so can’t say if they are any good

🙏 1
flowthing18:03:13

That Ruby library doesn't seem like a whole lot of code — maybe just port that to Clojure? 😛

rgm18:03:46

Hah it might come to that.

emccue19:03:07

example of using the generic object pool

rgm19:03:48

good example!

rgm19:03:43

can’t promise a Clojure wrapper lib but I suspect a reasonable interop example is within my abilities.

Jeff Evans19:03:04

How would pooling work for "plain" TCP? Wouldn't you have to do a bunch of programming just to get the pooling library to understand your protocol? How would it know whether a connection can safely be returned and used by another thread (i.e. it's not in the middle of framing a message or waiting on a server response)?

rgm20:03:20

Yeah, looking at the ruby one it’s relying on every “connection” being an object so presumably in Clojure terms each “connection” would probably also have to be an object conforming to some sort of lifecycle protocol, with the actual connection buried deep in there someplace.

rgm20:03:58

something’s got to be doing the bookkeeping.

p-himik22:03:45

taoensso.carmine.commands has this code:

(defonce ^:private command-spec
  (if-let [edn (enc/slurp-resource "taoensso/carmine/commands.edn")]
    (try
      (enc/read-edn edn)
      (catch Exception e
        (throw (ex-info "Failed to read Carmine commands edn" {} e))))
    (throw (ex-info "Failed to find Carmine commands edn" {}))))
If I start a REPL and run
(require '[taoensso.encore :as enc])
(enc/slurp-resource "taoensso/carmine/commands.edn")
I get a proper EDN. Not a nil. But when I run (require 'taoensso.carmine.commands), I get Failed to find Carmine commands edn. And it only happens on Heroku. I cannot reproduce it at all locally. How is this possible?

p-himik22:03:29

Of course. Despite all the thinking and staring at the above text for a good minute before posting the message, only after hitting Enter did I realize that I'm compiling Clojure code on Heroku, along with some of the libraries.

p-himik22:03:33

Seems to work without compilation. But frankly, I still have no idea why io/resource wouldn't work after compilation. Strange.

p-himik22:03:42

Aaaand now I'm getting "App boot timeout" errors without the compilation. Sigh.

andy.fingerhut22:03:40

Are you perhaps neglecting to deploy to Heroku the directory taoensso/carmine containing a file commands.edn?

andy.fingerhut22:03:20

I guess you probably are deploying such a file, if a remote REPL returns a good result for (enc/slurp-resource "taoensso/carmine/commands.edn")

p-himik22:03:42

Yep, it's all there.