Fork me on GitHub
#clojure
<
2023-10-01
>
kingcode14:10:37

How dow one slurp a URL content using slurp? I tried => (slurp "") => ""

p-himik14:10:01

What do you use for the REPL? With plain clj it worked on my end just fine.

kingcode14:10:30

I tried both from within Emacs/Cider REPL, and from a Linux prompt ’clojure command REPL, with the same result

jjttjj14:10:41

I get "" as well from clj on windows with java19 fwiw

Nundrum14:10:54

Hm I get the empty result from babashka.

kingcode14:10:44

Aw OK, so I am not crazy 🙂 There may be a more Java’ish way to construct an Http connection, but slurp is so much more convenient

kingcode14:10:40

;; CIDER 1.5.0 (Strasbourg), nREPL 1.0.0 ;; Clojure 1.12.0-alpha3, Java 1.8.0_112

kingcode14:10:13

Same result using clj

p-himik14:10:42

Heh, I don't even have JDK 8 anymore, so can't test on that. But with both CLJ 1.11 and 1.12.0-alpha3 and with both JDK 11 and 18 I got the same result - the response was full of text. So perhaps it not on the side of CLJ or JDK.

kingcode14:10:53

Hmmm strange! Thanks @U2FRKM4TW…will have to investigate.

p-himik14:10:38

Oh, I get an empty string on my laptop, perhaps will find something.

kingcode14:10:05

Oh? I have a laptop as well 🙂

p-himik14:10:11

Oh, hold on. I used HTTPS instead of HTTP when I got the text. I also get an empty string on my main PC when using HTTP.

p-himik14:10:51

So perhaps slurp doesn't follow redirects?..

kingcode14:10:20

That was it! 🙂 Thanks. I don’t know why it’s quirky, but this makes sense.

Ingy döt Net16:10:52

Is it common for a library foo to have a foo.core primary namespace?

p-himik16:10:50

Yes, very.

Ingy döt Net16:10:17

I thought so but before I asked just now I checked a couple that came to mind off the top of my head and they didn't. (pomegranate and eastwood).

Ingy döt Net16:10:33

Is there any docs that promote this?

p-himik16:10:09

Not that I know of, and I doubt that. Might be stemming from clojure.core.

Ingy döt Net16:10:44

It seems like a good idea to use a core ns. Helps people find the API quickly I would think.

ghadi16:10:43

It’s not a great ns name, doesn’t convey much

ghadi16:10:04

I wouldn’t assume it’s there

ghadi16:10:36

It’s certainly common but it’s a weak default

☝️ 2
p-himik16:10:44

Hmm. Upon seeing a random source code, I would definitely look at core.{clj,cljs,cljc} first. Assuming it's a public library and not an app or a tool. I wouldn't expect for it to be there, but if it is, I'd look there first.

1
Ben Sless16:10:00

You'll also find today things like foo.api or ingy.foo

Ingy döt Net16:10:20

@UK0810AQ2 not sure what you mean. Example?

p-himik16:10:39

foo/api.clj or ingy/foo.clj instead of foo/core.clj.

1
Ingy döt Net16:10:27

Yeah that aws thing looks like an actual API. I more meant the main public functions (when I said API).

Ben Sless16:10:41

The main user facing api can absolutely live in a namespace which ends with .api or .your-lib

Ingy döt Net16:10:47

Sure. Seems obvious that it may. I was just wondering if there were strong conventions here.

practicalli-johnny06:10:43

core is a name from lisp Many names from lisp have been kept, some were not included and a few are used less. lein new creates a namespace core, arguably a common reason why core appears as a namespace in many projects (especially older than a few years) As people stepped away from lein new and Leiningen itself, core as a name was used far less Whist core is a long established lisp convention for the Clojure language itself, clojure.core, I find it doesn't convey much meaning in application development E.g. when writing a Clojure service, I use practicalli.project-name.service as the main namespace I am not involved in designing languages, so I don't use core anywhere in projects. I would argue that there is a strong convention to not use core outside of the language design and Leiningen After 10 years of Clojure, I still think of core dump when every I hear the name 😭. So happy not to use the name

p-himik12:10:50

Thanks, didn't know the about older Lisp and lein new. Happy not to have car and cdr in Clojure. :D

Ingy döt Net17:10:04

Ironically my project is a language. And it will have a core standard library. So I maybe I'll use .core for that.

Joshua Suskalo17:10:52

From what I've seen and basic conventions, it's very common for single-ns projects to use project.core. If a project grows into multiple namespaces from that it will likely keep the core ns to keep backwards compatibility. Many projects which are larger than a single ns by the time they hit the first release though will not have a core at all. They might have project.module1 and project.module2. Also as mentioned, you'll sometimes see rdns names like tld.company-or-person.project for example com.rpl.specter. This is more common in very large projects, or people coming from a very strong java background. The main issue I personally have with name.project as a library though is that it doesn't play nice with forks-as-drop-in-replacements. If someone wanted to fork a library and make it as a drop-in replacement for the original, that name is now incorrect.

Joshua Suskalo17:10:25

In particular for these single-ns projects, I think the argument that "core is a weak name" is a bit of a distraction. In these cases a lot of times you just want to have a ns tied to your project and you only need one, but the convention to use multi-segment namespaces only trumps the desire to just use the project name as the ns.

Ingy döt Net16:10:45

If I have a library with one main public function and dozens of private functions supporting it, is there a way to make all defn s private by default, and just mark the public one. iow, I could defn- all the private ones, but I'd rather just defn them and make them private by default somehow.

p-himik16:10:47

I'd just put them in some impl namespace if you really care about such separation.

3
vemv16:10:34

You can run ! on each var

p-himik16:10:47

Personally I wouldn't bother with {:private true} (what defn- does) at all.

p-himik16:10:32

No, of course not. I always maintain a 1-to-1 relationship between files and namespaces.

vemv16:10:47

The impl pattern is nice, that way when you visit the 'main' namespace, all what you see is the intended API It's often nicer to read for anyone learning about your library

Ingy döt Net16:10:28

@U45T93RA6 the alter-meta! seems like a good trick. You could wrap that in a public-defns 🙂 But I guess if I have main/core ns I can just put the public stuff in that.

Ingy döt Net16:10:47

I'll get rid of my defn-s and know that I can easily do the alter-meta! thing later on if I feel it is needed.

p-himik17:10:00

Why would it be needed though? Making something fool-proof is impossible. As well as making vars inaccessible (of course vars can always be replaced with local bindings).

Ingy döt Net17:10:07

So the clojure concept of "private" is pretty much a waste of time?

Ingy döt Net17:10:33

I'm fine with that btw

p-himik17:10:07

Both the concept and the overall attitude towards it, I feel. But at the very least, the public stuff should be documented and be obviously public. I think in some of the talks Rich even mentioned that defn- was a mistake. But I could be wrong.

Ingy döt Net17:10:43

Cool. Thanks. I'll run on that for now.

lread18:10:19

My take: Whatever strategy you choose, it's nice for you and the users of your lib to clearly differentiate your public (and supported) API from its implementation details. If I haven't moved an implementation detail fn to an .impl namespace I use defn- to make it very clear to users that the fn is subject to change. I also mark namespaces (and individual public fns, if needed) that are not part of my public and supported API with :no-doc metadata so API documentation tools will not include them.

Bobbi Towers18:10:59

> Guido Van Rossum, best known as the author of Python, once said: "We're all consenting adults here" justifying the absence of such access restrictions.

lread18:10:42

Interesting! I see it as a kindness to myself and others to clearly delineate my public APIs.

p-himik18:10:28

Heh, Python in particular has long been supporting user-created dunder methods that can't be accessed from outside of the class where they're defined.

Ingy döt Net18:10:55

My project has a dozen namespace files and one public function. So eveything else is essentially private. I don't want to add impl to every ns so I'll just put my public api function in the .core ns and call it a day,

Ingy döt Net18:10:59

I've met Guido a couple times...

Ingy döt Net18:10:30

Felt nice to get rid of all the defn- s!

lread19:10:46

To consider: Clearly defining your public API reduces the surface area of what you promise not to break.

Ingy döt Net19:10:01

For sure. That's what I was going for. Just not sure how to best do that. Seems like all the ceremony of impl and defn- is likely not worth it, when I can just clearly document 1 or 2 core functions as the API.

lread19:10:32

I like to use defn- :no-doc meta and .impl namespaces as conventions that are pretty well understood. For me, the ceremony is worth it. But I'm not everyone!

Ingy döt Net19:10:46

@UE21H2HHD can you point me at a few of your libs that use these conventions?

Ingy döt Net19:10:53

The :no-doc thing seems good because it's just one small change per ns

Ingy döt Net19:10:14

If I understand it correctly

lread19:10:47

I'm an adoptive parent (and often co-parent) of many libs now under the loving care of clj-commons. These all now clearly define their public API: https://github.com/clj-commons/clj-yaml, https://github.com/clj-commons/etaoin, https://github.com/clj-commons/clj-http-lite, https://github.com/clj-commons/pomegranate, https://github.com/clj-commons/rewrite-clj. Oh, and my own https://github.com/lread/test-doc-blocks.

lread19:10:12

I used to also like that defn- reminded me that a fn was only used from its own namespace. I found this helpful to understand what namespaces would be impacted by a change. But these days, tools like clojure-lsp awesomely give fn usage info, so this benefit is diminished a bit, at least when working from an IDE.

Ingy döt Net19:10:40

Nod. Each of my namespaces in this project has one function that gets used outside the namespace (but internally, not intended for public usage) and only one ns has functions that are user facing. So the defn- s just felt a bit overkill.

Ingy döt Net19:10:04

But I can always revisit this after getting more experience

lread19:10:27

Yeah, there is no right or wrong. Just trade-offs. And sometimes personal preferences. Hopefully, this thread has helped!

Ingy döt Net19:10:05

It has. Thanks.

hifumi12304:10:46

I personally dislike private functions because they make testing tedious and I reject the idea that private functions shouldn’t be tested. With that said, private functions have one minor benefit: • if someone uses your public function and changes break their code, the person will likely file a bug report • if someone uses your private function and changes break their code, the person will not file a bug report, they will know its their fault for relying on implementation details that can change without warning My personal approach for libraries is to manage everything under a versioned namespace like foo.api.v1.thing and implementation details either go under something like foo.api.v1.thing.impl or exist outside of foo.api entirely. This allows me to create granular tests of implementation details without needing to constantly deref vars everywhere and work my around things being private.

hifumi12304:10:43

One more thought that just occurred to me: public/private as a means of encapsulation sort of goes against Clojure’s overall stance on encapsulation — the huge focus on immutable data defeats a lot of the purpose of encapsulation. So I think rather than encapsulating things with public/private, we should create namespaces that are as specific as possible while also being well-organized

Ingy döt Net04:10:11

I'm just going to document how my library works and not worry about it for now

👍 3
hifumi12304:10:28

FWIW Clojure’s Java API takes a very similar stance as your current one

hifumi12304:10:31

From https://clojure.org/reference/java_interop#_calling_clojure_from_java, emphasis mine. > The https://clojure.github.io/clojure/javadoc package provides a minimal interface to bootstrap Clojure access from other JVM languages. It does this by providing: > 1. The ability to use Clojure’s namespaces to locate an arbitrary var, returning the var’s clojure.lang.IFn interface. > 2. A convenience method read for reading data using Clojure’s edn reader > IFns provide complete access to Clojure’s APIs. You can also access any other library written in Clojure, after adding either its source or compiled form to the classpath. > The public Java API for Clojure consists of the following classes and interfaces: > • https://clojure.github.io/clojure/javadoc/clojure/java/api/Clojure.html > • https://clojure.github.io/clojure/javadoc/clojure/lang/IFn.html > All other Java classes should be treated as implementation details, and applications should avoid relying on them.

1
practicalli-johnny06:10:12

Public and private is essentially a documentation artefact in Clojure, which is also honoured by some tools. Private is one way to narrow down how a particular namespace was intended to be used by a developer, but given how easy it is to over-ride private, it's not that strong as a scope constraint (as with other languages) I use line comments to group functions only used by other functions in the current namespace I'll only use ^private if required for tooling used to generate externally facing APIs and docs or if the team I work with has agreed that is an approach they find valuable

borkdude10:10:19

@U05H8N9V0HZ Don't know if this has been said before but I usually split the public API and private API into different namespaces. The private API is then names "library-name.internal" and it has {:no-doc true} metadata, so everybody knows that this namespace should not be used by anyone else

borkdude10:10:50

I read now that it has been said before. I found "internal" an even better name than "impl" since somehow "impl" was still not clear enough to other users that they should not rely upon that code :P

borkdude10:10:10

Also, like @UE21H2HHD has done, I cannot emphasize enough how important it it to separate your public API from your internal, both for yourself and your users. For yourself, you have more wiggle room to make breaking changes in the private area of your lib and for others: you won't break them if you narrow down the API surface area.

lread12:10:08

Hmmm, yes @U04V15CAJ, you have mentioned using .internal instead of .impl namespace naming in the past, I forgot this - and I also like this extra clarity.

flowthing12:10:50

I'll add one more vote for the foo.api and foo.impl.*/`foo.internal.*` separation and just note that I think ^:private does have its place: I think it's useful to mark vars as private even in impl namespaces that are not meant to be used outside that namespace, if only to prevent editor autocompletion (and ns-publics) from suggesting those vars.

💡 1
lread12:10:42

From practical experience: When adopting libraries to clj-commons, figuring out their public APIs is challenging. None of the libraries I've been involved with had clearly defined a public API. This made it difficult to understand where we could safely make changes without impacting users. Unless it is made very clear, people will assume everything is part of a library's public API, use it, and expect it not to break. This is not good for anyone. If you clearly delineate your internal implementation details, people will still sometimes use internals directly, but in this case, they know they are doing so at their own risk.

👍 1