Fork me on GitHub
#clojure
<
2019-01-31
>
ahungry03:01:43

is there an existing library for converting a swagger.json file or json schema file into spec definitions? I know spec-tools supports spec to both formats, but I'm more interested in the opposite direction

LukeDonnet09:01:05

Hello, does anyone know of a way to decode php laravel sessions in clojure?

Jakub Holý (HolyJak)09:01:22

FYI I couldn't find any documentation (outside of the clojuresque api docs :)) on how to use prepl so I wrote some https://blog.jakubholy.net/how-to-use-clojure-1.10-prepl/

👏 5
cupello14:01:33

Hi folks! My current task is to profile a Clojure app (with Aleph and core.async) and after googling for some time it seems that is not an easy task like in Golang... I need some metrics like "how many goroutines are running", memory, and cpu. Does anyone know a way to get these metrics?

Jan K15:01:25

@lucelios I don't know about any tools to get stats for core.async specifically, but for memory and CPU profiling there's the usual JVM tools like https://visualvm.github.io/

👍 5
chrisblom15:01:41

@lucelios also aleph exposes metrics of its thread-pool

cupello16:01:06

Thanks! I will search for that now

borkdude16:01:39

ehm, I used metadata on a fn like this:

(defn ^String foo ([] "") ([a] (str a)))
(set! *warn-on-reflection* true)
user=> (.length (foo))
0
Apparently that works too, although I can’t find it in the docs like this.

Alex Miller (Clojure team)16:01:22

you’re putting it on the var there, not the fn

borkdude16:01:42

why don’t I get a reflection warning then?

Alex Miller (Clojure team)16:01:52

because it’s using the var meta

bronsa16:01:11

non primitive return type hints work on the var too

borkdude16:01:17

so this is cool, unless you use direct linking?

Alex Miller (Clojure team)16:01:55

your use of “too” above seems like there is some prior conversation this is in reference to, but I haven’t read that

Alex Miller (Clojure team)16:01:10

just mentioning that in case there is context I should be aware of

borkdude16:01:29

AFAIK there isn’t a prior conversation about this

Alex Miller (Clojure team)16:01:57

so “too” above implies there is something else that works, but not sure what that is

bronsa16:01:14

I was referring to type hints on the argvec

borkdude16:01:27

to summarize, should I change my type hint, or does this work correctly?

Alex Miller (Clojure team)16:01:29

not your “too”, @borkdude’s “too”

bronsa16:01:42

we need name qualified words in english

bronsa16:01:49

borkdude/too

👍 20
borkdude16:01:03

oooh, sorry 🙂 I mean instead of putting the return type hint on each arity

bronsa16:01:14

it works, I hate it

bronsa16:01:27

it's fine as is, but it makes me sad

Alex Miller (Clojure team)16:01:32

yeah, it works, but I agree with bronsa that putting it on the ret type is better

borkdude16:01:43

so I need to repeat the type hint for each arity?

borkdude16:01:51

and why you sad?

Alex Miller (Clojure team)16:01:59

not every arity has to have the same return type

borkdude16:01:13

no problem, I’ll do it, just want to know why

Alex Miller (Clojure team)16:01:19

(not that I would recommend having different return types)

bronsa16:01:39

it's just a matter of consistency, tagging return types on the argvec is guaranteed to work with any legal type hint and Do The Right Thing

bronsa16:01:51

there is a string of cases where tagging return types on the var doesn't do what you want, silently

Alex Miller (Clojure team)16:01:53

I think it’s also the tightest scoping to your intent

borkdude16:01:37

ok - thanks {::keys [:bronsa/too :alexmiller/too]}

Alex Miller (Clojure team)16:01:09

Syntax error macroexpanding clojure.core/let at (REPL:1:1).

bronsa16:01:53

who said that's code, it's just data bruh

Santiago18:01:42

how can I spit :append below whatever text already exists in a file?

Santiago18:01:24

@misha thanks, interesting

markw18:01:28

Anybody aware of a walk-through describing how reagent components that deref an atom register themselves as watching that atom?

noisesmith18:01:02

@markw my memory is that r/atom deref itself does the registration, since the deref happens inside the component data literal

noisesmith18:01:24

but I haven't touched reagent in ~year

noisesmith18:01:25

I think this is the line that literally answers your question: when you deref an ratom, your context is captured as a watcher https://github.com/reagent-project/reagent/blob/master/src/reagent/ratom.cljs#L145

noisesmith18:01:43

or, more precisely: no walkthrough that I know of, but the code isn't that hard to read

markw18:01:18

@noisesmith thanks... yeah you're right I shouldn't be lazy and just read the source

noisesmith18:01:25

well, hopefully at least knowing the precise place the behavior is defined is a good start

noisesmith18:01:48

I could imagine it eg. being part of the mounting code or the component loading code

markw18:01:49

definitely saved me some trouble, appreciate it

kwladyka19:01:06

https://github.com/seancorfield/depstar https://github.com/juxt/pack.alpha https://github.com/luchiniatwork/cambada I found 3 uberjar maker for deps.edn with clj. Do we need 3 as community? I always like simplicity about Clojure tools and making choices. Which one is recommended?

borkdude19:01:34

if you want simplicity, make no uberjar 😜

kwladyka19:01:58

What then? I want it for Docker image, so I don’t want to share source code

borkdude19:01:22

just mount the source code?

kwladyka19:01:57

> I want it for Docker image, so I don’t want to share source code 😉

borkdude19:01:00

oh you DON’T want to share the source code

borkdude19:01:22

so you want to AOT it and publish without the .clj files?

kwladyka19:01:24

no problem 🙂

hiredman19:01:02

I believe most of that tooling just zips up the source code and calls it a jar

kwladyka19:01:11

It was working perfect with lein, but I am moving with all new stuff to deps.edn

borkdude19:01:35

I’m still using lein to deploy for some projects, next to clj, I guess you could use both

hiredman19:01:37

the clj tooling has been explicitly called "not a build tool" over and over again, and aot compiling seems like a clear cut build step (of course so does building jars)

hiredman19:01:35

the pack.alpha has some suggestions for adding aot compilation on top of it, but it seems pretty adhoc

kwladyka19:01:23

Hmm I can use lein only to build jar from deps.edn, but I feel strange about that.

hiredman19:01:15

cambada actually defaults to aot compiling everything because it larger looks it is for building graal-native stuff, which requires aot

dpsutton19:01:44

no worries about that. there's a lein plugin to gather deps from deps.edn so you can just use lein for build tooling and deps.edn for what you use it for now.

kwladyka19:01:08

@dpsutton sure I know that plugin

hiredman19:01:47

it just seems like, if you have some weird corner case (building aot jars) and it is well served with lein, why make the change to clj?

kwladyka19:01:35

I like clj more than lein 🙂

dpsutton19:01:37

i understand lein and deps.edn use subtly different dependency resolution so i'm not sure that your built artifact is guaranteed to reflect what you developed. Need to find out what that plugin does, if it resolves everything in the way tools.deps.alpha would do (and i suspect this is what it does)

kwladyka19:01:15

And clj solve other corner cases easy like git dependency or local folder dependency

seancorfield19:01:35

Why, specifically, do you not want a JAR containing .clj source files? That's how we package everything for production deployment at work.

victorb19:01:47

distributing paid binaries?

victorb19:01:06

(I'm not doing that so no idea really, just guessing)

borkdude19:01:15

like Datomic

seancorfield19:01:17

(we use my fork of depstar to build uberjars from source, and run them with java -cp path/to/the.jar clojure.main -m entry.point ... in production)

seancorfield20:01:36

Because depstar does what I want and is nice and minimal. And @U050ECB92 made depstar. I just forked it, fixed some bugs, and added the "thin JAR" creation feature.

seancorfield20:01:54

Also, I'm pretty sure Cambada didn't exist when depstar was created?

seancorfield20:01:30

Juxt's pack.alpha came first (back in December 2017, it seems). Then depstar in early 2018. Then Cambada. Based on initial commits, not public releases.

seancorfield20:01:06

Not sure what you mean by that question... that's exactly what depstar does.

kwladyka20:01:45

Sorry if I am asking about something stupid. But does it only pack source files or compile them into different format? As I understand compile to different format. But hiredman wrote “just zips up the source code and calls it a jar” and it make me confuse.

seancorfield20:01:18

A JAR file is just a ZIP file with the extension .jar

seancorfield20:01:47

Not sure what you mean by "compile them into different format".

kwladyka20:01:40

hmm so how AOT is added?

kwladyka20:01:37

Does it still need .clj files after that?

kwladyka20:01:56

So then what source code doing there?

seancorfield20:01:26

When you compile Clojure source .clj files, you get .class files. You don't need the .clj file present if you have all the .class files that it compiles into -- unless it does some action at runtime that would require the source (such as loading a specific source file). AOT is transitive (which is considered a bug by most people -- there is at least one open JIRA issue about it) which means it compiles the namespaces you specify and any namespaces that are reached by loading those namespaces (recursively) -- which means that if your AOT'd code uses require/`resolve` at runtime on namespaces that were not part of the AOT pass, you still need their source files.

kwladyka20:01:14

So all solutions like lein uberjar etc. include source files?

seancorfield20:01:28

It depends on whether you configure them to AOT everything or just specific namespaces. And even at runtime, Clojure code can still pull in new dependencies (as source code) and load them (which compiles them into class images in memory).

seancorfield20:01:16

I think it's a long-standing mistake that Leiningen chose to have as the default in its basic app template, a compiled main namespace and a :gen-class directive.

seancorfield20:01:43

Leiningen does a lot of stuff by default that is not necessary.

kwladyka20:01:19

So… do you suggest to always include source code? What about business?

kwladyka20:01:41

Is there any right solution? How do you protect source code?

seancorfield20:01:37

Licensing. As several people said.

kwladyka20:01:03

It doesn’t protect source code in any way. It makes potential penalty if you can prove it.

kwladyka20:01:46

ok so I assume you make source code public for all your apps (unless it is on cluster only, so they can’t read anyway)

kwladyka20:01:55

Thanks for explanation

seancorfield21:01:17

All the code I have written for myself, that I own the rights to, has been open source -- for about thirty years. The code I write for companies, in general, is owned by those companies and it's up to them whether they release it publicly. Some code I wrote for Macromedia was open sourced (in the early 2000's). Some code I wrote for a client after I quit Adobe in 2007 was open sourced. Various companies I've worked for have open sourced some of their code. My current employer supports all of its developers contributing to existing open source projects -- and has allowed me to release a number of libraries as open source that were written internally, as well as having its own GitHub organization with some open source libraries.

seancorfield21:01:31

We have some code at work that is proprietary in terms of containing internal business logic and/or algorithms -- but we wouldn't release that in compiled format. Nothing to do with source code.

kwladyka21:01:55

> All the code I have written for myself, that I own the rights to, has been open source -- for about thirty years. I envy you have such opportunity to do it 🙂

seancorfield21:01:18

A couple of our developers have given conference talks, going into some depth about how we use certain software and sharing code examples publicly.

seancorfield21:01:10

The real reason most companies won't release their code as open source is that they don't want to have to maintain it as an open source project -- because that's a big commitment. We've had a lot of discussions about that internally.

kwladyka21:01:07

oh yes, it is

seancorfield21:01:18

For a long time, we only had one official company library available as open source, and it was on BitBucket (an internal repo made public), and that was all about the potential drain on our time for maintaining it for the community if we put it up on GitHub for "everyone".

seancorfield21:01:49

With the primary delivery platform now being the web, there's no deliverable for backend code to clients, so compiling (to whatever format) is far less important. And of course all your client-side code is public as JavaScript and the entire world can read it.

seancorfield21:01:50

Anything delivered as a desktop app built on Electron etc -- all your source code is public there too. Because JavaScript.

kwladyka21:01:07

hmm unless we use solutions like electron to make desktop app it is mostly true

seancorfield21:01:55

Companies like Microsoft are now releasing huge swathes of code as open source. Closed source is pretty much dead as a concept as far as modern companies are concerned.

kwladyka21:01:06

so I would say it is issue / feature of Java / JavaScript world

kwladyka21:01:53

> Closed source is pretty much dead as a concept as far as modern companies are concerned. That is interesting topic. I see this trend more and more, but I have never seen rationale for that.

kwladyka21:01:04

Do you know good article about that?

seancorfield21:01:21

PHP, Python, Ruby... all delivered as source code, regardless of whether the code is open source or restricted by licensing.

kwladyka21:01:58

licensing is ok, but is is hard to deal with people who don’t respect it

kwladyka21:01:12

Maybe in US it is easier, I don’t know

seancorfield21:01:39

That's always been true tho' even with proprietary closed source systems -- look at how widely pirated desktop software from Microsoft and Adobe is...

kwladyka21:01:45

Yeah, it is interesting move from MS to let run they system without cracks.

kwladyka21:01:49

I think they did it

seancorfield21:01:00

Adobe has a specific software license enforcement team that work with law enforcement worldwide and they conduct raids on pirating organizations, as well as sue companies that use pirated versions of their products. I'm sure Microsoft, Oracle, et al all do as well.

kwladyka21:01:17

But on the other hand they power is in number of users, also this who crack they system

seancorfield21:01:01

Unless you're a giant mega-corp, with billions in revenue from closed source desktop systems, piracy is really not worth worrying about tho'...

kwladyka21:01:06

I mean more people use they product, better for them, because more people will buy a licence.

seancorfield21:01:36

Your ideas in software are not likely to be novel 🙂

seancorfield21:01:16

There's much less true innovation in software than a lot of people seem to think.

seancorfield21:01:47

So the software you produce -- there's almost no benefit to keeping it "secret" in the vast majority of cases.

kwladyka21:01:08

Still if somebody will do copy & paste your system for which you spent a lot of money you will feel bad about that

kwladyka21:01:29

This things can easy happen in Russia or China

seancorfield21:01:50

If you really "spent a lot of money" and have a large amount of business riding on it, sue for license infringement. That's what licenses (and courts) are for.

seancorfield21:01:18

And if it's closed source, they'll just crack the product anyway. It's no defense.

kwladyka21:01:26

I think Russia and China don’t care about that too much

seancorfield21:01:43

Really, if you think closed source vs open source is some magic defense against piracy/cracking you're delusional 🙂

kwladyka21:01:45

but it is fair point

kwladyka21:01:56

no, I don’t

seancorfield21:01:19

Exactly, Russia and China et al will do whatever they want with your software regardless of whether you release the source or not!

kwladyka21:01:40

Reversing situations: so why companies close they source code? Everything should be public.

seancorfield21:01:13

Example: a Clojure open source library out there was made by a Russian as a copy of one of my libraries and they removed the license and copyright notices and passed it off as their own code. They announced the library on the Clojure mailing list and I called them out for license infringement -- and they recanted because they were caught in public breaking the rules.

😯 5
seancorfield21:01:40

Closed source is the old-fashioned model. Companies with old-fashioned thinking still follow that model. Companies still use COBOL, for example 🙂

seancorfield21:01:20

This is an interesting article on the thinking behind keeping things closed: https://opensource.com/business/13/10/seven-reasons-closed-better-than-open-source

kwladyka21:01:29

To be honest I think I don’t have strong opinion in that topic

kwladyka21:01:38

I think there is no answer for that

kwladyka21:01:07

But for example turquoise model of managing business is new “fashion”, but I don’t believe it fits everywhere

kwladyka21:01:03

> You never have to fix components when something goes wrong. haha I like this one

kwladyka21:01:19

Thanks for conversation. I think it changed my point of view a little.

seancorfield21:01:00

You're welcome. As you can imagine, I'm very much an open source advocate, having been doing this for about thirty years 🙂

👍 5
kwladyka21:01:34

I can imagine. I would like to do more open source, but not so easy. Trends from US go to Poland after a few years 😉

hiredman19:01:24

but you have to ask yourself, what are you trying to protect against

kwladyka19:01:35

for example security reasons and not free software

hiredman19:01:42

like, jvm bytecode is super high level and easy to decompile anyway

seancorfield19:01:54

Folks can always ... yeah, what @hiredman said...

dpsutton19:01:12

bronsa built a pretty good decompiler from class files to clj in addition to the classfile -> java decompilers

kwladyka19:01:40

sure, but decompile Clojure is probably much more difficult to debug, than Clojure code

seancorfield19:01:46

There's no security provided by AOT compiling Clojure code.

seancorfield19:01:52

(or compiling Java code)

kwladyka19:01:21

I know, just sharing source of the code sounds risky for me and second reason: not all code is for free 😉

borkdude19:01:43

@kwladyka with who are you doing to share it?

hiredman19:01:26

but, I mean, plenty of not free code is distributed everyday, right? that is what a license agreement is for

kwladyka19:01:30

I don’t want to go deep into security topic, that was not the point of my question 😛

seancorfield19:01:42

Software licensing is ... yeah, dammit, what @hiredman said! (again)

hiredman19:01:48

I dunno, I mean it is standard industry practice and whatever, but it is so silly

kwladyka19:01:30

then you could make everything public 😉 For example source code of bank systems.

kwladyka19:01:50

So I can agree only partially with you

hiredman19:01:50

the idea that not giving out the source code is some kind of barrier is silly

hiredman19:01:15

banks absolutely should be able to publish their source code

Alex Miller (Clojure team)19:01:32

next thing, you’ll want the source code for voting machines

😆 10
😁 5
noisesmith19:01:36

I'm sure somebody out there has a Classloader implementation that loads encrypted resources based on a key that needs to be present at runtime, I bet it's a pain to actually use though

ghadi19:01:57

this is a neat idea

hiredman19:01:49

linux is insecure because its source code is publicly available

kwladyka19:01:26

that is why we have distributions like alpine

hiredman19:01:53

what does that even mean?

hiredman19:01:06

I know what alpine linux is, I just don't see how it anyway addresses my point

hiredman19:01:32

alpine linux is insecure because the source code is publicly available

kwladyka19:01:37

I mean because code is public and everybody know what system cointain, we needs solutions like alpine

kwladyka19:01:11

I am afraid if bank will publish they source code many people will lose they money 😉

Alex Miller (Clojure team)19:01:26

don’t tell anyone, but banks use a lot of open source software

hiredman19:01:29

@noisesmith which is a key distribution problem not a source code problem

hiredman19:01:00

@kwladyka that is based on a deeply flawed understanding of the basis of computer security

noisesmith19:01:02

worse than that, they use fundamentally insecure things like FTP

☝️ 5
noisesmith19:01:11

they use legal means to fix the ensuing problems

hiredman19:01:28

if you look at crypto these days, no one wants to use a crypto system that relies on the implementation being secret, only on the keys being secret, they want everyone to know the implementation

kwladyka19:01:37

So do you want to say me you publish source code of your all projects? Free and pay? I don’t believe.

borkdude19:01:05

@kwladyka there’s a difference between a business model and security

kwladyka19:01:21

@hiredman blockchain idea is to be public. It is different topic.

hiredman19:01:30

legally I am unable to publish the source code to the stuff I do for pay because the rights are not mine

hiredman19:01:54

@kwladyka by crypto I mean "cryptography" not digital ponzi schemes

kwladyka19:01:30

I am sure there is dependency between code publicity and number of trying hack. But it is besides of our topic.

hiredman19:01:48

there are reasons not to provide source code, but if you put security on that list you have failed to secure anything

kwladyka20:01:33

I believe it reduce probability. Not making system secure. Just reduce attention and make things harder to hack.

kwladyka20:01:29

But things are different for very popular public projects where security start to go on higher level.

hiredman20:01:29

"Security experts have rejected this view as far back as 1851"

lilactown20:01:52

I think it's pretty proven-out that open vs closed source doesn't have much bearing on security

lilactown20:01:04

usually people want to keep their source closed for business reasons

lilactown20:01:25

i.e. I don't want a competitor to steal my codez

kwladyka20:01:55

Anyway business reasons are enough to do it

lilactown20:01:16

but it's kinda silly, because that's what licensing and legal teams are for 🙂

lilactown20:01:33

it's just putting a barrier in place. it's fairly easy to knock down

kwladyka20:01:07

So whole world is silly doing that 😉

lilactown20:01:15

plenty of businesses are based around JS applications that everyone downloads the source for ¯\(ツ)

lilactown20:01:40

anyway, not saying that everyone should make their code publicly available. but that going to great lengths to obfuscate it is probably not worth the time/money

lilactown20:01:12

AOT is great for perf if you need it. as obfuscation from a business/security perspective, it's not that great and in the case of Clojure can be work to make sure all code deals with it well

seancorfield20:01:44

AOT is great for startup time -- not performance.

☝️ 15
seancorfield20:01:52

There is no difference in performance since Clojure source code is compiled when it is loaded anyway and the JVM will do its JIT thing over time as the application runs -- AOT just saves that initial compile time on load.

borkdude20:01:21

yeah, considering that we’re doing AOT for each deploy, it kind of defeats the purpose (you only benefit from it once) - except that we maybe have a little less downtime between deploys

kwladyka20:01:23

not always true, if you run app in kubernetes to do a job or create new containers when needed, then it is not once

borkdude20:01:59

I was referring to an app we run in production. In general (for libs, etc) you can benefit from it more.

seancorfield20:01:21

For us, the biggest chunk of startup time is New Relic's instrumentation agent processing all of the classes as they load. The difference between AOT and non-AOT for us is noticeable but relatively small in comparison.

kwladyka20:01:52

Just to summary up what I wanted to say and avoid misunderstanding: Not publish code doesn’t make code more secure. But publish code increase a risk to be hacked. For example bot looking known bugs in public code, which will notice hacker you use dependency with security issue, have overflow bug etc.

todo21:01:22

Anyone have a recommendation of a smalltalk written in clojure? I'm mainly interested in the "liveness" and "object protocol", not so much the parser. EDIT: removed incorrect statement regarding Clojure / Protocols

hiredman21:01:11

clojure doesn't have an mop, and protocols are not based on smalltalk

todo21:01:20

I removed in correct statement regarding Clojure / Protocols. Regarding original question -- is there any readable impl of Smalltalk in Clojure ?

Alex Miller (Clojure team)21:01:34

re smalltalk, I think http://cloxp.github.io/ is smalltalk inspired

seancorfield21:01:16

> But publish code increase a risk to be hacked. For example bot looking known bugs in public code, which will notice hacker you use dependency with security issue, have overflow bug etc. A good, relevant example here is Slack. The JavaScript code is not open source but it is public because of the way web apps work and it has a stupid exploit in it that allows hackers to bypass security checks in the client. Slack have known about it for years but refuse to fix it. If their source code was actually open source, you can bet they'd be deluged with pull requests to fix security issues like that.

seancorfield21:01:56

That's why open source code is often more secure and less hackable than closed source code -- more eyeballs, more developers wanting to help fix vulnerabilities.

kwladyka21:01:32

I agree about all popular projects. Just not sure if it will work the same if you will publish the system, which only you maintenance. For example small SaaS or system which you use only in company X. I think true is not black and white. But I agree. It is fair point.

jrychter21:01:03

I've spent the last hours hunting down an obscure compilation problem with a cryptic error message. I still don't understand it, but it seems that if I mark a (def q (into (sorted-set) [1 2 3])) with a ^:const, then other files that need q for their (def)s will not compile. And I now know that sorted-set and ^:const are needed. A vector won't cause it.

jrychter21:01:24

Just wanted to check if this rings a bell with anyone.

Alex Miller (Clojure team)21:01:31

const means inline at point of use (that explains the scope of where you see the issue)

Alex Miller (Clojure team)21:01:22

I don’t think sorted-sets can be const - they have an embedded comparator function that’s not representable as data

jrychter21:01:21

Hmm. I didn't know there are things that can't be const.

jrychter21:01:48

The error messages are… suboptimal.

Exception in thread "main" Syntax error compiling at (/tmp/form-init5251724450573718419.clj:1:73).
 [...]
Caused by: java.lang.ClassCastException: Cannot cast clojure.lang.PersistentVector to clojure.lang.ISeq

Alex Miller (Clojure team)21:01:02

https://dev.clojure.org/jira/browse/CLJ-944 is in the ballpark of this - it’s a really tricky corner of the compiler

jrychter21:01:07

This is way beyond me. I searched the bugtracker before asking, but I wouldn't have recognized this issue as related. Anyway, it isn't a major problem, it's just that finding the cause is very difficult.

Alex Miller (Clojure team)21:01:08

can you repost the root cause of that exception chain (the last one one)?

jrychter21:01:31

Sure. I just abbreviated it, because it seemed rude to clog people's screens:

Caused by: java.lang.ClassCastException: Cannot cast clojure.lang.PersistentVector to clojure.lang.ISeq
        at java.lang.Class.cast(Class.java:3369)
        at clojure.lang.Reflector.boxArg(Reflector.java:552)
        at clojure.lang.Reflector.boxArgs(Reflector.java:585)
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:132)
        at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
        at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:325)
        at clojure.lang.LispReader$EvalReader.invoke(LispReader.java:1322)
        at clojure.lang.LispReader$DispatchReader.invoke(LispReader.java:853)
        at clojure.lang.LispReader.read(LispReader.java:285)
        at clojure.lang.LispReader.read(LispReader.java:216)
        at clojure.lang.LispReader.read(LispReader.java:205)
        at clojure.lang.RT.readString(RT.java:1874)
        at clojure.lang.RT.readString(RT.java:1869)
        at partsbox.preferences__init.__init1(Unknown Source)
        at partsbox.preferences__init.<clinit>(Unknown Source)
        ... 34 more
The last 2 lines reference my file: it's the one that uses the ^:const definition.

Alex Miller (Clojure team)21:01:25

to get this you have to think about what the compiler is doing. it’s trying to inline a value rather than look it up via the var.

Alex Miller (Clojure team)21:01:58

it’s going to include that value in the class compiled for the function using the inlined value

hiredman21:01:56

I, and I am sure @alexmiller will disagree, would say never use ^:const

hiredman21:01:52

as far as I can tell, people reach for it because it has a similar name to things they use in other languages when they want things to behave correctly

jrychter21:01:04

@alexmiller Right. I understand why this happens in the including file.

hiredman21:01:15

but clojure already behaves correctly, and const almost certainly doesn't do what you intuitively expect

Alex Miller (Clojure team)21:01:19

it has to store the constant value in a field of the class, but there is a limit to what you can store in bytecode. For simple values, this is easy, but iirc the fallback case is to pr the value to a string, that can be re-read back to a value in the consuming case.

jrychter21:01:55

Hmm. But sorted sets do not have a pr representation.

Alex Miller (Clojure team)21:01:04

sorted sets and maps don’t have a literal print syntax and no way to print/read the comparator function.

Alex Miller (Clojure team)21:01:10

user=> (binding [*print-dup* true] (pr s))
#=(clojure.lang.PersistentTreeSet/create [1 2 3])

Alex Miller (Clojure team)21:01:47

this gets into increasingly murky areas where reader eval is used

Alex Miller (Clojure team)21:01:12

I’m not sure I’m getting 100% of the details correct on all this (it’s been a few years since I looked at it)

Alex Miller (Clojure team)21:01:42

as @hiredman says, there are fewer places where this is worth doing than you might expect (I wouldn’t say none though)

Alex Miller (Clojure team)22:01:11

in particular using ^:const in tandem with ^long or ^double can be useful for inlining primitive long or double values

hiredman22:01:12

if you don't say none, everyone wants to argue that they are special and should use, and they never should

jrychter22:01:37

In general, I really wish sorted sets and sorted maps with default comparators had a printable/readable representation. I run into this quite a bit, and I don't have a single custom comparator function.

jrychter22:01:40

As to this problem, what next? Should I work on a minimal reproducible test case, or is it unnecessary? It seems this is a known issue, and all that could possibly be improved is the error message, right?

Alex Miller (Clojure team)22:01:29

I don’t think what you’re doing is a useful thing to do (that is, I can’t imagine how marking this ^:const well help you in any way)

Alex Miller (Clojure team)22:01:42

so I would recommend just removing that :)

Alex Miller (Clojure team)22:01:23

if you want to file a jira re the error message (or a clojure-site issue for better docs around ^:const), that seems reasonable

Alex Miller (Clojure team)22:01:59

the docs around ^:const are, well, largely non-existent. there are a couple lines in the changelog when it was added

noprompt00:02:32

Seems like it’d be easy enough to copy and paste the lines from changes.md over. Where would those go? I’ve seen this discussion come up enough that it’d be easier to just point someone to the site for reference. There are also several other meta questions that come up too (myself and others have asked about ^:inline).

Alex Miller (Clojure team)01:02:51

the inline stuff is primarily intended for use inside Clojure itself and is not considered a public feature that we wish to encourage (it may be removed in the future)

noprompt01:02:49

Thanks! I’ve already forked the repo to patch my name/github name. 🙂

Alex Miller (Clojure team)01:02:19

if you’re adding your name, you need to complete the CA form (but if you’re fixing a typo or something, that’s fine)

noprompt01:02:45

I completed the CA years ago. 🙂

noprompt01:02:47

Also, regarding ^:inline, I’m aware that it is not intended for public consumption; my suggestion regarding it is to make that fact public in documentation so that folks (you included) don’t have to keep having the same discussions.

jrychter22:01:19

I did, I don't much care about the ^:const. I just checked, and I have 253 ^:const declarations. I use them mostly as a comment, really.

jrychter22:01:04

I guess I can simply remove them all and assume that every def is a constant.

hiredman22:01:10

I'd suggest also maybe a re-evaluation of your relationship with sorted sets, if they are causing issues, maybe try and get a long without them (as a data point I am not sure if I recall ever using clojure's built in sorted sets outside of toy projects)

jrychter22:01:16

@alexmiller By the way, I really enjoy your weekly journal blog posts. If this doesn't take too much of your time, it is worth doing and is much appreciated.

Alex Miller (Clojure team)22:01:06

I’ll +1 reevaluating sorted sets

Alex Miller (Clojure team)22:01:44

I find using them means you are usually relying on order and linear searches rather than building indexes which will be a better solution

jrychter22:01:08

Hmm. They are actually quite useful in certain places. I use them for things like a list of quantities (build/order/whatever). A list like that has 1-20 numerical items. A sorted-set is perfect, as it lets me declaratively specify that it should always be sorted. I could stop using it, but then I have to worry about re-sorting in a number of places.

jrychter22:01:19

But — indeed if they cause more problems than they solve, I can just make sure I always sort manually.

Lennart Buit22:01:47

What’s the complexity of operations in a sorted set, membership and insert and such? Is it worse than a regular set? (I got intrigued… :) )

Alex Miller (Clojure team)22:01:29

they’re red-black trees, so mostly O(n log n) I think

Lennart Buit22:01:46

Cool! I remember having quite a difficult time getting my head around RB trees when I got them in CS

andy.fingerhut23:01:22

The constant factors for Clojure's sorted sets/maps are typically a fair amount larger than sets/maps that are not sorted, e.g. n log_2 n is a constant factor 5 more than n log_32 n.

hiredman22:01:31

or treat it more like a graph, {somevalue #{next} next #{nnext} etc}

dpsutton22:01:01

is this really easier than using a simple data structure like a sorted set?

hiredman22:01:09

if you are sorting just for display (as a todo list or whatever) then you don't need sorted data, you need a sorted view over something

hiredman22:01:34

it is more flexible in that it encodes the sort as data in the graph

hiredman22:01:37

vs. as a function

hiredman22:01:03

so it is better suited to things like serialization

dpsutton22:01:37

is it? seems like you run into the same issue as the sorted set. you can serialize its current order but how do you serialize the instructions for inserting new elements into the graph? that would be the comparator in the sorted set and would seem to be an analogous point for the graph insertion

hiredman22:01:01

that is the point, you don't serialize instructions at all

hiredman22:01:28

is order a property of your data/information or is it a property of the particular data structure you are using to represent the data

ghadi22:01:16

(I would also agree about :const earlier -- 90% of the time I've seen it in the wild, it is incorrect.)

hiredman22:01:49

1. sorted sets are cool, most of my favorite things are built on them to some degree or other (https://git.sr.ht/~hiredman/dset) 2. if something is causing you trouble stop using it 3. always have a third thing

jrychter22:01:07

I would humbly suggest that the word "incorrect" is too strong. ^:const, at a first glance, seems like an indication that something is a constant and will never change. Marking things this way is entirely reasonable

jrychter22:01:52

As for sorted sets, of course I don't require them. I can make sure that every function that needs the data sorts it before using (and no, it's not just "a view", there are multiple views, there is also logic that assumes the data is sorted). It's a tradeoff between declarative and procedural.

noisesmith23:01:13

but that's not how const works

noprompt00:02:02

@U0511PZ1R here’s a link to the aforementioned changelog entry explaining ^:const https://github.com/clojure/clojure/blob/master/changes.md#215-const-defs

jrychter00:02:52

Well, as an aside, I do understand how const works. The inlining is not critical in the code that I currently write, but I am aware of it. What I wasn't aware of are pitfalls, e.g. that not every Clojure data structure can be treated this way.

noisesmith00:02:23

no data clojure structure does anything useful with const - it's an optimization for primitives

noisesmith00:02:35

some data structures also error, none of them benefit

noprompt00:02:35

@U0511PZ1R I was trying to be helpful. I didn’t see anyone send you a link, etc.

jrychter00:02:13

Yes, "primitives" is what I was missing, I guess.

jrychter00:02:54

As for "useful", isn't it true that if initialization is expensive, it would happen at compile time and the result would be inlined?

noprompt00:02:00

This is the kind of the discussion that needs to be distilled into reference material so these discussions can just be collapsed into a URL.

noisesmith00:02:16

I don't want to come off as too strident here - I think given what const means in other languages it is poorly named and under documented

💯 5
noisesmith00:02:49

if your form is at the top level of the file, it's initialized during compilation, in clojure compilation happens when you load the file

jrychter00:02:16

Right. And if I build an uberjar with AOT?

noisesmith00:02:28

the file is loaded when making the uberjar, anything at the top level is done again on app startup runtime load of that class file, otherwise def with side effects in it would break when using AOT

noisesmith00:02:07

so you aren't saving as much time as you might think for defs at least

noisesmith00:02:50

eg. (def tracking (get-tracking-data (fire-missiles))) - fires missiles when you make the uberjar, then does it again when you load the resulting class file in an app

jrychter00:02:18

Thanks for explaining. I didn't think these definitions were evaluated again.

noisesmith00:02:08

it's surprising, but it's the least surprising choice - think of all the potential bugs when people use def on eg. a db connection pool, or an http server etc. (of course best practice for devs is never do anything stateful in def)

jrychter00:02:31

No, what surprises me is that the same is done when there is a ^:const metadata flag. I would not expect re-evaluation then.

noisesmith00:02:19

const doesn't really do anything with data structures, it enables a fast path for compiled code that uses the const var, if the const var is a primitive

noprompt00:02:32

Seems like it’d be easy enough to copy and paste the lines from changes.md over. Where would those go? I’ve seen this discussion come up enough that it’d be easier to just point someone to the site for reference. There are also several other meta questions that come up too (myself and others have asked about ^:inline).