Fork me on GitHub
#clojure
<
2020-07-15
>
chadhs13:07:52

perhaps i’m asking the wrong question here, but when i run a function in a REPL and it returns a function, is there any way to “inspect” or see the returned function, what args it takes, etc?

Alex Miller (Clojure team)13:07:24

no, function objects are opaque

chadhs13:07:21

thanks alex, wanted to make sure i wasn’t missing anything :thumbsup:

Daniel Stephens14:07:27

Hi all, does anyone know if this is the intended behaviour for clojure.java.data/from-java on maps? I kind of expected it to recurse through them. I guess I can override it in my case but wondering if I should raise an issue/pr to the library

(clojure.java.data/from-java (Quantity.))
=> {:additionalProperties {}, :amount nil, :format nil}
(clojure.java.data/from-java {"a" (Quantity.)})
=> {"a" #object[io.fabric8.kubernetes.api.model.Quantity 0x7865e2b8 ""]}
I found the line in the code that does the work:
(defmethod from-java java.util.Map [instance] (into {} instance))
but expected it to be more like:
(defmethod from-java java.util.Map [instance] (->> instance
                                                   (map (fn [[k v]] [(from-java k) (from-java v)]))
                                                   (into {})))
Is this just not the job of from-java?

seancorfield01:07:28

@dstephens I just released org.clojure/java.data {:mvn/version "1.0.86"} which adds from-java-deep which is a) deep in all cases including this one with a java.util.Map and b) accepts (requires) an options hash map, allowing the same options as from-java-shallow, to support adding :class, telling the function to ignore certain properties to avoid calling problematic getters, and controlling how to handle exceptions thrown by getters.

Daniel Stephens07:07:06

@U04V70XH6 Oh nice, someone read my mind then! I'll update now, cheers for all the help, I should have checked the source a bit closer 👍

seancorfield16:07:34

@dstephens Those changes were in response to your bug report last week -- from-java-deep is brand new! 🙂

Daniel Stephens16:07:13

Oh sweet, thanks for the work and swift turnaround!

seancorfield16:07:07

It spawned a 200 message thread about SemVer and backward compatibility and how to fix a bug where people might actually be relying on the buggy behavior 🙂

Daniel Stephens16:07:49

I did read that for quite a way and felt somewhat terrible for having mentioning anything 😂 but this seems like a good decision!

Alex Miller (Clojure team)14:07:56

seems like a philosophical question, not sure there is a right answer

Alex Miller (Clojure team)14:07:17

as you said, you can make your own defmethod to override

Daniel Stephens14:07:18

That's fair, I just thought someone might have had some reference to the original intent of the library, I found a test case for a map but it's just of string to string, so thought it might be a bug that hadn't been noticed yet. Thanks though, I'll just change things locally!

Michael J Dorian16:07:45

Hey friends, I have a library that I'm importing in a lein project.clj file. I suspect that I've found both a bug and a solution to said bug in the code, is there a quick way to swap the "release" version of the lib out for one that I have cloned locally in my project.clj file?

Ian Fernandez16:07:33

change the version of the lib

Ian Fernandez16:07:52

and do lein install

Ian Fernandez16:07:16

to install the "new version" of the lib onto your ~/.m2

Michael J Dorian16:07:34

thanks, I'll look into lein install

clj 3
Michael J Dorian16:07:14

Aha, it worked, and I was right about the bug!

clj 3
worlds-endless17:07:33

What's the idiomatic way to remove non-consecutive duplicates from a seq?

worlds-endless17:07:36

Nevermind; found (distinct)

seancorfield17:07:32

@dstephens I've taken over java.data as maintainer and I must admit I'm a bit surprised that from-java is shallow on a java.util.Map. It's recursive on every other type, I believe. Because there's from-java-shallow if you only want a shallow conversion...

✔️ 3
👍 3
Daniel Stephens18:07:48

Thanks for the reply @U04V70XH6! Great that the library is written such that I can change it for myself but I'll definitely be interested to see what you find/come up with. Not sure if there's anything I can do but if you need a hand let me know 🙂

seancorfield18:07:09

Feel free to DM me with questions about java.data. Knowing more about your usage will help inform how I change it.

Daniel Stephens18:07:51

So far I don't have many questions, it's proved very easy to use! My scenario is interop with a https://github.com/fabric8io/kubernetes-client library, they have a lot of POJOs that get created from their use of jackson. I wanted to avoid the jackson bit entirely but it doesn't seem very easily swapped out and most calls are simple enough to just call myself directly but the library has a lot built in to handle watching for changes on websockets which I need. The simplest solution at the time seemed to be just to use it as is and then convert the POJOs afterwards. The only issue is they are pretty deep with POJOs containing maps, containing lists, containing POJOs etc, hence the issue

Daniel Stephens18:07:08

I can definitely find alternative solutions to using from-java (or using it with my override as current) if it does appear to be a pain to change, but it did seem like a bug when I found it so thought I'd ask!

seancorfield17:07:28

(checking the git history indicates it has always behaved like that, but it surprises me -- it would be hard to change it now without risking breaking backward compatibility)

seancorfield17:07:42

I guess I could introduce a dynamic Var to control that behavior, and just default to the current shallow behavior, but that would allow for users to switch from shallow maps to deep maps...

p-himik17:07:46

Would it make sense to instead just add from-java-deep?

seancorfield17:07:24

from-java is already deep on everything except java.util.Map

p-himik17:07:09

Yeah, I understand. But that makes it not deep. :) IMO, a new name is more explicit and unambiguous as compared to a dynamic var.

seancorfield18:07:51

It would be a lot of duplication just to "fix a bug"... 😞

p-himik18:07:03

Not much more than it would be with a var, no? You'd just extract all common stuff into a separate "private" function and make from-java and from-java-deep call that function with different values of a single boolean flag. Or maybe I misunderstand the way it would work with a dynamic var. But I would agree that fixing from-java itself is better if you find sufficient evidence that such a fix will not be a breaking one for someone.

seancorfield18:07:10

The dynamic var would affect a single defmethod (out of a dozen or so) so it would be a very localized change. But perhaps adding from-java-deep would be worth all the extra code because it could take an options hash map that could control some of the behaviors, the way from-java-shallow does already.

p-himik18:07:07

Ahhh, right. I didn't realize it was defmethod. Thanks!

p-himik18:07:15

But your argument about having the same set of options for both -deep and -shallow is a solid one.

Alex Miller (Clojure team)17:07:07

It may be worth reviewing common actual use on github. Maybe most people don’t care

Alex Miller (Clojure team)17:07:12

The library description talks about recursive translation so that does seem to be the default mindset

Romit Gandhi17:07:29

Had anyone integrated stripe payment gateway with Clojure? I need some help for that.

noisesmith17:07:40

it was years ago, but I recall it just being a standard REST API?

Romit Gandhi17:07:32

Can you provide some demo kind of thing, it will be much helpful for me.

noisesmith17:07:41

sorry, it was for a closed project, and I haven't worked on it in years. I used clj-http as the rest client, if you have specific questions I might be able to fill in details, but I don't have a working example

Romit Gandhi18:07:52

I want to create subscriptions.

Romit Gandhi18:07:45

`(let [url (str api-root "/subscriptions")] (json/read-str (:body (client/post url {:basic-auth [token] :query-params {"customer" "XX" "items" [{"plan" "XX"}]}}))))`

Romit Gandhi18:07:55

This is my code for that.

Romit Gandhi18:07:43

I tried with valid customer I'd and plan I'd still I'm getting 400 request.

lukasz18:07:23

Might be a long shot, but you're using query params, whereas Stripe's API uses from data (check the curl example in their docs) Try :form-params instead

3
Romit Gandhi18:07:46

I have tried it but same error

noisesmith18:07:05

I would expect a post to take a request body rather than query-params, and for the string for the query to be a string, and not auto-converted from clojure data

Romit Gandhi18:07:35

:query-params works for all other things like creating customer, plans etc

noisesmith18:07:44

you can't just give the form body a clojure data structure here - you need to construct that "items[0][price]" string

noisesmith18:07:28

check out the full result of the call, and not just the body, often there's informative data about what went wrong

Romit Gandhi18:07:37

Can you write query for that to do this thing @U051SS2EU?

Romit Gandhi18:07:27

`{ "error": { "message": "Invalid array", "param": "items", "type": "invalid_request_error" } }` This is the response body.

noisesmith18:07:20

that's exactly what I was saying - you can't hand it a clojure data structure for "items" - you need to use the same pseudo-array string that is in the example

Romit Gandhi18:07:10

So, is there any way to do that?

noisesmith18:07:13

{"customer" "cus_HeY0QGjNIjqzM8"
 "items[0][price]" "price_1H5Auy2eZvKYlo2CsZrpDrGC"}

Romit Gandhi18:07:52

Done, thank you so much. 😊

seancorfield17:07:40

FWIW https://clojure.atlassian.net/browse/JDATA-21 for from-java on java.util.Map but I'll see how it is used via http://grep.app and see if I can figure out whether such a change would really be breaking or not.

❤️ 3
seancorfield18:07:53

Only @tolitius and @magnars appear to be using clojure.java.data/from-java in a context where they might be expecting a deep conversion... but those calls might only ever happen on shallow maps...

jjttjj20:07:22

fwiw I'm using this in a context where I would expect the map to be deeply converted. Haven't actually hit this yet but it seems like something that will inevitably come up.

3
Alex Miller (Clojure team)18:07:03

I suspect that's often the case (but maybe not always)

Daniel Stephens18:07:06

it seems hopeful that most wouldn't call from-java only to then expect a java object somewhere in the depths! 🤞 Thanks for looking into it

seancorfield18:07:58

One possibility (suggested by @p-himik) would be to add from-java-deep that did the deep map conversion (but otherwise behaved just like from-java) although that would be a lot of code duplication because of the tree of paths through all the conversions. Part of me feels like just calling this a bug and fixing it -- which would break code that depended on a shallow conversion for hash maps (but a deep conversion everywhere else -- an odd combination, I agree).

deactivateduser18:07:52

Might this be an opportunity for cough SemVer to come to your rescue? 🤡

deactivateduser18:07:30

(yes yes I’m leaving, no need to call security 😉)

3
andy.fingerhut20:07:48

It doesn't come to the person's rescue who wants to update to the latest version of the library, if they somehow depended on the current behavior.

deactivateduser21:07:17

No, but why would they move to the latest version in that case? This kind of change is exactly what SemVer uses the MAJOR component of a version number to communicate. And if there’s some unrelated bug that effects both the old and new version, and users of both versions want it fixed, that could be fixed in a new PATCH version of both the old MAJOR version, and the new MAJOR version (provided that that bug fix preserves backwards compatibility in both).

deactivateduser21:07:26

And if there are other reasons to move to the new MAJOR version (e.g. it contains new, though backwards compatible features that they wish to use), then either they could investigate (or request) that those features be backported to the old MAJOR version (as a new MINOR version), or they could take on the more substantial work of moving to the new MAJOR version, and updating their code to work with whatever backwards-incompatibilities it introduced.

deactivateduser21:07:30

Either way, SemVer, when properly used*, clearly communicates the types of changes the consumer of the library should expect to have to deal with when they see a new “version” of a library they’re using (and many other versioning schemes simply don’t - you have to go and read lengthy prose or read code or whatever to figure this stuff out). *and yes it’s a given that some library authors will get it wrong, but human fallibility is a constant independent of SemVer

seancorfield22:07:31

The problem is that a huge number of library authors do get it wrong and a huge number of library authors don't actually follow SemVer despite their library apparently following MAJOR.MINOR.PATCH -- and the vast majority of libraries not even stating whether they follow SemVer or not -- which all goes toward making any assumptions about SemVer based on a library's version number changes being almost completely meaningless. That's the problem.

deactivateduser22:07:56

None of which are problems specific to SemVer.

deactivateduser22:07:10

As I said above, “human fallibility is a constant”

andy.fingerhut01:07:20

I think when it comes to version numbers that look like X.Y.Z, it isn't merely humans making mistakes, it is humans using things that don't even resemble SemVer, either because they do not care about SemVer but want to use version numbers that look like other software's version numbers, or they claim to use SemVer, but when push comes to shove, they don't truly want to follow it. It is willfully not following SemVer's rules.

Ruy Valle02:07:08

@U0MDMDYR3 What if you write some code in a library you maintain and you don’t realize there is a bug and your users come to rely on that bug being there, often not realizing it’s a bug? Wouldn’t it be a better policy to make a new function, new namespace or something along those lines, even if it’s more work? I don’t have a strong stance on this, but I think I agree with the point Rich makes. With Semver, when you see there is a new major version, the only thing you know is you might have to fix some breakage if you upgrade. You still have to either try out the new version or read through some prose. Which should only be the case by mistake if the library authors have committed to not making breaking changes.

deactivateduser02:07:04

@U015GL869TQ sure, but what about MINOR and PATCH releases? The alternatives I keep hearing about (dates, “RELEASE”, etc.) basically force me to assume that every version breaks something.

deactivateduser02:07:49

@U0CMVHBL2 agree that that’s a problem, but I don’t see alternative approaches attempting to even give what little comfort SemVer attempts to give. The argument, boiled down to an extreme statement, strikes me as “communication is hard and humans are fallible, so we shouldn’t attempt to communicate meaning”, and I guess I just fundamentally reject the idea that an attempt to communicate isn’t worth something, however flawed the execution of that intent is.

Ruy Valle02:07:03

yes I agree that some attempt at communicating meaning is better than none

deactivateduser02:07:28

As @U04V70XH6 discovered today, breaking changes are sometimes the best of a bad lot of options, and being clear about communicating that a new version is or is not breaking is important.

deactivateduser02:07:05

And yes, that doesn’t diminish the importance of not breaking compatibility in the first place (something that the Clojure ecosystem is remarkably good at, ime!).

deactivateduser02:07:24

But, to paraphrase folks much wiser then myself, shit happens. 😉

Ruy Valle02:07:53

I am not sure if that is the best option. Then again, I am not sure what the best option is and I’m not sure what I would do in the same situation.

seancorfield03:07:34

SemVer is a noble ideal but, frankly, it's a failed experiment. People either deliberately don't follow it or, worse, they claim to follow it and then don't. It is not even just a neutral failure, it's actively harmful -- for all the reasons discussed here, for the reasons Rich mentioned, for every single library upgrade everywhere that "should have been safe" based on SemVer that has caused hours and hours of bug chasing, frustration, and quite often production downtime.

👎 3
seancorfield03:07:37

I applaud library authors who at least have the honesty to say they're not following SemVer and then explain how they identify types of changes to their users.

seancorfield03:07:45

As for the original issue under discussion, if everyone's assumption of a library's behavior is X but it turns out that the behavior is Y and the library's documentation implies the behavior is X, is the really a breaking change to release a new version of the library with behavior Y? Is this "just" a bug fix (patch in SemVer)? Is it a big enough change to warrant a minor version bump? A major version bump? A new name for the fixed feature?

seancorfield03:07:39

Anyone who considers it a bug fix would be okay with a patch bump in SemVer -- and yet you'd be breaking everyone's code that relied on the behavior being incorrect (despite being documented to the contrary). And this is again why SemVer is actively harmful: a bug fix would be a patch bump and SemVer says that's a "safe" upgrade. But anyone relying on the bug's existence has their code broken. SemVer is just a fairy tale.

andy.fingerhut03:07:01

@U0MDMDYR3 An alternative to SemVer, not necessarily one that is easy to follow, but possible I believe in many situations, is: Don't break backwards compatibility. Introduce new names/APIs/libraries/whatever, rather than changing the behavior of existing ones in known-to-be-incompatible-ways.

deactivateduser03:07:02

@U04V70XH6 as I mentioned last time this topic came up, behavioural backwards compatibility is indeed a lot more difficult to reason about than syntactic backward compatibility. But that is true independent of SemVer.

deactivateduser03:07:23

What I don’t see are alternatives attempting to communicate either.

seancorfield03:07:50

> But that is true independent of SemVer. Right, which is why SemVer doesn't actually help.

seancorfield03:07:31

> What I don’t see are alternatives attempting to communicate either. Andy has described an alternative, more than once now, in this thread. You just don't seem to like his answer 🙂

seancorfield03:07:32

Also, just because SemVer is the "only" choice you see, doesn't make it good. It's entirely possible for a common "standard" to be bad "independent of" whether there are any alternatives, let alone better alternatives.

seancorfield03:07:03

If we end up depending on everything via :git/url then we have only a :sha and only the system "knows" whether a given SHA is more recent than another (or whether they are incomparable), so "version numbers" would no longer matter 🙂 I think a lot of this is about adjusting people's expectations.

andy.fingerhut03:07:17

I could probably be convinced that there are bugs that have been fixed in software, e.g. Clojure's implementation, where you could maybe make a convincing case that what the developers call a bug fix could be relied upon by someone somewhere, and be considered not a bug fix but a breaking change.

andy.fingerhut03:07:26

Often software systems with a specification that is separate from their current implementation can be used for more strongly arguing that a change in the implementation is a bug fix, i.e. moving it closer to the specification, even if someone, somewhere, happened to rely upon the buggy behavior. The closest Clojure has to that separate specification are the doc strings.

deactivateduser03:07:14

> Also, just because SemVer is the “only” choice you see, doesn’t make it good. Come on @U04V70XH6 you’re better than to resort to straw man arguments. Nowhere have I implied that I’m a SemVer zealot (in fact I consider it the best of a bad lot of alternatives), and I’ve read Andy’s alternative and it gives up on the idea of “version values as communicators of ‘classes of change’“. I’m not convinced (yet) that giving up on that is justified.

deactivateduser03:07:54

(and apologies for fragmented responses - I’ve been dealing with offspring schooling crises and will likely have to continue doing so on and off this evening)

deactivateduser04:07:14

And to your later point, git+SHA coordinates don’t communicate anything about “classes of change” to the consumer. Basically that implies a state where every change in version (i.e. SHA) needs to be treated as if it were a breaking change by consumers. That’s quite an extreme position, and worse than a world where “SemVer is used, but for various reasons X% of version identifiers incorrectly identify the ‘class of change’” for all values of X < 100.

deactivateduser04:07:31

Are you, in fact, saying that 100% of the time, SemVer versions incorrectly identify the “class of change”? Because if so I’d ask for evidence, since that’s a very strong assertion.

seancorfield04:07:19

Re: straw man -- just remember that you're the one that started this thread by being "deliberately controversial" Peter... and I haven't "called security" yet 🙂

seancorfield04:07:23

Re: every change needs to be treated as if it were a breaking change -- which is exactly why advocating for only accretive and fixative change is so very important. And, yes, I agree that SHAs communicate nothing -- but if there is a strong commitment to never making breaking changes, that's OK I think, yes?

deactivateduser04:07:00

You’re considering making a breaking change in java.data - you tell me. 😉

seancorfield04:07:08

Re: 100% -- no, not "100%" but that's too high a bar. I think the reality is "some arbitrary percentage" but whatever it is in actuality, I think it's too high. In other words, I think SemVer has suffered a death of a thousand cuts from all those projects not following its lofty ideal -- but advocates don't want to admit the patient is terminal yet 🙂

deactivateduser04:07:11

If I were in your shoes I’d prefer to have made a commitment to attempt to communicate that via well defined (i.e. standardised, via SemVer) changes in version values, rather than assuming any downstream consumer is going to RTFM.

deactivateduser04:07:52

Oh and by having a FM, you’re in a small minority of library authors already. 😉

deactivateduser04:07:08

Look I get the frustration with poor adherence to SemVer - I share it. But the wild west of “do what you want” version values (including SHAs) is worse. I lived that back in my C/C++ days in the late 80s and early 90s, and it’s hell.

seancorfield04:07:10

Re: java.data -- yes, and that's exactly why I'm agonizing over it in Clojure's climate of "never make breaking changes". Frankly, in a SemVer climate, I'd just declare it a bug -- the treatment of just one Java type doesn't match the documentation, while all the other types are treating in a consistent manner, per the documentation. "Clearly" a bug. Fix it. Make a patch release. I'm following SemVer, just like "everyone else".

deactivateduser04:07:05

Welcome to the thorny thicket of “is a behavioural breakage backwards compatibility breaking or not”? 😉

seancorfield04:07:16

But I'm not in a SemVer climate so fixing a bug such that it changes the behavior of something people have asked questions about makes me want to consider extremely carefully how I address the problem.

deactivateduser04:07:47

Again that’s neither here nor there. I would expect the same of a library author who was a SemVer zelout.

deactivateduser04:07:57

Expect, but not assume.

deactivateduser04:07:16

Perhaps we just disagree about the potentialities of human communication. 😉

seancorfield04:07:08

Re: C/C++ days -- yup, I lived that too. I've been doing software for 35 years and I spent eight years on the ANSI C++ Standards Committee in the 90's. I remember that pain 🙂

deactivateduser04:07:23

Copypasta code reuse FTW!

deactivateduser04:07:28

Anyhoo, if you were to simply say “reliably identifying breaking changes is a pipe dream and therefore every change should be treated as if it were breaking by consumers” I’d be interested to hear more. That’s an argument that somewhat resonates with me.

deactivateduser04:07:04

But arguments that try to say “some changes are breaking and some aren’t, and SemVer has zero role in helping communicate that” just don’t resonate at all.

seancorfield04:07:10

I am a cynic. Or at least I have become a cynic. I view most human's ability to communicate with a jaundiced eye. I think SemVer contributes to misunderstanding (because it is not an accurate communication in enough cases that I consider it actively harmful). So maybe you are simply more optimistic about humans than I am?

deactivateduser04:07:42

I am a cynic too, but that doesn’t stop me believing in the power of aspiration.

seancorfield04:07:15

And also I think the behavioral semantics are important and are definitely worth discussing. Far more important (and interesting) than version numbers.

deactivateduser04:07:58

Agreed. However if you agree that there are different classes of change (such as “breaking” vs “non-breaking”), then I will continue to argue that version values can (and probably should) be used as a communication medium for those different classes of change.

deactivateduser04:07:50

And yes, they’re not interesting in and of themselves, except in what they can be used to communicate succinctly.

deactivateduser04:07:09

I mean a SHA doesn’t communicate anything. They’re not even ordered.

seancorfield04:07:38

@U0MDMDYR3 OK, how about COMMITS.CHANGE_TYPE ? 🙂 Where CHANGE_TYPE is breaking or safe perhaps?

deactivateduser04:07:52

That’s just SemVer, minus the differentiation between PATCH and MINOR (which could be argued is a less valuable classification). 😉

seancorfield05:07:18

The CHANGE_TYPE can be anything to reflect the "important" aspects of the release compared to the previous release. And COMMITS is a clear ordering.

deactivateduser05:07:19

And yes I get that syntactically it’s not a SemVer version value. ;-)

seancorfield05:07:04

OK, and if you commit to never breaking anything, then it seems like MAJOR.MINOR.COMMITS is a viable version system in your book?

deactivateduser05:07:47

Absolutely. And if MAJOR and MINOR never increment (because of no new features or breaking changes), awesome! You’ve done a better job than 99.9999% of library authors! 😉

seancorfield05:07:08

We can all argue over whether any given change is "major" or "minor" but as long as it is purely accretive or purely fixative (and not breaking) then the intent is ... acceptable?

deactivateduser05:07:11

Yes, though I think accretive changes are (or can be) materially different from fixative changes. An example is in resource constrained environments.

deactivateduser05:07:39

e,g, I want to know if my “binary” is about to double in size because of a bunch of accretive changes I don’t want or need in some upstream dependency. It might influence my decision about whether to use that new version or not.

seancorfield05:07:42

Yup, like I said, we can argue all day about what's minor and what's major and what additional information users might need...

deactivateduser05:07:16

And (perhaps another day) whether it’s even possible for a library developer to know how their changes should be classified. 😉

seancorfield05:07:42

...but the key thing here is if you take breaking changes out of the equation, you get an honest version system: it reflects forward motion (an ordering), it reflects some notion of "small change" vs "big change", and it never breaks your code.

deactivateduser05:07:10

If it exhausts limited resources due to accretion in upstream, then yes, it’s broken my code.

seancorfield05:07:20

The classification is the hardest part. And SemVer doesn't help with that (nor does any other versioning system).

deactivateduser05:07:24

And I want to know about it.

deactivateduser05:07:38

Especially vs fixation, where typically resource consumption won’t change substantially due to the fix.

deactivateduser05:07:01

In fact that’s all that SemVer does.

seancorfield05:07:04

You're moving the goalposts a bit there -- any non-breaking new feature could break your system under those constraints.

deactivateduser05:07:11

It provides a way to construct version values that communicate that information.

deactivateduser05:07:19

It doesn’t help you obtain that information, mind you.

deactivateduser05:07:29

That’s on the (human, fallible) developer.

seancorfield05:07:33

Well, we simply disagree on this.

seancorfield05:07:42

This = SemVer.

deactivateduser05:07:28

SemVer is nothing more and nothing less than a way to construct a version value that communicates one (or more) of three things: 1. Breakage (MAJOR) 2. Accretion (MINOR) 3. Fixation (PATCH)

deactivateduser05:07:56

It doesn’t help the developer decide, for any given code level change they make, which “bucket” that change should fall into.

deactivateduser05:07:26

That inherent complexity exists no matter how version values are constructed (even meaningless ones like SHAs).

seancorfield05:07:16

As we've already established -- or at least most of us agree -- SemVer isn't actually useful because people don't get it right: because they don't agree on those categories of change.

deactivateduser05:07:35

That’s true of all version systems, including ones based on changelogs and other prose.

deactivateduser05:07:09

The difference, the only difference of SemVer, is that I can read it at a glance and determine what the author thought they were doing in a given change.

seancorfield05:07:19

And that's why I said git SHAs would be just as useful 🙂

deactivateduser05:07:21

In prose form I have to slog through a bunch of human-language text.

seancorfield05:07:44

But you can't determine that. That is the entire point of this discussion.

deactivateduser05:07:08

SHAs aren’t though - they don’t give the author the ability to describe which of three classes of change I outlined above they thought they made.

seancorfield05:07:15

Apparent conformance to SemVer is a lie in enough cases as to make it useless.

deactivateduser05:07:25

Instead I have to go elsewhere to find that out - either a changelog, or docs, or (worst case) the code itself.

seancorfield05:07:27

You are completely missing the point.

deactivateduser05:07:53

And you’re missing my point - those “lies” (your words not mine) exist in all version systems.

seancorfield05:07:04

You said: > That’s true of all version systems, including ones based on changelogs and other prose. And that is why I responded that SHAs are therefore as good as any other.

deactivateduser05:07:23

But that does NOT mean there isn’t value in an author ATTEMPTING to communicate what they THOUGHT their changes were - breaking, accretive, or fixative.

deactivateduser05:07:36

Even though they’re going to be wrong X% of the time!

deactivateduser05:07:53

Do you see the difference?

deactivateduser05:07:02

Or do you really think that trust is not a thing? 😉

deactivateduser05:07:14

That there is only “verify”?

seancorfield05:07:25

If you can't trust the version number, you're not gaining anything from it.

deactivateduser05:07:43

Forget about version numbers - this is about human to human communication.

deactivateduser05:07:49

Do you trust the author of a library?

seancorfield05:07:07

Practical experience has shown that we can't trust SemVer -- it's a lie in enough cases that we can't rely on it.

deactivateduser05:07:18

i.e. to correctly inform you that a) they’ve made a change and b) that change is one or more of breaking, accretive, fixative.

seancorfield05:07:45

This is a pointless discussion at this point.

deactivateduser05:07:23

If you say so. It’s a shame you’re so myopically focused on the syntax of version values, instead of seeing that SemVer’s intent is simply to use such values to facilitate communication between (fallible) humans (i.e. library authors and library consumers).

deactivateduser05:07:30

We seem to agree that there is value in a library author (fallibly trying to) communicate breaking vs accretive vs fixative changes to library consumers, and I’m sorry I can’t seem to explain how version values can be an effective part of that communication (vs, say, SHAs or dates, which can’t communicate any of that information).

andy.fingerhut13:07:11

How can one be myopically focused on the syntax of version values, when their arguments are all about how many software authors claiming to follow the semantics of SemVer, but for one reason or another changing their minds when it is convenient for them to do so?

andy.fingerhut13:07:52

@U04V70XH6 I am curious on your take on the following idea. It seems that one way to follow the advice of 'no backwards compatibility breakage', AND follow SemVer versioning rules, is "never bump the major version number, while following SemVer versioning rules". That is, if you really followed SemVer semantics, then all of your version numbers would be "1.x.y".

deactivateduser15:07:45

@U0CMVHBL2 that argument applies to any form of communication made by those same software authors. If they weren’t messing up SemVer, they’d be messing up changelogs, or prose in a manual, or whatever they use to try to communicate the nature of their changes to consumers of their library. Again, “human fallibility is a constant”.

andy.fingerhut15:07:47

There are people who have at least heard the idea of SemVer, but actively avoid following it. If by "fallibility" you mean "accident", then that is not what is happening in those cases. They want to advertise big new additions which don't break backwards compatibility? Bump the major version! They made some backwards incompatible changes, but are embarrassed about it, and don't want to call attention to it? Bump the minor but keep the major the same.

andy.fingerhut15:07:16

If you are including such practices in "human fallibility", best to be explicit about it

deactivateduser15:07:04

Those people aren’t using SemVer then, are they?

andy.fingerhut15:07:21

Tell me someone who is?

deactivateduser15:07:51

Various open source foundations mandate it.

andy.fingerhut15:07:52

Not sure if you've heard of the "No true Scotsman" fallacy

andy.fingerhut15:07:57

I work with one open source group where I know they explicitly "fixed" something from version 1.0 to 1.1, and didn't want to call it 2.0 even though there was an open discussion verifying that yes, this breaks backwards compatibility, but we don't want to move to 2.0 because it would be embarrassing.

deactivateduser15:07:03

But I don’t consider software authors who happen to use MAJOR.MINOR.PATCH[-SUFFIX] to be using SemVer, unless they explicitly say so.

deactivateduser15:07:15

Given that that format was in use well before SemVer was a thing (especially in the Maven ecosystem).

andy.fingerhut15:07:38

And they know about SemVer, and pay lip service to it

andy.fingerhut15:07:07

I recommended going to 2.0. No dice.

andy.fingerhut15:07:34

There were no accidents, no mistakes. It was an explicit decision said out loud, after it was pointed out.

andy.fingerhut15:07:54

And in my previous sentence I could remove "mistake" as being redundant in the way I meant it with the word "accident".

andy.fingerhut15:07:33

I know that is one anecdote, not a mountain of data, but I suspect if you do a poll of people asking if they have used a library that did such a thing, you would find that most software developers have encountered this practice.

deactivateduser15:07:32

Sure. And my point is that would have happened whether those libraries used SemVer or some other versioning scheme.

andy.fingerhut15:07:42

So, please excuse me if I have a very strong belief that if you dig into the open source foundations that mandate it, I would be completely unsurprised to find many occurrences of similar events within their projects.

deactivateduser16:07:16

Of course. Humans are fallible.

andy.fingerhut16:07:08

And you include in the word "fallible" the cases of "knows what SemVer and their organization mandates, but chooses not to follow it when it would be embarrasing" ?

deactivateduser16:07:25

Ultimately, the point I’m trying to make is that those kinds of things happen regardless of the mechanism by which software authors communicate the classes of change (fixation and/or accretion and/or breakage) that they’ve made.

deactivateduser16:07:38

SemVer doesn’t magically fix that, but neither does any other version labeling scheme.

deactivateduser16:07:02

> And you include in the word “fallible” the cases of “knows what SemVer and their organization mandates, but chooses not to follow it when it would be embarrasing” ? Yes. It’s very difficult to determine intent from the outside (a good general lesson in life, fwiw).

andy.fingerhut16:07:03

And I think ultimately, the point that those discussing this with you is that, if it is actually correctly followed 0.001% of the time, of what utility is it to consumers of such software?

andy.fingerhut16:07:23

At best, it is a very rough hint that is often wrong. At worst, it is an outright lie.

deactivateduser16:07:32

And to reiterate, if someone is embarrassed about having made a certain change (of any class) and consciously decides to be less than honest about communicating it, then it doesn’t matter how they’re communicating it (SemVer, changelog, prose in a manual, etc.) - they’re going to “lie”.

deactivateduser16:07:14

And my point is that authors still communicate that information somehow.

andy.fingerhut16:07:26

Oh, the case of doing 1.0->1.1 instead of 1.0->2.0 was for a specification, rather than an implementation of that specification, and anyone who knew the two specifications could very clearly see it was not backwards compatible. There was no hiding it.

deactivateduser16:07:16

So let’s argue the other case then - let’s say this specification used SHAs for version labels. What mechanism would they have used for communicating fixation vs accretion vs breaking, instead of the version label?

deactivateduser16:07:42

Changelog, prose in a manual, …?

andy.fingerhut16:07:26

They have prose changelogs.

deactivateduser16:07:55

Ok cool. So in those prose changelogs, did they attempt (in most cases) to communicate fixation and/or accretion and/or breaking changes?

andy.fingerhut16:07:05

I would have to go back and check whether they explicitly mentioned the backwards-incompatibility property of the change. It was definitely made because the older version was considered "not what we actually intended to specify".

deactivateduser16:07:15

Ok. So in the SemVer case, why did they bump MINOR instead of MAJOR for this breaking change? Embarrassment, incompetence, accident, …?

deactivateduser16:07:36

(and I realise you may not know - despite what I said earlier, this is trying to determine intent 😉 )

andy.fingerhut16:07:57

Embarrasment that 1.0 had a mistake they didn't catch before releasing 1.0, and to release a 2.0 so soon after release 1.0 would have been a slight bit of egg on their face.

andy.fingerhut16:07:29

Nothing that impacted them in any major way that I know of -- just probably didn't want to answer questions like "why did you release 2.0 so soon after 1.0?"

deactivateduser16:07:09

Ok. So let’s assume that that embarrassment is a given (meaning, they would have been embarrassed anyway, whether they were using SemVer or not), and let’s return to our hypothetical (that they’re only using prose in changelogs to communicate classes of change). Don’t you think it’s possible, even probable, that that embarrassment would have caused them to obfuscate the breaking change in the prose in their changelog?

andy.fingerhut16:07:11

They listed it as a "major overhaul" of the portion of the spec that was affected in the prose of the changelog. Seems technically accurate to me.

andy.fingerhut16:07:38

The folks involved I've known for years, and they are conscientious careful engineers. You know, typical stance of preferring clear truth to hiding behind obfuscation or trickery.

deactivateduser16:07:54

And yet without SemVer, the folks who felt embarrassment acutely enough to “lie” about the nature of the changes would have nowhere else to focus their revisionist eye, correct?

deactivateduser16:07:31

I hope you see where this is going - that “fallibility” results in the same problems, no matter how “classes of change” are communicated.

deactivateduser16:07:41

SemVer doesn’t solve that problem, but then neither does anything else.

andy.fingerhut16:07:57

They know they were making backwards incompatible changes. They don't use the words "backwords incompatible change" in the change log, but I don't think they were trying to hide that fact in the change log, either. They knew the project was nominally supposed to be following SemVer. They heard my recommendation to change the version to 2.0. They chose to use 1.1 instead.

deactivateduser16:07:10

BTW I’ve been in a very similar position with another spec (OASIS CMIS - I was on the TC for some years), so I have some experience of the politics involved when this kind of “embarrassing screw up” happens. I found that it was very hard for the “honest” faction to fight against the “embarrassed” faction.

andy.fingerhut16:07:25

And I am sorry, but I really do not get your meaning with this question "And yet without SemVer, the folks who felt embarrassment acutely enough to “lie” about the nature of the changes would have nowhere else to focus their revisionist eye, correct?" I am not saying I don't know the answer -- I don't know what the question means.

deactivateduser16:07:26

If, in our hypothetical case, SemVer was not in use (let’s say they used SHAs for version labels), then they they would have been using prose in changelogs to communicate classes of change (fixation and/or accretion and/or breaking). Agree so far? Then, it follows that if they had a screw up big enough to cause an embarrassing breaking change, and the “embarrassed” faction won the internal political debate and chose not to communicate that change accurately, then, in our hypothetical example, the changelog would be obfuscated.

deactivateduser16:07:01

After all, that’s exactly what did happen in the real case, except that instead of communicating it via prose in the changelog, they happened to (mis)use SemVer.

andy.fingerhut16:07:16

I can imagine a scenario where such a thing could happen, sure. I don't think it happened here.

deactivateduser16:07:26

My point is that removing SemVer won’t “fix” fallibility.

deactivateduser16:07:52

Sure, but they had SemVer as a crutch for the “embarrassed” faction. Take that away and where do they focus their “revisionist eye”?

andy.fingerhut16:07:01

And my point is what I said above: if SemVer is only in truth used accurately in a vanishingly small fraction of cases, of what utility is it?

deactivateduser16:07:41

And my point is that all forms of communication of classes of change (whether it’s via version labels, changelogs, or War and Peace in a manual) suffer from the exact same problem!

andy.fingerhut16:07:28

So do you use that conclusion as an argument to advocate FOR the use of SemVer? If so, I don't see how that can possibly follow.

deactivateduser16:07:14

No, but there are other arguments in favour of SemVer that other “change labeling” mechanisms don’t have. Prior art (as crappy an argument as it is) being one of the primary ones.

andy.fingerhut16:07:34

"A large fraction of people will mislead you about changes in their software and systems. I propose that we use SemVer to do so."

deactivateduser16:07:40

As I said way back when, I’m not a fan of SemVer - it’s just the best of a bad lot of options.

deactivateduser16:07:58

Not my straw man, but if we’re going there here’s mine: “Some fraction of people will mislead you about changes in their software, for a variety of reasons (mostly inadvertently, some due to malice, though the difference is immaterial to the consumer). I propose that we adopt the most widely understood method of communicating change, simply because it reduces the cognitive burden on our consumers. No, it doesn’t solve the problem of fallibility, but then nothing can do that, so it’s a wash and can be ignored as a discriminating factor.”

andy.fingerhut16:07:40

Sorry, I should have prefaced that statement with a question such as "Would you consider this a brief statement of your position?"

deactivateduser16:07:09

Ok. And the answer would have been “no that does not reflect my position”.

andy.fingerhut16:07:56

It seems to me that what we currently have in practice, even among a significant fraction of people claiming they follow SemVer, is "major.minor.something", where those numbers and when they change follows different rules per project.

deactivateduser16:07:37

Right. And that’s even more true of projects that use some other method to communicate “classes of change” too, since in that landscape, everything is a special snowflake.

andy.fingerhut16:07:46

And the only thing you can actually tell from those is that lexicographic sorting of those vectors of numbers is a tree structure that communicates "laterness"

deactivateduser16:07:09

That’s the only thing that’s guaranteed, correct.

deactivateduser16:07:39

There is also an intent to describe classes of change, an intent that needs to be “trusted but verified” by consumers.

deactivateduser16:07:01

(as is equally true of “special snowflake” approaches)