Fork me on GitHub
#clojure
<
2022-03-17
>
aratare07:03:39

Hi there. I have a seemingly silly question which I hope someone can enlighten me: What is the difference between the Clojure deps in a project (e.g. org.clojure/clojure {:mvn/version "1.10.3"}) and the Clojure CLI itself (e.g. clj/`clojure` command) and how are they related? What happens when there is a mismatch between these two versions? Thanks in advance 🙂

borkdude08:03:06

@rextruong clj / clojure are essentially bash scripts that build a call to an uberjar which packages tools.deps.alpha. tools.deps.alpha then calculates a classpath and returns this to the script. The script then starts a java process with this classpath. This classpath can contain any Clojure version, completely unrelated to the clj / clojure version you have installed.

aratare08:03:04

I see. Thank you for answering 🙂 This has always been a bit confusing for me after coming from Java land 😅

borkdude08:03:30

I've also translated those bash scripts to Clojure, you can see those here: https://github.com/borkdude/deps.clj

1
Harsha08:03:18

Hi. I am trying to use google’s protocol buffers in a clojure project. lein javac to compile .java files, produced by protoc compiler, is failing. I am using protoc of version 3.19.4 and [com.google.protobuf/protobuf-java "3.19.4"] and proto2 . one of the errors is error: cannot find symbol com.google.protobuf.Descriptors.OneofDescriptor

user=> (import com.google.protobuf.Descriptors)
com.google.protobuf.Descriptors

user=> (import com.google.protobuf.Descriptors.OneofDescriptor)
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:445).
com.google.protobuf.Descriptors.OneofDescriptor

user=> (import com.google.protobuf.Descriptors$OneofDescriptor)
Execution error (IncompatibleClassChangeError) at java.lang.ClassLoader/defineClass1 (ClassLoader.java:-2).
class com.google.protobuf.Descriptors$OneofDescriptor has interface com.google.protobuf.Descriptors$GenericDescriptor as super class 
https://javadoc.io/static/com.google.protobuf/protobuf-java/3.19.4/com/google/protobuf/Descriptors.OneofDescriptor.html I never used protocol buffers with java or clojure before. Can someone please help me out

p-himik08:03:46

If it's a nested class, replace the last . in its FQN with $.

Harsha09:03:36

This .java code is generated by the protoc compiler. There are other errors too. This . notation is inside .java file, so it is a valid java code. Shouldn’t lein javac take care of it itself.

error: OneofDescriptor is not public in Descriptors; cannot be accessed from outside package                                                                                                         

Harsha09:03:04

replacing . with $ did not work

Harsha09:03:08

error: OneofDescriptor is not public in Descript ors; cannot be accessed from outside package com.google.protobuf.Descriptors$OneofDescriptor

p-himik09:03:03

Ah, I'm still waking up - managed to miss that you actually did use $, my bad. >

class com.google.protobuf.Descriptors$OneofDescriptor has interface com.google.protobuf.Descriptors$GenericDescriptor as super class 
Weird - that's not in the source code, it's public abstract static class GenericDescriptor there. Are you trying the latest version?

Harsha10:03:07

protoc is of version 3.19.4 - it is the latest one protobuf-java is of version 3.19.4 - but it is not latest, but it looks same as latest version https://javadoc.io/doc/com.google.protobuf/protobuf-java/latest/com/google/protobuf/Descriptors.OneofDescriptor.html FQN of OneofDescriptor in java is com.google.protobuf.Descriptors.OneofDescriptor in clojure it should be com.google.protobuf.Descriptors$OneofDescriptor Since OneofDescriptor is used in .java file, com.google.protobuf.Descriptors.OneofDescriptor is the right FQN I don’t even understand the error now

Conor10:03:14

I would make sure that the protobuf-java JAR is on the classpath. protoc will generate whatever it wants to generate without regard to that

Harsha10:03:59

protobuf-java jar is on the classpath

Harsha10:03:47

output of lein classpath has /Users/harshakiranboyapati/.m2/repository/com/google/protobuf/protobuf-java/3.19.4/protobuf-java-3.19.4.jar

Conor11:03:47

If you want to make sure, you can run javac with the same arguments that lein javac generates and verify it works correctly. If it doesn't, then lein is misleading you. 😅

Harsha11:03:57

lein javac `lein classpath`   did not give any errors

Conor11:03:04

No, I mean work out what javac call lein javac is making, then do the same thing to see if your code compiles

Ben Sless14:03:34

Check your protoc version

Ben Sless14:03:20

I stumbled across it when the proto dependency and compiler versions weren't the same

Harsha15:03:58

protoc is of version 3.19.4 and protobuf-java is of version 3.19.4

Harsha16:03:59

lein version
Leiningen 2.9.8 on Java 17.0.2 OpenJDK 64-Bit Server VM

protoc version
libprotoc 3.19.4
Attached files of a trivial project, which is giving the same error while trying to compile generated java code to java bytecode Example.java is the file generated by protoc on compiling person.proto lein-javac-error.txt has the console output of lein javac project.clj only has [com.google.protobuf/protobuf-java 3.19.4] other than clojure as dependency

Harsha16:03:21

protoc -I=resources --java_out=src/java resources/proto/person.proto is the command used to generate .java from .proto

Conor16:03:46

→ lein javac
Retrieving org/clojure/clojure/1.10.3/clojure-1.10.3.pom from central
Retrieving org/clojure/spec.alpha/0.2.194/spec.alpha-0.2.194.pom from central
Retrieving org/clojure/core.specs.alpha/0.2.56/core.specs.alpha-0.2.56.pom from central
Retrieving nrepl/nrepl/0.8.3/nrepl-0.8.3.pom from clojars
Retrieving org/nrepl/incomplete/0.1.0/incomplete-0.1.0.pom from clojars
Retrieving org/clojure/core.specs.alpha/0.2.56/core.specs.alpha-0.2.56.jar from central
Retrieving org/clojure/spec.alpha/0.2.194/spec.alpha-0.2.194.jar from central
Retrieving org/clojure/clojure/1.10.3/clojure-1.10.3.jar from central
Retrieving org/nrepl/incomplete/0.1.0/incomplete-0.1.0.jar from clojars
Retrieving nrepl/nrepl/0.8.3/nrepl-0.8.3.jar from clojars
Compiling 1 source files to /home/conor/dev/Coats/temp/target/classes
Note: /home/conor/dev/Coats/temp/src/java/person/Example.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Conor16:03:52

:man-shrugging:

Harsha17:03:42

But how do u get a success message in first try and I have been stuck with this error for days what is your protoc version

Harsha17:03:01

Also what is your leiningen version Does the generated java code matches with what has been generated for me? Anything else that could be the reason?

Conor17:03:40

protoc --version
libprotoc 3.19.4

Conor17:03:49

lein --version
Leiningen 2.9.8 on Java 11.0.11 OpenJDK 64-Bit Server VM

Harsha06:03:33

javac is compiling the java code generated by protoc just fine but lein javac is failing for me

Harsha09:03:56

Is anyone working with protocol buffers in clojure? If you are, did you not face any issue while compiling generated .java files to .class files? if you did not face any issues please share the steps u have followed. If you did face issues please share the steps u have taken to resolve them. Thanks in advance:slightly_smiling_face:

Ed21:03:42

I've done this in the past. We created a separate project with the protobuf files that compiled to class files, then required that as a dependency in the Clojure project. The protobuf files were a schema defining data provided by another team, so we deployed them as an artifact to our internal artifactory.

Ed21:03:16

But I think that if you're using the new Clojure build tools, there's a way to include preparation steps like this.

Harsha06:03:31

javac is compiling the java code generated by protoc just fine but lein javac is failing for me

timo16:03:14

Anyone knows how to get what https://github.com/clj-time/clj-time/blob/master/src/clj_time/periodic.clj does but with clojure.java-time/Date-Time API?

lukasz17:03:20

Yes, Tick can do that

👍 1
henryw37417:03:22

yeah tick has a range function which is basically the same thing

henryw37417:03:23

... which has an example in those docs linked by lukasz, if you search within the page... no direct link

timo17:03:51

cool, thanks! didn't look at that one.

jumar22:03:02

Just found a production bug related to clojure.set/union - was quite surprised by that

(set/union [1 2 3] #{3 4 5})
  ;; => [1 2 3 4 3 5]

universe_guy 1
R.A. Porter22:03:59

I agree that it's not great that it silently does the wrong thing.

jumar22:03:09

Yep, and there's no mention about it in the docstring.

p-himik22:03:00

An answer that can be often seen from the core team in such situations - if it's not in the docstring, then the behavior is undefined. In this case, the docstring talks about "input sets" and doesn't talk about any other types. So using instances of any other types as inputs is undefined behavior.

jumar22:03:26

Oh, yeah - I know about this "GIGO approach" but it's quite easy to miss and make a mistake 😞

andy.fingerhut23:03:48

Many people have pointed this out over the years, but it seems that once you know about it, no one seems to care about it so much that they use a library like this: https://github.com/jafingerhut/funjible

andy.fingerhut23:03:19

I wrote a big old fat warning about this here years ago: https://clojuredocs.org/clojure.set/union

andy.fingerhut23:03:54

(scroll down a little bit to see my comment)

andy.fingerhut23:03:03

funjible downloaded 96 times in 4 years on Clojars (practically unknown & unused)

phill23:03:25

In theory, couldn't someone write a spec with which to instrument clojure.set?

andy.fingerhut03:03:55

Definitely. I have done it. Performance numbers in the funjible project README, but sure, as long as you only turn spec checking on during testing, not deployment, the significant performance hit won't matter as much.

jumar05:03:46

I'm wondering if at least expanding the docstring or maybe even checking the inputs or coercing to sets would be doable here

p-himik08:03:15

It's doable but has been discussed a number of times, both here and on Google Groups - it won't happen. The reasoning is that the current state is sufficient and that changing it would require for it to be changed everywhere to be consistent - that would be a whole paradigm shift.

andy.fingerhut14:03:53

@U06BE1L6T If a library like https://github.com/jafingerhut/funjible already exists and does nothing additional except check the inputs, would you consider using it?

andy.fingerhut14:03:01

Or does it HAVE TO BE IN CORE in order to consider it?

andy.fingerhut14:03:47

I am curious, since despite pointing out alternative docs (http://clojuredocs.org) and alternative libraries, they never seem to satisfy people.

Joshua Suskalo15:03:24

I think that people are generally pretty averse to adding in new dependencies, and if the dependency just does something that you can already do with core, even if it's a little better or less error prone, people just won't add it even if they know about it.

andy.fingerhut16:03:48

@U5NCUG8NR I agree that people should be averse to adding dependencies willy-nilly. This one is one that you cannot do with core as it is. Or if you can, it is by adding something functionally equivalent to funjible in your own code and using that instead. I do understand the wish that this stuff was in core, but unless the core developers change their minds in a very significant way, do not hold your breath on that.

Joshua Suskalo16:03:16

Oh yeah, I agree with you on that, and I do conceptually like having more checking available. Something I've noticed in my own code and in others' as well though is that Clojure programmers seem averse to error-like situations, and are more likely to program code in such a way as to make errors just not happen rather than use or write code that handles errors better, so honestly I expect more to happen when the core team eventually gets around to speccing core (which is a stated eventual goal) than I expect to happen from any library that adds additional error checking.

vemv06:03:17

https://github.com/borkdude/speculative seems a sound, non-disruptive approach to me

andy.fingerhut16:03:16

As long as the performance hit of spec (which is significant -- see the perf stats in the funjible library readme) is OK for you. Perhaps during testing it is. I doubt it would be for production use with many calls to those functions.

andy.fingerhut16:03:47

Even very simple sanity checks in spec are very slow compared to a hand-coded type check.

zimablue22:03:47

Hi random question, if I have two independent async operations that might be called, and I want them never to both happen at the same time (eg. say they're both messing with the same file and would corrupt it if they ran together), the only way I can think of to do that in clojure (without using java locking) is to put the resource they're contending behind a channel and pipeline-async with parallelism 1 to basically serialize the access. Is that a reasonable way to do it and what others exist? Also is my scenario understandable?

zimablue22:03:45

I realize I could also use a go macro and manual <! but I'm counting that as basically the same thing as that paralellism 1 pipeline-async although if anyone has a strong preference for one over the other, that would be interesting

zimablue22:03:28

sorry, also let's say it has to be clojurescript compatible, so no STM or other clojure-specific async tools, although I'm curious which would be right

hiredman23:03:53

I would definitely prefer a single go loop to a pipeline when the whole point is to have no parallelism

hiredman23:03:36

pipeline-async is also has some subtlety to it, a single go loop over a channel is less likely to be screwed up

hiredman23:03:05

it is already the case that js is singled threaded, so you can maybe use reader conditionals to noop whatever concurrency control is needed in clojurescript

zimablue23:03:08

js is single threaded but if the two operations both have async parts they might still get interleaved as operations on [thing that must be worked on by one thing at a time]? Like you don't have shared mutable state INSIDE the language/runtime but might still want this logical exclusion of a resource? Is that right?

hiredman23:03:18

which is why I said "maybe", depending on what you are doing you may be able to take advantage of the single threaded nature of js