Fork me on GitHub
#clojure
<
2019-02-21
>
johnj04:02:53

is with-open clojure's TWR version?

hiredman04:02:01

more or less, with-open predates try with resources by some number of years

hiredman04:02:29

so with-open predates autocloseable

👍 5
dominicm06:02:21

I think that you're right hiredman, doing a require of clojure.core is causing the global hierarchy to reset. I think one of the manifold namespaces aliases clojure.core, so that's why the finger is pointing there. Why would clojure.core be loaded twice?!

Alex Miller (Clojure team)06:02:50

it's loaded on RT initialization and should never be loaded again after that

Alex Miller (Clojure team)06:02:49

(unless you're in an isolated classloader chain)

dominicm06:02:10

I'm using onejar class loader. So I think I will need to figure out the details.

Alex Miller (Clojure team)06:02:08

that seems like an important detail you omitted :)

dominicm06:02:09

You're right, I was a bit hyper focused on "require is causing my program to break" and was a bit lost.

dominicm06:02:21

I'm aware of https://dev.clojure.org/jira/browse/CLJ-971 and I'm setting the factory

dominicm06:02:05

But I'm not aware of any other things I need to do for clojure to work with onejar.

Alex Miller (Clojure team)07:02:45

you could collect info about the classloader chain at points where things look good and bad and compare

dominicm08:02:34

If I turn on verbose logging, I can see it searching for clojure__core when I do require

dominicm08:02:51

But I'm a bit confused why it wouldn't be the same as what clojure.main is using, maybe I need to do a custom build of clojure with some logging in.

dominicm08:02:17

Into clojure.main/main I added a print of the context class loader, then I compared it to what I have at the repl, exactly the same:

#object[com.simontuffs.onejar.JarClassLoader 0x7a79be86 "com.simontuffs.onejar.JarClassLoader@7a79be86"]
Clojure 1.11.0-master-SNAPSHOT
user=> (.getContextClassLoader (Thread/currentThread))
#object[com.simontuffs.onejar.JarClassLoader 0x7a79be86 "com.simontuffs.onejar.JarClassLoader@7a79be86"]
user=> 

dominicm08:02:41

#object[com.simontuffs.onejar.JarClassLoader 0x7a79be86 "com.simontuffs.onejar.JarClassLoader@7a79be86"]
Clojure 1.11.0-master-SNAPSHOT
user=> (clojure.lang.RT/baseLoader)
#object[clojure.lang.DynamicClassLoader 0xeccaf1b "clojure.lang.DynamicClassLoader@eccaf1b"]
clojure.lang.Compiler/LOADER is bound then, but I don't really know what that means?

dominicm08:02:49

(I confirmed the binding)

dominicm08:02:11

Oh, but that isn't too useful anyway:

user=> (.getParent @clojure.lang.Compiler/LOADER)
#object[com.simontuffs.onejar.JarClassLoader 0x7a79be86 "com.simontuffs.onejar.JarClassLoader@7a79be86"]

dominicm08:02:53

I'm a bit defeated really, I'm going to need some direction on the clojure internals to puzzle out what's going on here I guess.

dominicm10:02:52

I modified doInit so I could check the class loader before load("clojure/core") is called, and com.simontuffs.onejar.JarClassLoader@7a79be86 is the classloader there, and #object[com.simontuffs.onejar.JarClassLoader 0x7a79be86 "com.simontuffs.onejar.JarClassLoader@7a79be86"] is the one loaded when clojure.main/repl is running. I also modified load so that I could see which ClassLoader it was using when loading "clojure/core", and I got com.simontuffs.onejar.JarClassLoader@7a79be86

dominicm10:02:03

So class loader doesn't seem likely to me. I can't see where the caching is.

dominicm11:02:19

loaded-libs doesn't contain clojure.core at the point of me having a REPL...

user=> @*1
#{clojure.core.protocols clojure.core.server clojure.core.specs.alpha clojure.edn clojure.instant clojure.java.browse  clojure.java.javadoc clojure.java.shell clojure.main clojure.pprint clojure.repl clojure.spec.alpha clojure.spec.gen.alpha clojure.string clojure.uuid clojure.walk}

dominicm11:02:53

Looks like clojure.core is never in loaded-libs:

❯ clj                    
Clojure 1.10.0
user=> @@#'clojure.core/*loaded-libs*
#{clojure.core.protocols clojure.core.server clojure.edn clojure.instant clojure.java.browse  clojure.java.javadoc clojure.java.shell clojure.main clojure.pprint clojure.repl clojure.spec.alpha clojure.spec.gen.alpha clojure.string clojure.uuid clojure.walk}

dominicm11:02:21

So, I think I'm onto something here. OneJar always returns "0" for the last modified. I have a suspicion this might be because I'm not setting the last modified of the jar itself, and maybe that's what onejar uses. Anyway, when I use clj, this passes as true:

if((classURL != null &&
	    (cljURL == null
	     || lastModified(classURL, classfile) > lastModified(cljURL, scriptfile)))
	   || classURL == null) {
However, in my jar lastModified is 0, hence 0 > 0 is false. So we end up going to if(!loaded && cljURL != null) { which I'm guessing doesn't look at the same caching path as the first if to determine whether the namespace is already loaded or not

Alex Miller (Clojure team)13:02:59

Isn’t this the same issue as the ticket you linked to earlier?

Alex Miller (Clojure team)07:02:25

the classloader thing is a hypothesis, so try to prove/disprove

dominicm07:02:01

I have a clojure.main repl in my jar, so this should be easy. How could I collect info about the chain?

dominicm07:02:46

The context class loader is the same before and after requiring clojure, for example.

gypsydave514:02:12

Small rant: I've had the hardest time working with deps.edn over the last few days. Nothing to do with the workflow or the principle - it's the tooling that did for me. Was trying to build an uberjar for a Cloud Foundry deploy - this pretty much means that it needs to have AOT compilation and run with java -jar uberjar.jar. I tried using depstar, cambada and Pack as listed on the tools page of the deps wiki. Depstar doesn't support AOT or a manifest file out of the box and I'm nowhere near skilled enough to work out how to add them in. Both pack and cambada did not stop building the uberjar - literally, both of them filled the disk up and exited them. Thought it might be the code, but then I try lein uberjar and it just worked. Really disappointed - kinda think it'd be worth packaging up the leiningen uberjar code as a separate library? </moan> Other than that I'm having a lovely day. 😄

Alex Miller (Clojure team)14:02:33

someone was just using lein as a library under deps.edn to call lein stuff so that might be a path

gypsydave514:02:56

That's what I was thinking - who, where, how?! 😄

Alex Miller (Clojure team)14:02:08

having tools be reusable things rather than bound into plugins for a single build tool is generally a great idea imo

gypsydave514:02:05

I agree as well - it's instinctively where I want to get to as I've always been put off by Leiningen.

Alex Miller (Clojure team)14:02:02

that's from @sekao if you have questions for him

gypsydave514:02:24

I like it - makes sense!

sekao14:02:53

yep i started using this technique in most of my projects. if you want to use other lein commands you can make a multimethod like this https://github.com/oakes/Nightlight/blob/master/prod.clj

10
gypsydave514:02:47

Have you considered extracting the pattern out into a library after the fashion of depstar or Pack - or do you consider it too trivial?

sekao15:02:59

for me i’d rather use it directly and avoid the layer of indirection. i’m ok with a little code duplication. might be an interesting library idea if others want to try it though

ghadi15:02:08

it's also really easy to AOT with depstar, but it's not "baked in"

gypsydave515:02:35

could you show me an example?

gypsydave515:02:57

(as I liked depstar - in that it actually finished building...)

dtsiedel16:02:26

Is there a way to create a function literal of a Java function? The following does not work:

user=> (def func Integer/parseInt)

Syntax error compiling at (form-init730605430373361324.clj:1:1).
Unable to find static field: parseInt in class java.lang.Integer
So I've been playing with making Clojure wrapper functions and using those for higher-order needs. Ex:
(def parse-int #(Integer/parseInt %))

erwinrooijakkers16:02:22

I don’t think there is one because it’s a static method

erwinrooijakkers16:02:27

For instance methods you can use memfn

erwinrooijakkers16:02:40

(map (memfn charAt i) ["fred" "ethel" "lucy"] [1 2 3])
;; => (\r \h \y)

dtsiedel16:02:04

Ah cool, I hadn't seen memfn. Thanks!

erwinrooijakkers16:02:49

Me neither before I tried to answer your question 😉

dtsiedel16:02:59

I had even made it to that Java Interop page before but must have missed it

Alex Miller (Clojure team)16:02:31

note that memfn introduces reflection so an anonymous literal wrapper is often better

erwinrooijakkers22:02:15

Interesting. Thanks

dtsiedel16:02:12

Interesting. Just because of overhead of the reflection?

Alex Miller (Clojure team)17:02:53

presumably if you are setting this up, you're going to invoke it more than once

dtsiedel17:02:54

I just didn't know that reflection was costly enough to make a difference. But on looking it up, it does appear to be very very slow

Alex Miller (Clojure team)17:02:32

if you're calling it once, you probably don't care, but if you're calling it a lot, you will

dtsiedel17:02:25

Got it, thanks

ghadi17:02:24

memfn can take a hint on the symbol and avoid reflection, no?

ghadi17:02:55

No one uses memfn in the wild

craftybones18:02:17

Any test runners that can spit out JSON?

dominicm18:02:28

memfn is something I used a lot to implement a datafy thing.

mars0i19:02:57

@seancorfield or anyone: tools.cli will parse e.g. -for as -f -o -r. Is there a way to turn this functionality off? I personally like this GNU behavior, with double-dash for verbose options, but there's also Java convention that uses single-dash for multi-character options. I'm using a Java lib that does that. I use tools.cli for the options that my code provides, but I don't want to reinvent the wheel for the options that the library provides. However, this means that if I use tools.cli to find errors using parse-opts, my code has to distinguish between real errors in option specification on the command line, as opposed to "errors" due to cli/parse-opts thinking that the user has provided -f -o -r. I am trying to add code to filter out the Java lib options, but without turning off the single-dash/multi-character expansion, it's very difficult. (I'm also trying not to simply reinvent tools.cli's parsing.) Thanks-

Lennart Buit19:02:54

many linux utils take multiple “flags” together

Lennart Buit19:02:01

for example tar -xvf

Lennart Buit19:02:23

really, Java is the oddball here

seancorfield19:02:24

Yes, that's the "standard" GNU/*nix behavior.

seancorfield19:02:49

I agree. And tools.cli was changed deliberately in the past to the GNU/*nix behavior 🙂

Lennart Buit19:02:48

is there a story here btw, Unixes were widespread when Java came about right

Lennart Buit19:02:30

Your google fu is better than mine!

Lennart Buit19:02:48

or better filter bubble 😛

mars0i19:02:16

Yeah. Well it helps to know that Java is following Posix. I didn't realize that.

mars0i19:02:53

Oh no that's wrong.

Lennart Buit19:02:28

yeah, X11 style 😛

seancorfield19:02:26

My Bing fu 🙂

seancorfield19:02:50

I haven't used Google for years.

Lennart Buit19:02:20

‘google fu’ as in ‘velcro’ 😉.

mars0i19:02:08

Or maybe I should look more closely at the tools.cli innards and maybe I can build something from the functions there....

seancorfield19:02:03

@mars0i I'm not sure that you could support both styles of options -- it would be ambiguous -- and this is a change that was made explicitly in tools.cli to support GNU-style options (prior to me taking the library over).

seancorfield19:02:58

So I think my position here is that tools.cli supports the GNU behavior by design and will not support the Java convention.

mars0i19:02:38

Right, one couldn't support both in the same application. But you could have a flag that turns the single-dash/multichar behavior on and off. But I (doh!) just realized that I can address my problem pretty simply by just requiring that the Java lib options be put after -- to stop option processing by tools.cli. The lib ignores --.

seancorfield19:02:22

Ah, OK. So this was just a case where you need to pass additional command-line options directly to an embedded library? I missed that.

mars0i19:02:26

The rationale for turning it off would specifically be compatibility with Java conventions. But maybe it's not worth the trouble.

mars0i19:02:00

Oh, sorry. I didn't explain clearly enough then. Yes, the library also sees the command line, and ignores options it doesn't know, but processes the ones it does know.

mars0i19:02:19

Maybe it will only be a real problem if the -- trick isn't enough for a situation.

seancorfield19:02:09

I wouldn't expect command-line parsing libraries to be very composable if they're both parsing the same command-line data 🙂

mars0i19:02:18

Yah, I know. It actually works just fine in this case, as long as you ignore misspelled options and that sort of thing. And I have been misspelling them lately, and then thinking I have a bug. 🙂

mars0i19:02:42

Thanks. That's helpful to read. I may think about it further and come back with some ideas for tools.cli down the road if I think there's something worth doing that's potentially worth the trouble. Obviously an isolated use case isn't worth trouble.

mars0i19:02:17

I think this application is unusual in other ways too.

didibus20:02:30

Quick question about gen-class. The :name value in the doc says it has to be a package-qualified name of the class to be generated. Which I assume means it is not a Clojure namespace name, but a Java one. Which means, since that it should not contain hyphens, but underscores instead, since Java doesn't allow for hyphens. Now, here's the thing, it seems that if you put hyphens, it will work, and generate a file with hyphens in the folder and classname, and somehow, that will work from Java as well. What's happening here?

didibus20:02:03

For example, the following:

(:gen-class :name screening.whirlpool.update-investigation-poller.Main)
Will generate the directories: screening/whirlpool/update-investigation-poller/Main And the Main class generated has as its package name: screening.whirlpool.update-investigation-poller But it does: Util.loadWithClass("screening/whirlpool/poller/update_investigation_poller", Main.class); in its static initializer. Yet, somehow, I can use that class fine form Java.

noisesmith21:02:49

that looks like something accidentally working, at first glance

Lennart Buit21:02:11

I am trying to check whether some val that implements a protocol via metadata (satisfies? ...) that protocol. That sadly doesnt work. then I found https://dev.clojure.org/jira/browse/CLJ-2426, and I tried inlining that proposed satisfies? function in my code, but also no success.

ghadi21:02:03

the most straightforward solution is to check for the symbol inside the meta, as it does around line 9

Lennart Buit21:02:49

sorry, which symbol, there are only function names in the meta?

ghadi21:02:06

no they're literal symbols inserted -- see line 21

ghadi21:02:12

`bla-bla == test/bla-bla

ghadi21:02:00

the function is checking for the existing of the correct symbol within the metadata, on Line 10

ghadi21:02:26

I don't think using satisfies? is a good idea in general

ghadi21:02:37

it runs counterproductive with polymorphism

ghadi21:02:49

the whole point of protocols is not to care about doing type-tests

Lennart Buit21:02:50

I dont use it as a type test, I use it in s/fdef’s to do some rudimentary argument checking in development. Maybe not the best usecase, but it has found some bugs for me ^^

ghadi21:02:52

in general

Lennart Buit21:02:58

Right, but (symbol (:var proto)) is test/BlaFactory, thats not test/bla-bla right?

ghadi21:02:23

I didn't read the details, but check for the name of the protocol function, not the protocol var

Lennart Buit21:02:44

hmm I’ll try that. Thanks as always!

robertfw22:02:36

Hi all - I have a spec question. I'm writing a spec describing some JSON that is getting posted to one of our endpoints (which we do not have control over, it is a callback from another system). The callback can be in one of two shapes, depending on if the operation was successful or not. So I have my spec looking essentially like (s/def ::callback-post (s/or :success ::success-spec :failure ::failure-spec)). All that is fine and dandy - what I am stuck on, is that the callback has a success flag, which will be either true or false, and I'm not sure of the best way to represent that within the ::success-spec and ::failure-spec specs

robertfw00:02:50

Yup, that was one adjustment, though the major problem (choosing a spec based on the value of that boolean) was solved using multi-spec on suggestion from butterguns

robertfw22:02:35

I have a higher level (s/def ::success (s/org :true true? :false false?)) spec, but without using some manual namespacing I'm not sure how best to specify within the ::success-spec that ::success ought to be only true

Lennart Buit22:02:20

First of all, (s/def ::success boolean?) ^^

robertfw22:02:47

yup, that's a picard-facepalm

butterguns22:02:08

... where the success field indicates the "type", and hence which spec to use

robertfw22:02:07

yeah, that looks like it could be the answer, thanks 🙂 I hadn't come across multi-spec before

butterguns22:02:36

You're the second person I've suggested it to this week. I'm a big fan of it!

Lennart Buit22:02:06

I used it to do payload validation before, mighty useful

butterguns22:02:36

I posted this in #clojure-spec, it might be useful as an example

butterguns22:02:40

(s/def ::email string?)
(s/def ::username number?)
(s/def ::request-type #{:change-email :show-username})

(s/def ::change-email-request (s/keys :req [::email] :opt [::username]))
(s/def ::show-username-request (s/keys :req [::username] :opt [::email]))


(defmulti request-type ::request-type)
(defmethod request-type :change-email [_] ::change-email-request)
(defmethod request-type :show-username [_] ::show-username-request)

(s/def ::body-params (s/multi-spec request-type ::request-type))

(s/valid? ::body-params {::request-type :change-email ::email "boop"})
=> true
(s/valid? ::body-params {::request-type :change-email ::username 1234})
=> false
(s/valid? ::body-params {::request-type :show-username ::email "boop"})
=> false
(s/valid? ::body-params {::request-type :show-username ::username 1234})
=> true

robertfw22:02:44

thanks, I was just doing a little headscratching over the docs

robertfw22:02:51

Thanks, that did the trick 🙂 Hopefully spec2 makes this kind of thing a little more straightforward

butterguns23:02:31

:thumbsup: you're welcome!

Braden Shepherdson22:02:00

I'm looking for a way to mock out a defrecord that implements several protocols. empty implementations that just return nil are fine for most functions; a few others need to be replaced with code I will supply. https://github.com/alexanderjamesking/spy has protocol/spy but that only seems to work for one protocol. (though its code looks like it could handle more? maybe I should patch it)

Braden Shepherdson22:02:14

is there an alternative library, or am I misapprehending something?

noisesmith22:02:15

@braden.shepherdson what's the use value for a stubbed method with no real logic vs. not implementing the method at all?

noisesmith22:02:48

because clojure is happy to let you define only the methods you care about and throw an error at runtime if others are called

Braden Shepherdson22:02:33

no particular reason, then. in this case, I need three functions that are split across two protocols; nothing else is called anyway.