Fork me on GitHub
#juxt
<
2019-02-20
>
p-himik19:02:07

How do you guys produce an uberjar? uberjar.adoc says to use bin/onejar but it doesn't seem like the specified -A:prod is enough. And bin/uberjar simply doesn't work: java.io.FileNotFoundException: Could not locate io/dominic/krei/alpha/main__init.class or io/dominic/krei/alpha/main.clj on classpath

dominicm19:02:47

bin/uberjar is probably broken

dominicm19:02:55

We use onejar in production though

dominicm19:02:59

What's wrong with it?

p-himik19:02:50

Well, using -A:prod creates a jar that's not runnable since nrepl is missing. Also, there are no generated web resources. Can you tell me what arguments you pass to bin/onejar?

p-himik19:02:08

Just tried running ../bin/onejar -A:prod:build:prod/build:build/once --args '-m edge.main' project.jar from main. Running java -jar project.jar still results in Could not locate clojure/tools/namespace/repl__init.class or clojure/tools/namespace/repl.clj on classpath.

dominicm19:02:04

I need to document this better.

p-himik19:02:15

And probably some tests won't hurt. 🙂

dominicm19:02:21

I don't know what is requiring tools.namespace via edge.main

p-himik19:02:38

user.clj is.

dominicm19:02:14

Nothing should be, unless your code is?

p-himik19:02:56

As I said - prod/user.clj requires [clojure.tools.namespace.repl :as repl].

p-himik19:02:22

I'm trying to pack and run vanilla edge, there are no changes.

p-himik19:02:33

Edge/main that is.

dominicm19:02:38

Don't do anything with edge/main

dominicm19:02:42

That's deprecated

dominicm19:02:07

prod/user.clj is a bad idea, for example.

dominicm19:02:34

I need to mark main as deprecated somehow, without breaking the yada documentation.

p-himik19:02:55

Wait, what am I supposed to be using then?

dominicm19:02:27

../bin/onejar -A:prod --args '-m edge.main' onejar.jar is how I produce uberjars btw, but that doesn't factor in cljs/css. But will circle back to that.

p-himik19:02:47

Yes, I've seen that documentation. By "what to use" I meant "is there any test project using edge?" 🙂

dominicm19:02:32

inside of edge? a good one might be tutorial.vent

dominicm19:02:32

It may well be that we lack a good example anymore though, vent hasn't been taken into production for example.

dominicm19:02:00

I often deploy the out-of-the-box fresh app that's generated though.

dominicm20:02:39

the default app has this which can compile the assets: clojure -A:build:build/once

dominicm20:02:02

okay, vent fails in a way I would basically expect 😄

p-himik20:02:44

OK, just did:

$ ./bin/app acme/test --sass --cljs
$ cd acme.test
$ ../bin/onejar -A:prod --args '-m edge.main' project.jar
$ java -jar project.jar
It failed with:
22:07:31.889 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.simontuffs.onejar.Boot.run(Boot.java:342)
        at com.simontuffs.onejar.Boot.main(Boot.java:168)
Caused by: clojure.lang.ExceptionInfo: Missing definitions for refs: [:acme.test/index :edge.yada.ig/classpath-name], [:acme.test/assets :edge.yada.ig/resources] {:reason :integrant.core/missing-refs, :config {:acme.test.foo/string "Hello, test!", [:edge.yada.ig/classpath-name :acme.test/index] {:name "index.html"}, [:edge.yada.ig/resources :acme.test/assets] {:path "public/"}, :edge.yada.ig/listener {:handler #integrant.core.Ref{:key :edge.bidi.ig/vhost}, :port 7200}, :edge.bidi.ig/vhost [["" ["" [["/" #integrant.core.Ref{:key [:acme.test/index :edge.yada.ig/classpath-name]}] ["/hello" #integrant.core.Ref{:key :acme.test.foo/string}] ["" #integrant.core.Ref{:key [:acme.test/assets :edge.yada.ig/resources]}]]]]], :edge.kick/builder {:kick.builder/target "target/prod", :kick/sass {:builds [{:id "test", :source "test.scss", :target "public/test.css"}]}, :kick/figwheel-main {:builds [{:id "app", :main acme.test.frontend.main, :output-to "public/frontend.js", :output-dir "public/frontend.out", :asset-path "/frontend.out", :optimizations :advanced}], :figwheel-config {:ring-server-options {:port 3535}}}}}, :missing-refs ([:acme.test/index :edge.yada.ig/classpath-name] [:acme.test/assets :edge.yada.ig/resources])}
        at integrant.core$missing_refs_exception.invokeStatic(core.cljc:191)
        at integrant.core$missing_refs_exception.invoke(core.cljc:190)
        at integrant.core$build.invokeStatic(core.cljc:320)
        at integrant.core$build.invoke(core.cljc:303)
        at integrant.core$init.invokeStatic(core.cljc:418)
        at integrant.core$init.invoke(core.cljc:410)
        at integrant.core$init.invokeStatic(core.cljc:415)
        at integrant.core$init.invoke(core.cljc:410)
        at edge.main$_main.invokeStatic(main.clj:10)
        at edge.main$_main.doInvoke(main.clj:8)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.applyToHelper(AFn.java:152)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at clojure.lang.Var.applyTo(Var.java:705)
        at clojure.core$apply.invokeStatic(core.clj:665)
        at clojure.core$apply.invoke(core.clj:660)
        at clojure.main$main_opt.invokeStatic(main.clj:493)
        at clojure.main$main_opt.invoke(main.clj:487)
        at clojure.main$main.invokeStatic(main.clj:598)
        at clojure.main$main.doInvoke(main.clj:561)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:705)
        at clojure.main.main(main.java:37)
        ... 6 more

dominicm20:02:04

yeah, I just hit that. I'm a bit confused actually.

dominicm20:02:46

especially as this is one of the simpler parts of the codebase, and I can see in the failing system that it has those keys.

p-himik20:02:54

Why does config.edn use both [:edge.yada.ig/classpath-name :acme.test/index] and [:acme.test/index :edge.yada.ig/classpath-name]?

p-himik20:02:09

Doesn't feel right.

dominicm20:02:42

copy/paste error I would suspect 🙂

dominicm20:02:33

I mean, really, the #ig/ref shouldn't use the composite key I think.

dominicm20:02:00

but yeah, they should match in the template, else integrant will generate different composite keys for them.

dominicm20:02:37

does it work if you switch them?

dominicm20:02:27

Very strange, it's failing to find the edge.yada.ig.resources integrant component when I make them match.

p-himik20:02:56

Yep. And using just :acme.test/index as a reference fails as well.

p-himik20:02:22

I think it's a recent failure since this part works in my other project that I started earlier.

dominicm20:02:41

I would assume it is 6de7f049fe26d80c02cfbc339d3479508be0538a

p-himik20:02:29

But it just changes the template. Whereas the main failure is with matching.

p-himik20:02:04

I think 8a5be082f72fbdb1ec88035b373d9da49820e329 doesn't have this issue since that's what I use in the other project.

dominicm20:02:59

Reverting 6de didn't help.

p-himik20:02:47

As expected. 🙂 Since the issue with with #ig/ref matching, something must've changed in integrant, and then the new version must've been used in edge.

dominicm20:02:18

maybe the composite keys method changed and I've been doing it wrong the whole time 🙂

dominicm20:02:20

annoyingly, it works in dev?

p-himik20:02:25

Just checked the versions - I use the same versions of integrant and aero. Hmm. Indeed it works in dev.

dominicm20:02:38

I can see my println firing indicating that edge.yada.ig is loading from the jar.

p-himik20:02:45

Ahh, a good old "Bohr bug".

dominicm20:02:05

that's a novel name for it 😄 I like it

dominicm20:02:59

heisenbug is the name I know

p-himik20:02:27

Yep, also mandelbug and schroedinbug.

dominicm20:02:34

mandelbug looks useful too, hanging onto that one

dominicm20:02:59

neat trick with onejars, if you pass -r you get a REPL 😄

p-himik20:02:32

Will keep in mind. 🙂

dominicm20:02:25

going to use that to figure out what is going on with the methods

dominicm20:02:19

hmm, this isn't right

dominicm20:02:38

user=> (ig/composite-keyword [:acme.test/assets :edge.yada.ig/resources])
:integrant.composite/acme.test.assets+edge.yada.ig.resources_10849
user=> (ancestors :integrant.composite/acme.test.assets+edge.yada.ig.resources_10849)
nil
user=> (descendants :integrant.composite/acme.test.assets+edge.yada.ig.resources_10849)
nil

dominicm20:02:49

that's particularly suspect as composite-keyword calls derive

dominicm20:02:56

okay, parents does give me better luck

dominicm20:02:05

no it doesn't

dominicm20:02:18

it works for random things like :a/a and :b/b, but not for [:acme.test/index :edge.yada.ig/classpath-name]

dominicm20:02:17

user=> (ig/composite-keyword [:a/a :c/c])
:integrant.composite/a.a+c.c_45572
user=> (parents *1)
#{:a/a :c/c}
user=> (ig/composite-keyword [:acme.test/index :edge.yada.ig/classpath-name])
:integrant.composite/acme.test.index+edge.yada.ig.classpath-name_10892
user=> (parents *1)
nil

dominicm20:02:42

I can't even explain this one

dominicm20:02:56

the code is simple, it shouldn't even be possible to fail.

dominicm20:02:03

nor is it consistent

dominicm20:02:12

could it be clojure 1.10 vs 1.9 maybe?

dominicm20:02:27

and I know that's a stretch, but I'm grasping at straws a little 🙂

dominicm20:02:36

it's a change that was made after your version

p-himik20:02:03

Certainly feels like a CLJ bug.

dominicm20:02:29

still failing on 1.9

dominicm20:02:57

Hmm, this time I managed to get parents though

dominicm20:02:08

maybe it's related to requiring the namespace first or not :thinking_face:

p-himik20:02:31

Just running the same code in a regular CLJ repl works fine.

dominicm20:02:23

Okay, if I call (edge.system/system-config {:profile :prod}) then when I do parents I get nil.

dominicm20:02:21

something about requiring the namespaces into the project causes this issue.

p-himik20:02:36

Not sure it that helps, but running a dev repl and (edge.system/system-config {:profile :prod}) doesn't reproduce the error.

dominicm20:02:50

it does help, it's very interesting

p-himik20:02:41

Was running just ../bin/rebel -A:dev:build.

p-himik21:02:01

From derive doc: "if [hierarchy] not supplied defaults to, and modifies, the global hierarchy". Just a wild guess - maybe at some point the global hierarchy is somehow replaced/wiped.

dominicm21:02:45

I can't think of anything that would clear the global hierarchy

p-himik21:02:39

underive madness. 😄

dominicm21:02:16

If I remove load-namespaces from edge.system, the composite key is valid

dominicm21:02:33

I guess I try calling load-namespaces now, and see if that clears the global hierarchy anywhere.

dominicm21:02:19

user=> (parents (ig/composite-keyword [:acme.test/assets :edge.yada.ig/resources]))
#{:edge.yada.ig/resources :acme.test/assets}
user=> (ig/load-namespaces (edge.system/system-config {:profile :prod}))
21:05:53.810 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
Hayo from edge.yada.ig
(acme.test.foo edge.yada.ig edge.bidi.ig)
user=>  (parents (ig/composite-keyword [:acme.test/assets :edge.yada.ig/resources]))
nil

dominicm21:02:31

well, that is enlightening indeed.

dominicm21:02:52

I still can't explain it, but we seem to have something.

p-himik21:02:39

Ideas: 1. Test with manually loading relevant namespaces 2. If that fails as well, maybe try defmethod or some other ways those keys are used.

dominicm21:02:58

this is what the global-hierarchy is after calling load-namespaces: {:parents #:ctx{:return #{:ctx/expr}}, :ancestors #:ctx{:return #{:ctx/expr}}, :descendants #:ctx{:expr #{:ctx/return}}}

dominicm21:02:10

I wonder what is doing something with :ctx

p-himik21:02:54

In clojure.tools.analyzer: (derive :ctx/return :ctx/expr).

dominicm21:02:09

that was quick 😄

p-himik21:02:19

Full-text search in IDE. 🙂

p-himik21:02:35

Almost - IDEA + Cursive plugin.

p-himik21:02:09

That's kinda strange that they used just :ctx as a namespace there. Seems to have a high chance of colliding with other code.

dominicm21:02:27

requiring acme.test.foo is sufficient to cause this issue.

p-himik21:02:02

So now for #2. Does defmethod ruin everything as well?

p-himik21:02:30

If not, it's probably some of the requires within acme.test.foo.

dominicm21:02:32

I'm going to comment out yada in acme.test.foo

p-himik21:02:09

OK man, I'm gonna go get some sleep. Have fun and good luck debugging it.

dominicm21:02:46

Thanks for the bug report, appreciate you spending so much time with edge 🙂

dominicm21:02:55

Looks like it is yada. Requiring yada causes the hierarchy to clear.

p-himik21:02:08

Yeah, definitely - I find it to be a rather interesting project.

dominicm21:02:23

It's a bug in yada 1.3.0-alpha7. We haven't put that into production yet, so looks like we have a reason not to 🙂

dominicm21:02:33

bug isn't present in alpha5

dominicm21:02:10

alpha6 is good too it seems, great. So a very small set of commits to review.

dominicm21:02:16

reverting the aleph upgrade solves it

dominicm21:02:28

aleph 0.4.6 has been so much trouble 😞

dominicm21:02:28

still a problem in 0.4.7-alpha4

dominicm21:02:00

Nope. I'm just confused. Bug is present in alpha6.

dominicm22:02:10

Bug is supposedly present all the way back to 1.2.15, so I've found something weird, but I have no idea what.

dominicm22:02:42

and there's no code outside of clojure/core.clj which is modifying global-hierarchy...