Fork me on GitHub
#component
<
2017-10-06
>
stex20:10:01

Didn't know that there was a component channel 🙂. I've been loving the idea of component, and structured all my projects with it. However, the namespace refresh seems quite unstable in my projects. Quite frequently it just breaks because it's unable to find some (random?) namespace. Does anyone here have similar problems? How did you solve them?

hiredman20:10:50

stex: refresh can often reveal structural problems that already exist

hiredman20:10:41

e.g. cyclical dependencies of one k ind or another, missing dependencies, using import and java constructors for records instead of require and the factory functions

stex21:10:22

@hiredman Interesting! Any idea how to debug such problems?

hiredman21:10:40

I would start by looking at the namespace named in the error message, and then trying to load that namespace with require in otherwise clean repl

stex21:10:53

@hiredman Starting a new repl 🙂. Hmm..

hiredman21:10:01

it is very possible you'll find that the code on disk isn't valid and won't compile in a fresh repl, and there was some left over state that was allowing it to compile for whatever reason

hiredman21:10:53

you can also, without a fresh repl, try (require 'whatever.ns :reload)

hiredman21:10:44

the refresh error messages tend to just point at the namespace that is a problem, without telling you want the problem is, where as if you manually load the code you will get a real error message that tells you what the error is

stex21:10:07

I'm quite sure the code in the namespaces is working correctly. I think it's probably more like a cyclical dependency or something.

hiredman21:10:40

does the code load in a fresh repl?

stex21:10:51

I've started a new REPL repl and required the namespace, which worked, than loaded everything and than required with the :reload flag and works too.

stex21:10:21

when I refresh it doesn't... But with manually requiring with reload works

hiredman21:10:05

if its a circular dependency in general, if you rare requiring via the ns form, you will get an error for circular deps, if you are doing something more complicated then you have taken your fate in to your own hands

stex21:10:28

What is "more complicated"? I only require via the ns form but never seen a warning

hiredman21:10:34

if you are only using ns, then I wouldn't worry about it

hiredman21:10:08

are you defining records, types, or using multi methods or protocols?

stex21:10:34

Only records, as components. But I don't use them directly, only the helper function which calls map->Web for example.

stex21:10:00

Hmmm. So, I've been trying to remove the require in the ns form of the function which cannot be found. It fixes that specific error-case, but than another namespace doesn't load. Have done this 8 times now and doesn't seem to stop

hiredman21:10:45

you may be walking back through a cycle, the require code tries to detect those and throw an error, but I dunno, there may be something that circumvents that

hiredman21:10:40

what build tool are you using? is refresh able to correctly find you source code?

stex21:10:19

I'm just using a simple leiningen setup, no special build steps. Well, refresh isn't able to find namespaces, so I don't think it can find all the source code...?

stex21:10:22

Not sure 😛

noisesmith21:10:23

@stex any chance your namespace is using a namespace that it doesn’t require, but is available because another namespace required it?

noisesmith21:10:36

this could easily go bad when refreshing (and is a bad idea anyway)

stex21:10:29

@noisesmith Sounds plausable. I'll start grepping the source files

noisesmith21:10:32

also, what is the specific error message you are seeing for the missing namespace?

stex21:10:37

@noisesmith I cannot find any usages of a namespace without requiring it first. Only clojure.string and http://clojure.java.io, but that is fine, right?

noisesmith21:10:52

no, but that isn’t your problem

noisesmith21:10:13

clojure.string isn’t loaded by default - so it can work to use it without a require in dev, but break when running from a jar

noisesmith21:10:47

http://clojure.java.io is used by clojure itself in bootstrapping, so that won’t break the same way (currently) but there’s no promise that implicitly relying on it being present won’t break in the future

noisesmith21:10:06

considering that the fix is just one line in a require form, it’s better to add it for consistency’s sake anyway

noisesmith21:10:42

so you are using slamhound to invoke c.t.n/reload ? if not, it could be that something slamhound is trying to do gets broken by the refresh

stex21:10:42

@noisesmith Yep, I've replaced both occurrences right away 👍. Doesn't solve reset though.

noisesmith21:10:00

what’s the slamhound process doing exactly?

noisesmith21:10:50

iirc slamhound messes with your namespaces, and I could see it going bonkers if someone deletes a namespace (and that’s something refresh does in order to get a clean state)

stex21:10:46

Hmm.. Seems like slamhound is something of cljr-refactor in cider... Let's try a simple lein repl session...

noisesmith21:10:33

OK - if you aren’t explicitly using slamhound to drive your refresh, I bet refresh and slamhound are getting in a spat 😄

stex21:10:01

I don't know what slamhound is actually, I'm just using cljr-refactor in cider 😛

stex21:10:15

Well, lein repl gives the same issue, no stacktrace though

noisesmith21:10:16

yeah, clj-refactor is built on slamhound iirc

noisesmith21:10:26

(pst) will show you the stack trace

noisesmith21:10:41

wouldn’t lein repl still pull in cider if you have it set up in your profiles.clj?

noisesmith21:10:56

or is the new cider injecting the plugin manually?

stex21:10:15

Cider is injecting the manually

stex21:10:21

:error-while-loading project.erp.agp.client

CompilerException java.lang.Exception: namespace 'project.erp.agp.parsers' not found, compiling:(project/erp/agp/client.clj:1:1)
user=> (pst)
CompilerException java.lang.Exception: namespace 'project.erp.agp.parsers' not found, compiling:(project/erp/agp/client.clj:1:1)
	clojure.core/throw-if (core.clj:5656)
	clojure.core/load-lib (core.clj:5742)
	clojure.core/load-lib (core.clj:5717)
	clojure.core/apply (core.clj:648)
	clojure.core/load-libs (core.clj:5774)
	clojure.core/load-libs (core.clj:5758)
	clojure.core/apply (core.clj:648)
	clojure.core/require (core.clj:5796)
	clojure.core/require (core.clj:5796)
	project.erp.agp.client/eval28368/loading--5569--auto----28369 (client.clj:1)
	project.erp.agp.client/eval28368 (client.clj:1)
	project.erp.agp.client/eval28368 (client.clj:1)
Caused by:
Exception namespace 'project.erp.agp.parsers' not found
nil

stex21:10:51

When I require them manually (directly after the other output) it just works...

stex21:10:56

user=> (require 'project.erp.agp.parsers)
nil
user=> (require 'project.erp.agp.parsers :reload)
nil

noisesmith21:10:56

@stex do you have a user.clj in that project by any chance (eg. it might have been created by a template)

stex21:10:09

Yep, i do use user.clj

stex21:10:33

There I load the (reset) functions and some other functions

noisesmith21:10:42

@stex I just found this > Adding a :main to your project.clj will enable AOT compilation and thus break clojure.tools.namespace/refresh. I worked around this by declaring my main in the :uberjar profile.

stex21:10:25

Yep, I've seen that. I've only added the main to the :uberjar profile.

noisesmith21:10:53

so you have no main or aot in other profiles?

stex21:10:11

Nop, just in the uberjar (just checked it)

stex21:10:35

Is it fine to load setup the reset function in the user.clj?

noisesmith21:10:44

sometimes there are shenanigans going on with user.clj, in particular if it helpfully preloads some of your namespaces, so it’s worth double checking it’s not trying to be too clever

noisesmith21:10:00

@stex the other gotcha with aot / refresh is that even if you only have aot / main set for uberjar, if you run a repl after uberjar without running clean first, you’ll get that same error

stex21:10:04

😮. For some reason the error disapears when I move the code from the user namespace to a new dev namespace

stex21:10:16

Oke, just to be sure it isn't the lein clean + uberjar thing I'll revert 🙂

stex21:10:28

Sweet, but weird 😄

stex21:10:33

Curious why this works

noisesmith21:10:01

do weird things, get weird results, some templates (eg. luminus) put sketchy stuff in user.clj

stex21:10:26

I don't like templates 🙂. Created user myself and it's super minimal

noisesmith21:10:51

oh, OK - well regardless of why I helped, glad I could help haha

stex21:10:18

(ns user
  (:import java.lang.Runtime)
  (:require [figwheel-sidecar.system :as sys]
            [com.stuartsierra.component.repl :refer [set-init system start stop reset]]
            [com.stuartsierra.component :as component]
            [project.utils.env :refer [env]]
            [project.components.system :refer [create-system-map]]))

(defrecord ScssCompiler []
  component/Lifecycle
  (start [this]
    (if-not (:process this)
      (do
        (println "Starting SCSS watch process...")
        (assoc this :process
               (.exec (Runtime/getRuntime) "bin/watch-sass")))
      this))
  (stop [this]
    (when-let [process (:process this)]
      (println "Stopping SCSS watch process...")
      (.destroy process))
    this))

(defn cljs []
  (sys/cljs-repl (:figwheel-system system)))

(defn dev-system [_]
  (assoc (create-system-map env)
         :scss-compiler (->ScssCompiler)
         :figwheel-system (sys/figwheel-system
                           (assoc-in (sys/fetch-config) [:data :build-ids] ["dev" "figwheel-test"]))
         :css-watcher (component/using
                       (sys/css-watcher {:watch-paths ["resources/public/css"]})
                       {:figwheel-server :figwheel-system})))

(set-init dev-system)

stex21:10:27

That's my user.clj, Do you find anything weird?

stex21:10:18

Oh, oke! It works now also when I lein clean and keep everything in the user.clj

noisesmith21:10:40

aha, so it was a caching issue

noisesmith21:10:50

that makes a lot more sense, that user.clj looks fine

noisesmith21:10:30

@stex instead of remembering to run lein clean, you could make sure the target-path of uberjar isn’t in the repl’s classpath

stex21:10:03

How do you do that specifically?

noisesmith21:10:08

by eg. setting a custom compile-path or target-path (forget the config name) under the uberjar profile

noisesmith21:10:18

lein help sample should reference it somewhere

stex21:10:40

Thanks man!!

stex22:10:18

Super happy I know it's a caching problem. I've been having these weird issues in all my projects for months now and was really really fed up with it

noisesmith22:10:40

@stex I’ve hit enough random leiningen caching problems (and wasted enough time on them) that I now double check if lein clean fixes my issue before even thinking about the actual bug if I have any problem with finding namespaces or classes

stex22:10:11

Haha, will remember that for the next time 😉

noisesmith22:10:18

but it’s so automatic that I sometimes forget to recommend it for others

stex22:10:19

Could have saved me a lot of time

stex22:10:56

Yep, i've tried a uberjar, ran reset and the issue reappeared. Fixed by lein clean

stex22:10:58

nice 🙂

stex22:10:25

It's late here already, going to bed. Thanks @noisesmith and @hiredman!! Much appreciated.