Fork me on GitHub
#component
<
2017-10-06
>
Stefan Roex20:10:01

Didn't know that there was a component channel 🙂. I've been loving the idea of component, and structured all my projects with it. However, the namespace refresh seems quite unstable in my projects. Quite frequently it just breaks because it's unable to find some (random?) namespace. Does anyone here have similar problems? How did you solve them?

hiredman20:10:50

stex: refresh can often reveal structural problems that already exist

hiredman20:10:41

e.g. cyclical dependencies of one k ind or another, missing dependencies, using import and java constructors for records instead of require and the factory functions

Stefan Roex21:10:22

@hiredman Interesting! Any idea how to debug such problems?

hiredman21:10:40

I would start by looking at the namespace named in the error message, and then trying to load that namespace with require in otherwise clean repl

Stefan Roex21:10:53

@hiredman Starting a new repl 🙂. Hmm..

hiredman21:10:01

it is very possible you'll find that the code on disk isn't valid and won't compile in a fresh repl, and there was some left over state that was allowing it to compile for whatever reason

hiredman21:10:53

you can also, without a fresh repl, try (require 'whatever.ns :reload)

hiredman21:10:44

the refresh error messages tend to just point at the namespace that is a problem, without telling you want the problem is, where as if you manually load the code you will get a real error message that tells you what the error is

Stefan Roex21:10:07

I'm quite sure the code in the namespaces is working correctly. I think it's probably more like a cyclical dependency or something.

hiredman21:10:40

does the code load in a fresh repl?

Stefan Roex21:10:51

I've started a new REPL repl and required the namespace, which worked, than loaded everything and than required with the :reload flag and works too.

Stefan Roex21:10:21

when I refresh it doesn't... But with manually requiring with reload works

hiredman21:10:05

if its a circular dependency in general, if you rare requiring via the ns form, you will get an error for circular deps, if you are doing something more complicated then you have taken your fate in to your own hands

Stefan Roex21:10:28

What is "more complicated"? I only require via the ns form but never seen a warning

hiredman21:10:34

if you are only using ns, then I wouldn't worry about it

hiredman21:10:08

are you defining records, types, or using multi methods or protocols?

Stefan Roex21:10:34

Only records, as components. But I don't use them directly, only the helper function which calls map->Web for example.

Stefan Roex21:10:00

Hmmm. So, I've been trying to remove the require in the ns form of the function which cannot be found. It fixes that specific error-case, but than another namespace doesn't load. Have done this 8 times now and doesn't seem to stop

hiredman21:10:45

you may be walking back through a cycle, the require code tries to detect those and throw an error, but I dunno, there may be something that circumvents that

hiredman21:10:40

what build tool are you using? is refresh able to correctly find you source code?

Stefan Roex21:10:19

I'm just using a simple leiningen setup, no special build steps. Well, refresh isn't able to find namespaces, so I don't think it can find all the source code...?

Stefan Roex21:10:22

Not sure 😛

noisesmith21:10:23

@stex any chance your namespace is using a namespace that it doesn’t require, but is available because another namespace required it?

noisesmith21:10:36

this could easily go bad when refreshing (and is a bad idea anyway)

Stefan Roex21:10:29

@noisesmith Sounds plausable. I'll start grepping the source files

noisesmith21:10:32

also, what is the specific error message you are seeing for the missing namespace?

Stefan Roex21:10:37

@noisesmith I cannot find any usages of a namespace without requiring it first. Only clojure.string and http://clojure.java.io, but that is fine, right?

noisesmith21:10:52

no, but that isn’t your problem

noisesmith21:10:13

clojure.string isn’t loaded by default - so it can work to use it without a require in dev, but break when running from a jar

noisesmith21:10:47

http://clojure.java.io is used by clojure itself in bootstrapping, so that won’t break the same way (currently) but there’s no promise that implicitly relying on it being present won’t break in the future

noisesmith21:10:06

considering that the fix is just one line in a require form, it’s better to add it for consistency’s sake anyway

noisesmith21:10:42

so you are using slamhound to invoke c.t.n/reload ? if not, it could be that something slamhound is trying to do gets broken by the refresh

Stefan Roex21:10:42

@noisesmith Yep, I've replaced both occurrences right away 👍. Doesn't solve reset though.

noisesmith21:10:00

what’s the slamhound process doing exactly?

noisesmith21:10:50

iirc slamhound messes with your namespaces, and I could see it going bonkers if someone deletes a namespace (and that’s something refresh does in order to get a clean state)

Stefan Roex21:10:46

Hmm.. Seems like slamhound is something of cljr-refactor in cider... Let's try a simple lein repl session...

noisesmith21:10:33

OK - if you aren’t explicitly using slamhound to drive your refresh, I bet refresh and slamhound are getting in a spat 😄

Stefan Roex21:10:01

I don't know what slamhound is actually, I'm just using cljr-refactor in cider 😛

Stefan Roex21:10:15

Well, lein repl gives the same issue, no stacktrace though

noisesmith21:10:16

yeah, clj-refactor is built on slamhound iirc

noisesmith21:10:26

(pst) will show you the stack trace

noisesmith21:10:41

wouldn’t lein repl still pull in cider if you have it set up in your profiles.clj?

noisesmith21:10:56

or is the new cider injecting the plugin manually?

Stefan Roex21:10:15

Cider is injecting the manually

Stefan Roex21:10:21

:error-while-loading project.erp.agp.client

CompilerException java.lang.Exception: namespace 'project.erp.agp.parsers' not found, compiling:(project/erp/agp/client.clj:1:1)
user=> (pst)
CompilerException java.lang.Exception: namespace 'project.erp.agp.parsers' not found, compiling:(project/erp/agp/client.clj:1:1)
	clojure.core/throw-if (core.clj:5656)
	clojure.core/load-lib (core.clj:5742)
	clojure.core/load-lib (core.clj:5717)
	clojure.core/apply (core.clj:648)
	clojure.core/load-libs (core.clj:5774)
	clojure.core/load-libs (core.clj:5758)
	clojure.core/apply (core.clj:648)
	clojure.core/require (core.clj:5796)
	clojure.core/require (core.clj:5796)
	project.erp.agp.client/eval28368/loading--5569--auto----28369 (client.clj:1)
	project.erp.agp.client/eval28368 (client.clj:1)
	project.erp.agp.client/eval28368 (client.clj:1)
Caused by:
Exception namespace 'project.erp.agp.parsers' not found
nil

Stefan Roex21:10:51

When I require them manually (directly after the other output) it just works...

Stefan Roex21:10:56

user=> (require 'project.erp.agp.parsers)
nil
user=> (require 'project.erp.agp.parsers :reload)
nil

noisesmith21:10:56

@stex do you have a user.clj in that project by any chance (eg. it might have been created by a template)

Stefan Roex21:10:09

Yep, i do use user.clj

Stefan Roex21:10:33

There I load the (reset) functions and some other functions

noisesmith21:10:42

@stex I just found this > Adding a :main to your project.clj will enable AOT compilation and thus break clojure.tools.namespace/refresh. I worked around this by declaring my main in the :uberjar profile.

Stefan Roex21:10:25

Yep, I've seen that. I've only added the main to the :uberjar profile.

noisesmith21:10:53

so you have no main or aot in other profiles?

Stefan Roex21:10:11

Nop, just in the uberjar (just checked it)

Stefan Roex21:10:35

Is it fine to load setup the reset function in the user.clj?

noisesmith21:10:44

sometimes there are shenanigans going on with user.clj, in particular if it helpfully preloads some of your namespaces, so it’s worth double checking it’s not trying to be too clever

noisesmith21:10:00

@stex the other gotcha with aot / refresh is that even if you only have aot / main set for uberjar, if you run a repl after uberjar without running clean first, you’ll get that same error

Stefan Roex21:10:04

😮. For some reason the error disapears when I move the code from the user namespace to a new dev namespace

Stefan Roex21:10:16

Oke, just to be sure it isn't the lein clean + uberjar thing I'll revert 🙂

Stefan Roex21:10:28

Sweet, but weird 😄

Stefan Roex21:10:33

Curious why this works

noisesmith21:10:01

do weird things, get weird results, some templates (eg. luminus) put sketchy stuff in user.clj

Stefan Roex21:10:26

I don't like templates 🙂. Created user myself and it's super minimal

noisesmith21:10:51

oh, OK - well regardless of why I helped, glad I could help haha

Stefan Roex21:10:18

(ns user
  (:import java.lang.Runtime)
  (:require [figwheel-sidecar.system :as sys]
            [com.stuartsierra.component.repl :refer [set-init system start stop reset]]
            [com.stuartsierra.component :as component]
            [project.utils.env :refer [env]]
            [project.components.system :refer [create-system-map]]))

(defrecord ScssCompiler []
  component/Lifecycle
  (start [this]
    (if-not (:process this)
      (do
        (println "Starting SCSS watch process...")
        (assoc this :process
               (.exec (Runtime/getRuntime) "bin/watch-sass")))
      this))
  (stop [this]
    (when-let [process (:process this)]
      (println "Stopping SCSS watch process...")
      (.destroy process))
    this))

(defn cljs []
  (sys/cljs-repl (:figwheel-system system)))

(defn dev-system [_]
  (assoc (create-system-map env)
         :scss-compiler (->ScssCompiler)
         :figwheel-system (sys/figwheel-system
                           (assoc-in (sys/fetch-config) [:data :build-ids] ["dev" "figwheel-test"]))
         :css-watcher (component/using
                       (sys/css-watcher {:watch-paths ["resources/public/css"]})
                       {:figwheel-server :figwheel-system})))

(set-init dev-system)

Stefan Roex21:10:27

That's my user.clj, Do you find anything weird?

Stefan Roex21:10:18

Oh, oke! It works now also when I lein clean and keep everything in the user.clj

noisesmith21:10:40

aha, so it was a caching issue

noisesmith21:10:50

that makes a lot more sense, that user.clj looks fine

noisesmith21:10:30

@stex instead of remembering to run lein clean, you could make sure the target-path of uberjar isn’t in the repl’s classpath

Stefan Roex21:10:03

How do you do that specifically?

noisesmith21:10:08

by eg. setting a custom compile-path or target-path (forget the config name) under the uberjar profile

noisesmith21:10:18

lein help sample should reference it somewhere

Stefan Roex22:10:18

Super happy I know it's a caching problem. I've been having these weird issues in all my projects for months now and was really really fed up with it

noisesmith22:10:40

@stex I’ve hit enough random leiningen caching problems (and wasted enough time on them) that I now double check if lein clean fixes my issue before even thinking about the actual bug if I have any problem with finding namespaces or classes

Stefan Roex22:10:11

Haha, will remember that for the next time 😉

noisesmith22:10:18

but it’s so automatic that I sometimes forget to recommend it for others

Stefan Roex22:10:19

Could have saved me a lot of time

Stefan Roex22:10:56

Yep, i've tried a uberjar, ran reset and the issue reappeared. Fixed by lein clean

Stefan Roex22:10:25

It's late here already, going to bed. Thanks @noisesmith and @hiredman!! Much appreciated.