Fork me on GitHub
#speculative
<
2020-01-24
FiVo12:01:46

Hey there, I am running into some issues with speculative. Here is a small example to reproduce the issue

(set! *print-level* 10)
  (set! *print-length* 10)
  (require '[clojure.tools.analyzer.jvm :as ana])
  (require '[clojure.tools.deps.alpha.util.maven :as mvn])
  (require '[clojure.tools.deps.alpha.repl :refer [add-lib]]) ;; using the add-lib branch of tools.deps.alpha
  (require 'speculative.instrument) ;; without loading speculative it works
  (add-lib 'edos {:mvn/version "0.1.0-SNAPSHOT"})
  (ana/analyze-ns 'edos.core)
I am actually not using speculative in my project itself but rather analyzing some clojure code and one of the projects is speculative. I am doing something along the lines of
(require 'speculative.instrument) 
(ana/analyze-ns 'speculative.instrument)
(remove-ns 'speculative.instrument)
Is speculative like the clojure namespace and can not be removed?

borkdude12:01:04

> ;; without loading speculative it works what works?

borkdude12:01:15

what is going wrong?

FiVo12:01:48

@borkdude I am getting an ClassCastException

FiVo12:01:58

Execution error (ClassCastException) at speculative.core/fn (core.cljc:274).
class clojure.lang.LazySeq cannot be cast to class java.util.Map$Entry (clojure.lang.LazySeq is in unnamed module of loader 'app'; java.util.Map$Entry is in module java.base of loader 'bootstrap')

FiVo12:01:49

The last line of the first code snippet fails.

borkdude12:01:01

speculative.core/fn ... hmm, not sure what that is and this is at core.cljc 274: https://github.com/borkdude/speculative/blob/master/src/speculative/core.cljc#L274 I have no clue if I have to go by just that snippet. Maybe you can post the entire exception somewhere?

borkdude12:01:22

You can also try to call (speculative.instrument/unstrument) before removing the ns, maybe that helps

borkdude12:01:02

remove-ns just deletes an in-memory object, but maybe spec instrumentation breaks when you try to do that

borkdude12:01:19

also it seems you're using jdk11. do you get the same error with jdk8?

FiVo13:01:31

It's the spec for map

FiVo13:01:45

One of the key access fails

borkdude13:01:49

can you put a println before that key access to see what is the value of args or ret?

FiVo13:01:44

output of having added (println args ret)

[:seqable {:f #object[clojure.tools.analyzer.utils$dissoc_env 0x24f3d208 clojure.tools.analyzer.utils$dissoc_env@24f3d208], :colls [[{:env {:locals {}, :ns edos.core, :file jar:file:/home/fv/.m2/repository/edos/edos/0.1.0-SNAPSHOT/edos-0.1.0-SNAPSHOT.jar!/edos/core.cljc, :end-column 33, :column 1, :line 156, :once false, :end-line 162, :context :ctx/expr}, :form events, :name events, :variadic? true, :op :binding, :arg-id 0, :local :arg}]]}] ({:form events, :name events, :variadic? true, :op :binding, :arg-id 0, :local :arg})

borkdude13:01:31

so it seems ret here is a lazy-seq instead of a map-entry...

FiVo13:01:58

no changes with respect to jdk8

FiVo14:01:54

The edos lib I am analyzing depends on orchestra. Is this somehow incompatible? But more of a guess?

borkdude14:01:13

ehm, yeah could very well be. can you try with plain spec?

borkdude14:01:35

fwiw the specs in speculative are tested quite well, even using generative testing

borkdude14:01:26

user=> (instance? clojure.lang.MapEntry (s/conform (s/or :seqable seqable? :transducer any?) (map inc [1 2 3])))
true
user=> (instance? clojure.lang.MapEntry (s/conform (s/or :seqable seqable? :transducer any?) (map inc)))
true

borkdude14:01:54

@finn.volkel There was an issue about this exact problem here: https://github.com/jeaye/orchestra/issues/26

FiVo14:01:22

Ok. I am not familiar enough with spec. But why would this only surface if one adds speculative?

FiVo14:01:37

What do you mean by "trying with plain spec"?

borkdude14:01:10

because speculative has specs for clojure.core. if a library patches spec incorrectly, you'll get problems when you run core functions

borkdude14:01:55

what are you trying to accomplish? do you actually want to run with clojure.core specs instrumentation or do you just want to analyze code?

FiVo14:01:09

Just analyze code and learn something along the way, but I get your point that issue is somewhere else.

borkdude14:01:33

speculative.instrument is a "special" namespace that enables instrumentation when required.

borkdude14:01:54

so it would be better to not require that namespace if you don't want instrumentation

FiVo14:01:55

The main thing is I guess why speculative still shows up even if I used remove-ns

borkdude14:01:13

because by then the namespace has already been loaded

borkdude14:01:39

sorry my bad, it doesn't do that automatically, but only when you call i/instrument

borkdude14:01:30

I think your workflow is as follows: you load all specs from speculative (by requiring speculative.instrument) and orchestra instruments them. Now when you call a core function, you run into a bug with orchestra.

borkdude14:01:38

removing namespaces does not unstrument functions.

borkdude14:01:42

but maybe you can get rid of the problem by calling i/unstrument before doing anything else after loading speculative.instrument

borkdude14:01:00

(still not clear to me why you would load this in the first place)

FiVo14:01:56

I ran into the issue because I analysed speculativewith analyze-ns itself. I am just pulling jars with the clojars api, analyze the namespaces and then remove them again. I guess the easiest thing is to blacklist speculative in this case. I don't know though if the problem will show up again if some project depends on speculative .

FiVo14:01:36

thanks for the help anyway

borkdude14:01:06

@finn.volkel but if you're not actively instrumenting the clojure.core specs from speculative, how could you run into a bug with one of them?

FiVo14:01:30

I should maybe add that analyze-ns also evaluates the stuff in the namespace.

borkdude14:01:08

does it execute snippets from comment sections?

borkdude14:01:50

or does orchestra have a hook that once you enable instrumentation and a new spec gets loaded it's automatically instrumented? something fishy there

FiVo14:01:57

The end of the stacktrace looks as follows:

[[clojure.core$key invokeStatic core.clj 1567]
  [clojure.core$key invoke core.clj 1567]
  [speculative.core$fn__106968 invokeStatic NO_SOURCE_FILE 102]
  [speculative.core$fn__106968 invoke NO_SOURCE_FILE 101]
  [clojure.spec.alpha$spec_impl$reify__2059 conform_STAR_ alpha.clj 923]
  [clojure.spec.alpha$conform invokeStatic alpha.clj 164]
  [clojure.spec.alpha$conform invoke alpha.clj 160]
  [orchestra.spec.test$spec_checking_fn$conform_BANG___391943 invoke test.clj 97]
  [orchestra.spec.test$spec_checking_fn$fn__391949 doInvoke test.clj 129]
  [clojure.lang.RestFn invoke RestFn.java 421]
  [clj_astminer.retrieve$analyze_from_clojar_map invokeStatic retrieve.clj 133]
don't know if that helps. Goes through orchestra as well as speculative

borkdude14:01:50

yeah, so somehow something has instrumented the specs in speculative.

FiVo14:01:59

I don't think analyze-ns evaluates anything in the comments.

borkdude14:01:44

maybe disable orchestra 🙂

borkdude14:01:08

if you have a repro using vanilla clojure spec and speculative, an issue is welcome

borkdude14:01:27

which version of orchestra are you using?

FiVo14:01:29

well orchestra is just loaded via edos which uses [orchestra "2018.09.10-1"]

borkdude14:01:40

ah, that explains it then

borkdude14:01:52

it's an old version which still has that bug from the issue I linked to earlier

FiVo14:01:02

I see thanks