Fork me on GitHub
#beginners
<
2020-01-21
>
derpocious04:01:41

Hello, I am getting a strange error trying to write a spec for a function

derpocious04:01:46

This is the function:

derpocious04:01:56

(defn power [base exponent]
  (reduce * (repeat exponent base)))

derpocious04:01:17

and the spec things:

(s/fdef power
  :args (s/cat :base int? :exponent int?)
  :ret int?)

(stest/instrument `power)
(stest/summarize-results (stest/check `power))

derpocious04:01:39

but I get this Java number overflow error 😞

derpocious04:01:59

{:spec (fspec :args (cat :base int? :exponent int?) :ret int? :fn nil),
 :sym armstrong-numbers/power,
 :failure #error {
 :cause "integer overflow"
 :via
 [{:type java.lang.ArithmeticException
   :message "integer overflow"
   :at [clojure.lang.Numbers throwIntOverflow "Numbers.java" 1576]}]
 :trace
 [[clojure.lang.Numbers throwIntOverflow "Numbers.java" 1576]
  [clojure.lang.Numbers multiply "Numbers.java" 1960]
  [clojure.lang.Numbers$LongOps multiply "Numbers.java" 501]
  [clojure.lang.Numbers multiply "Numbers.java" 173]
  [clojure.core$_STAR_ invokeStatic "core.clj" 1016]
  [clojure.core$_STAR_ invoke "core.clj" 1008]
  [clojure.lang.Repeat reduce "Repeat.java" 74]
  [clojure.core$reduce invokeStatic "core.clj" 6823]
  [clojure.core$reduce invoke "core.clj" 6810]
  [armstrong_numbers$power invokeStatic "armstrong_numbers.clj" 10]
  [armstrong_numbers$power invoke "armstrong_numbers.clj" 9]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$apply invoke "core.clj" 660]
  [clojure.spec.test.alpha$check_call invokeStatic "alpha.clj" 293]
  [clojure.spec.test.alpha$check_call invoke "alpha.clj" 286]
  [clojure.spec.test.alpha$quick_check$fn__3069 invoke "alpha.clj" 309]
  [clojure.lang.AFn applyToHelper "AFn.java" 154]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$apply invoke "core.clj" 660]
  [clojure.test.check.properties$apply_gen$fn__1078$fn__1079 invoke "properties.cljc" 16]
  [clojure.test.check.properties$apply_gen$fn__1078 invoke "properties.cljc" 16]
  [clojure.test.check.rose_tree$fmap invokeStatic "rose_tree.cljc" 78]
  [clojure.test.check.rose_tree$fmap invoke "rose_tree.cljc" 74]
  [clojure.test.check.rose_tree$fmap$fn__544 invoke "rose_tree.cljc" 78]
  [clojure.core$map$fn__5851 invoke "core.clj" 2755]
  [clojure.lang.LazySeq sval "LazySeq.java" 42]
  [clojure.lang.LazySeq seq "LazySeq.java" 51]
  [clojure.lang.Cons next "Cons.java" 39]
  [clojure.lang.RT next "RT.java" 709]
  [clojure.core$next__5371 invokeStatic "core.clj" 64]
  [clojure.core$next__5371 invoke "core.clj" 64]
  [clojure.test.check$shrink_loop invokeStatic "check.cljc" 99]
  [clojure.test.check$shrink_loop invoke "check.cljc" 79]
  [clojure.test.check$failure invokeStatic "check.cljc" 125]
  [clojure.test.check$failure invoke "check.cljc" 112]
  [clojure.test.check$quick_check invokeStatic "check.cljc" 70]

ScArcher04:01:21

your return value may be larger than an int

derpocious04:01:21

do you know how I can say "is (Number. Java)" as a spec predicate?

ScArcher04:01:12

I'm not familiar with this stuff, I just noticed the int return type. If you pass valid ints in you could generate a larger than int return value.

ScArcher04:01:32

I think you'd want to use a "bigint" but I'm not sure how to spec that.

seancorfield04:01:46

Your function isn't a "total function" as written -- on primitive ints -- because <large int> to the <large int> power can be bigger than a <large int> can hold and generative testing is going to pump arbitrary ints into that function and so it will overflow.

seancorfield04:01:53

So you can't write a spec for it, as it stands, that will succeed under generative testing...

seancorfield04:01:06

You have to decide what you want the behavior to be here: should the multiplication promote to bigger integer types automatically? Should the input arguments be restricted to smaller numbers? Should the function catch the overflow and return some other value?

derpocious04:01:03

hmm ok, let's say I wanted it to take int's and return a Java bigInt

derpocious04:01:29

and maybe cap the generative testing at something large but not too large

derpocious04:01:48

is the with the :fn part of the spec?

seancorfield04:01:18

That's... cheating...

seancorfield04:01:49

If you want int's in and bigint out, you need to change how the function actually works @derpocious

seancorfield04:01:27

You can use *' instead of * to get promoting multiplication.

seancorfield04:01:41

Then you can specify a return type of decimal? I think

seancorfield04:01:28

Hmm, actually that won't be quite enough...

seancorfield04:01:42

The result could be either int? or decimal?

seancorfield04:01:12

Er, nope. decimal? is the wrong one... hang on...

solf04:01:38

number? works

seancorfield04:01:46

Yeah! That's what I'm looking for @andy.fingerhut!

seancorfield04:01:37

@derpocious Here we go:

PS C:\Users\seanc\clojure> clj -A:test
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (defn power [base exponent] (reduce *' (repeat base exponent)))
#'user/power
user=> (power 10 Long/MAX_VALUE)
4455508415646675013373597242420117818453694838130159772560668808816707086990958982033203334310070688731662890013605553436739351074980172000127431349940128178077122187317837794167991459381249N
user=> (s/fdef power
  :args (s/cat :base int? :exponent int?)
  :ret integer?)
user/power
user=> (stest/instrument `power)
[user/power]
user=> (power 10 Long/MAX_VALUE)
4455508415646675013373597242420117818453694838130159772560668808816707086990958982033203334310070688731662890013605553436739351074980172000127431349940128178077122187317837794167991459381249N
user=> (stest/summarize-results (stest/check `power))

seancorfield04:01:17

(I'm still waiting for it to complete... (reduce *' (repeat <large int> <value>)) can take a very long time!)

seancorfield04:01:09

(of course, it's possible I may run out of heap here too...)

Matti Uusitalo04:01:17

This makes a good case of whether you actually want to allow arbitrarily large numbers in your program. Often the results won't be useful anyway.

derpocious04:01:45

Thanks @seancorfield. When I run line test with int? int? and integer? the terminal just hangs...

derpocious04:01:18

should I limit so the numbers aren't arbitrarily large?

seancorfield05:01:36

It's not a very space-efficient function... it isn't "hanging", it's just taking a long time to run the test

seancorfield05:01:05

This is why generative testing should be separate from "unit testing"

andy.fingerhut05:01:30

The number of bits in the result could easily be proportional to 64 bits times the value of the exponent, and take O(exponent^2) time

derpocious05:01:13

ok but suppose for what I'm building right now I only need this function to work for numbers between -100 and 100

andy.fingerhut05:01:14

So you probably want to limit exponent to be at most a few thousand or so in testing

seancorfield05:01:22

Yup, gives you a good appreciation of Big O notation 🙂

seancorfield05:01:25

As written, base needs to be limited since it is the repeat factor.

derpocious05:01:30

hmm is that something you do in spec? I don't see anything about limiting in the docs

seancorfield05:01:39

You can specify that an int argument must be in a given range (s/int-in min max)

seancorfield05:01:56

But the question is all about what your function's behavior should be.

seancorfield05:01:01

That's up to you.

seancorfield05:01:23

What do you want the function to do. Then specify that behavior.

derpocious05:01:50

I do want to specify that min max behavior, but I'm not sure where to put it

seancorfield05:01:56

You can specify that base and exponent are only legal in a specific range.

derpocious05:01:03

(s/fdef power
  :args (s/cat :base int? (s/int-in 0 5)
               :exponent int?)
  :ret number?)

seancorfield05:01:42

Not a very useful function if it can't do (power 6 2) tho', eh?

derpocious05:01:24

lol this doesn't even compile though

seancorfield05:01:44

Oh, because your syntax is wrong.

seancorfield05:01:57

Use (s/int-in 0 5) instead of int?

derpocious05:01:10

ah ok thanks. 👍

derpocious05:01:36

hmm when I run the test it still doesn't finish though. How many is it generating?

seancorfield05:01:06

Hmm, I'm a little surprised that with (s/int-in 0 5) for :base it isn't terminating -- but generative testing can take a while

seancorfield05:01:03

I think maybe you didn't follow all the steps

PS C:\Users\seanc\clojure> clj -A:test
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (defn power [base exponent] (reduce *' (repeat base exponent)))
#'user/power
user=> (power 10 Long/MAX_VALUE)
4455508415646675013373597242420117818453694838130159772560668808816707086990958982033203334310070688731662890013605553436739351074980172000127431349940128178077122187317837794167991459381249N
user=> (s/fdef power
  :args (s/cat :base (s/int-in 0 5) :exponent int?)
  :ret integer?)
user/power
user=> (stest/instrument `power)
[user/power]
user=> (power 10 Long/MAX_VALUE)
Execution error - invalid arguments to user/power at (REPL:1).
10 - failed: (int-in-range? 0 5 %) at: [:base]
user=> (power 4 Long/MAX_VALUE)
7237005577332262210834635695349653859421902880380109739573089701262786560001N
user=> (stest/summarize-results (stest/check `power))
{:sym user/power}
{:total 1, :check-passed 1}
user=>
That completed almost immediately for me.

seancorfield05:01:26

But the important point here is: what do you actually want the function to be and what restrictions do you want to put on it?

seancorfield05:01:45

Then you have to determine how to spec that behavior and then how to test it.

seancorfield05:01:33

It's an illustration that an apparently simple function actually isn't -- because computer math isn't as simple as it looks 🙂

derpocious05:01:17

When I do it like this my repl hangs:

(s/fdef power
  :args (s/cat :base int? :exponent int?)
  :ret int?)

derpocious05:01:41

but if it limit the exponent also then I can get it to finish quickly

seancorfield05:01:12

That's because you've swapped base and exponent compared to the version you first posted.

seancorfield05:01:50

you originally had (repeat base exponent) but in the version you just linked to you have (repeat exponent base)

seancorfield05:01:04

My REPL session above mirrored your original code.

seancorfield05:01:42

Given the example-based test suite, I'm not sure why you're even trying to spec it. I mean, you could instrument it as part of the example-based (unit) tests and that will "work" -- because it won't run any generative tests.

derpocious05:01:33

hmm but I wanted to check that the it was returning the right thing

seancorfield05:01:19

:ret isn't just about "returning the right thing (type/range)" -- :fn and :ret are about function behavior which is what stest/check is testing, generatively.

derpocious05:01:28

isnt that the same thing though?

derpocious05:01:45

if the purpose / behavior of the funtion is to return the thing?

derpocious05:01:06

This is working for me now:

(s/fdef power
  :args (s/cat :base (s/int-in 0 10) 
               :exponent (s/int-in 0 20))
  :ret int?)

seancorfield05:01:51

Sure, that "works" but now you can't pass in negative arguments or anything "useful" in terms of values.

derpocious05:01:12

hmm what do you mean by useful values?

derpocious05:01:19

I can change it to this:

seancorfield05:01:21

For the purposes of Exercism, I'm sure your code works fine without Spec, and I expect it will get exercised with values outside that 0-10 and 0-20 range

derpocious05:01:27

(s/fdef power
  :args (s/cat :base (s/int-in -9 10) 
               :exponent (s/int-in 0 20))
  :ret int?)

derpocious05:01:58

but then won't that give the Java out of range error?

seancorfield05:01:54

user=> (defn power [base exponent] (reduce *' (repeat base exponent)))
#'user/power
user=> (power -9 0)
1
user=> (power -9 20)
1
user=> (power 10 0)
0
user=> (power 10 20)
10240000000000
user=>

seancorfield05:01:22

user=> (power 100 200)
126765060022822940149670320537600000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000N
user=>
This is also fine -- without the Spec.

seancorfield05:01:15

Even (power 1000 1000) will work just fine -- with *'

seancorfield05:01:51

With *, this is also OK

user=> (power 13 27)
4052555153018976267

seancorfield05:01:21

(in fact (power 13 28) is OK too)

derpocious05:01:34

hmm so what should I do? 😕

derpocious05:01:47

should I not run spec/check?

derpocious05:01:59

should I not put :ret in the fdef?

seancorfield05:01:10

Like I said, you need to think carefully about exactly what semantics you want to guarantee for your function and Spec it accordingly.

seancorfield05:01:26

And then determine how you want to test it.

derpocious05:01:15

I just want it to say that it takes some integers and returns some integers. I don't understand why spec makes it so difficult

derpocious05:01:51

If I'm not running the tests then can't anyone change the return value from int? to :ret string? and then nothing will catch it?

seancorfield05:01:56

Spec isn't making this difficult. "takes some integers and returns some integers" isn't a useful definition of the behavior.

seancorfield05:01:57

instrument can check how a function is called. check exercises the function in isolation according to its Spec by trying lots of different conforming arguments and verifying that the result conforms. It's much more than "checking the result type"

seancorfield05:01:19

Since you're writing the function, you know it can't possibly return a string.

seancorfield05:01:53

What benefit would a :post assertion to that effect give you, switching from Spec to another feature of Clojure that is built-in?

seancorfield05:01:57

{:pre [#(and (int? base) (int? exponent))] :post [#(integer? %]} would be assertions to just check the input types and result types.

seancorfield06:01:03

Spec is more than that.

seancorfield06:01:29

This comes down, in part, to the whole issue of static types vs predicate-based specifications.

seancorfield06:01:17

Spec is not a type system. But even in this case, what value do the types on arguments and return values give you here? If you pass input values that can't be used in repeat and *' you'll get an exception. If you pass input values that can be used in repeat and *' then you will get an integer? out: that's the only possible return type. You can't ever get a string and you've assured now that you can't get an exception either (which was the original issue with *, regardless of the return "type").

derpocious06:01:49

yes but you only know that after understanding the entire function. what if I want my friends to call my power function? How do they know if it returns an int or an integer or a bool or something else?

derpocious06:01:35

then they have to understand the magical tick-star multiplication and whatever else could be in the function

andy.fingerhut06:01:09

You can use spec to document such properties as types such as int? boolean?, certainly. Perhaps another way to say what Sean is saying is that you aren't getting the full value of spec using it only for that.

andy.fingerhut06:01:36

You don't have to get the full value of it in order to get a little value from it.

andy.fingerhut06:01:09

Just realize that it can do quite a bit more than that.

seancorfield06:01:13

As written, your function works for natural number exponents but for arbitrary number bases -- not just int?.

seancorfield06:01:39

user=> (defn power [base exponent] (reduce * (repeat exponent base)))
#'user/power
user=> (power 3.3 4)
118.59209999999999
user=>
So your Spec isn't right anyway, trying to restrict base to int?

andy.fingerhut06:01:03

well, it is not as general as the code supports.

seancorfield06:01:13

And exponent could be a big integer too

user=> (power 3.3M 4N)
118.5921M
user=>

andy.fingerhut06:01:40

Some Clojure doc strings promise less than they implement.

seancorfield06:01:20

Right, but my point here is that the Spec isn't much good unless you think about the actually semantics of the function -- do you intend for it to work for "anything" that the code supports? Just a subset of those types? Just a subset of those values? (two different things -- 4 vs 4N for example). Can the function throw an exception? (Spec only specifies valid behaviors -- it can't spec invalid behaviors)

seancorfield06:01:01

And "math" functions are notoriously hard to specify correctly -- because of overflow, and representation limits.

seancorfield06:01:04

I would probably never even try to specify a mathematical function because of that. Especially a one-line, simple function.

seancorfield06:01:40

I'd just write a docstring.

andy.fingerhut06:01:08

I suspect we very well might be in violent agreement on the fundamentals here -- I doubt I would bother using spec for fairly simple functions, either, except perhaps for beginning practice in using spec.

andy.fingerhut06:01:47

It does often seem worthwhile to me for a spec of a function to restrict its inputs more than the function implementation supports, simply because that is all you need the function to do, and all you plan to support for the foreseeable future (until and unless requirements change). Also, the restrictive inputs you put into a spec might be all you want to spec's random test case generation to exercise.

seancorfield06:01:16

Right, which was why I questioned Jim's "takes some integers and returns some integers" -- because that doesn't say enough about the function inputs or outputs to be useful for Spec -- it's fine for type-based pre/post assertions.

seancorfield06:01:18

Anyways, I need to go and help my wife (recently out of knee surgery) so I'm calling it a night. I hope this was a useful discussion and didn't just seem argumentative...

derpocious14:01:51

Thanks! sorry I disappeared last night. I will check out :pre / :post and truss... man it seems every day I discover a new thing in Clojure that is kind of an alternative to static types sorta but not really... 😅

andy.fingerhut06:01:50

Likewise from me. Take care of her.

Eric Ihli12:01:20

Am I misunderstanding something about module resolution?

(defonce json-data (js/require "../../assets/number-to-word-tree.json"))
(defonce number-to-word-tree (js/require "../../assets/number-to-word-tree-module")) ;; This file has a .js extension. I've tried including the extension. Same result.
The first line works. The second line errors with:
Error: Unable to resolve module `../../assets/number-to-word-tree-module` from `app/index.js`:

None of these files exist:
  * ../assets/number-to-word-tree-module(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
  * ../assets/number-to-word-tree-module/index(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
The first file is just plain json. The second is a module version of it.
const foo = <stringified-json>;
export default foo;
I'm using shadow-cljs and react native. Both files exist and have the same owner/permissions. I've cleared every cache and reset every process I can think to try.

Eric Ihli15:01:58

For anyone who comes across this: it was resolved. ReactNative (metro?) can't import modules from the assets path. Moving the module to the same directory of my compiled output fixed the problem.

dharrigan13:01:31

When destructuring and testing a map, this doesn't work as I expect (if-let [{{:keys [foo]} :bar} baz] :found :not-found) if baz is not empty, i.e., (def baz {:bar {:not-foo "wibble"}}). Onlly if the map itself (`baz` in this case) is nil would the result be :not-found. Is there a way to destructure a map and test if the key is contained within the map (and pull out the value) in one go?

dharrigan13:01:06

(I've resorted to using (let [....] (if foo :found :not-found))

dharrigan14:01:25

This will work - using the (get-in ....) function

practicalli-johnny14:01:50

I am sure there is a simple function for the transformation I want. I have a data structure like this:

[{:a {:x 0 :y 5}
  :b {:z 2 :zz 1}}
 {:a {:x 0 :y 5}
  :b {:z 7 :zz 8}}]
and want just the maps containing the :b key
[{:b {:z 2 :zz 1}}
 {:b {:z 7 :zz 8}}]
I thought it was just a reduce with the key, but it doest seem to do anything... any suggestions?

Luke Schubert14:01:45

that looks like you want map

Luke Schubert15:01:27

unless you want to filter out maps like {:a {:x 0}} that don't have a :b key

practicalli-johnny15:01:17

Oh, yes thats so obvious now. One day I wont get map and reduce mixed up :)

lispyclouds15:01:09

something like (->> d (map #(select-keys % [:b])) (filter seq)) ?

lispyclouds15:01:17

d is the data

practicalli-johnny15:01:04

I just used

(map :b ,,,) 

4
practicalli-johnny15:01:08

I think I was trying to overthink it again 🙂 Thanks for the help

Luke Schubert15:01:02

is using seq to detect an empty sequence idiomatic?

oliver18:01:41

Hi, I thought I'd post this in the beginners' channel since I'm in need of some basic orientation right now… I've had quite a blast developing my first website with ClojureScript (http://violina-petrychenko.de/en/) using the Macchiato framework for the server (https://macchiato-framework.github.io/). The frontend being very basic vanilla JS I was now planning to use ClojureScript for the frontend as well… now I'm in tooling hell: It seems that Macchiato's Leinigen template is not geared towards CIDER (I've articulated this in more detail here: https://stackoverflow.com/questions/59832391). My options seem to be: 1. either tweak the template until it plays nice with CIDER (swapping figwheel for fighweel-main to begin with) or 2. build my own template from scratch. I've been trying the first approach and just found myself dabbling at something I didn't really understand… so as for the second approach I'm unsure if Leiningen is even the best fit for my setup. Before opting for macchiato (and its lein template) I had begun learning Boot, so I actually have a better understanding of its workings, but I'm unsure if I'll ever be able to make a build.boot that would match Macchiato's impressively intricate project.clj (including its custom lein plugin)… do you have any suggestions on how a rookie like me should proceed at this point? Many thanks! Oliver (Clojure(script) is my first language requiring a build tool so I have vey little experience to draw on in this department)

dpsutton18:01:55

i'm gonna clone the template at lunch and investigate

oliver18:01:19

Thanks a lot!

oliver19:01:11

Oh, I see CIDER is your repo… I'm really grateful you're taking the time for this.

dpsutton19:01:26

Not mine :) I just love contributing and helping

oliver19:01:13

Ok, still you seem to quite knowledgeable about the CIDER codebase… I'm really impressed by the Clojure (and Elisp) Community in general… let me know if I can specify anything beyond my StackOverflow question (and be sure to use the +browser profile when cloning the template… the problems only become relevant when trying to run different profiles from within Emacs)

dpsutton19:01:42

so i'm unable to get this to work. The readme states you can run npm start but there's no package.json file. There seems to be no index.html file that this can serve. presumably this is served by the js backend . I ran lein build-browser but that again compiles the frontend, no idea how to get the backend running. But I think i can help you with two points to get going. Remove all references to cemerick/piggieback, both the wrap suff and the dependency itself. Then, your other problem is how to jack in with a lein profile. To do so, add a prefix argument. Here, cider-jack-in-cljs is C-c M-J so add C-u to the beginning C-u C-c M-J and you can edit the jack in command. update it so its lein with-profile +browser-dev [all the rest of the stuff] or whatever profile you need

dpsutton19:01:32

if this were me, i would start from scratch with shadow-cljs and slowly add in the things you want. get a simple macchiato route responding with text and build out from there. shadow-cljs will make this all much easier since you just need to select which build you need

oliver20:01:08

Hi, many thanks for looking into this! Ive actually managed to start both backend and frontent using three terminals (Im afk right now, so i hope I'm getting this right) 1. lein build 2. lein build-browser 3. node target/out/nameofapp.js (to run build 1) I can then perfectly open localhost:3000 and find the HTML dynamically generated by build 1 which refers to the backend of build 2. I've even managed to connect to these repls with cider-connect-cljs... but it complains about cider-nrepl missing (tried adding it to no avail)... and while i get the repls within Emacs the buffers are bot connected

oliver20:01:52

Ive also tried lein with-profile +browser-dev...(only without the +, I'll try that as soon as I'm back home)

oliver20:01:47

Again thanks for having taken the time... I'm even somewhat relieved that this didn't turn out to be a no-brainer