Fork me on GitHub
#beginners
<
2016-12-28
>
roelof09:12:51

I m thinking of doing the spec files into there own namespace like paintings.api-get-specs

roelof09:12:38

Can I then in paintings.api-get use require to use api-get-specs and in api-get-specs require api-get

roelof09:12:54

or do I getr a circular errror ?

agile_geek10:12:04

You'd usually keep Specs in the same namespace as the functions that use them so that the keywords in the specs are in the same namespace

roelof10:12:04

oke, then I have to find a way to seperate the specs from the normal code. The file gets very big then ?

agile_geek10:12:01

Shouldn't be that big

roelof10:12:03

and I have to make input validation and testing output still in it

roelof10:12:56

@agile_geek Do you still think I can put all the specs into this one file ?

agile_geek10:12:48

That namespace isn't very big

roelof10:12:18

oke, and maybe use a (comment ) to seperate the two

agile_geek10:12:06

I would keep specs together with fn's that use them and if the ns gets to big split them a different way. For example, you could split the fns that call the api from the functions that destructure the data from the calls if you wanted. However, at the moment that code is just a few fn's so I would just leave them together for now.

agile_geek10:12:11

If you want put your specs at the top of the file and the fns after. If you want to separate with comments use ;; not (comment ...) as the first version is a better visual cue and it tends to highlight in different colours from a normal fn or macro in some editors which makes it stand out even more.

roelof10:12:36

oke, thanks, I will try it that way

roelof10:12:13

so ;; these are the specs files and later ;; these are the source files

agile_geek10:12:43

Something like that but most ppl reading it will have no problem separating the two anyway. Here's what I mean about highlighting comments.

roelof10:12:10

What do I do here not right :

;; the specs files for input validation

(def id-regex #" [a-z]{2}-[a-z]-\d{1,2}")

(s/def ::id-type (s/and string? #(re-matches id-regex %)))

(s/def ::id ::id-type)

(s/def ::id-list (s/keys :req [::id]))

(defn read-numbers
  "Reads the ids of the paintings"
  [response]
  (let [response-checked (s/conform ::id-list response)]
    (response-checked)))
 

roelof10:12:37

repl :

(read-numbers 1)
IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
(read-numbers "1")
IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
(read-numbers {:roelof 1})
IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
 

roelof10:12:05

read-numbers has only 1 parameter so why a wrong number of args ?

agile_geek10:12:37

Your spec for ::id-list says your argument has to be a vector of ::id's

roelof10:12:07

oke, but this is also not working :

IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
(read-numbers [:id 1])
IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
(read-numbers [1])
IllegalArgumentException Wrong number of args passed to keyword: :clojure.spec/invalid  clojure.lang.Keyword.throwArity (Keyword.java:97)
 

agile_geek10:12:36

Hmm, I miss read your code...

roelof10:12:39

then I have to change the spec for id-list. It schould check for this [{ :id 1}{:id 2}]

agile_geek11:12:24

Question: Why do you define ::id-type and then redefine it unchanged as ::id?

agile_geek11:12:35

Not sure it's wrong but don't see the point

roelof11:12:07

oke, I was doing that because I followed the spec guide guide : http://clojure.org/guides/spec#_entity_maps

agile_geek11:12:18

Ah ok like I said don't think it's wrong

roelof11:12:18

oke, I have no problem asking for why I do things

roelof11:12:40

but if I can work without it , it's also no problem

roelof11:12:48

I ask to learn things

agile_geek11:12:10

Break it down. Start by calling re-matches on that regex with something you expect to pass.

agile_geek11:12:24

looking at the regex it looks odd.

agile_geek11:12:47

You are expecting a space as the first character? Is that correct?

roelof11:12:41

no, I expect to validate this SK-C-5 so two characters , a - , one character , a - and then 1 or two numbers

agile_geek11:12:49

OK so that regex looks wrong as it's got a space between the " and the [

agile_geek11:12:58

Even after you correct that your spec is specifying that you pass a map as an argument and the map has a key of ::id with a value that conforms to a string and the regex.

agile_geek11:12:03

is that correct?

agile_geek11:12:34

i.e. once you remove that space in the regex this will pass

agile_geek11:12:44

(s/explain ::id-list {::id "aa-s-12"})

roelof11:12:15

oke, I tried that in repl but saw something wierd :

(s/explain ::id-list {::id "aa-s-12"})
In: [:paintings2.api-get/id] val: "aa-s-12" fails spec: :paintings2.api-get/id-type at: [:paintings2.api-get/id] predicate: (re-matches id-regex %)
=> nil
(s/explain ::id-list {::id "aa-s"})
In: [:paintings2.api-get/id] val: "aa-s" fails spec: :paintings2.api-get/id-type at: [:paintings2.api-get/id] predicate: (re-matches id-regex %)
=> nil 

roelof11:12:35

I was expecting that the second one would fail

roelof11:12:59

because the regex would not be right for that one

roelof11:12:21

oke , both are failing

agile_geek11:12:38

It does fail - it will be in sys out

agile_geek11:12:50

explain prints to console

agile_geek11:12:23

Start by just running the re-matches

agile_geek11:12:41

i.e. (re-matches id-regex "aa-s-12")

agile_geek11:12:09

see what that does and then try explaining the ::id-type spec next and so on

agile_geek11:12:19

one step at a time

roelof11:12:51

then I see this :

(re-matches id-regex "aa-s-12")
=> nil 

agile_geek11:12:09

Right which means your regex is not matching

agile_geek11:12:29

as you should see the string again if there's a match

roelof11:12:48

and this I see this with a non-matching string :

(re-matches id-regex "aa-s")
=> nil  

agile_geek11:12:06

They both don't match

agile_geek11:12:27

Your regex is wrong - did you remove the space?

roelof11:12:24

I did remove the space

agile_geek11:12:36

Then your repl is probably confused and hasn't reloaded the pattern properly. Restart it and try again. As you can see @clojurebot thinks it's ok

roelof11:12:38

aha, I have to restart or reload the file

roelof11:12:54

now everything is working fine

agile_geek11:12:29

Look at this as well:

(defn read-numbers
  "Reads the ids of the paintings"
  [response]
  (let [response-checked (s/conform ::id-list response)]
    (response-checked)))

roelof11:12:07

see here :

(s/explain ::id-list {::id "aa-s-12"})
Success!
=> nil
(s/explain ::id-list {::id "aa-s"})
In: [:paintings2.api-get/id] val: "aa-s" fails spec: :paintings2.api-get/id-type at: [:paintings2.api-get/id] predicate: (re-matches id-regex %)
=> nil  

agile_geek11:12:10

What is response-checked? I.e. what type of thing and then look at what you do with it (response-checked)

roelof11:12:13

What I tried to do it check if the input is valid. So if it have a field with a :id

roelof11:12:43

so it schould have the contents of true or false because of conform

roelof11:12:17

but I have to change something. the input is [{:id}] not {:id}

agile_geek11:12:54

Run s/conform on a map that conforms and see what it does

roelof11:12:13

oke, so something like (s/conform ::id-list [{:id 1 :name "Roelof"}]

agile_geek11:12:47

Also remember that ::id is a namespace qualified keyword, so you can't pass :id you need to pass a fully qualified namespaced keyword. So if you namespace is paintings.api-get you need to pass {paintings.api-get/id "aa-s-12"}

agile_geek11:12:25

::id is shorthand for <current-ns>/id

agile_geek11:12:50

It's not expecting a vector of maps. Just a map

roelof11:12:10

yep, see here :

=> :clojure.spec/invalid
(s/explain ::id-list [{:id "SK-C-5" :name "Roelof"}])
val: [{:id "SK-C-5", :name "Roelof"}] fails spec: :paintings2.api-get/id-list predicate: map?
 

roelof11:12:27

but response is a vector of maps

roelof11:12:44

so that one schould conform

agile_geek11:12:47

OK so you need to change your spec defn

roelof11:12:52

that is what I try to say

roelof11:12:22

yep, I have to change this line (s/def ::id-list (s/keys :req [::id]))

agile_geek11:12:37

OK - I've got to go now but take it step at a time and try it out in repl using s/conform or s/explain

roelof11:12:57

I will, thanks for the pointers

agile_geek11:12:28

Don't forget it's expecting a fully qualified id key

roelof11:12:23

thanks, you said that also a few minutes ago

roelof11:12:59

Can someone explain this :

(in-ns 'paintings2.api-get)
=> #object[clojure.lang.Namespace 0x171df08 "paintings2.api-get"]
(s/explain ::id-list [{:id "SK-C-5" :name "Roelof"}])
IllegalArgumentException Key must be integer  clojure.lang.APersistentVector.invoke (APersistentVector.java:292)
(s/explain ::id-list [{paintings.api-get/id "aa-s-12"}])
CompilerException java.lang.ClassNotFoundException: paintings.api-get, compiling:(C:\Users\rwobb\AppData\Local\Temp\form-init8511580151742996713.clj:1:1) 
 

roelof11:12:05

agile-geek have said to use a fully qualified id key but now it cannot find the class

roelof13:12:55

here more examples where I cannnot see what im doing wrong ?

(s/explain ::id-list {:id 1 })
val: {:id 1} fails spec: :paintings2.api-get/id-list predicate: (contains? % :paintings2.api-get/id)
=> nil
(s/explain ::id-list [{paintings.api-get/id "aa-s-12"}])
CompilerException java.lang.ClassNotFoundException: paintings.api-get, compiling:(C:\Users\rwobb\AppData\Local\Temp\form-init3968416759727215223.clj:1:1) 
(s/explain ::id-list {paintings.api-get/id "aa-s-12"})
CompilerException java.lang.ClassNotFoundException: paintings.api-get, compiling:(C:\Users\rwobb\AppData\Local\Temp\form-init3968416759727215223.clj:1:1) 
 

roelof13:12:29

I already switched to the right namespace :

(in-ns 'paintings2.api-get)
=> #object[clojure.lang.Namespace 0x171df08 "paintings2.api-get"] 

dpsutton13:12:45

you're caling paintings.api-get

dpsutton13:12:52

but you are working in the namepsace paintings2

roelof13:12:35

@dpsutton thanks. Now this problem :

(s/explain ::id-list {paintings2.api-get/id "aa-s-12"})
CompilerException java.lang.RuntimeException: No such var: paintings2.api-get/id, compiling:(C:\Users\rwobb\AppData\Local\Temp\form-init5817193040681417679.clj:1:1) 
 

dpsutton13:12:48

i don't know spec at all

dpsutton13:12:56

but please take some time to experiement

dpsutton13:12:01

poke around at things at the repl

dpsutton13:12:10

use your repl. evaluate paintings2.api-get/id

dpsutton13:12:19

figure out why it isn't resolving

dpsutton13:12:23

go to it in source

roelof13:12:48

@dpsutton I experiment the whole afternoon with it

roelof13:12:56

but get no further

dpsutton13:12:40

well, the first bug was not listing your namespace correctly

dpsutton13:12:56

did you go to the source and find where you define paintings2.api-get/id

dpsutton13:12:08

go look at somewhere in code where you defined def id ...

dpsutton13:12:47

because your reader is saying No such var: paintings2.api-get/id. So go investigate and see if you are wrong or it is wrong

roelof13:12:04

id is defined here : (s/def ::id ::id-type)

roelof13:12:30

where s is defined here : [clojure.spec :as s]

dpsutton13:12:52

ok, and how do you refer to specs that are put in the registry?

dpsutton13:12:33

and do you pass it in as a map to s/explain like you did above?

roelof13:12:23

Right now I do this : (s/explain ::id-list {::id "aa-s-12"})

dpsutton13:12:11

ok, so that's defined as a keyword

dpsutton13:12:24

but your paintings2.api-get/id is not used as a keyword

dpsutton13:12:29

maybe you need to refer to it as such

roelof13:12:34

but agile-geek says this :

Also remember that `::id` is a namespace qualified keyword, so you can't pass `:id` you need to pass a fully qualified namespaced keyword. So if you namespace is `paintings.api-get` you need to pass `{paintings.api-get/id "aa-s-12"}  ::id` is shorthand for `<current-ns>/id  

roelof13:12:22

so I try to do it that way

agile_geek13:12:40

I missed the :

agile_geek13:12:49

It's a keyword

dpsutton13:12:42

you need to be clear you are referencing a keyword rather than a var

dpsutton13:12:53

if you had a var in that namespace, you would refer to it in the way that you are

dpsutton13:12:00

so how do you refer to keywords from different namespaces

roelof13:12:33

@agile_geek @dpsutton then I see the same error :

(s/explain ::id-list {paintings2.api-get/:id "aa-s-12"}) 

CompilerException java.lang.RuntimeException: No such var: paintings2.api-get/:id, compiling:(C:\Users\rwobb\AppData\Local\Temp\form-init5817193040681417679.clj:1:1) 

dpsutton13:12:35

i believe the colon precedes the whole expression

dpsutton13:12:47

:ns/keyword rather than ns/:keyword

roelof13:12:57

so somehow I do not fully understand fully what agile-geek is meaning

dpsutton13:12:05

And roelfw, I am googling these things

dpsutton13:12:10

I don't know about namespaced keywords

dpsutton13:12:13

I am learning

dpsutton13:12:18

so try to do what I am doing

roelof13:12:20

@dpsutton thanks, this is working (s/explain ::id-list {:paintings2.api-get/id "aa-s-12"})

agile_geek13:12:22

:my-namespace/id is fully namespaces kw

agile_geek13:12:08

If you reference ::id inside the NS my-namespace you get same thing

agile_geek13:12:46

Namespaces keywords are used to give context and avoid name clashes

agile_geek13:12:48

So u can have :ns1/id and :ns2/id and they are different

agile_geek13:12:48

You can see with a common name like id why that might be useful

roelof13:12:56

time for a break and then trying to find out how to change this : (s/def ::id-list (s/keys :req [::id]))

roelof13:12:23

Yes, before you know you have bugs because the wrong id is passed on

roelof13:12:41

to something that takes a vector of maps

roelof13:12:00

time for google and some experimenting with repl

agile_geek13:12:03

Yes and you can use id in addresses, person, car, etc

roelof13:12:46

Both thanks

agile_geek13:12:41

This might help you think about how you name namespaces too

agile_geek13:12:54

One tip I give ppl is "every developer spends over 80% of their time reading and comprehending code not writing it so optimise for readability". That means formatting matters, naming matters and following common ideoms matters too. All those things allow the reader to shortcut some comprehension. I can write all my English sentences without punctuation but they'd be hard to read!

roelof14:12:32

yes. I have someone is the family which is not using punctuation and now capital characters and that is not nice to read

roelof14:12:19

and sentence with the wrong order.

roelof14:12:27

I try with my code to be as clear as possible to other knows what the code does and for myself when I see the code after some months I still know what the code does exactly

roelof15:12:51

Why can the id not be found here :

(s/def ::id ::id-type)

(s/def ::id-list (s/coll-of
                   (s/keys :req-un [::id])
                   :kind vector?))  

roelof15:12:24

but when I do : (s/explain ::id-list [{:paintings2.api-get/id "aa-s-12"}])

roelof15:12:52

it fails with this message : In: [0] val: #:paintings2.api-get{:id "aa-s-12"} fails spec: :paintings2.api-get/id-list predicate: (contains? % :id)

roelof15:12:04

but it contains a id

dpsutton15:12:45

if i'm reading this correctly, can you tell me what part of "aa-s-12" would have the key :id?

dpsutton15:12:00

what is ::id-type

roelof15:12:22

here are all the specs :

(def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")

(s/def ::id-type (s/and string? #(re-matches id-regex %)))

(s/def ::id ::id-type)

(s/def ::id-list (s/coll-of
                   (s/keys :req-un [::id])
                   :kind vector?))  

roelof15:12:01

I mean to test if the map of vectors have a id like this ' SK-C-5'

roelof15:12:40

so this is valid [{ :id "SK-C-5"} {:id "SK-D-45"}]

roelof15:12:14

and this schould not be valid [ { :id 1 }]

dpsutton15:12:01

do you know what :req-un means?

dpsutton15:12:12

and it explains why the predicate is looking for (contains? % :id)

dpsutton15:12:17

rather than a namespace qualified key

dpsutton15:12:54

also, your id-list spec is just saying it expects a vector of maps with an unqualified keyword :id

dpsutton15:12:14

I don't know how to say a vector of things that conform to this spec ::id

dpsutton15:12:26

but make sure your code is saying what you want it to say

roelof15:12:08

found it . This works

def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")

(s/def ::id-type (s/and string? #(re-matches id-regex %)))

(s/def ::id ::id-type)

(s/def ::id-list (s/coll-of
                   (s/keys :req [::id])
                   :kind vector?))  

roelof15:12:39

see this :

(s/explain ::id-list [{:paintings2.api-get/id "aa-s-12"}])  
Success! 

dpsutton15:12:17

pass a non-valid ::id conforming thing

dpsutton15:12:21

I think there may still be an error

dpsutton15:12:44

(s/explain ::id-list [{:paintings2.api-get/id "wont conform"}])

roelof15:12:21

it works fine as far as I can see :

(s/explain ::id-list [{:paintings2.api-get/id "wont conform"}])
In: [0 :paintings2.api-get/id] val: "wont conform" fails spec: :paintings2.api-get/id-type at: [:paintings2.api-get/id] predicate: (re-matches id-regex %)  

roelof15:12:25

but one think I do not understand. I have this function :

(defn read-numbers
  "Reads the ids of the paintings"
  [response]
  (let [response-checked (s/conform ::id-list response)]
    (response-checked)))  

roelof15:12:53

which I call it like this :

(read-numbers [{:paintings2.api-get/id "aa-s-12"}] )  

roelof15:12:31

and then I see this error : ArityException Wrong number of args (0) passed to: PersistentVector clojure.lang.AFn.throwArity (AFn.java:429)

roelof15:12:44

found it. response-checked is not a function but a variable ,

roelof15:12:01

I have another problem . I now have to use ::id but the reponse which I want to input validated has [{:id "xxxx"}]

roelof15:12:14

and that is still not valid

roelof15:12:33

where x is for example XX-X-5

roelof15:12:40

so or I have to rewrite the specs or parse the response first to have :paintings2.api-get/id instead of just :id

roelof16:12:46

Is there a way I can convert keywords like :a to named keywords "::a"

roelof16:12:13

or is there another way to describe a vector of maps , where a map has a key named :id

seancorfield17:12:01

Use :req-un in your spec — for required, unqualified.

seancorfield17:12:13

You have :req which means required, qualified.

roelof17:12:55

now the input validation seems to work well :

(read-numbers [{:id "aa-s-9"}] )
=> [{:id "aa-s-9"}]
(read-numbers [{:id "aa-s"}] )
=> :clojure.spec/invalid  

roelof17:12:28

now time to think what schould happen when the spec is invalid

roelof17:12:43

When there are no id's the rest of the code is not working

roelof17:12:06

so I think to use a 500 error page somehow

roelof17:12:33

but I do not know if in normal code I can use (render .... )

roelof17:12:44

or only in the routes part