Fork me on GitHub
#clojure-spec
<
2020-06-06
>
practicalli-johnny07:06:04

Question about spec keys. It seems I can use either qualified keys with :req or unqualified keys with :req-un . I cannot use qualified keys with :req-un. So if I have keys that could be qualified or unqualified, would I have req and req-un sections in the keys

(spec/def ::customer-details
  (spec/keys
    :req [::first-name ::last-name ::email-address ::residential-address ::social-security-id]
    :req-un [::first-name ::last-name ::email-address ::residential-address ::social-security-id]))
Or I assume I should just choose to use one or the other. I could just add the #:: auto-resolve macro to hash-maps with unqualified keys...

practicalli-johnny07:06:57

Adding a specific namepace in front of the hash-map, #:practicalli.bank-account-spec seems to work too and feels like a better approach

(spec/explain :practicalli.bank-account-spec/customer-details
              #:practicalli.bank-account-spec
              {:first-name          "Jenny"
               :last-name           "Jetpack"
               :email-address       ""
               :residential-address "42 meaning of life street"
               :postal-code         "AB3 0EF"
               :social-security-id  "123456789"}  )

Aron07:06:30

I haven't really had time to watch all the videos, is there one where the organization of the namespaces (especially in relation with models, specs, entities)? I am coming from js where circular dependencies are allowed in certain circumstances and I find it a bit too much work to create new files to create new namespaces (note, it's not the problem that I have to create files, I love lots of tiny files, I just don't like the now additional work that means I have to constantly rewrite all my files if I want to reorganize just a couple.)

practicalli-johnny07:06:51

@ashnur I will be covering a bit of organisation in the broadcast today in about 30 minutes https://youtu.be/jUgm4zh-vF4 - I create a src, test and spec namespace and have a design-journal namespace to show my working 🙂 It still fairly basic, so you dont need to watch the other videos

Aron07:06:33

thanks, I will be watching

👍 4
dev.4openID11:06:05

There seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated. So you s/def uses :: and can compose other s/defs with :: However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere. At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough". I see similar being asked above

Alex Miller (Clojure team)14:06:00

First, spec know nothing about :: - this is just a Clojure shorthand to fully resolve a keyword within the current namespace at read time. Spec will just see the full kw as if you had typed :user/myspec. s/keys will always validate all fully qualified keys in the map with their associated registered specs. It will check that the map has all of the specified :req keys (match fully qualified) or :req-un keys (match only the unqualified name). And specially for :req-un it will verify the values validate against the spec for the qualified key.

dev.4openID20:06:57

@U064X3EF3 Hi Alex

(s/def ::timestamp string?)  ;; => :parceltrax.tracking.spec/timestamp
(s/def ::status string?) ;; => :parceltrax.tracking.spec/status
(s/def :o/status (s/keys :req-un [::timestamp ::status])) ;; => :o/status
(s/valid? :o/status {:timestamp "2020-09"
                     :status "abc" }) ;; => true
(s/explain-str :o/status {:timestamp "2020-09"
                          :status "abc" }) ;; => "Success!\n"

(s/def ::mapx (s/keys :req-un [:o/status])) ;; => :parceltrax.tracking.spec/mapx
(s/valid? ::mapx {:status {:timestamp "2020-09"
                           :status "abc" }}) ;; => true
(s/explain-str ::mapx {:status {:timestamp "2020-09"
                                :status "abc" }}) ;; => "Success!\n"
I accept is works and maybe I'm being a little thick here: The solution depends of the "temp" creation of a namespace as a workaround. In this case a :o/status in order to differentiate from the actual :req-un namespace of :status. My perception is this in not very elegant or consistent with the spec model. As is encouraged by the clojure team, developers are to use :: more often. In the simple example provided - the code is not "portable" as such. I recognize it may seem may seem nitpicking - perhaps I have missed the understanding or this is just accept "it works this way" In the real workplace there are many occasions of "recursive dup fields in JSON data. Could you provide a comment or two on this? Much appreciate your efforts

Alex Miller (Clojure team)21:06:41

I’m not trying to encourage more use of :: and it’s typically not what you need (but it’s very convenient for examples)

Alex Miller (Clojure team)21:06:17

Personally I most often use full namespaces for data specs

dev.4openID09:06:03

Hi, I was not stating you were the advocate - the spec literature does - just to be clear Any thought about my point of "making up kw names" in conjunction with the point of recursive/nested JSON map structures that employ a keword such as my example :status - is there no recursive strategy for example? I don't know thus I ask.

dev.4openID09:06:12

Thanks for your patience

tianshu13:06:50

Is it expected if :a/b is not defined in this case?

(s/def ::c
  (s/keys :req [:a/b]))


(s/valid? ::c {:a/b {}})
;; => true

dev.4openID13:06:01

No. Imagine they are defined It's not about a/b nut when :req or :req-un is appropriate

Alex Miller (Clojure team)13:06:24

I would say yes, this is expected. :req means it’s required (and it’s there). If there are specs for any keys, they must be followed.

tianshu03:06:11

For the case (s/def ::a ::b), ::b must be declared before use, but in this case it is not. Should it be unified? (I mean always define before use or always check at runtime)

Alex Miller (Clojure team)13:06:35

Yes, in general resolution should be delayed for all specs and it’s not entirely true right now. Not going to make any updates on this in spec 1 though, will work on it in spec 2

hadils13:06:58

I am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!

dev.4openID13:06:23

I have just done this and it is OK for code and spec to be in different files I assume you have converted the JSON to a structure. Now define the spec "structure" accordingly Apply s/conform ::spec "JSON structure" - note the spec has :: and your JSON map has : for all elements - in spec make sure all :req are :req-un and it will work

dev.4openID13:06:50

in my case the file my namespace is ns: myproj.trail.clj and myproj.trial.spec.clj

dev.4openID13:06:11

Lesson learnt model the API JSON map example in spec file fisrst and all becomes more clear. I started from leaves to trunk and not trunk down as seemingly many do

hadils13:06:43

What about for multiple JSON outputs in the same spec file? How do I do that?

hadils13:06:20

That's the real problem I'm having.

dev.4openID13:06:47

If you break down the JSON maps into components and composed elements that are reusable. You then compose complete "trees" for all the responses for all the API calls. There are commonalities in the APIs returns and it goes faster in the compositions as you do more. i.e do 1 at a time. I went with the most complex one (in order to lean spec) and it just became easier to gen the rest.

hadils13:06:23

How do you qualify your spec keys?

dev.4openID14:06:04

It becopmes more and more obvious. Just build the leaves and branches slowly - use s/valid? and s/explain-str against sample data - extracted from the JSON map eventually build more complex branches and tada! you will compose the trunk. It seem tedious at first (as I was leaning) but it become faster

hadils14:06:08

I am hung up on the qualified keys like ::foo instead of using :node/foo. The latter seems more appropriate for my needs. How should I approach this?

dev.4openID14:06:17

look at the toy example I made first here <script src="https://gist.github.com/dev4openid/5c9320fb8ef2f8f5383836f379ff507e.js"></script> It is tedious at first BUT it gave me insights on how to do things I will leave it up for a while

dev.4openID14:06:26

in the spec file use :: format for your definitions - saves typing

dev.4openID14:06:00

note the :reg-un !!

hadils14:06:04

Your Gist does not show up properly on my browser.

dev.4openID14:06:34

(def exampl {:abc [{:loca {:addr {:country "USA"}}
                    :dets {:some{:timestamp "2020-04-09T15:27:00"
                                 :Url ""
                                 :prod {:name "this product"}}}
                    :totalNumber 399
                    :pieceID ["ABC1" "DEF2" "GHI3"]}]})


(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))

(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca  {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca  {:addr {:country "USA"}})  ;; => true
(s/check-asserts?)
(s/check-asserts true)                       ;; by default false!!
(s/assert ::loca  {:addr {:country "USA"}})  ;; => {:addr {:country "USA"}}
(s/assert ::loca  {:addr {:countryx "SA"}})  ;; note the exception error BUT look at the repl

(s/def ::timestamp (and not nil?
                        string?))     ;; this needs work for T-time issue
(s/def ::Url string?)                 ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))

(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
                   :Url ""
                   :prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
                           :Url ""
                           :prod {:name "this product"}}})

(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
                      vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]

(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
                   :dets {:some{:timestamp "2020-04-09T15:27:00"
                                :Url ""
                                :prod {:name "this product"}}}
                   :totalNumber 399
                   :pieceID ["ABC1" "DEF2" "GHI3"]}])

;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}

(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
                                 :dets {:some{:timestamp "2020-04-09T15:27:00"
                                              :Url ""
                                              :prod {:name "this product"}}}
                                 :totalNumber 399
                                 :pieceID ["ABC1" "DEF2" "GHI3"]}]})

(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)

hadils14:06:28

Thanks a lot @UEGT2541J!!!!

hadils16:06:51

@UEGT2541J what do you do when you are trying to spec 2 different JSON outputs with same names -- some of the fields are identical and some are not, but have the same name?

hadils16:06:15

Do I need to split them out into separate files?

dev.4openID16:06:02

@UGNMGFJG3 Do you mean 2 fields in the JSON structure, where 1 is subordinate to another? {:name {:person "abc" :name}} or do you mean recurring pasterns that are in a map?

dev.4openID16:06:42

For the former: (s/def :inner/status string?) ;; => :inner/status (s/def :inner/timestamp string?) ;; => :inner/timestamp (s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status])) ;; => :outer/status (s/def :outer/map (s/keys :req-un [:outer/status])) ;; => :outer/map (s/valid? :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => true (s/explain-str :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => "Success!\n"