This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-04
Channels
- # announcements (3)
- # babashka (14)
- # beginners (151)
- # calva (14)
- # cider (9)
- # clj-kondo (24)
- # cljdoc (12)
- # cljs-dev (195)
- # cljsjs (3)
- # cljsrn (13)
- # clojars (12)
- # clojure (234)
- # clojure-dev (3)
- # clojure-europe (9)
- # clojure-greece (1)
- # clojure-italy (2)
- # clojure-japan (4)
- # clojure-nl (4)
- # clojure-spec (89)
- # clojure-taiwan (1)
- # clojure-uk (16)
- # clojuredesign-podcast (2)
- # clojurescript (17)
- # conjure (11)
- # core-async (4)
- # core-typed (31)
- # cursive (9)
- # datomic (8)
- # emacs (17)
- # figwheel (1)
- # fulcro (5)
- # ghostwheel (42)
- # graphql (3)
- # hugsql (5)
- # jackdaw (3)
- # jobs-discuss (93)
- # joker (4)
- # meander (6)
- # mount (1)
- # off-topic (23)
- # pathom (10)
- # re-frame (23)
- # reitit (7)
- # remote-jobs (18)
- # shadow-cljs (153)
- # spacemacs (24)
- # sql (30)
- # tools-deps (14)
- # vim (12)
- # xtdb (1)
I have a map that has this shape {"account" {"active" true "balance" 30}}
, what's the best way to spec the "account"
key?
I've tried these and I'm using the first one.
(s/def ::account-key (s/with-gen (s/and string?
#(= "account" %))
#(gen/return "account")))
(s/def ::account-key (s/with-gen (s/and string?
#{"account"})
#(gen/return "account")))
You can use #{"account"}
to spec a known set of string values.
(and that will also generate correctly)
(s/def ::account-key #{"account"})
I was being redundant here:
(s/def ::account-key (s/with-gen (s/and string?
#{"account"})
#(gen/return "account")))
Inventive... but certainly "overkill" ๐
Learner. Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps) When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails. I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter. Any clarifications on this? :thinking_face: <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>
The conversion from JSON will probably give you unqualified keywords, since no such concept exists there.
You have to either qualify your keywords on a separate post-processing step, or make a spec for the payload that takes unqualified keys (e.g. :req-un
instead of :req
)
@U3UFFB420 Thanks for the explanation. But..... why is the :timestamp not found in the namespace or is it global? If global than how do I relate it to my incoming data - as it were. Because the spec is not of use right now. Obviously I do not see the connection
So that is the purpose of req-un then? Does that make the :keyword global?
Maybe you are mixing concepts. Keywords are just keywords, they can be qualified (`:test/timestamp`) or unqualified (`:timestamp`). ::
It's a shortcut to write a qualified keyword with the same namespace as the current file.
Keywords can be qualified or unqualified.
Unqualified can be thought of as "global", but it's better to stick with "lacking a namespace".
When you type in ::timestamp
, Clojure uses the current namespace and creates a qualified keyword.
req-un
means that the spec will match unqualified keys to the qualified keys, comparing just the name
(i.e. discarding the namespace
)
So in a more complex map structure, where I have several collections do I have to req-un for all definitions or only at the top level and it is inferred?
'Cos I req-un all my defs and now it does not work
Yes, you'd have to req-un all of them. Try breaking it into smaller pieces to find where it's failing
here is the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>
It is detailled
seems to work.... what do you mean?
If it is all :req for keywords exampl2 works fine and conforms. So I am confused here?
Unless this is spec2, I don't think this is a valid definition: (s/def ::loca (s/keys :req-un {::addr [::country]}))
Might have worked by accident, or because you had a correct definition earlier that was stored in the registry
would (s/def ::loca :req {::addr [::country]}) do it? no it does not
Surely the country spec forces a s/keys?
2 rows above and it works -> (s/def ::addr (s/keys :req [::country])) (s/conform ::addr {::country "USA"}) ;; => #:myproj.trial.spec{:country "USA"}
It is effectively a map of map, no?
and :loca is the map of the above. But if I do not have s/keys it seems now not to work
These two are not the same:
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un {::addr [::country]}))
You're feeding a vector to the first req, but a map to the secondAgreed but I tried ::loca {:: addr {::country and it will not accept it
@U1UQEM078 BTW what do you mean as spec2? I am using clojure.spec.alpha latest version
@U3UFFB420 I agree with you, how does one relate the one to the other? I changed my spec to have :reg-un at the highest level and it fails spec is at <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>
Try first working with simpler examples, try to create a spec for a map with unqualified keys
If you look at the gist that Is what I am doing? No?
When all the keys are NOT reg-un then all my test cases pass
Well, https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19 you need to change it to (s/def ::loca (s/keys :req-un [::addr]))
and so my spec seems OK ??
OK how? I know it is indicating a map but if I do not use s/keys it fails
https://app.slack.com/team/UEGT2541Jย ย [2:57 PM] Agreed but I tried ::loca {:: addr {::countryย ย and it will not accept
Even though I initially figured it wasa map of map of map
Oh oh!!!! I see it now!
I see several places you use :req-un
instead of :req
, which is the correct way for conforming/validating these examples
As I said, try creating a simpler spec for a non nested map and then try validating maps
Ok. But how do I use the spec against the data map (ex JSON) which is :keyword and not ::timeline (IYKWIM)
Here's an example for a qualified map
(ns myproj.trial.spec)
(s/def ::country string?)
(s/def ::street string?)
(s/def ::addr (s/keys :req [::country ::street]))
(s/valid? ::addr {::country "USA"
::street "A street"})
;; => true
(s/valid? ::addr {:country "USA"
:street "A street"})
;; => false
(s/valid? ::addr {:myproj.trial.spec/country "USA"
:myproj.trial.spec/street "A street"})
;; => true
yes, without changing the above, how do you apply it to a map ex JSON) {:addr {:country "USA :street "A street"}}
As aisamu said before, if you are working with JSON you loose the namespace of the keywords, so you probably want to use :req-un
in your specs.
(ns myproj.trial.spec)
(s/def ::country string?)
(s/def ::street string?)
(s/def ::addr (s/keys :req-un [::country ::street]))
(s/valid? ::addr {:country "USA"
:street "A street"})
;; => true
I was advise to use :req-un to allow for unqualif namesapces
Right, but then when you use valid?
or conform
you also have to use unqualified keywords for the map https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L69-L81
(s/def ::country string?) (s/def ::addr (s/keys :req-un [::country])) (s/def ::loca (s/keys :req-un [::addr ::country ])) (s/conform ::country "USA") ;; => "USA" (s/conform ::addr {:country "USA"}) ;; => {:country "USA"} (s/conform ::loca {:addr {:country "USA"}}) ;; => #:myproj.trial.spec{:addr #:myproj.trial.spec{:country "USA"}} (s/explain-str ::loca {:addr {:country "USA"}}) ;; => "{:addr {:country \"USA\"}} - failed: (contains? % trial.spec/addr) spec: :trial.spec/loca\n{:addr {:country \"USA\"}} - failed: (contains? % trail.spec/country) spec: :parceltrax.tracking.spec/loca\n" ??why this??
I updated the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script> If you look at line 16 When I look sat it it is a map of map
This line is wrong https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19
Yes, in theory that will work, however the data looks like this :pieceID ["ABC1" "DEF2" "GHI3"]}]}) (def exampl2 {::abc [{::loca {::addr {::country "USA"}} ::dets {::some{::timestamp "2020-04-09T15:27:00" ::Url "https://mywebpage.com/answer"
Look at line 1 of the gist
so loca is a map of map of map
The trouble I a,mm having is the compounded map that does not seems to play nicely
OK - it does conform to the definition Thx. However it is not as per line 1 - which is how the data is meant to be. So extracting from line 1: :loca {:addr {:country "USA"}}
Yes, you need to modify ::loca
so it looks like this (s/def ::loca (s/keys :req-un [::addr]))
@U3UFFB420 OK !!!! I think I have it ! I must take out the ::country in the s/def as the ::country is already defined (s/def ::loca (s/keys :req-un [::addr]))
I think that is what you were trying to get me to see
Appreciated
@U3UFFB420 Thanks for your patience and help. I got it now!! my complete script now works! Kudos for you ๐
The difference is that ::
expands to the current namespace. So if you're working in a ns called my.test
then ::timestamp
expands to :my.test/timestamp
. The spec fails with :timestamp
because it's also expecting the namespace.
Hi fellow clojurians, is anyone aware of a way to parse a swagger json to valid specs that can be used to generate test data?
That's a great idea and something I'm interested in, too
This is what I did to learn spec. From this point on one can gen data. What you see is a map converted from a JSON structure. Now this is not a "take a swagger spec and convert" but is the basis of how to clojure spec based on a swagger that I did.
(def exampl {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))
(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca {:addr {:country "USA"}}) ;; => true
(s/check-asserts?)
(s/check-asserts true) ;; by default false!!
(s/assert ::loca {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/assert ::loca {:addr {:countryx "SA"}}) ;; note the exception error BUT look at the repl
(s/def ::timestamp (and not nil?
string?)) ;; this needs work for T-time issue
(s/def ::Url string?) ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))
(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}})
(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]
(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}])
;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}
(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
:dets {:some{:timestamp "2020-04-09T15:27:00"
:Url ""
:prod {:name "this product"}}}
:totalNumber 399
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)
Also look at https://www.youtube.com/watch?v=f2hNQdS2VxQ to watch a conversion - the nearest I have found.
Good luck with this ๐hey @UEGT2541J thanks for joining in. actually the challenge is not to create a clojure map from the json and then writing the spec. but parsing the json into specs . the difference being that I don't want to write them manually. but you are absolutely right, the chain is: json -> clojure-map -> transform clojure map entries to spec entries and compose them to upper level spec. then we can use the gen functions to create test data
if we have a working parser for that, we can just call the endpoint on webservices, slurp the swagger json and create specs for data models
which is a handy little tool ๐
I'm not an expert for specs, but searching a swagger yaml/ json for the definitions
and then using the type information there and maping those and the field names to specs sounds doable ๐
definitions:
Order:
type: "object"
properties:
id:
type: "integer"
format: "int64"
petId:
type: "integer"
format: "int64"
quantity:
type: "integer"
format: "int32"
shipDate:
type: "string"
format: "date-time"
status:
type: "string"
description: "Order Status"
enum:
- "placed"
- "approved"
- "delivered"
complete:
type: "boolean"
default: false
xml:
name: "Order"
i like structured data ๐
I'll work on that a bit an will put a link to a gist here if I can create something useful
@U0J8XN37Y im really interested in this, we just rolled out swagger to our APIโs both internally and externally, I would love to be able to have our apps handle specs for us
will keep you posted too.
I think it's easier than I thought.
.. I mean until I hit the brick wall ๐
at least the basics. translating the conditions on properties can take some time ๐
OK, similar to what you are stsing then: look st the library serene - It autoigenerates the specs for REST and graphql APIs.
I'll shut up now ๐
no, please don't. never heard of that library and it looks like they are doing the same thing, but for a different format. :thumbsup:
That's a great idea and something I'm interested in, too