you could try writing your own with gen/recursive-gen

or...is this a DAG?

or I guess it's a proper tree?

I did try with recursive-gen and got the same error. seems like it should be a proper tree yeah, which I guess is a type of DAG

yeah I think recursive-gen would work

can you share the recursive-gen code?

this is what I tried:

boot.user=> (spec/def ::posts
       #_=>   (spec/with-gen
       #_=>     (spec/map-of ::post-id ::post)
       #_=>     #(tc-gen/recursive-gen (fn [post-gen] (tc-gen/map (spec/gen ::post-id) post-gen)) (spec/gen ::post))))

I think the problem there is the (spec/gen ::post)

you can't use that generator as part of what you're building, since it's your original problem

if you had a ::post that had everything but the ::posts key, so that it didn't recurse, then you could use that

::post-with-empty-posts

but a post without the ::posts key is not valid, the valid value for the leaf would be {}

Yeah I meant a version where the key always has {}

@gfredericks out of curiosity, could you try: (s/with-gen ::post #(s/gen (s/or :post ::post :empty #{{}})))

what would be the best way to spec a collection of tuples, e.g.

[[:k1 :v1] [:k2 :v2]]

the alternative i came up with was (s/* (s/or (s/cat ...) (s/cat ....))), but that's also ugly

(as a bit of background, the reason i'm not using a map is that the ordering of the elements is important)

Maybe this?

(s/coll-of (s/tuple keyword? keyword?))

yeah that's similar to the cat use case... the problem is that for a specific key, i need a specific value so then you would end up with

(s/or :k1 (s/tuple #(= :k1 %) ::v1)
           :k2 (s/tuple #(= :k2 %) ::v2)

etc

that is, if you want to have a similar effect as

(s/keys :opt-un [:k1 :k2])

Ok. I also don’t know a better way than your s/or way

ok. guess i'll wrap this in a macro or something to make it pleasant.

maybe a multi spec is the answer though...

yes, i definitely think that can work

you can shorten it a bit with (s/tuple #{:k1} ::v1)

otherwise maybe keys* can be used

basically (s/keys* :req-un [(or ::k1 ::k2)])

conforming it will return a map tho

(s/valid? (s/keys* :req-un [(or ::foo ::bar)]) [:foo 1]) -> ok (::foo is #{1} here for the example)

but it will happily take "longer" tuples so not sure it's the best thing tbh

seems like keys* exists (among other things) to validate unspliced kw args

@mpenet thanks! that looks like what i want

:data #:clojure.spec.alpha{:problems [{:path [:args], :reason Extra input, :pred (clojure.spec.alpha/cat :docstring (clojure.spec.alpha/? clojure.core/string?) :attr-map (clojure.spec.alpha/? clojure.core/map?) :clauses :clojure.core.specs.alpha/ns-clauses), :val ((:require-macros [respo.macros :refer [defcomp div span input <> cursor->]]) (:require [clojure.string :as string] [hsl.core :refer [hsl]] [respo.app.comp.task :refer [comp-task]] [respo.core :refer [create-comp]] [respo.comp.space :refer [=<]] [respo.comp.inspect :refer [comp-inspect]] [respo.app.comp.zero :refer [comp-zero]] [respo.app.comp.wrap :refer [comp-wrap]] [polyfill.core :refer [text-width* io-get-time* set-timeout*]] [respo.app.style.widget :as widget])), :via [], :in [1]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x60330e39 clojure.spec.alpha$regex_spec_impl$reify__1200@60330e39], :value (respo.app.comp.todolist (:require-macros [respo.macros :refer [defcomp div span input <> cursor->]]) (:require [clojure.string :as string] [hsl.core :refer [hsl]] [respo.app.comp.task :refer [comp-task]] [respo.core :refer [create-comp]] [respo.comp.space :refer [=<]] [respo.comp.inspect :refer [comp-inspect]] [respo.app.comp.zero :refer [comp-zero]] [respo.app.comp.wrap :refer [comp-wrap]] [polyfill.core :refer [text-width* io-get-time* set-timeout*]] [respo.app.style.widget :as widget])), :args (respo.app.comp.todolist (:require-macros [respo.macros :refer [defcomp div span input <> cursor->]]) (:require [clojure.string :as string] [hsl.core :refer [hsl]] [respo.app.comp.task :refer [comp-task]] [respo.core :refer [create-comp]] [respo.comp.space :refer [=<]] [respo.comp.inspect :refer [comp-inspect]] [respo.app.comp.zero :refer [comp-zero]] [respo.app.comp.wrap :refer [comp-wrap]] [polyfill.core :refer [text-width* io-get-time* set-timeout*]] [respo.app.style.widget :as widget]))}

how to read this error please?

@jiyinyiyong try https://github.com/bhb/expound

There is a good sample of it in https://github.com/metosin/reitit/blob/master/README.md#validating-route-trees. Original :problems was totally unreadable (recursive specs). Would like it just to report the deepest error, but still it's awesome.

@jiyinyiyong According to the :args and :in [1], it's not expecting that :require-macros form -- so I'm thinking you tried to run this in Clojure, rather than ClojureScript? (I think :require-macros is ClojureScript-only?)

(but, yes, some of ns spec output is a bit overwhelming)

Extra input generally means a sequence (`s/cat`) ran into a form that was not recognized by the syntax specified in the spec.

:value (respo.app.comp.todolist (:require-macros [... is the value that failed and :in [1] gives you the position within that sequence.

Is validating data with runtime context in the purview of spec? For example, instead of just saying that :foo/id is an integer, can I specify that "it's an integer that corresponds to an entity in the provided database?"

not recommended

From what I can tell, even though specs are being checked at runtime, there's no idiomatic way (short of dynamic var based hacks) to pass additional context into the validation logic.

@ghadi Is there any kind of rationale or discussion on this topic somewhere that you know of? I haven't been able to find anything in my research more specific than "don't use spec as a runtime transformation system"

which is sort-of-related, but not quite.

if you look up in a DB your predicates / specs might change their contract

the meaning of a spec might change based on state, which loses a whole bunch of value (inverting a spec into a generator)

I think specs should not depend on any ambient state

besides the implicit spec database

e.g. just check that the email address is valid looking, don't try to open up an SMTP connection

Yeah, but that's something that can't be technically enforced by the API of spec

given that it accepts arbitrary predicate fns

i get that, it's just a terrible idea to curry in state into your predicate

nothing I can do to prevent it ¯\(ツ)/¯

= )

Right, I don't mean to challenge the underlying design decision

as a consumer, it just wasn't clear to me whether what I wanted to do was in the design space at all

I thought it might be, since conceptually it's just further validation

i think you'd use spec to check data at the door, then do extrinsic checks afterwards

i don't think it's a design goal to be a total system

At that point, it's easier to do the entire validation / generation myself, rather than using spec.

since in my domain pretty much all of the incoming data is just ids

there's so many more benefits you'd be dropping

so I need to realize those IDs into entities, then check properties of those entities.

ditto on the generation front, just getting integers won't help. I need to be able to generate integers that correspond to entities of type X

or whatever.

don't you need to check the entities?

unless I'm missing something where spec might still be able to help in here?

you're right the IDs are trivial, but I doubt that's the extent of the applicability of spec for your use case

I'd certainly want something to describe one of the "entities" and get a free generator

A non-trivial generator

I can walk you through a concrete example if you have 5 minutes

go for it; I might not be able to help, but there are many 👀 in here

cool, thx = )

I'm using datascript as the db, so it's all in-memory.

the application architecture is event sourced

basically, I have (step db event) ;;=> new-db

basically, create a new database from the old database, given an event

The events implement a few protocol methods like (valid? [this, db]) which says whether or not the event is valid for a given database.

So, for example, #events.AddChild{:parent-id 5} is a record that represents the event "add a child to node 5". The valid? function needs to look and see if node 5 exists in the db and, if it does, make sure it's the type of node that is allowed to have children

All this make sense so far?

Anyway, I don't understand how I can implement valid?w/ spec, since everything depends on having the runtime db value. Likewise, I cannot generate these events with spec, because it'd need to only generate integer IDs that correspond to nodes that can have children.

If wrote some specs to define what it means for a node to be allowed to have children, then I could turn the ID into a datascript entity at runtime and validate the entity using spec.

but it doesn't seem like those specs would help me generate events to test against the system

your scenario makes sense. I think the fact that you and spec named something the same thing (valid?) but has a different contract is confusing the situation

Ah, I only did that once I gave up on spec =P

You might make a spec to check a JWT. (s/def :my.security/jwt ....) Doesn't mean that the token is good enough to get through authentication. Spec will check that it is structurally sound

The spec might be pretty involved, but you can't have something that is valid? one minute and not the next

spec predicates should not be hooked up to state. a point of the specs is to have enduring, sharable meaning

Right, that makes sense

I wish I had a more succinct rationale. Maybe @alexmiller will chime in later

I didn't realize that it was a deliberate decision to prioritize that sharable meaning by disallowing runtime context.

yeah, I hope I don't sound cranky about spec at all --- I think it's a really interesting idea and solves some hard problems.

it doesn't disallow it, it's the runtime context (AKA mutable state) that will work against spec's goals

that's why I'm trying to use it in this context

I don't want to rely on implicit runtime context or mutable state

I'm just trying to pass that in

Anyway, thanks for the chat and all your work on this stuff

There are some other places where I've very happily used spec for validation/generation = )

I don’t have a lot to add to Ghadi’s comments which I mostly agree with. Some people are doing runtime generation and registration of specs (by loading values from a db for example). That seems fine to me (it does get a little trickier with s/keys).

in constrast, specs that use stateful stuff (dynvars, atoms, etc) seem bad to me

what is spec buying you in this case vs just writing code to assert whatever predicate you have?

@alexmiller I have the same kinds of problems that spec solves very well: composing predicates and giving back specific errors about why things failed validation. E.g., the AddChild event I mentioned earlier actually needs to take multiple parent-ids, and it would be great to get the detailed validation errors from spec.

"Cannot add child because the third item in the list is of type X, but you can only add children to types U, V, W"

ok

you can easily express that in spec if you write a predicate

it's the db extrinsic stuff that is a anti-recommendation

@ghadi The predicate can only run against reified entities from the DB. Are you suggesting something like (->> parent-ids #(d/entity db %) #(s/valid? (s/coll-of ::parent))?

Where ::parent is a spec that has the relevant predicates?

sure, that's better -- fetch the data before asking spec

(I was suggesting just building plain, offline predicates)

if there are different types of nodes, some that have parent-ids and some that don't, you should be able to represent that in the data directly, without querying something external

Just to clarify, all of this is happening in an in-memory database, so there is no network fetching or anything like that.

Right but if that thing can change, YMMV

it's immutable.

ok as long as a payload that is valid? right now, is valid? in the future, that's good.

If that can flip, bad kitty

i'm assuming you wouldn't call it a database if the data could never change

this will probably get me in trouble: specs should only check stuff intrinsic in the data.

(s/def :my.expiration/date #(re-matches #"\d{4}-\d{2}-\d{2}" %)

(aside: are regexes valid specs?)

Spec will generate you a bunch of :my.expiration/date's, some might be expired, some not

Another way to phrase that question: Should spec allow you to validate {:time #inst"..." :expr #inst"..."}

part of the spec is that both values should be dates

but should you check in spec whether the :expr date is before or after the :time date.

sure

it's a bad idea to call (Clock/now) while validating the expiration

because that's extrinsic

Yeah, I understand that part

I've been drinking the clojure kool aid for a while, don't worry = )

heh

Then you should have no problem extrapolating (Clock/now) to a database 😃

calling (d/entity some-db some-id) essentially turns my integer id into an immutable view into the whole database

so reifying the IDs into entities, then calling spec makes sense.

it's weird though, when you check {:time #inst"..." :expr #inst"..."} it's a totally different entity that you're checking. Something like a token-request

we're in sync

i think originally I was thinking the db lookup was happening within the spec definition

well, that was my question, if it'd be possible to do that.

but you can't without dynamic var tomfoolery

which I don't want to do.

It's still awkward, though, since I can't really spec out my events, since I can't spec their integer-id arguments

The best I can do is write my own validation fn that hydrates the ids into immutable entities, then calls spec on those entities.

ditto for generation, I can't use spec to generate anything.

The best I could do is, for a given DB, enumerate all entities, see which ones are valid? against a specific spec, then return their ids

Anyway, I'll give that a shot and see how tidy I can make things. I may come back in a week or two with a follow up report + gist example

Thanks again for the conversation!