yeah, I felt tension speccing pull-patterns vs pulled data too, but I think pull-pattern should not be specced with s/keys, which kinda resolves the situation

In my case, we only do two layers of transformation before passing the data out. Input is big, mangled and flaky. Given all that it felt the domain - if there is one indeed - was rather thin, and thus there were few benefits to be gained in namespacing everything. But then we're using it for data engineering over Kafka, so we're basically talking about transit transformations, rather than domain manipulations.

> spec 2 select just says that the notion of required/optional keys is contextual ... but there is no explicit syntax of marking some keys as optional in select? Rather implicit "If the key if not part of the select, it's optional", right?

there's no equivalent for the opt declaration in spec 1? "if present, it has to be this shape" thing

Schemas in spec 2 are all optional

Select specifies what’s required in a particular context

I have a question that I think is related to the conversation above. If I have the following datomic-style schema:

{:db/ident       :user/email
    :db/unique      :db.unique/identity
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/many}

(s/def :email/address ::non-empty-string)
(s/def :user/email (s/coll-of :email/address))    ;; matches the schema in name and shape
(s/def :user.input/email (s/or :one :email/address :many :user/email))  ;; matches what the boundaries of the system would accept

with specs it's always best to try to "state the truth" so I'd go with something that took either

for the last question, might be best to ask that in #datomic, I'm not well-versed enough to have a good answer

That makes sense. The truth here being that, indeed, the input is a different thing than what comes out of the database?

I suppose I could restate the question as: if I want my specs to “follow” the schema, should I spec what comes out, or what goes in?

By “follow” I mean “possess the same semantics”

I think I may have answered my own question…I’m realizing that spec’ing input is much more useful than having a truthful “spec+schema” (the latter of which is likely redundant)

In other words, spec’ing what comes out of the database is pointless…at that point it’s too late. Spec efforts should be focused at the input boundaries of the system.

Interestingly tho this makes spec and the :db/ensure feature of datomic feel a bit incompatible, because predicate functions given to :db/ensure have the signature (defn my-pred [db eid])…and so they can only validate what comes out

I think it’s better to think of the transaction as a DSL. You can spec a dsl’s grammar, but not its meaning

so you can’t spec :user/email specifically as input to d/transact, only as an opaque key in a transaction map

you should either spec a level higher, e.g. some function that takes well-formed input and produces a set of transaction assertions/retractions/maps that do what you want; or a level lower with data-aware assertions, e.g. perhaps the DSL produces an ast which is easier to analyze, or in the case of datomic :db/ensure to guarantee your invariants.

it’s a little less obvious how to leverage spec in the “level lower” cases

combining spec with datomic, I’ve had more luck specing what you would want to see out of d/pull, because that’s what’s flowing through the functions in an application

that feels like where spec shines the most

and I’ve just accepted there’s no silver bullet for the cases where the data approaches the edges, e.g. to serialize/deserialize or to transact to the db