clojure-spec 2023-05-17 | Slack Archive

I'm trying to understand if this is an okay usage for spec, and if it's possible. I have user spec like so:

(s/def ::email string?)
(s/def ::id string?)
(s/def ::user (s/keys :req [::email ::id]))

I'm going to store this user in DynamoDB, but to do that I need to transform from DynamoDB's shape to my ::user spec. So far I have record->user and user->record helper functions to transform from one to the other. It works, but I need to manually type in the mapping if I add new attributes. Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user, and have the transformation functions auto-created as well? Or is there another approach I should look into?

phronmophobic16:05:15

> have the transformation functions auto-created as well Are the transformation doing something more than removing the namespace of the keyword?

phronmophobic16:05:22

> Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user, and have the transformation functions auto-created as well? Spec is agnostic to where the data comes from or is going to. It's possible to write this sort of functionality in clojure, but I might worry about artificially creating an https://en.wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch. It's hard to recommend anything specific without knowing more about the use case. • How do you plan to use Dynamo? • Are interacting with existing data or do you have more control over the format used to store data? • Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option?

john2x23:05:48

> Are the transformation doing something more than removing the namespace of the keyword? Basically this

(defn user->record
  "Transform from ::models.user/user to DynamoDB record data that can be written"
  [user]
  {:id {:S (::models.user/id user)}
   :email {:S (::models.user/email user)}})

(defn record->user
  "Transform raw DynamoDB record data to ::models.user/user spec"
  [data]
  {::models.user/email (get-in data [:email :S])
   ::models.user/id (get-in data [:id :S])})

I guess this is my OOP background leaking. But I would prefer if I'm using and passing around ::user maps everywhere, and only use the DynamoDB {:id {:S "foo"}} shape on the edges.

john2x23:05:06

> • How do you plan to use Dynamo? > • Are interacting with existing data or do you have more control over the format used to store data? > • Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option? Nothing fancy. Just intending to use it as a document store. I do have full control over the data. And I'm using com.cognitect.aws/api

phronmophobic23:05:18

What is the :S key?

john2x23:05:01

It's just dynamodb's way of saying that the value is a string

phronmophobic23:05:51

It seems like the translation to/from could be automatic rather than manual. Is there a reason not to just iterate through all the keys in the provided map and strip/append the keyword namespace info?

john2x23:05:28

yeah I think that could work. It might get tricky if I use more complex specs/shapes (e.g. nested). Fortunately I don't see this work doing anything too fancy with the schema for now

phronmophobic00:05:28

For inspiration, you can check out how https://github.com/seancorfield/next-jdbc interfaces with databases. Another higher level API that might provide some ideas is datalevin, which also provides a clojure interface over a key/value store, https://github.com/juji-io/datalevin#use-as-a-key-value-store.

Ed14:05:34

I've definitely written something that converts back and forth between DDB's representation and clojure's literal data. A quick google found this that you could use for inspiration: https://github.com/doo/clj-dynamodb/blob/master/src/clj_dynamodb/convert/to_dynamodb.clj (nothing to do with me - all my impl's have been closed source)

Ed14:05:04

But I think it's generally a better idea to explicitly convert your data structures from DDB land to Clojure data and back with function calls, rather than rely on something like spec to coerce the data from one structure into another. I think that reduces the utility of the spec.

john2x22:05:53

Yeah, I'm starting with just manually writing the conversion for now. Thanks for all the suggestions

2023-05-17

Channels