Fork me on GitHub
#clojure
<
2019-11-21
>
didibus02:11:05

I seem to still struggle with organizing and naming things when trying to use qualified keyword keys? Does anyone else feel that way?

didibus02:11:02

Getting something: (-> m :db/user :db.user/profile :db.user.profile/name)

didibus02:11:59

Maybe I'm wrong trying to outline the hierarchy inside the namespace, but otherwise it gets tricky. Say I also have a :db.group.profile/name

didibus02:11:45

And since I have specs for all these, and specs are in a global namespace, clashes can happen

didibus02:11:09

If I just said :profile/name it would seem way less heavy handed, but someone somewhere else would overwrite its spec by accident when creating group profile names.

didibus02:11:11

Another issue I sometime face is that maybe I have :db/user be a spec with a qualified key of say :db.user/name and maybe I have some API called create-user and I don't want to tie its spec to the db spec. So I want a :create-user/user key spec, but for now that's the same as the db, except if I say: (s/def :create-user/user :db/user) it means that the API has to take a map of the form: #:db.user{:name "John"}

vemv03:11:33

I advocate something along these lines: • all specs are ns-qualified (`::foo`) • these ns-qualifed names are also used as Datomic attribute names • each ns represents one model • each ns is an API , solely dedicated to containing specs. That way I know to not alter the ns name • each model has one "write model" (a keysspec) which is the sole thing allowed to be persisted, and N optional "read models" that are derived from the write model • DRY specs (the problem you indicate in Yet I don't want to have to redefine a whole new spec) are achieved through metaprogramming I've worked with these to different degrees throughout the last 2 years

didibus03:11:02

I think I need details of each ns represents one model

didibus03:11:16

What about my case where I have:

{:db {:user {:profile {:name "John"}}}}

didibus03:11:36

That's my entity?

didibus04:11:22

Would that be 3 models in your case? A db ns, a user ns, and a profile ns ?

vemv04:11:19

I see a user model and a profile model. Just like I would in something like Rails (two SQL tables, two ActiveRecord models)

didibus04:11:00

So you'd just rely on aliases to make that pleasant? Like

(-> m ::db/user ::user/profile ::profile/name)

didibus04:11:19

And what about your API specs? This defines your data specs for what goes in your DB. If you have APIs, like Rest APIs, do you have them share specs with the DB ones, or you just create new ones for that?

didibus04:11:39

And map them internally

vemv04:11:48

> So you'd just rely on aliases to make that pleasant? Yes Probably I'd code it like this (-> m ::user/model ::user/profile ::profile/name) i.e. get rid of the db prefix. In Datomic I don't use that prefix - I leave it reserved for Datomic internals

didibus04:11:49

Oh, I think that was an accident. I actually don't use Datomic. Was just a coincidence my key is also a datomic key it seems.

👍 1
didibus04:11:50

Ya, I can try that approach. You kinda have to require a lot of models 😛 to get all those aliases. But its better then what I currently had where I just typed it all out.

didibus04:11:56

I also had all my models in one namespace. And I wasn't using ::, I was creating namespaces by hand as I spec things. So that meant I couldn't rely on aliasing to make the names smaller.

vemv04:11:25

wrt REST, here's an example: (ns invoices.write-model) (spec/def ::foo int?) (spec/def ::bar int?) (spec/def ::model (spec/keys :req [::foo ::bar])) (ns invoices.rest-api) (spec/def ::GET (spec/keys :req-un [:invoices.write-model/foo])) ;; I can skip bar.

vemv04:11:50

my advocated pattern works well at scale, it's thought out for a landscape of multiple apps, modules etc might be overkill for other scenarios

didibus04:11:02

Right, I see, so you define a new keys spec for the API, using unqualfied versions of the write spec. Not bad.

didibus04:11:21

I like it.

didibus04:11:32

All of it does seem to address some of my pain points

✌️ 1
didibus04:11:37

Its kind of an interesting topic. I feel there would be a lot of different strategies out there. Everytime I start something new I feel I change slightly how I spec things 😛

didibus04:11:02

Thanks for talking about yours.

🙌 1
vemv04:11:26

yeah it's part of the beauty of spec - not being too prescriptive, allowing a variety of designs/purposes etc :)

jjttjj04:11:04

I faced these same issues with my library Iboga, which wraps my stock broker's java api client. Basically I use relflection to turn their method calls into data messages to be passed around. So the method

com.ib.client.EWrapper/historicalData(int id, com.ib.client.Bar bar)
becomes (fully qualified version:)
:iboga.recv/historical-data {:iboga.recv.historical-data/id 123 :iboga.recv.historical-data/bar {:iboga.bar/time 111 :iboga.bar/close 222 ....}]

👀 1
jjttjj04:11:56

[arg this new slack formatting will take some getting used to]

💯 1
👍 1
jjttjj04:11:21

orginally the user interface I presented eliminated the need to use qualified keywords, and the structure was walked and qualified to be used internally/spec'd

jjttjj04:11:42

But then I started dog-fooding the library myself and found that increasingly I wanted the verbose fully qualified keys and switched back to just requiring them (in the unpublished current version)

jjttjj04:11:15

the qualified keys names are generated automatically from java reflection, and there are hundreds of "methods" with 0-10 args (average probably 2-3)

didibus04:11:39

Do you use aliases to help out? Or you fully type them out always?

jjttjj04:11:45

I could probably get away with less qualified keys, like qualifying the method name keys but not the arg name keys, but I think I've grown to accept that it's just going to be somewhat verbose typing these things. Internally they all need to be fully qualified so just forcing them to be input that way means there's less code the library has to maintain

jjttjj04:11:59

I end up typing them out fully

jjttjj04:11:36

the map qualifyier tag thingy, ie #:x.y{:z 1} does help a lot though

jjttjj04:11:54

I also kind of feel that I'm using keywords too "hierarchically" when they're not particularly great to use that way

jjttjj05:11:31

when representing something that's a graph, I don't really get any leverage from structuring my keywords that way. But then when I also want them to be globally unique, I end up having to name them their full graph path anyway

jjttjj05:11:27

these are just some ramblings about keywords I don't really have a point. But I too have wondered if other people had pain points here

didibus05:11:11

Ya, I share some of your troubles for sure

didibus02:11:33

But I want it to take a map of the form: #:create-user{:name "John"}

didibus02:11:59

Yet I don't want to have to redefine a whole new spec, since for now they're the same... except for their namespaces

didibus02:11:12

All this tempts me to go fully unqualified

jjttjj04:11:44

I very much relate to this as well

Gerome09:11:27

Hi! I’m trying to solve a problem for localization testers of an SPA. I constantly have to create breadcrumbs, so they know how to find the texts that I added or changed. So, I was wondering, If I could use a webcrawler for that. I looked at two of them but the way it looks they just follow links and create an index of the pages they’ve visited. They can collect information from the page but if I want to create an index of breadcrumbs for all texts, the crawler would have to click buttons and write texts and stuff like that. Is this something webcrawlers can do?

weavejester11:11:37

I’m struggling to remember the name of a Clojure library that generates data structures for testing databases. I have a vague recollection it used spec? Does anyone have any idea which library this might be?

grav13:11:45

I'm looking for a style guide or some discussion around using _foo for unused vars that are kept for the sake of documentation. It might be in a defmulti like this:

(defmulti [kind _ctx] kind)
Can't find anything on Google

alexmiller13:11:21

its just a convention to mark unused locals with a leading underscore

alexmiller13:11:01

oftentimes that will show up in abstraction methods (interrfaces, multimethods, protocols) b/c you might not need all the params in every method impl

alexmiller13:11:31

but they are not special in any way, just regular symbols treated exactly the same as any other local symbol

grav11:11:33

Cool, I do see several tools supporting this convention (Cursive, as well as some linters), so I just thought it might be more than just a convention. It's something that I use myself, but if I'd like to get a whole team to use it, it's always good to have some kind of reference at least 🙂

alexmiller13:11:30

In case it helps...

👍 1
kah0ona13:11:46

Hi folks, i’m using @seancorfield’s dot-clojure file, and trying out building an uberjar. It builds succesfully, but upon running, it throws ClassNotFoundException for clojure.pprint. I run it as clj -A:uberjar:1.10 MyUberJar.jar

kah0ona13:11:24

should my deps.edn contain clojure.pprint?

alexmiller13:11:05

you can't run uberjars through clj like that

alexmiller13:11:27

clj will make a command line like java -cp ... clojure.main ...main-args...

alexmiller13:11:47

for an uberjar, you want java -cp MyUberJar.jar

alexmiller13:11:19

or java -jar MyUberJar.jar depending on how the jar is created and whether it has a manifest with a main

kah0ona13:11:38

ah just tried adding clojure.pprint in the require of the main’s ns, rather than using (clojure.pprint/pprint {:my :data} in my code directly … works

kah0ona13:11:21

which is a pretty awesome concept, having such a dot-file, thanks Sean!

1
bmaddy18:11:40

Does anyone remember the name of the site where you can put sample input and output for a clojure function and it will tell you what function it might be? I believe it used spec to guess what would work.

Joe Lane06:11:06

If you want it as a desktop application, you know who to ask ;)

parrot 1
samedhi20:11:25

Hah, that is pretty cool.

bmaddy18:11:33

Totally! Thank you @dpsutton!

josh_tackett21:11:47

Anyone know how to get around recaptcha with selenium?

dominicm22:11:23

You can't really, that's the point

dominicm22:11:07

The reliable solution is usually to pay people in poor countries to fill it out.

mattsfrey22:11:28

yeah, you make a mechanical turk app that receives captcha images and the turker enters the captcha text and it posts back and you proceed