Clojurians Log v2

The project structure is taking shape. If I’m making a mistake, it’s taking on a project of these dimensions. It’s moot though. I didn’t choose the project. For better and worse, it chose me. In some ways, it’s my _<https://en.wikipedia.org/wiki/Illusions_(Bach_novel)|Illusions>_. This is a very data-driven system. Very. Boiled down, it is in fact an anatomy-driven system. Anatomy is very complex, particularly neurocognitive anatomy.

I’m deeply grateful. I imagine I’ll have more to say/ask once I metabolize what you’ve shared. Again, thank you. Please brain dump and shamelessly plug as the spirit moves. Drive-bys are so encouraging.

I’ve been more a script writer that a developer. Test-driven thinking takes a minute to grok.

I’m digging into malli. It is the way. I’m just no good at it. I’m so ignorant. Onward.

I’m only trying to do what the Romans do at this point. I’m reading through malli. Thanks.

Do you ever need spec or does malli do everything you’ve needed so far?

I find spec is great for speccing clojure code - eg smarter parsing and error messages in custom macros.

But anything to do with validating user/external data - I prefer to reach for malli

Rich Hickey talked about spec2 in one of his talks, where they separate schema vs select, which solves some of the problems of context-localized specs; but spec2 is yet to be released and malli is available :)

So helpful. Thank you.

Gold. RDF made practical. SPARQL++. If that is your version of shamelessness, run naked. Thank you. I have an RDF vs LPG question. Sometimes I want an unreified relationship (RDF-esque via :db/ref). Sometimes I need relationship info (LPG). To boot, I know my wants/needs will flip-flop. My current thinking is I can have both using the same ref attribute. One hop is RDF, two is LPG with my properties hanging off the interstitial entity. How have you dealt with not knowing what you need of relationships ahead of time?

Thank you for the kind words. gratitude

It's a good question... and appropriately "it depends". XD But here's some coffee thoughts:

First of all, my current day job is dealing with SQL spaghetti, so take everything I say with a grain of salt, since I've been out of the trenches for a while.

As you point out, you can make everything RDF/LPG by just changing the number of hops. So, the good news is, you can start with one or the other and just change your mind later. P*ick a new name* (I especially love the looks people give me when I just tack a 2 suffix onto something if I can't be more clever) and move on with your life. Don't try to reuse names (especially in Datomic, but also in your APIs) and you'll be fine.

Another rule of thumb is how important this thing is to your domain. If you use it as a noun when you're talking about it out loud, or you have some notion of a platonic identity when you think about it, it's probably safer to make it an LPG (even with just one attribute).

Another thing to consider is the database/storage/querying strategy. Nothing exists in a vacuum - so knowing your tooling may help steer your decision. For example, some things that I would model as a separate entity in Datomic, I may model as a property label in Neo4J, just because the querying/indexing has certain optimizations that take better advantage of one or the other.

Another trick is to remember that not everything needs to be neatly denormalized; if you create a single hop :db/ref and later realize you need one more piece of info, just add a second attribute to the parent entity. Just like in SQL, 3NF is what you learn in school, but not what you use in production. 😂

Last thing I would mention is to be open to ideas. People become fixated on what they know best. I once listened to a complicated schema model in Neo4J that had lots of labels hanging off the edges and suggested that perhaps they can just create an actual node that represents the <thing> in the domain. You could see their eyes light up when they realized it could simplify their lives (and they were in a local maxima fixated on LPGs). Same could be said for RDF style modeling (where we don't have labels, so obviously we will fight anyone that says that they are kind of useful in certain situations).

Helpful thinking. Thank you. I’m tempted to do two hops for now, even when I’m just traversing. Therefore, the intermediate entity will always have a ref. If only space and time didn’t matter. My first-pass thinking was :<domain-object>/<relationship> (RDF) and :<domain-object>/reified/<relationship> (LPG). The trouble I see there is I’m duplicating data and need a reification of the unreified and reified edges. Yikes. We’re back to your quite-accurate “it depends”. I need more experience with my model. Maybe I’ll double-hop my way to the understanding I need. edit: removed “always”

You may end up in a place where you realize you've got lots of "meta" attached to the relationships and perhaps a DB that more accurately helps you work with that structure is actually more convenient. That's OK, especially if you're open to recognizing it... Datomic is not a panacea.

Indeed, nothing is a panacea. Me thinks it’s flexibility I need most right now. Datomic seems like the place to start anyway.

My first gut reaction, given no other context, is to create explicit entities if I can visualize that <thing> is some kind of <essence> in my domain (that I can try to point to, try to name, etc). That <essence> can even just be a reified "relationship" or "verb" instead of a noun. But for some reason I find it may be important in the domain

Actually, the exercise of trying to give the thing a name is often useful in and of itself, even if I end up scratching the idea.

^ The corollary of the above is that if the thing I'm thinking of is a piece of data (in the Clojure sense of a persistent data structure) and doesn't have it's own identity, it may be best as a value attribute. Even if it's a nested/complicated piece of data, you can just serialize some EDN to a string and store it like a blob. EDIT: Not :db/ref but regular value attribute.

Very helpful thoughts. Risking that I’ve missed a subtlety in your words, I’m starting with two hops (LPG). It leaves things more flexible. I’ll optimize when experience shows I don’t need the extra structure.

You are so generous. Whether I pay back when I can or pay forward, I’m so grateful for the connection.

If you build indirections in your code between your logic and your DB storage, you'll find it easier to change your mind and consider alternatives when you hit a wall.

Reminds me of a book which you may enjoy - https://elementsofclojure.com/

Zach is really good at waxing poetic about thinking about indirection, abstraction, etc.

I’m not smart enough yet to understand what “indirections in my code” look like. Sounds like the book is what I need. Thank you.

> Whether I pay back when I can or pay forward, I’m so grateful for the connection. The pleasure is all mine. Looking forward to future threads. :)

I will benefit from this conversation for a long time.

Another book I'm always quick to recommend is https://grokkingsimplicity.com/

It may feel a little "too easy", because Eric wrote the book for "beginners", but he has some really good insights on software development and FP. Alternatively, you can try to gleam the insights e.g. from his blog / podcast / etc.

Alright, I'm off to get some work done. It's been a real pleasure and looking forward to hearing more about the project. And remember - Clojurians Slack has a LOT of smart and caring people around, so I'm in the same boat as you - just here to learn and take part in this fine community. :)

You are a gem. I can’t thank you enough.

For prompt engineering and chat bot advice, you may want to hang around #ai and #clojurellm - it’s not exactly a 1:1 correspondence, but I bet people there can help you with at least the broad strokes.

Thanks for joining. I’m honored.

I support every initiative to learn in public! ❤️

Thank you. If you are an American like me, you’re an unusual one. It’s so American to say “You’re already supposed to know that!!” Appearing informed is a cultural norm that deeply undermines education — and bores me. edit for clarity

I dislike that attitude too. I don't think it's helpful for learning. It's very important to be able to say "i don't know how this works" without punishment. Otherwise we can't talk about the things we don't know yet! It's also hard to get help from others if they don't know what you don't understand. I've learned a lot about this from @borkdude in #babashka and other channels. I'm Norwegian! Though I'm not sure I'm a typical Norwegian either 😄

I can’t wait to visit Norway. Takk.