Fork me on GitHub
#datomic
<
2021-04-02
>
Oliver George06:04:29

I'm researching how Datomic cloud functionality fits with a user extending or constraining the schema. Simple things like adding attributes is fine. Attribute and entity predicates sound useful. Question about entity predicates... it seems they are designed to reference predicates implemented as code and deployed. Are there ways the predicate could be defined or configured without redeploying (e.g. transacting an s-expr to use as a predicate)?

Oliver George08:04:44

Pretty sure the answer is no for sensible reasons. One workaround might be some generic predicates configured by data accessible via the db arg.

Joe Lane14:04:28

Hi @U055DUUFS, I think what you're asking for is "How can I supply an arbitrary predicate in a transaction along side the data it should operate upon" Correct?

Oliver George20:04:17

Sounds like an interesting approach

enn13:04:50

Good morning. Can someone confirm that I’m correctly understanding the following quality of composite tuple attrs? Let’s say I have a composite tuple attr :thing/foo+bar of two attrs :thing/foo and :thing/bar. Once this attr exists, I can never create a new attr :thing/foo, even if I first rename the existing :thing/foo to something else beforehand, because the original :thing/foo participates in the composite tuple attr under its original name. In practice this seems to be the case--I get a :db.error/cannot-retarget-ident. I’m wondering if there is any workaround here.

enn13:04:19

For example, maybe I could: 1. rename :thing/foo to :thing/foo-old 2. update the :db/tupleAttrs of :thing/foo+bar from [:thing/foo :thing/bar] to [:thing/foo-old :thing/bar] (is this possible?) 3. add new attr :thing/foo 4. update the :db/tupleAttrs of :thing/foo+bar back to [:thing/foo :thing/bar] But I can’t tell from the documentation if it’s even possible to change the :db/tupleAttrs on an existing attribute.

prnc14:04:27

Hello 👋 Assuming that I have quite a bit of urls i.e. :db.type/uri in Datomic and want to be able to query them, say by host. Is this OK (1) (idiomatic, performance etc.) or should I be “unpacking urls” into attributes (2)?

;; e.g. (1) "query the object"
  '[,,,
    :where
    [(.getHost ?url) ?h]
    [(= "" ?h)]]

  ;; OR e.g. (2) "store expanded", and query `:uri/host`
  {:uri/host "",
   :uri/path "/wiki/Safari_(web_browser)",
   :uri/full-uri
   #object[com.cognitect.transit.impl.URIImpl 0x58ec6b0c ""]}
  

Joe Lane14:04:53

It depends on: • How many is "quite a bit"? • Will you be filtering on any other criteria first? • What is your SLA (if you have one)? • The exchange of time vs space (like any algorithm) An interesting thought exercise would be to think about 1. How much "work" is needed between the approaches and 2. When do you want to do that work? Approach 1. Creates lots of objects at query time, which can consume memory and take CPU time (maybe a negligible amount, depends on the size of "quite a bit") but you do it every time you query for every URI you need to compare against (including ones that won't end up in the results). Maybe you don't want to create 3x as many datoms (approach 2) so you're ok with that. Approach 2. performs the work only once, but exchanges space for that time. Things to consider: • How big is "quite a bit" • How big is your database going to get? Do you have a plan for this? • With production workloads (I assume you'll have these?) how much memory will one of these queries consume, and, how many queries do you expect to be serving at a time. This will determine how much memory you might need per peer (on-prem) / query group node (cloud). THIS MAY DECIDE YOUR ANSWER FOR YOU • You should MEASURE THE DIFFERENCES YOURSELF! Only you can determine which solution will be performant enough!

prnc15:04:02

Thanks Joe! It’s a new project so I’m just getting a sense of what’s needed, experimenting w/ cloud/solo topology atm—to learn how things work. Just wanted to get a sense if both approaches are valid and get a better grasp of trade-offs, which you kindly provided 😉 So, thanks again!

Joe Lane15:04:30

Glad to help, always happy to talk about this kind of stuff!

Joe Lane14:04:53

It depends on: • How many is "quite a bit"? • Will you be filtering on any other criteria first? • What is your SLA (if you have one)? • The exchange of time vs space (like any algorithm) An interesting thought exercise would be to think about 1. How much "work" is needed between the approaches and 2. When do you want to do that work? Approach 1. Creates lots of objects at query time, which can consume memory and take CPU time (maybe a negligible amount, depends on the size of "quite a bit") but you do it every time you query for every URI you need to compare against (including ones that won't end up in the results). Maybe you don't want to create 3x as many datoms (approach 2) so you're ok with that. Approach 2. performs the work only once, but exchanges space for that time. Things to consider: • How big is "quite a bit" • How big is your database going to get? Do you have a plan for this? • With production workloads (I assume you'll have these?) how much memory will one of these queries consume, and, how many queries do you expect to be serving at a time. This will determine how much memory you might need per peer (on-prem) / query group node (cloud). THIS MAY DECIDE YOUR ANSWER FOR YOU • You should MEASURE THE DIFFERENCES YOURSELF! Only you can determine which solution will be performant enough!

shields17:04:49

Hello, I have a question about using Datomic Cloud along with https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html. I've https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html the Kafka clusters in my Datomic VPC and added them to the subnets. I understand that the bastion just gives access to the Client API if I'm not mistaken. Is there a way to dev locally w/ the created "bootstrap.servers" in MSK? Any additional steps? Or is it not possible? Apologies if I'm missing anything obvious, but any suggestions would be helpful. Thanks.

csm19:04:30

AIUI MSK is only accessible within your VPC, so to connect from your local you either need your own bastion or a VPN connection

kenny18:04:51

Under "https://docs.datomic.com/cloud/releases.html#current" there is this bullet point: > Ion libraries (`ion` and `ion-dev`) are available on the https://docs.datomic.com/cloud/ions/ions-reference.html#libraries. The link https://docs.datomic.com/cloud/ions/ions-reference.html#libraries seems to take you to the wrong stop (just the top of the "Ions Reference" page). That doesn't seem like the desired location.