Are there are any long-term future plans to allow extending Datomic with custom data types, such as sparse matrices to d o the matrix math needed for collaborative filtering, or is this something that could be handled by database functions already + caching?

Hello @marshall, found a dead link. “ion :allow list” under https://docs.datomic.com/cloud/query/query-data-reference.html#calling-clojure-functions

@henrik fixed - thanks!

Any conformity people here? I’m trying to use c/ensure-conforms to set up my db schema, and I’m getting “:db.error/not-a-data-function Not a data function: 10”. I don’t have “10" anywhere in my norms, and there’s no reference to any conformity code in the stack trace. This is on an in-memory db (datomic on-prem), so no previous schema in the db.

Here’s the norm:

{::fz.location
   {:txes [{:db/ident :fz.location/name
            :db/valueType :db.type/string
            :db/cardinality :db.cardinality/one
            :db/doc "Location name"}]}

Co-worker helped me figure this out: :txes has to be a vector of vectors of maps, not just a vector of maps.

Hey, datalog question here. I have an entities with a cardinality-many keyword attribute :x/colors. I want to write a datalog query that accepts a set of color keywords, and returns every entity for which the entire set of :x/colors for the entity is the same as the set I passed in.

I'm a little confused about how to do this - using collection binding I only match on a single :color at a time

And then in the context of a datalog clause I'm not sure how to get the entire set of :x/colors for a single entity e.

I'm not sure how the iterable binding approach would work, but I'd probably do something like this:

(defn get-colors
    [db eid]
    (let [results (->> eid
                       (d/pull db '[:x/colors])
                       (map :x/colors)
                       set)]
      results))

  (d/q '[:find ?e
         :in $ ?search
         :where
         [?e :x/colors]
         [(foo/get-colors $ ?e) ?search]]
       (d/db conn) #{"blue" "green"})

I can see how that would work, but I'm still hoping for a way to do it inside of the query itself

I guess I'm hoping for a way to write an (every?) predicate or something

I found this StackOverflow answer which seems like it might be heading in the right direction: https://stackoverflow.com/questions/23352421/for-all-in-datalog

Is it possible to do a Datomic restore across AWS accounts? I've given the other account (i.e. the one that owns the destination dynamo table) access to the backup bucket in S3, and I can do aws s3 ls from the CLI w/ those creds, but Datomic's bin/datomic restore-db still crashes with S3 access errors.

you’ll need to make sure the role/credentials used by the restore-db call (peer application) have read permissions to the s3 bucket

that should be all that’s required

keep in mind, aws s3 ls isnt’ the same as being able to read all the objects in that bucket

IAM has separate permissions for getting objects and listing bucket contents

@marshall OK I think you clued me into the right thing here. I need to set this up using IAM, not ACLs.

yeah, it may be possible with ACLs, but I think you’ll have better luck / more control with IAM

also, I believe S3 ACLs are a legacy tool that pre-dates IAM or bucket policies

Hmm, the only stuff I'm seeing around this involves assuming roles. I'm not having any luck combining that with a Datomic restore-db command.

run the instance with a role

whatever instance you’re running the restore job on

alternatively, give the role to a user

and run with that user’s credentials

i.e. if you’re running the restore on your local machine

Is there a way to give a role to a user other than aws sts assume-role? I tried that, then tried using the creds it gave me, but those aren't working.

you can assign a user to a group and put the role on that group

actually i may be thinking of policies

Oh, I think I may have figured it out. I needed to set the AWS_SESSION_TOKEN env var too. And now it looks like I need to give this role permission to write to the destination dynamo table b/c it drops existing privs when you assume the role.

Hmm, no. That doesn't work either. It tries to restore to the DynamoDB table in the account that the assumed role gave it access to. So I'm back at square one. I'm not sure how to grant one side of a restore access to another account's S3 bucket and the opposite side access to the primary account's DynamoDB table. 😕

Using a bucket policy that gave permissions to the other account in the Principal field fixed this ^^^. And with no more need to mess with assuming roles.