@UJZ6S8YR2 Also checkout https://github.com/markbastian/replion ! Will get you a lot closer to the Cloud/Ions based environment

Thanks @UNRDXKBNY 🙂

I guess I could use d/with to some degree as showed here http://jamespw.com/clojure/datomic/testing/2016/09/17/testing-datomic-queries-and-pulls-using-with.html

@UJZ6S8YR2 I made a development solo system which I used for over 2 years successfully without a local offline system. I wouldn't complicate things with the datomic-client-memdb or any other replacement for just having an internet connection until you run into an actual pain point. If cost is the concern you can use the datomic CLI Tools to turn on your bastion+solo node when you begin doing development for the day and shut it off at the end of the day (or set an Autoscaling group to do it for you on a schedule).

The last thing you want to do when just getting started is have divergent library behavior between local development and production.

FWIW, I wrote that library and agree with @U0CJ19XAM. It's very unfortunate there is no local Datomic Cloud solution. If you can get away with using a solo deployment for your needs, you most certainly should. You will need to build some tooling around how to use a single solo deployment with multiple developers but that isn't too difficult (we prefix DB names). We have the ability to easily switch to that library for those that have a poor/no internet connection. We run integration tests against Datomic Cloud.

Thanks @U0CJ19XAM and @U083D6HK9! I didn’t know you could turn on/off the bastion and node like that with CLI Tools, that sounds quite handy approach. I was also concerned about the integration with CI-pipeline but it should be okay just to use the Cloud like you are doing, Kenny.

It took some work to get it to work on CI 😬

When you set up CI, do it in the us-east-1 region.

Otherwise you will have headaches...

> When you set up CI, do it in the us-east-1 region. What does this mean?

Datomic Cloud region or CI region? We use CircleCI and I don't think they have a region selector.

Sorry, I assumed codebuild+codepipeline. If you use those, set them up in the us-east-1 region.

Its the classic "cross-region s3 bucket access from within a vpc" issue :)

you can also avoid creating the bastion and connect directly to the node, set a firewall rule to allow and protect access

I’d suggest against that due to the security implications.

as?

You have created a hole in your VPC to the outside world 🙂 unless you have 100% certainty (and a security policy that allows such a thing) that your hole will always hold true (e.g., reserved static IP), you have an issue.

yep, for solo, restricting access to an IP in the security group

how will that won't always hold true?

You just be 100% sure that the IP is owned by you. Typically an IP provided to you by an ISP is not static. Most hosted CI solutions (e.g., CircleCI) do not make any guarantees about the IP addresses of their worker nodes.

An example of an IP owned by you would be an EIP.

Yeah, I understand, but for a solo dev on solo playing on the weekends it might be enough

anyway, I just saw that now there are some CLI tools to simplify access

so there's less reason to avoid bastion

Never do that.

u would need to have a valid DB reference to that state of the DB which contained the datoms still before the restore, then u can do something like (seq (d/datoms (d/history db-value-before-restore) :eavt)) , but not sure if it would work

Then I could import them with which command?

nearly done writing an import script anyway — but want to try this first

you have to program that yourself

gist online of something like this about 200 lines.. looks sketchy

u just get back a sequence of Datom objects, which you can either thread through (map (partial into {})) or (map (partial into [])) , then if u want to transact them back to the DB, you would need to massage them into [:db/add ....] or [:db/retract ...] form

i haven't seen any off-the-shelf solution for this kind of situation, but technically you want those datoms transacted back into your DB what you see thru the history DB. you might even want to use d/since to just get the tail of the history db after the time of restoration.

it's a bit unrelated, but i often work with the aggregate differences of a database between 2 points in time. for that i use this function:

(defn db-diff [db-before db-after]
  (-> db-after
      (d/since (d/basis-t db-before))
      (d/history)))

if a box will always have a name when not empty, then I believe you could do something along the lines of:

[:find ?box
 :where
   (missing? $ ?thing :thing/name)
   [?thing :thing/container? ?box]
   [?box :box/id ?box]]

thx for trying!

Efficiency of what though?

You are not required to generate a tempid unless you want to in datomic cloud. https://docs.datomic.com/cloud/transactions/transaction-processing.html#tempid-resolution https://docs.datomic.com/cloud/transactions/transaction-data-reference.html

Actually I'd like to generate an actual entity id, not a tempid. I'd like to save a file to S3 including the entity id before I commit the entity to datomic, so the file is available for other systems.

I could use a generate my own uuid instead, and use :db/unique, but was wondering if there as a way to do this with :db/id

Make a unique attribute with a UUID