Fork me on GitHub
#datomic
<
2017-05-24
>
jeff.terrell01:05:43

@erichmond - No, but I'm wondering if somebody accidentally put a literal <bucket> instead of the actual bucket name. Maybe search your code for the literal <bucket> string?

erichmond02:05:20

@jeff.terrell sorry, that was a real bucket name, I removed it tho, so you guys didn't see where our production backups are ;D

jeff.terrell02:05:50

😆 d'oh! Makes sense…

uwo02:05:38

forgive me for asking, I feel like the answer should be obvious to me: why is it important to ensure that schema changes are applied once and only once?

pesterhazy08:05:14

no, afaik you can retransact the schema on each connection without any harm

val_waeselynck08:05:49

@uwo: most of the time (e.g when installing attributes that don't change and database functions), you don't need to ensure schema installation runs only once.

uwo12:05:55

@pesterhazy @val_waeselynck thanks! why is that the focus of conformity then?

pesterhazy12:05:00

for some migrations, like altering attributes, installing the schema only once may be necessary

val_waeselynck14:05:59

Another common use case is creating a new attribute and populating it with default data for existing entities

uwo16:05:36

Is there any reason I might be getting db.error/transactor-unavailable besides overwhelming the transactor?

uwo16:05:16

I’m not in an import situation. And there should be very few writes going on (unless of course something in our system is going haywire)

uwo16:05:31

we’re using the dev store in staging at the moment. Could it be that??

favila17:05:24

you get this error when connecting or when transacting? @uwo

devth17:05:55

side note: it'd be amazingly useful if there was a doc of all/most of Datomic error messages explaining what they actually mean

uwo17:05:48

@favila ah, yes. sorry. when transacting

uwo17:05:55

we handle db.error/transactor-unavailable during imports with exponential backoff to allow it to recover, but I wasn’t anticipating it during normal use

marshall17:05:47

@uwo could definitely be due to dev storage Dev is an integrated storage engine that shares resources with the transactor process. It’s intended strictly as a development convenience

marshall17:05:57

not intended for any kind of production or scale use

marshall17:05:37

for several reasons, including the fact that it doesn’t have its own independent resources to handle peers and transactor concurrently connected and using read and write bandwidth

uwo17:05:48

thanks. that sounds about right. we were certainly not intending to use it for prod, just been using it in staging for a little bit because of timelines I guess

uwo17:05:06

I’m make it a priority to add real storage then

uwo17:05:57

If I might add a question. we’re bringing our own storage (mssql). Silly question I know, but should we put storage on a separate box from the transactor?

uwo17:05:23

dopey question. sorry. they’re separate 😄

marshall18:05:05

yea, it’s definitely recommended to be on separate instances

marshall18:05:10

for the same kinds of reasons

timgilbert19:05:53

Is there a good reason why EIDs come back as #uuid from (d/q) but as java.lang.Long from (d/entity)?

favila19:05:56

@timgilbert Are you sure you are not confusing your domain-specific id (the uuid) with the internal entity id (the long?) What attribute are you reading in each case?

favila19:05:06

entity ids (from :db/id, or the :e field in a datom) are always longs

timgilbert19:05:47

Ah, yes, you're right, my mistake

timgilbert19:05:10

I was looking at an external domain UUID, as you suspected. Thanks!

Lone Ranger20:05:47

anybody here using datomic in production?

Lone Ranger20:05:15

I understand that it's designed so that transactor, peer server, storage, etc are usually on different machines (virtual or physical), curious what setups folks are using

Lambda/Sierra20:05:18

@goomba Yes, definitely those should all be on different machines. Most production installations are set up this way.

potetm20:05:20

:raised_hand: Yeah that's the setup we use.

Lone Ranger20:05:44

are you guys doing physical servers or VMs? or microservices?

marshall20:05:18

@goomba are you likely to be on AWS?

Lone Ranger20:05:36

or local hosting

Lone Ranger20:05:10

well, ultimately both

Lone Ranger20:05:19

depending on the project/company

marshall20:05:52

so you’ll likely be limited to whatever infrastructure is available at the on-prem sites; in general that is something that Datomic supports fairly well

marshall20:05:21

you’ll want independent instances, either real or virtual, for transactor and peers (or peer server)

marshall20:05:31

also separate storage

marshall20:05:48

but you can use different storages in the different deployments

marshall20:05:52

depending on what is availble

marshall20:05:21

i.e. you might use Oracle for one customer and cassandra for another if that is what they already have avaliable

marshall20:05:33

i’d highly recommend a memcached instance for production as well

marshall20:05:43

it will significantly improve performance across the board

Lone Ranger20:05:32

okay, got all that. Is the primary way of communicating server information with jdbc strings etc? or is there a config file I should be looking at?

Lone Ranger20:05:26

although I don't need it now ultimately I'd like to be doing some sort of microservices thing where where the whole shebang can be treated as a bunch of ephemeral instances and would like to avoid hardcoding resources locations or at least be able to do it programmatically

potetm20:05:41

The call to d/connect requires the storage location. So in the case of jdbc, yeah it requires a jdbc url.

potetm20:05:40

The transactor location is looked up from storage.

favila20:05:52

@goomba we use instance or project metadata

favila20:05:36

then the instance pulls metadata, spits into app-specific config (eg an edn or properties file, or systemd environments file, whatever), and it's accessible to app on startup

Lone Ranger20:05:41

okay, gotcha. So the storage at least needs to be fairly static

potetm20:05:54

Yeah, or rigged up yourself.

Lone Ranger20:05:12

cool, thanks guys