Fork me on GitHub
#datomic
<
2021-06-15
>
greg12:06:42

Hi, I'm trying to build a first app using datalog/datascript. In the db I want to store FX exchange rates. For each pair (eg. GBP/USD, GBP/EUR), for every day, one value. I'm wandering how to design schema for such application. I'm struggling with what should be marked as an entity. Currency, currency pair, currency pair for given date, or all actually? I'd much grateful for some ideas or sample schemas that you think might make sense. Thanks

Joe Lane13:06:52

Hi @U023TQF5FM3, are exchange rates directional? e.g. [#inst "2021-01-01" :GBP :USD 10] but [#inst "2021-01-01" :USD :GBP 8]?

greg14:06:47

@U0CJ19XAM yes, this kind of situation is possible: [#inst "2021-01-01" :GBP :USD 2] [#inst "2021-01-01" :USD :GBP 0.4] In addition there might be more then one source of rates, so there might be: [#inst "2021-01-01" :GBP :USD [:name "BoE"] 2.01] [#inst "2021-01-01" :GBP :USD [:name "HMRC"] 2.03]

Joe Lane17:06:49

To give better advice I'd need to know more about the rest of the app, but for those fx entities I'd lean towards representing :GBP->:USD as an entity with a composite tuple of :loc/from and :loc/to where the composite tuple represents identity. Then for the rates i'd want to know what the access patterns will be and the growth of the dimensions. The rate entity could be something like:

{:time/at #inst "2021-01-01
 :rate/from-to {:loc/from :GBP :loc/to :USD}
 :rate/source {:source/name "BoE}
 :rate/amount 2.01}
That would then have a composite tuple of
[#inst "2021-01-01" 123 456 2.01]
Where 123 is the directed exchange and 456 is the rate source. You can add additional composite tuples to allow different access patterns in exchange for space. That being said, if the number of datoms stays small ( sub 1-billion) then who needs the extra tuples.

👍 2
👏 2
jdkealy17:06:31

What's the best way to get two strings like this into an instant that i can save in datomic? "2021-06-08T16:30:12" "America/New_York" I tried using clj-time but no dice

(let [datetime-vec (take 8 (parse-timestamp vector date-str))]
     (-> (apply zoned-date-time datetime-vec)
         (with-zone UTC-offset)
         (instant)))
I get
:db.error/wrong-type-for-attribute Value 2021-06-08T16:30:12-04:00[America/New_York] is not a valid :inst for attribute :session/start_time
` even though the output of my function is an instant Trying to insert
#inst "2021-06-08T20:30:12.000000000-00:00"

Joe Lane17:06:25

@U1DBQAAMB For now you gotta turn that into a j.u.Date before you persist it in Datomic.

jdkealy17:06:16

what's a j.u.date a google search is giving me linkes to jdate 🙂

jdkealy17:06:11

i guess, Java Util Date ?

jdkealy17:06:26

(java.util.Date. (tz/to-instant "2021-06-08T16:30:12" "America/New_York"))

jdkealy17:06:55

This seems to work

([date-str, UTC-offset]
   (let [datetime-vec (take 8 (parse-timestamp vector date-str))]
     (-> (apply zoned-date-time datetime-vec)
         (with-zone UTC-offset)
         (instant)
         inst-ms
         (java.util.Date.))))

jdkealy17:06:53

input

"2021-06-08T16:30:12" "America/New_York"
output
#inst "2021-06-08T20:30:12.000-00:00"
looks legit 🙂

🎸 2
Joe Lane17:06:47

FWIW, if you care about the TZ when you query obviously you'll need to stash that as another attribute and reconstitute a ZonedDateTime on the way out. Avoid querying with zones if you can, it can make simple queries slow and complicated.

Joe Lane18:06:40

Oh also, you can use (Date/from your-inst)

jdkealy19:06:07

cool thanks! Yes, I'm saving 3 attrs, the timezone, the original string, the converted instant (just in case i screw something up, i can regenerate them )

jdkealy19:06:56

I'll only be querying for stuff in the immediate future, so i think I'll just be passing (> (java.util.Inst))

greg22:06:12

When accessing raw index (listing datoms using ), is there a way of substituting numbers by entities itself? I was checking https://docs.datomic.com/cloud/query/raw-index-access.html, but there is nothing about such a thing. Example: when I preview datoms for an example data set

(d/datoms db-solar-system {:index :eavt})
I receive something like that:
...
 #datom[74766790688845 73 "Sun" 13194139533319 true]
 #datom[74766790688845 74 696000.0 13194139533319 true]
 #datom[74766790688846 73 "Jupiter" 13194139533319 true]
 #datom[74766790688846 74 69911.0 13194139533319 true]
 #datom[74766790688847 73 "Saturn" 13194139533319 true]
 #datom[74766790688847 74 58232.0 13194139533319 true]
...
And I would like to see attribute names instead of its number. Is there some Datomic API to do that?

thumbnail10:06:58

the numbers are eids, so you can just pull them using d/pull. You could query all attribute names, and use the query-result to map over the 4th entry to make it more efficient. depending your usecase.