Fork me on GitHub
#biff
<
2023-08-23
>
Martynas Maciulevičius08:08:19

Hello. I'd like to know how to get user's time zone from the client in the first HTML request. I can do something similar to this (it adds header to subsequent requests but not the first one that fetches the HTML (and I didn't bother to make it actually work)): [:div {:hx-headers "{\"TZ\": \"javascript: Intl.DateTimeFormat().resolvedOptions().timeZone\"}"} Is there a better way than setting user's time zone into his profile upon login? I'd like to not save user's timezone at all.

Mario Trost08:08:40

My thoughts: For the first request (= a first visitor), Biff itself can't help you much here, apart from having a middleware resolving the user's IP to a timezone, probably using an external Api. I also used Cloudflare for this in the past (https://developers.cloudflare.com/support/network/configuring-ip-geolocation/) Then for recurring users, I'd store the timezone in the session cookie while checking for changes (with the code snippet you shared) and updating it accordingly. > I'd like to not save user's timezone at all. 👆 I took this to mean you don't want to save it in the DB, but cookie storage is okay.

Martynas Maciulevičius08:08:29

> setting user's time zone into his profile upon login I meant upon signup. I think this is the simplest option :thinking_face: I can still use the DateTimeFormat header to send it when user is logging in for the first time and I create the user I don't want to hardcode cloudflare's header too 😕 I don't yet know if I want to use cloudflare. And using it solely for this is too much.

Mario Trost09:08:36

Understandable! Last option for initial, first user requests, using something like ipstack or ipapi. IPStack comes with 1000 free requests per month: https://ipstack.com/product Hm, ipgeolocation has 30k for free: https://ipgeolocation.io/pricing.html This reminds me that I used multiple of these in my last job to always stay in the free tier. Probably not what you should do 😓

Martynas Maciulevičius10:08:41

I found a problem with the session-based approach. The best location to set the timezone is when the user verifies the token or code. So it's here: https://github.com/jacobobryant/biff/blob/master/src/com/biffweb/impl/auth.clj#L139 and here: https://github.com/jacobobryant/biff/blob/master/src/com/biffweb/impl/auth.clj#L190 But those two places don't allow a way to add my own data.

Martynas Maciulevičius10:08:44

But that is also a problem because what if a user didn't even bother to log in... 😐 Oh no

Martynas Maciulevičius11:08:37

Welcome to localhost. Where delay was supposed to be max 20ms but now it's 230ms 😄

Mario Trost11:08:30

Regarding that Github issue you linked: Wouldn't that be solved by combining it with your earlier snippet?

[:div 
  {:hx-get "/example"
   :hx-headers "{\"TZ\": \"javascript: Intl.DateTimeFormat().resolvedOptions().timeZone\"}"}]

Martynas Maciulevičius11:08:01

Kind of. But it wouldn't still work for title page. Maybe I should rely on the host and just parse their header...

Mario Trost11:08:54

And regardless of that, can't you just make some arbitrary request after the dom is loaded to send the "real" TZ and store it in the cookie?

Mario Trost11:08:28

Stepping back from the implementation: What exactly are you trying to do?

Martynas Maciulevičius11:08:11

Just display a time variable in my own timezone. That's it. And forget about it. The forgetting bit is why I want to find a proper way. Basically make a renderer for every time variable in database.

Martynas Maciulevičius11:08:40

I'm rendering a screen with many buttons which I made forms for. So each form is... one TZ check. I can cache them but I don't yet know. For instance server-side caching for 1 second would work well here.

Mario Trost11:08:59

And all the HTML is server-side rendered. I see how that can complicate it, else you could just return the e.g. utc time string and let the client browser javascript handle timezone stuff:

time = new Date().toUTCString()
'Wed, 23 Aug 2023 11:16:42 GMT'
new Date(time)
Wed Aug 23 2023 13:16:42 GMT+0200 (Central European Summer Time)

Mario Trost11:08:48

Perhaps using HTMX / Hyperscript is a possibility. But it's still difficult for me to understand your exact problem, especially with your last sentence: > For instance server-side caching for 1 second would work well here. That sounds like a guarantee that multiple requests come from the same session and you could store the TZ information in the session cookie, which gets resent with every subsequent request from the same session/browser.

Mario Trost11:08:19

Could you provide a minimal repro? Gotta go back to work now but could could have a look at it in 3-4 hours

Martynas Maciulevičius11:08:05

Currently I saved it into DB. It's actually faster than 200ms when I reload. Now it's 80ms. But it's not 20 as before. I just call this about 20-30 times.

(defn get-user-timezone [{:keys [session biff/db] :as _ctx}]
  ;; TODO: optimize this into a session cookie or something
  ;; 
  (try
    (if (->> session :uid)
      (->> (q db
              '{:find (pull user [:user/timezone])
                :where [[user :user/timezone tz]]
                :in [user]}
              (:uid session))
           first
           :user/timezone
           zones/of)
      (zones/system-default))
    (catch Exception _e
      (zones/system-default))))

Martynas Maciulevičius11:08:00

Way faster, it's about 20-30ms again:

[clojure.core.cache.wrapped :as cachew]


(def user-tz-cache (cachew/ttl-cache-factory {}))

(defn get-user-timezone-for-uid [{:keys [session biff/db] :as _ctx} uid]
  ;; TODO: optimize this into a session cookie or something
  ;; 
  (try
    (if uid
      (->> (q db
              '{:find (pull user [:user/timezone])
                :where [[user :user/timezone tz]]
                :in [user]}
              uid)
           first
           :user/timezone
           zones/of)
      (zones/system-default))
    (catch Exception _e
      (zones/system-default))))

(defn get-user-timezone [{:keys [session] :as ctx}]
  (cachew/lookup-or-miss
   user-tz-cache
   (:uid session)
   (partial get-user-timezone-for-uid ctx)))
I was making 59 DB calls just to fetch the timezone crying-laughing-joy-blood

Mario Trost12:08:34

59 DB calls doesn't sound right. What is calling get-user-timezone-for-uid ?

Martynas Maciulevičius13:08:19

The bottom function. And then the bottom function gets called by something that is initializing each separate button. I don't need to fetch timezone for each of them but for some forms and date display fields I actually need to call it. So it's easier just to call it in several general places.

Mario Trost13:08:19

The initial trigger to all of those 59 calls is one single http request?

Martynas Maciulevičius14:08:23

I have 59 form elements on the page and each of them trigger one call

Mario Trost14:08:29

Then why don't you send the browser timezone along with the request?

Martynas Maciulevičius14:08:29

The problem is that this is not React. There is no "hook" and no "hook boundary". All I have is functions and macros. So all I can do is make a small scope and pass pre-rendered element to a body rendering a function. I don't want to use macros all over the place. It would be too much. If I'd use macros instead of functions then "scopes" would happen mostly at compile time and there would only mostly be "one scope". Also my design isn't too productive as I didn't want to pass the user's zone value down the line and instead I used def ^:dynamic to pass the time zone. I may refactor this later. Currently it's not a problem but I have too many params already and I don't want more.

Mario Trost14:08:39

I really think you already provided the solution in your first message: Use htmx to send along the user's timezone. The requests come from the user's browser, there is nothing stopping you from setting whatever htmx-attributes you want. For example, hx-headers that you used in your first message or hx-vals could work too perhaps: https://htmx.org/attributes/hx-vals/

Mario Trost14:08:44

A minimal repro would help a lot or at least the HTML/hiccup that gets rendered

Martynas Maciulevičius14:08:25

> I really think you already provided the solution in your first message Yes. But my code structure is why there are so many internal calls. It's not the UI that does it. The UI only does one request. I already gave you the repro and I already mentioned why there are so many calls. It's because I render a list of elements and they have at least two forms each. And each form requests the info separately. I solved this by caching. Now it's calculated only once per 2 seconds -- even for multiple requests of the same user.

Martynas Maciulevičius14:08:48

I would still have 59 calls to the function that calculates the zone-id itself. So by having a cache I solved that too. Now I have to somehow solve the time zone issue when the page loads and there is no account. And my fixes don't address this. Yes -- it could be addressed by sending the header but it doesn't work for the initial fetch. I could do a "redirect if there is no header" but then HTMX wouldn't even execute. So for now I don't know how to properly do it for the first refresh of a logged-out user.

Mario Trost14:08:55

That was not a repro but a code snippet showing only the db calls. And "each form requests" implies the requests come from the browser, where you have form elements. But now you cleared that up: The UI/browser only makes 1 request and the place where you render the form elements is calling this code. So use a middleware: HTTP request comes in and first thing you do is loading the timezone from the db and add this value to the request context map. Then you have it everywhere

Martynas Maciulevičius14:08:43

> add this value to the request context map I don't yet know how to do this. Is there a way? Because as Clojure is immutable then I don't know. Also if this solution only works for one request then it doesn't save that additional DB call.

Mario Trost14:08:01

> Yes -- it could be addressed by sending the header but it doesn't work for the initial fetch. > I could do a "redirect if there is no header" but then HTMX wouldn't even execute. > So for now I don't know how to properly do it for the first refresh of a logged-out user. (edited) That should be solvable too, some ideas: https://stackoverflow.com/a/69563926

Mario Trost14:08:34

Variations of solutions 2 or 3 should work

Martynas Maciulevičius14:08:10

Yes, but I went with solution 1 -- "force everything from back-end"

Martynas Maciulevičius14:08:05

For now I think that the API-call or Cloudflare-like header would work for the initial request and then I could somehow cache the TZ for some small amount of time. And then later the user will not be interested or will just log in. For instance cache by IP because I should know the user's call's IP.

Mario Trost14:08:51

Really the caching shouldn't be necessary: Just use a cookie

Martynas Maciulevičius14:08:05

It's still a form of cache. But yes. If I'd know how to set it then I'd try to also do it.

Mario Trost14:08:59

> For instance cache by IP because I should know the user's call's IP. That makes your application so much more complicated. Coming from a primary react background, I only ever learned about how to use cookies efficiently when I learned Remix

Mario Trost14:08:30

> It's still a form of cache. But yes. If I'd know how to set it then I'd try to also do it. Yes, but managed by the browser after you set it once. It's so liberating 😄

Mario Trost14:08:59

Just googled "Biff cookies" lol

Martynas Maciulevičius15:08:44

Uncle Biff's California Killer Cookies 😄

🍪 2
Martynas Maciulevičius15:08:22

This could be solved for the whole biffweb framework... It would be a nice thing. Anyway. This is a bit low priority. If you have this route:

["/hi" {:get (fn [{:keys [session biff/db] :as ctx}]
                           (println "session" session)
                           {:status 200
                            :body "hi"
                            :headers {"Content-Type" "text/html"}
                            :session (assoc session :mm-cookie "Hi!")})}]
And refresh twice (to set the cookie and then print it) then you get this:
{:ring.middleware.anti-forgery/anti-forgery-token ItSuRrd19P4mwKrfuD/+0+mHcOvW2o4JocxS5OkXeJhcl2yVSsnPGJ9EX8YOOEMYh6TEl8rh/EBjDLhu, :uid #uuid "0384ef51-973f-4430-8cfd-63f1b81b9a2e", :mm-cookie Hi!}
But I still don't yet know how to properly do it for the first request. Also it will mean that I'd be sending the time zone again and again if I'd do it by "user will eventually click somewhere" strategy.

Mario Trost15:08:27

> Also it will mean that I'd be sending the time zone again and again if I'd do it by "user will eventually click somewhere" strategy. That's not bad: You could for example update the timezone value in cookie if it changes. But if you don't want to send it all the time when the user clicks around, then you could set the hx-headers in the html elements only if you don't have a cookie stored timezone yet

Martynas Maciulevičius15:08:33

I think I could return some kind of hx-trigger from the back-end if there is no data in the cookie and no header. And then later load the webpage when all of it is present.

👍 2
Martynas Maciulevičius15:08:28

That would be two roundtrips for the client but the first roundtrip would be really fast. But if something caches the webpage into this noop form then oh my 😄 Also it will mean that the webpage won't be viewable without JS. Currently it is viewable without JS.

Mario Trost15:08:40

1. Check if cookie timezone is present 2. If present: Render everything 3. If not: You could still render everything except where you need the time values. Just render one container div with hx-get="/example" , hx-vals or hx-header with the timezone (your first message), and hx-swap : Your /example endpoint does 2 things a. Storing timezone in the cookie b. Returning all the innerHTML with all the forms and formatted date time strings which then gets swapped into the container div (or whatever element that ist) -> You don't have to store any timezone-info in your DB

Martynas Maciulevičius15:08:42

> -> You don't have to store any timezone-info in your DB (edited) Yeah... that's pretty nice... Less updates and less problems.

🙌 2
Martynas Maciulevičius15:08:43

I think the best place for it is the page fn. But not now. I want to implement my actual business logic. I handled like 70% of boilerplate that I needed to write.

Mario Trost15:08:17

Perhaps you could also always return dates and times in a span with a data-utctimestamp=<some-timestamp> value and ship a very small Javascript snippet that loops over all these spans, formats each date and does a span.innerHTML.

document.querySelectorAll("span[data-utctimestamp]").forEach(element => element.innerText = new Date(element.getAttribute("data-utctimestamp")).toLocalDateString())

Mario Trost15:08:55

Very much untested code, could have syntax errors

Martynas Maciulevičius15:08:13

> and ship a very small Javascript snippet HTMX comes with event system that can filter all of these before swapping them into the content (yes, if you write that custom JS). But for the first load you'd actually have to do the scan like in your example. I don't want to do it. I think cookies are cleaner. Edit: Also even then you need to allow JS in browser.

Jacob O'Bryant16:08:27

I just read/scanned through this--my feeling is that https://clojurians.slack.com/archives/C013Y4VG20J/p1692804677918819?thread_ts=1692778099.752249&amp;cid=C013Y4VG20J is probably the way to go. I do a similar thing in Yakread; I have this JS function:

function localizeTimes() {
  document.querySelectorAll('#send-time option').forEach(el => {
    let date = new Date();
    date.setHours((el.value - new Date().getTimezoneOffset() / 60 + 24) % 24);
    date.setMinutes(0);
    el.innerText = date.toLocaleTimeString([], {hour:'numeric', minute:'numeric'});
  })
}
which is called via hyperscript like so:
(ui/select {:id "send-time"
            :name "send-time"
            :default (:user/send-time user 15)
            :options (for [hour (range 1 24 2)]
                       {:value hour
                        :label (str hour ":00 UTC")})
            :_ "init call localizeTimes()"})
and looks like this:

Jacob O'Bryant16:08:25

To merge that with @ULA8H51LP’s example, you could do something like this:

;; clojure
[:span {:data-utctimestamp (inst-ms (java.util.Date.))
        :_ "init call localizeTimestamp(this)"}]

// javascript
function localizeTimestamp(element) {
  let millis = parseInt(element.getAttribute('data-utctimestamp'));
  let date = new Date(millis);
  element.innerText = date.toLocaleDateString() + " " + date.toLocaleTimeString();
}
And then you don't have to do a querySelectorAll

🙏 2
Martynas Maciulevičius16:08:21

And then what would you do when you have to revert the time zone to UTC(or other) in the back-end? Send the TZ in the header? :thinking_face: It could work

Jacob O'Bryant16:08:37

If you're making something like the select component in my screenshot, you can set each item's value to be the original UTC timestamp and only have the label be localized. If you're e.g. using a <input type="datetime-local" ..., then yeah, I would send the TZ in the header/in a hidden input element.

Jacob O'Bryant16:08:41

if you're only rendering dates and not accepting them as input values, then rendering them with JS might be enough; no need to ever send the TZ to the backend.

Martynas Maciulevičius18:08:48

Thanks. I don't yet know what I want to do. I kind of already did the implementation where I save the TZ into DB and the easiest way to refactor this would be to use a cookie. I'll try to think about this some time.

👍 2
Martynas Maciulevičius17:08:20

How do I find the ID of freshly inserted document? In my previous job with XTDB I could just generate the IDs myself and I didn't need to care. Should I generate the IDs here (I checked, I can do it) or should I not do it? If I don't do it then I want to retrieve the auto-generated ID somehow.

Jacob O'Bryant17:08:40

If you don't set :xt/id, biff just sets it to (random-uuid) . so if you need to do something with the ID immediately, you can just generate it with (random-uuid) yourself and set :xt/id explicitly.

Martynas Maciulevičius18:08:59

Ah. So it's not forced. That's good. I'm a little lost between the large macros when I want to understand what's going on. The submit-tx fn isn't big but I don't know what the transducers do for instance. They have too much logic.

Jacob O'Bryant20:08:14

yeah, that function is pretty complicated. I've been wanting to refactor the transformer fn stuff for a while, though there also is inherently just a lot of logic there that needs to happen. You can use the biff/biff-tx->xt function to help see what it ends up doing, e.g.:

(biff/biff-tx->xt
 ctx
 [{:db/doc-type :user
   :db.op/upsert {:user/email ""}
   :user/joined-at (java.util.Date.)}])

=>
([:xtdb.api/match #uuid "68cb1345-a4c7-480f-a711-3f2c8789a0bb" nil]
 [:xtdb.api/put
  {:user/joined-at #inst "2023-08-23T20:20:42.467-00:00",
   :user/email "",
   :xt/id #uuid "68cb1345-a4c7-480f-a711-3f2c8789a0bb"}]
 [:xtdb.api/fn
  :biff/ensure-unique
  #:user{:email ""}])

Jacob O'Bryant21:08:47

(I should also mention that you can always use plain xt/submit-tx instead of biff/submit-tx as well--the latter just gives you some convenience stuff like schema checking and merge/update operations)

Martynas Maciulevičius04:08:24

I want to use the schema checker. It gives me a little bit more sanity 😄 I think about also checking the FK so that the user won't be able to insert something that he shouldn't. Schema already encodes that by doing :xt/id -> :user/id. Maybe at some point I'll try to do this.

👌 1