I can tell you that Fulcro in general makes no promises if you work outside of UTF-8. Everything assumes that’s what you’re using, and I have no plans on dealing with other encodings in any of the internals.

So I strongly suggest you set up your html/js env to always use UTF-8

Perhaps routing history only works with latin1 in url?

This was a correct implementation https://github.com/fulcrologic/fulcro/blob/fulcro-3.5.10/src/main/com/fulcrologic/fulcro/algorithms/do_not_use.cljc#L133 This commit is the problem: https://github.com/fulcrologic/fulcro/commit/91fe23917d48eb7bfbe480eef2ed02ede3a4389b

test:

(require '[com.fulcrologic.fulcro.algorithms.do-not-use :refer [base64-encode base64-decode]])
  (require '[goog.crypt.base64 :as b64])
  (require '[goog.crypt :as crypt])
  (require '[clojure.string :as str])
;; correct
  (let [bytes (crypt/stringToUtf8ByteArray (clj->js "Dončić"))] ;; First convert our JavaScript string from UCS-2/UTF-16 to UTF-8 bytes
    (b64/encodeString (str/join "" (map char bytes))))
;; error
  (base64-encode "Dončić")

Good catch. I remember changing that because the old way was super slow on large values... I was hoping it would not matter but clearly I had thought otherwise earlier. Oh well. Should probably change it back and use the new implementation with a different name in the performances case.

Released in 3.5.25

Thanks, fast-base64, great so there are both implementations. Are you sure that you won't have problems not supporting non latin1 characters in your apps? I am just asking because I don't know how in US you are writing some foreign names? For example is HR recording names differently as they written in legal documents? This is probably the case and saves a lot of problems. I am going to ask some HR guys how they are handling foreign alphabets.

Well, the fast is only on if you turn it on. I had a use-case where it mattered a lot, and there were no encodings, so it can be useful, but this is a do-not-use ns, and should only affect internals like this one thing you found

Haha I originally wrote those b64 functions. I can imagine them being super slow. JS engines are not helping with their base64-decode-functions-being-only-defined-for-latin-1-characters.

Why would base64 have anything to do with whether it’s latin-1 or not? Base64 encode is by nature a byte encoding scheme, so what data it’s encoding should not matter. Look at any Base64 encoder in Java it takes byte[]. The source of bytes is where there can be a difference… ASCII string can be used as bytes directly, but UTF-8 string needs to be converted to bytes first.

I must admit that the details escaped me over the years 😛.

because it is base64 ecoding the bytes of the string, and the encoding of the string has to be known. In js, evidently, UTF8 internal strings are not the guaranteed default, so you have to first ask it to re-encode the string in UTF8 so that you have a consisitent known thing to decode later.