biff

jf 2024-11-08T13:14:34.760019Z

I am having issues with trying to run add-fixtures from dev/repl.clj after using schemas from https://github.com/metosin/malli?tab=readme-ov-file#malliexperimentaltime. The insidious thing is, I've been able to validate things (in dev/repl.clj itself!):

(let [malli-opts @(:biff/malli-opts (get-context))]
    (for [doc [{:xt/id #uuid "..."
                ... ; other attribs
                :loan/start-date (. java.time.LocalDate (parse "2024-09-10"))
                :loan/end-date   (. java.time.LocalDate (parse "2024-09-17"))}]
          schema [:loan]]
      {:loan     (:loan/title doc)
       :schema  schema
       :valid   (m/validate schema doc malli-opts)
       :explain (pp/pprint (m/explain schema doc malli-opts))}))
; => ({:loan "hello", :schema :loan, :valid true, :explain nil})
But if I try to run (add-fixtures) I get a complaint:
Unhandled clojure.lang.ExceptionInfo
   Doc wouldn't be a valid :job after transaction.
   {:tx-doc
    ...
    :explain
    #:loan{:end-date ["unknown error"], :start-date ["unknown error"]}
Not exactly sure, but it almost seems as if biff/submit-tx is not using the schema I've set up with malli.experimental.time. Surely not?

👀 1
2024-11-08T20:53:53.617159Z

"unknown error" looks suspicious. What does your fixtures.edn file look like?

2024-11-08T20:54:12.070579Z

and your schema? I can try to reproduce it on my end.

jf 2024-11-08T22:02:23.534769Z

this is a minimal setup that should reproduce it:

; schema.clj

(def schema
  { ; ...
   :loan/id :uuid
   :loan [:map {:closed true}
          [:xt/id           :loan/id]
          [:loan/title      :string]
          [:loan/start-date :time/local-date]
          [:loan/end-date   :time/local-date]]
  })
; fixtures.edn
; note that I switched to a different syntax for creating java.time.LocalDate; but this is merely a syntactical change and the issue occurs with the same dot form style as described in dev/repl.clj

[{:db/doc-type :loan
  :loan/title "hello"
  :loan/start-date (java.time.LocalDate/parse "2025-01-01")
  :loan/end-date   (java.time.LocalDate/parse "2025-05-31")}]
; eelchat.clj

(ns com.eelchat
  (:require [com.biffweb :as biff]
            ...
            [malli.core :as malc]
            [malli.registry :as malr]
            [malli.experimental.time :as met]
            ...)
  (:gen-class))

...

(def malli-opts
  {:registry (malr/composite-registry
              malc/default-registry
              (met/schemas)
              (apply biff/safe-merge (keep :schema modules)))})

jf 2024-11-08T22:05:35.792189Z

I am currently trying to see if this is an issue with xtdb somehow not accepting java.time.LocalDate (at least in terms of seeing if there's any docs regarding accepted data types!). I see https://docs.xtdb.com/reference/main/data-types.html for v2... but am not having success so far finding anything for v1

2024-11-08T23:55:08.381679Z

I think the problem is that (java.time.LocalDate/parse "2025-01-01") in the edn file doesn't get evaluated. e.g. if you run the following bit from inside add-fixtures:

(-> (io/resource "fixtures.edn")
        slurp
        edn/read-string)
then you get this:
[{:db/doc-type :loan,
  :loan/title "hello",
  :loan/start-date (java.time.LocalDate/parse "2025-01-01"),
  :loan/end-date (java.time.LocalDate/parse "2025-05-31")}]
whereas if you copy the contents of fixtures.edn into repl.clj and then evaluate it from your editor, you get something like this:
[{:db/doc-type :loan,
  :loan/title "hello",
  :loan/start-date
  #object[java.time.LocalDate 0x43fb729e "2025-01-01"],
  :loan/end-date #object[java.time.LocalDate 0x57e77a0f "2025-05-31"]}]
i.e. in the first case, you're getting a literal list starting with a symbol (`java.time.LocalDate/parse`), etc. the cleanest solution is probably to use this library: https://github.com/henryw374/time-literals and then you can do :loan/start-date #time/date "2024-01-01" etc. alternatively, as a bit of a hack you could add an eval to that bit inside add-fixtures:
(-> (io/resource "fixtures.edn")
        slurp
        edn/read-string
        eval)

1
jf 2024-11-09T01:02:13.280419Z

thank you, thank you. This worked!

2024-11-09T01:02:28.135269Z

awesome!

jf 2024-11-09T02:41:18.822119Z

thank you for the explanation as well. Very helpful!

Chip 2024-11-08T16:22:20.022119Z

What troubles do you think we might encounter working on Biff as a 2- or 3-member team if any? At our level of experience (very low), I'm sure our team workflow will be far more of an issue that anything in Biff. Figured I'd ask anyway.

jf 2024-11-08T16:32:46.829089Z

specifically with Biff, or also with Clojure? it makes a difference

Chip 2024-11-08T16:34:17.776969Z

Oh. Thank you for the question. I better understand both.

Chip 2024-11-08T16:35:11.938739Z

If you refer me to #beginners for the Clojure parts, maybe I could lift some burden on you.

jf 2024-11-08T16:50:38.583239Z

I'm not the author of Biff, fyi, in case you're thinking I am. Speaking from my experience as somebody still new to Clojure when I got into Biff, I'd say that the more familiar you are with Clojure and the rest of the parts that you might pull in as a result of Biff (Rum/HTML/CSS, XTDB, etc.), the better it'll be for you.

1
Chip 2024-11-08T16:52:43.552859Z

Thank you. Any comment on team workflow?

2024-11-08T20:44:36.125689Z

I think the main thing will just be deployment. The builtin server-setup.sh script is built with a solo developer in mind, especially the clj -M:dev prod-dev stuff--I don't recommend using that command in a team setting. You might want to set up git-based deploys so anyone can deploy by merging something to master. otoh if it's the early days and you don't want to deal with that it would also be perfectly fine to use biff's built in deployment stuff--you might just want to have some agreed-upon process around how to do deploys. e.g. you could have a designated deploy person who always does it, you could have a #deploys slack channel where you say if you're about to do a deploy (and then make sure to fetch master/main before deploying).

1
Chip 2024-11-08T20:46:03.277819Z

Excellent. Thank you.

2024-11-08T20:47:31.047879Z

The other thing that comes to mind is code organization as your codebase grows. Things can start to get a little hairy around 10k lines of code in my experience. compare to fulcro which is aimed more towards at managing that kind of complexity as your project grows. fwiw my current biff work is focused on this (inspired heavily from using fulcro at work), so hopefully... at some point... I'll have some more guidance/examples around using biff for large projects.

Chip 2024-11-08T20:50:49.630799Z

Hmm. Fulcro keeps coming up. Seems we will be there at some point. Don't know enough to guess how or when. Thank you always for your continued work on Biff and your ever-helpful words.

👌 1
adammiller 2024-11-08T22:43:59.372509Z

Related to that, one thing I'd like to see is Biff being a bit more friendly for working inside of the polylith architecture. I tried it for awhile but it was awkward. Don't recall all my exact issues right now, but I think if Biff could work nicely in polylith then the structure is mostly solved.

👀 1
1
2024-11-08T23:46:17.400609Z

I'd be interested to hear more about polylith. I haven't used it but looked into it a bit for work. It seemed like basically it was an approach for structuring your codebase as a collection of different libraries? We ended up just treating the top-level namespaces in com.example.lib.* as "libraries", with the top-level namespaces being public APIs and the descendant namespaces being private implementation namespaces. e.g. you might have a com.example.lib.foo namespace which exposes functions defined in com.example.lib.foo.impl.a etc. I'm planning to take that approach with Biff as well. It seemed like a lightweight approach to getting some of the benefits* of polylith without having to actually split your app up into a bunch of different deps.edn projects. But yeah, I'm really not that familiar with polylith, and there may be things about it that I'm missing. *not including being able to generate different artifacts with different sets of code files--though I'm a little skeptical this would matter for most biff apps. you can always have a single artifact but with different runtime config, e.g. so you can have web servers and workers. (Besides that, the main thing I'm experimenting with to make Biff better for large codebases is introducing Pathom)

adammiller 2024-11-10T14:48:40.490789Z

Just one small opinion here so take it with a grain of salt, but if you add in Pathom I hope you will do it more as an opt-in, rather than a default (or with an easy ability to remove it). We currently use Pathom in a server side only fashion and I contend that it adds lots of complexity for very little value. Not to say it's not a great library, because I think it is (especially when used in place of like GraphQL as an API endpoint for client side apps). But using it in a server rendered environment it just adds extra code, more complexity, and performance issues. Granted we use it in front of mostly PG database so maybe it's better with XTDB. Although, with XTDB you already have the datalog syntax so not sure what benefits Pathom would even add there.

adammiller 2024-11-10T14:54:54.143309Z

As far as polylith, I think the benefits are simply a prescribed structure/architecture that makes it easy to work with other developers. It's well documented on how things should be done and comes with a great CLI tool that provides lots of information about your codebase. It can help with incremental testing for larger projects. It's easy to swap components for testing or other reasons. I think it is mostly about providing some sane defaults when working with teams. All that said, I think reading Sean's blog post related to Polylith gives some good insight into it: https://corfield.org/tags/polylith/

👍 1
2024-11-10T23:10:07.879099Z

I am planning to have pathom be an add-on. E.g. I might write a document like "recommendations for large projects" that goes over pathom and some other things, with a standalone lib for any pathom helper code I write. I'm in the process of rewriting a server-rendered biff app with pathom. so far there are two main parts where I've used pathom resolvers (besides writing a generic resolver for pulling things out of XTDB): • UI components--E.g. there's a resolver for the sidebar that queries for the current user's roles, so it knows whether or not to display a link to the admin page. (the way I think about fulcro is that it's basically a bunch of plumbing so you can use Pathom's graph query model for frontend UI components--the idea here is that since biff focuses on server-side rendering, we can just use pathom directly without all the extra plumbing) • derived data--I've got another unreleased biff feature where you can create secondary indexes from the XTDB transaction log and store them in RocksDB. I add resolvers for the index data so that the rest of the app can query the data in the same way as the "regular" data in XTDB. (I expect I'll also use resolvers for non-indexed derived data, like the "first name + last name -> full name" example in the pathom docs, I'm just still early on in the rewrite and haven't had the need yet) I am guessing that keeping things performant is indeed easier with XTDB like you mention. It is an experiment for sure. after the rewrite is done I'm planning to take a few parts of the app and compare to a non-pathom implementation and see how they compare in performance and maintainability etc. I'm getting off on a tangent here but I've also wondered sometimes if I could/should write a mini version of pathom with less stuff in it, in an attempt to make it a lighter-weight layer... something to think about later.

👍 1
Chip 2024-11-08T20:30:14.839559Z

Searching for XDG in #biff doesn't turn anything up for me. Any Biff-specific thoughts or guidance with considering XDG standardization?

2024-11-08T20:49:52.277999Z

XDG as in this? https://specifications.freedesktop.org/basedir-spec/latest/ e.g. putting config in a standardized spot? I haven't thought about it at all. I think that's more relevant for programs that the user installs to their own machine rather than server-side web apps

Chip 2024-11-08T20:56:15.135249Z

Yup, that XDG. > I haven't thought about it at all. I think that's more relevant for programs that the user installs to their own machine rather than server-side web apps Exactly the distinction I needed pointed out. Thank you.

👍 1