Clojurians Log v2

Based on Tony's recommendation, I've been trying Claude code the last few days. I am similarly blown away! It's definitely much better working in existing clojure codebases where it has lots of examples. Versus JavaScript where it can fart out several hundred lines of a prototype application into an empty workspace dir. It is definitely expensive. I played a but with giving it more autonomy and trying to encourage it to find a productive feedback loop. The results were mostly positive. The biggest issue I saw was so laughably silly (for us humans with editors) and that is balancing parens. Once it spat out a bunch of quite decent code but missed a paren somewhere and spent 5+ minutes in a loop trying increasingly silly ways to figure out where the ) should go.

Also it can get into negative feedback loops where it's trying to solve a problem and is thrashing about trying to make its tests pass, all the while iterating already crappy code further into the ground. It doesn't seem to be able to stop, look at the ruins it's surrounded itself in and say "maybe I should start over on a different tack"

My understanding is that CC is kind of a new paradigm that incorporates "tool use", and when it uses tools is when you really get the costs up. I've seen some projects that let you get a CC-like thing with any LLM.

https://github.com/dnakov/anon-kode

I’m currently trying Cline. It’s a free vscode plugin. With a Claude sonnet 3.7 API key it has some things CC is missing, and it seems similarly good

I know someone who recently tried it as well and had a similar sentiment. The new ChatGPT code editor is okay. It's not directly integrated into your IDE but you can attach a running application to your prompt. Ones connected, it becomes aware of your code and can assist. Also, have you tried the code review feature? You can ask it to code review code it generated 😆. Had some fun building a simple landing page with it.

I used github copilot recently and it was able to gen a defsc and fulcro dom pretty well. But no, I’ve not tried much more yet…it’s on my TODO

I mostly use copilot. It does fairly well.

I've been using https://aider.chat/ a bit here and there. It does fairly well with clear instructions and a good context (provided through conventions) but sometimes messes up the edits. Sometimes a one shot prompt saves a good chunk of time but other times I find it leads you down the wrong direction and gets kind of annoying.

I just tried out Claude code on two different fulcro projects of varying size. I asked to do fairly complex edits that required modifying a state chart, editing web and native UI, and full stack operations. It did amazingly well. Like I was floored. It added things to component queries, it used my wrappers for gluestack in my react native app, it was able to add a custom attribute to the rad model correctly including using a custom macro I had written to make bit sets. The only downside was that it was quite expensive. A 1-hour session was about $5

I highly recommend spending five bucks on credits just to see how it works. I think it was probably doing really well just because there was existing source it could use for patterns, but the fact that it derived the correct way to modify components, including rad forms, and worked across multiple namespaces was beyond impressive. It's actually making me rethink my hiring strategies as a CTO

I just tried it again on a REALLY complicated RAD Form save middleware stack that needs optimization. My stack does CAS-based atomic retries in Datomic, malli schema generation from the RAD model, speculative entity generation with datomic with . The current middleware does a LOT of queries as a result, and my server performance suffers for this. I worked with Claude Code to refactor this, including writing tests. I did have to feed it nuances as it made mistakes, but it very quickly created something (with Fulcro Spec tests) that looks respectable. I use metadata on my tests to group them for parallel execution in CI, and it even noted that automatically when generating the tests. I ran out of money before I completed the task, but I was curious how the resulting code worked (e.g. would it even load in a REPL?) I loaded the code. There was one typo (a stray close paren). I fixed that, then ran the tests. 8 of them passed before a fail. The fail was due to the AI not fully understanding the schema of the entities it tried to use. E,g, it generated test data that it thought would be valid, but wasn’t. A reasonable mistake given that our schemas are generated from annotations on a RAD model. But this is an example of the beginning of one of the tests:

(specification "Batch validation of entities" :group1
  (let [c             (test-conn)
        user-id       (new-uuid 10)
        company-id    (new-uuid 100)
        _             (d/transact c {:tx-data [(new-company "ACME" :company/id company-id)]})
        env           (test-env c user-id company-id)
        id            (tempid)
        valid-address {:address/city       {:after "CUMARIBO"}
                       :address/department {:after "VICHADA"}
                       :address/line       {:after "123 Main"}}
        ident         [:address/id id]]

    (component "Collecting pull patterns"
      (let [pull-patterns (batch/collect-pull-patterns env {ident valid-address})]
        (assertions
          "Returns a map of id-key to pull pattern"
          (map? pull-patterns) => true
          "Contains pattern for address"
          (contains? pull-patterns :address/id) => true
          "Pattern is a vector"
          (vector? (get pull-patterns :address/id)) => true)))

• The :group1 is how we do the metadata grouping. It chose these randomly over the tests it wrote. • It used fulcro spec! • It wrote good behavioral testing strings, and tested one behavior at a time • The main bug here is that valid-address requires a country-code in our schema…but that is nested way deep in the RAD model, so it is an understanable oversight that I’m sure I could have pointed out and gotten fixed had I not ran out of my “experiment” money 😄

What are you putting in your context window? I haven't used Claude Code but have used other similar tools. Aider allows you to select what you include as context and also generates its own repo-map that it includes. Their docs do include suggestions to keep your context small - limited to the files you expect to edit - or it confuses the LLM.

I’m not!

it’s frikkin amazing. It’s a CLI REPL tool. It indexes the code base and figures all of this out for itself

It makes the other tools where you have to control the context window look like stone-age tools.

The main downside is that it costs like $5/hr in my experiments, but honestly for commercial coding, it is worth it

I wonder if you could parameterize it a bit to reduce the cost.

it’s in an experimental stage. I’m betting they will lower the cost as they improve the tool, but t hey are currently charging $15/M tokens I think, which is high…but it IS doing context window narrowing on its own

E,g, here’s part of the session:

in dataico.rad.save-middleware namespace, there is a RAD Form save middleware stack. It is rather complex, and includes validations at various layers. This validation creates a lot of queries, which cause
   the save to take a long time. Can you analyze this middleware and figure out how to refactor it to reduce the query load?

⏺ I'll analyze the RAD Form save middleware stack to identify opportunities for optimization. Let me first find the relevant file.

⏺ Search(pattern: "dataico.rad.save-middleware")…
  ⎿  Found 7 files

⏺ Let's examine the main save middleware file first:

⏺ Read(file_path: "src/main/dataico/rad/save_middleware.clj")…
  ⎿  (ns dataico.rad.save-middleware
       "The Dataico RAD save middleware."
       (:require
     ... (+99 lines)

⏺ Let's also check the test file to better understand how the middleware is used and tested:

⏺ Read(file_path: "src/test/dataico/rad/save_middleware_test.clj")…
  ⎿  (ns dataico.rad.save-middleware-test
       "Integration test against the production save middleware"
       (:require
     ... (+136 lines)

⏺ Let's look at the validation middleware since that's likely a significant part of the performance issue:

⏺ Search(pattern: "wrap-validations", include: "*.clj")…
  ⎿  Found 5 files

⏺ Read(file_path: "src/main/dataico/rad/validation_middleware.clj")…
  ⎿  (ns dataico.rad.validation-middleware
       "Middleware to run validations on SPECULATIVE ENTITIES. Requires Speculative entity middleware, or it CANNOT function
        properly."
     ... (+98 lines)

⏺ Let's check the schema enforcement middleware:

⏺ Search(pattern: "wrap-schema-enforcement", include: "*.clj")…
  ⎿  Found 4 files

⏺ Read(file_path: "src/main/dataico/rad/malli_validation_middleware.clj")…
  ⎿  (ns dataico.rad.malli-validation-middleware
       "Requires the use of speculative-application-middleware to pre-expand the entities (with refinements)"
       (:require
     ... (+113 lines)

⏺ Now let's look at the speculative application middleware since it seems to be a key component:

⏺ Read(file_path: "src/main/dataico/rad/speculative_application_middleware.clj")…
  ⎿  (ns dataico.rad.speculative-application-middleware
       "RAD save/delete middleware. Speculatively applies the env diff to the current database(s) (by schema) and then pulls the entities
        affected according to Malli pull pattern of that entity. These expanded
     ... (+42 lines)

⏺ Let's look at the speculative application implementation:

⏺ Read(file_path: "src/main/dataico/rad/speculative_application.clj")…
  ⎿  (ns dataico.rad.speculative-application
       (:require
         [clojure.set :as set]
     ... (+483 lines)

⏺ Let's also look at the refinement process since it's part of the middleware stack:

⏺ Read(file_path: "src/main/dataico/rad/refinement.clj")…
  ⎿  File does not exist. Did you mean refinement.cljc?

⏺ Read(file_path: "src/main/dataico/rad/refinement.cljc")…
  ⎿  (ns dataico.rad.refinement
       "Support for refining the form delta of a save via a sequence of dispatch keys that are co-located on the ID attribute
        of the entity in question. Add the `refinement-options/refinements` option to the ID attributes of entities that
     ... (+250 lines)

⏺ Let's examine the entity-snapshot-datomic file to further understand the middleware:

⏺ Read(file_path: "src/main/dataico/rad/entity_snapshot_datomic.clj")…
  ⎿  File does not exist. Did you mean entity_snapshot_datomic.cljc?

⏺ Read(file_path: "src/main/dataico/rad/entity_snapshot_datomic.cljc")…
  ⎿  (ns dataico.rad.entity-snapshot-datomic
       (:require
         #?@(:clj
     ... (+428 lines)

⏺ Let me analyze the malli-validation-middleware since it seems to be a key part of validation:

⏺ Search(pattern: "malli-schema", include: "*.cljc")…
  ⎿  Found 100 files

⏺ Read(file_path: "src/main/dataico/rad/malli_schema.cljc")…
  ⎿  (ns dataico.rad.malli-schema
       "Generate Malli schema from RAD model"
       (:require
     ... (+331 lines)

⏺ Let's look at the pathway flow for a save operation and understand where the performance issues might arise:

⏺ Task(Your task is to analyze the save middleware stack in dataico.rad.save-middleware to identify performance bottlenecks and find opportunities for optimization.…)…
  ⎿  Done (11 tool uses · 41.3k tokens · 46.6s)

⏺ Based on my analysis and the agent's detailed assessment, I can identify several key optimization opportunities for the RAD Form save middleware stack:

  1.

That's pretty cool. Going to give it a spin sometime. Do you have concerns about protecting your IP when you use it?

it went and found relevant files….but later when looking at particular functions it’ll just add the relevant PORTION of the file into the context

The fact that I asked it a very high level redesign task, and pointed it at the starting point, and then it analyzed it quite well was just crazy good

e.g.

⏺ Read(file_path: "src/main/dataico/rad/speculative_application.clj", offset: 308, limit: 20)…
  ⎿          #_#_seems-filled? (do
                                 (log/info [pattern '-> potential-value])
                                 potential-value)
     ... (+17 lines)

Here’s an example of a refinement I asked it to do:

Here's how we could approach this:

  1. Single-pass combined operation:
    - Take the input diff
    - Batch-collect all necessary entities from the database in a single operation
    - Apply the diff speculatively all at once
    - Run all validations in a single coordinated pass
    - Return a combined result
  2. Simplified approach:
  (defn validate-save-diff [env diff]
    (let [idents          (keys diff)
          id-keys         (map first idents)
          ;; Get all required patterns upfront
          pull-patterns   (reduce
                            (fn [acc id-key]
                              (assoc acc id-key (ms/pull-pattern env id-key)))
                            {}
                            (distinct id-keys))
          ;; Pull all entities in a single batch operation per schema
          initial-entities (reduce
                             (fn [acc [id-key pattern]]
                               (let [attr (attr/key->attribute id-key)
                                     schema (ao/schema attr)
                                     idents-of-type (filter #(= id-key (first %)) idents)]
                                 (reduce
                                   (fn [acc ident]
                                     (assoc acc ident (d/pull (senv/current-db env schema) pattern ident)))
                                   acc
                                   idents-of-type)))
                             {}
                             pull-patterns)
          ;; Apply speculative diff to all entities
          speculative-entities (apply-diff initial-entities diff)
          ;; Validate all entities in a single pass
          validation-results   (validate-entities env speculative-entities pull-patterns)]
      (if (valid? validation-results)
        ;; Return the validated result
        {:valid? true
         :entities speculative-entities}
        ;; Return validation errors
        {:valid? false
         :errors validation-results})))

  This approach:
  1. Extracts all needed patterns in a single pass
  2. Pulls all entities together by schema
  3. Applies the diff once
  4. Validates all entities together

  Would you like me to sketch a more detailed implementation based on your existing code, or is this high-level approach helpful?

> I think the batching would be better as a datomic q instead of a reduction on pull

⏺ You're absolutely right. Using a Datomic query would be much more efficient for batching, especially when pulling multiple entities of the same type.

  Here's a refined approach using d/q instead of individual d/pull operations:

  (defn validate-save-diff [env diff]
    (let [idents          (keys diff)
          id-keys         (distinct (map first idents))
          ;; Get all required patterns upfront
          pull-patterns   (reduce
                            (fn [acc id-key]
                              (assoc acc id-key (ms/pull-pattern env id-key)))
                            {}
                            id-keys)
          ;; Group idents by schema and id-key for batching
          idents-by-schema (reduce
                             (fn [acc [id-key _]]
                               (let [attr (attr/key->attribute id-key)
                                     schema (ao/schema attr)]
                                 (update-in acc [schema id-key] (fnil conj #{}) ident)))
                             {}
                             idents)
          ;; Pull all entities in batched queries by schema
          initial-entities (reduce-kv
                             (fn [acc schema id-key->idents]
                               (let [db (senv/current-db env schema)]
                                 (reduce-kv
                                   (fn [acc id-key idents]
                                     (let [pattern (get pull-patterns id-key)
                                           ids (map second idents)
                                           ;; Use a batched query to pull multiple entities at once
                                           query-results (d/q '[:find (pull ?e pattern)
                                                               :in $ [?e ...] pattern
                                                               :where [?e]]
                                                             db ids pattern)
                                           ;; Convert query results to entity map
                                           entity-map (reduce
                                                        (fn [m [entity]]
                                                          (assoc m [id-key (get entity id-key)] entity))
                                                        {}
                                                        query-results)]
                                       (merge acc entity-map)))
                                   acc
                                   id-key->idents)))
                             {}
                             idents-by-schema)
          ;; Apply speculative diff to all entities
          speculative-entities (apply-diff initial-entities diff)
          ;; Validate all entities in a single pass
          validation-results   (validate-entities env speculative-entities pull-patterns)]
      (if (valid? validation-results)
        ;; Return the validated result
        {:valid? true
         :entities speculative-entities}
        ;; Return validation errors
        {:valid? false
         :errors validation-results})))

  This approach provides several advantages:

  1. Uses Datomic's query capabilities to batch-pull entities of the same type
  2. Minimizes database round trips by grouping entities by schema
  3. Extracts all necessary patterns upfront to avoid redundant schema analysis
  4. Groups validations to minimize redundant work

All I typed in there was “I think the batching would be better as a datomic q instead of a reduction on pull”

and this was a global analysis of a larger problem it was optimizing

you still need to know what they heck you’re doing. It made some really bad suggestions through the session, but with a trained eye, I was able to point it in the right direction with small refinements like this.

Yeah, that's quite nice. It definitely helps that you can articulate it so well and your prompt description matches the language in the code. Did it examine library code at any point or just your source?

the library code isn’t in scope, so I think just local project

so I suspect that generating a brand new project with no local patterns would be much worse. This is 250k LOC in Clojure(script), so there are ample patterns for it to follow

but I bet if you split the Fulcro docs into files based on topics so you could refer to them, and we amped up examples there, it would be pretty good.

none of these tools could replace the need for a good engineer. But man for some of these complex refactorings or bug finding missions it is going to be a mistake to ignore them. I’ve been using copilot for a while, and chatgpt, and have had some luck on generation of complex things at times (with limited context and numerous iterations) but this is my first experience seeing things work well in such a large context. Even at $5/hr (which was my burn rate) that’s just $860/mo…I’d pay someone that to pair with me and be as effective as what I just saw…I’d actually pay a lot more

I keep musing on what could be if the LLM was given the project's clj-kondo analysis data https://github.com/clj-kondo/clj-kondo/tree/master/analysis. I think there's potential for something cool, but maybe if Claude Code and others like it do such a good job, it might not even be worth the effort. Might run an experiment for fun at some point. I've also thought that the RAD options namespaces would make for some really useful context, but might similarly not be worth the effort if Claude is good enough.

Effectively using these tools feels like it's a bit more in the realm of a project manager than just a straight up engineer. Clearly defining requirements, evaluating the results, requiring tests, etc.

With Aider there is also a /run command that take an arbitrary shell command, runs it, and then offers to feed the result into your LLM session. Good for tests and interactive queries. I'm guessing Claude has something similar.

@alex.sheluchin when using aider, what LLM do you have luck with. I tried a local deepseek-r1:7b and it kinda sucked

@tony.kay I've had decent results with:

model: openai/gpt-4o-mini
editor-model: openai/o1-mini
architect: true

Someone suggested using https://openrouter.ai/. It gives you access to some models you might not otherwise have access to (e.g. I don't have the spend for openai/o3-mini rn) and has some other benefits. Haven't tried it yet. Too many things lol

Pinging @genekim to check out this thread since the intersection of Fulcro and LLMs seems like your thing.

I tried your aider config and tried to repeat my experiment. It was no good. It totally screwed up the statechart, and was slow to boot. I tried to get it to fix the mistake and spent like 30 minutes just trying to get prompts right. The result: I’d rather do the edit myself. Claude Code was far superior….but your config was a LOT cheaper in API fees…that said, my time is worth way more than $5/hr. Yeah, basically on this higher level edit aider with opeai just mangled things to an unrecognizable mess 😞 The final edit it did, it just deleted everything from the namespace 😛

I tried again with o1-preview…and o1-mini as editor, and it failed to do the edit.

even with o1-preview as the only model, it still just mangled the code.

There are also edit mode settings and certain modes work better with certain models.

my claude code experience was pretty much flawless with zero config. It didn’t do a perfect job, but I felt like it wasn’t wasting my time. This feels like wasting my time and money 😕

It's kind of a can of worms when you have to figure out the best settings permutation and since this stuff is evolving so quick it could be wasted learning. If CC covers it will without much config, that would be really nice. Is it using Sonnet 3.7 under the hood?

but that’s part of the evaluation I’m doing 😄 Seeing what works. Yeah, it is sonnet 3.7, and the automation and code search was crazy impressive. It was doing multi-file changes, looking for trouble, etc.

I've read Sonnet can be a bit too keen and do too much at times. When it's making widespread changes, you have to deal with needle in the haystack bugs that it might introduce. You could try Aider with that model too.

My experience with AI code assist to date was pretty lukewarm (similar to what just happened). If I have some quick boilerplate and a good prompt, I get something helpful. At one point I had chatgpt generate an iOS and Android plugin for me, which was impressive, but again limited in scope and wasn’t code comprehension/refactoring, but a clear evolving code specification. Claude Code is the first experience I’ve had where I saw something that impressed me in a wider context. > I’ve read Sonnet can be a bit too keen and do too much at times. It was not my experience at all on CC because it asked me about each change before it made it, and didn’t do this annoying git commit at every code edit.

and it allowed me to give it additional context at those denied changes to get it to do the right thing. For something that just came out a week ago, it was surprisingly adept

The Aider stuff is all very configurable, probably to a fault. I'll try out CC and see how it does with zero conf. It's funny how these LLM providers are constantly eating their users' lunch.

Really appreciate you sharing your user experience trying things out against Fulcro!

yeah, I didn’t expect to find something that capable. I more expected what aider did: caused me more frustration than good. I don’t want my team sitting around fooling with an AI tool just to get it working. Using AI to code something new as sort of starting place and refining it, or talking about implementation ideas is great, but I was looking for something that could help my team deal with 250k lines of Clojure that would at least get them to the right spots. They would still need to know what they’re doing, but CC felt like leverage, not drag.

Completely reasonable take. I feel this tension between wanting to try the new tools to avoid falling behind, and on the other hand all the time sunk into anything other than writing code seems like lost time. Right now I'm pretty much just using it for refactoring and brainstorming. It feels like a decent balance. FYI there's also a service called http://phind.com that is NOT a code assist thing, but it is an LLM powered search that includes source links to github, docs, etc.. Haven't used Google once since subbing to it. It's quite good, but YMMV. Just thought I'd mention it while we're on the topic.