macro?

(-> (def var1 value)
    (reset-meta! (meta #'var2)))

So I have some monetary values, that then are processed through a pipeline of sums, percentages, multiplications, divisions, averages, etc. While a lot of it could be optimized I’m sure, I’m basically transforming an Excel sheet into a small program. There are quite a few variables to handle, not just one value that is being threaded, and I not only need to keep track of the final result but a lot of intermediary values as well. Since it’s complex enough, I’m also making a test suite to make sure the end results are accurate, and especially here, floats come short because 23.4 and 23.3999999999999 are not equal, leading to test failures (as a singular example). As a simplified example:

I have:
monetary value X
modifier percentage M1
modified monetary value Y = X times M1
modifier percentage M2
list of frequency percentages F, made up of [F1 F2 F3 F4…Fn]
Threshold T

I want to calculate (again, simplified, but just to show there are a lot of steps involved)
Ys = list obtained by multiplying Y by each of the elements of F
Ya = sum of Ys divided by modifier M2
final result R = if Ya < T then Ya otherwise T

Perhaps useful, but haven't used it myself: https://javamoney.github.io/

Right, seems very similar to clojurewerkz/money. Seems in either case I have to re-think/re-write some of my code to properly support either one of those libs/formats, and hope it ends up with the correct results... gonna experiment

hmm also having trouble using org.javamoney/moneta {:mvn/version "1.4.2"} in my deps.edn ¯\(ツ)/¯ either way looks like I need to rethink my life my algorithm...

Well ok either one of those seem like they'll work. Guess I needed to convince myself to stop depending on unreliable float arithmetic and use libs that can handle currencies properly. Thanks!!!

You will have to do some rounding at some point, but you can use (with-precision 0 ,,,) to mean "unlimited" precision.

but yes, you absolutely will get the wrong answer if you use floating point numbers

huh why did I not think of (with-precision 0...) thanks @U0P0TMEFJ, @U2FRKM4TW! For now the money related libraries seem to work, with proper rounding to half-up. Will keep testing

Not exactly an IDE but more a tool that could be used to build such a thing: https://nextjournal.github.io/clojure-mode/

the only thing that comes to mind is http://app.klipse.tech/

This one is also not an IDE but it is a REPL: https://babashka.org/xterm-sci/

Then we have e.g. https://4clojure.oxal.org/ which allows you to evaluate code, but to solve puzzles. Still not an IDE, but might give you some IDEas.

(all client side)

Depending on what your goal is, this might help: I've become addicted to using Calva's "jack-in" REPL for VSCode. I will type one clojure expression in the file, then hit "alt-enter", and it immediately evaluates in vscode. I don't think that's what you're looking for, but if you are just after a certain unified dev experience, then that might help. However, you can run VSCode directly online: https://vscode.dev/. The Calva plugin isn't available in the online version, but maybe something else can work there and/or be created for it :man-shrugging:

it really depends on your codebase, there are several ways to inject the connection object so that f is aware of it. The simpliest and probably dirtiest: have a dynamic *conn* var that you rebind in tests. I tend to not care about it and just delete the test db only once at the end of the tests.

Hmm I’m just learning how web dev is done with clojure and probably worrying too much about doing things “the right way”. I guess just deleting everything after I’m done with it will be fine. Thanks for guidance @U02F0C62TC1!

Where I work we just truncate/delete all the rows in the db at the end of the fixture. It’s not quite as fast as doing the transaction trick, but it’s functional enough.

I have the same debugging needs too. My solution might not work for you, but because I use Calva VSCode plugin, I just type #break inside the let statement, then hit "alt-enter" (jack-in repl) to evaluate the let, and then it breaks there in vscode and I can see evaluate any variable or expression from there.

I've considered writing this macro before and never gotten around to it, there's a relevant tweet that i can't find which defines a scale from (println) (inline def) (tap) (portal)

(defmacro extract-let [let-binding]
  `(do
     ~@(->> let-binding
            (second)
            (partition-all 2)
            (map (fn [[k v]]
                   `(def ~k ~v))))
     ~@(drop 2 let-binding)))

Or honestly, this might be easier to understand and use

(defmacro let-as-def [vars & logic]
  `(do
     ~@(for [[k v] (partition-all 2 vars)]
         `(def ~k ~v))
     ~@logic))

(do
  (def x 100)
  (println "x is now" x)
  (let-as-def [x 5
               y (+ x 7)]
              (println "this" (+ x y))
              (println "that" (+ y y)))
  (println "x is now" x))

it may be an idea to have the macro walk over the body of a defn, and do this to all nested lets so to get the behaviour you change your defn to something like defn! or whatever to add the debugging features

There’s some interesting responses in here. For my particular use case, I want this:

(extract-let
  (let [x 5
        y (+ x 7)]
    (+ x y)))

[(def x 5)
(def y (+ x 7))]

For now, just printing is fine, but I’d like to improve my macro skills and was curious if what I’m hoping to achieve is possible

FlowStorm debugger can help with debugging this kind of stuff without having to write anything. For def functionality demo check: https://youtu.be/cnLwRzxrKDk?t=103 Repo : https://github.com/jpmonettas/flow-storm-debugger/

I believe this will solve those specific requirements

(defmacro extract-let [let-binding]
  `(vector
    ~@(->> let-binding
           (second)
           (partition-all 2)
           (mapv (fn [[k v]]
                   `(quote (def ~k ~v)))))))

Oh hey, that’s exactly what I was looking for. Thank you! Now I just need to figure out why the other versions of this macro I wrote didn’t work

Does this help? https://clojurians.slack.com/archives/C03RZGPG3/p1658980576819099?thread_ts=1658943785.295319&channel=C03RZGPG3&message_ts=1658980576.819099

The whole binding part of the let is just to support the defn's wide range of possible arguments.

Plus :inline and metadata, but those are minor things.

Not sure what else to add without actually describing every form in the binding, unless you have specific questions.

Starting from the top, it's: • interning a var called defn with the some metadata • the var's value is defined using the shorthand function macro fn, and it's then given the name defn • the :argslist in the metadata has a very different list than the nested defn fn - why?

The difference between the :argslist and the actual signature is due to two facts: • defn is a macro: (. (var defn) (setMacro)). That's where &form and &env come from • It has lots of optional positional arguments - that's why & fdecl is needed, along with all that value "pigeonholing" in the let.

What is fdecl?

just generally, because defn is defined before most of clojure.core exists, it can't use things you can normally use (like defmacro - that's why the setMacro)

Yes, thank you @U064X3EF3, I understand it's bootstrapping from the JVM and can't leverage much Clojure at this point.

fdecl is ultimately the the function declaration body that gets put in an fn

Ah, "function declaration", of course.

defn is really (def a (fn ...))

> What is fdecl? Given that it's preceded by &, it's a list of arguments that you pass to defn. So if you use (defn x "s" []), fdecl will end up being '("s" []). x is not there because it's bound to the name argument.

the let parses out the initial parts like docstring etc and reduces fdecl to just the function part

It makes sense from a high level. Would you guys say it's worth investing the time to learn internals like this? Does becoming more familiar with these areas provide much added understanding when implementing stuff in Clojure?

it kind of depends on what you're interested in. a lot of people will tell you that core is not idiomatic clojure, and that's often true, but I read core.clj anyway and don't really see the harm as long as you understand that up front

I think the early parts of clojure.core can be a bit weird and are probably sufficiently different from "normal" clojure that maybe not useful

will it make you a better clojure programmer? I don't know… maybe?

I think reading the source for other functions in core can be interesting and useful, just maybe not the defstuff so much

if your goal is to write idiomatic code, libraries are usually the better place to look. I like internals so I'm partial to reading core source, but ymmv

I also kind of liked learning early on how clojure core is built up

It's certainly interesting, but I'd like to first learn the things that will give me the biggest gains. I'm getting more familiar with using the core functionality and now I'm just wondering what should be next on the list.

Something like the content from https://insideclojure.org/2016/03/16/collections/ seems like a good next as well.

maybe you could use the diff command line

diff -qr directory-1/ directory-2/ 

Depends if you want a diff, or just want a finger print of a directory to compare

if you want to do everything in clojure, there is a lib call pallet that has custom merge functions, where you can dispatch on dir/file, but you have to do a little bit of work to tell pallet what those are before you merge

It's for a Clojure unit test, so I don't want to use shell utils. I don't need a full diff, just a yes/no answer.

you can sha all the files, and create a non-nested map, and then do a compare. map key = file path, val = sha

If you just want a fingerprint I would do a sort of hash tree. Hash each file, then a directory hash is a hash of a list of the files in a directory and their hash

https://git-scm.com/book/id/v2/Git-Internals-Git-Objects has some examples of git tree objects which are used like this

@U0NCTKEV8 Oh that's an interesting solution. I suppose that could be faster than directly comparing file content when there are lots of large files.

There are also libraries for hashing clojure data structures (Puget maybe?), so you could just build a data representation of the filesystem then hash that

I'll play around with that, thanks!

Hey @U0NCTKEV8 wrt to taking the hashing approach, is there any reason why I can't just have a :dir value for each directory key? Since all the normal file entries have to match, seems like I don't have to have a SHA for the directories. :thinking_face:

To clarify with fake SHAs:

{"index.html" "3lkj423lk4j3kj4",
 "about.html" "lkfjjkl3j4kjk3jj5",
 "blog" :dir,
 "blog/cool-post.html" "3j5jlh544kh5"}

I suppose taking the SHA of directory contents provides some cool optimization paths for certain operations but I don't think I need any for my use case

It looks like you are still making a flat map of files to hash values

Which is not what I suggested, I suggested a tree hash or a Merkel hash

In which case using a :dir marker would not be sufficient because then the contents of a nested directory would not contribute to the hash of its parent

Ok that makes sense I just don't think I need a recursive data structure in this case

As in I don't need to compare nested dirs so I don't need a SHA for them

Posting my solution in case anyone is interested. Not perfect but seems to work well enough for my use case which is unit tests (using clj-commons/digest lib):

(defn dir-contents-map
  "Create a datastructure representing a directory's structure and contents in
  order to compare it with another directory. Creates a map of: relative paths
  (string) -> md5 checksums (string) of all the files inside the directory.
  Nested directory paths are not checksummed and have a value of :dir"
  [dir]
  {:pre [(-> dir (java.io.File.) (.isDirectory))]}
  (let [files (-> dir (java.io.File.) file-seq)
        parent-path-name-count (-> files first (.toPath) (.getNameCount))]
    (reduce (fn [contents-map file]
              (let [abs-path (.toPath file)
                    path-name-count (.getNameCount abs-path)
                    rel-path (str (.subpath abs-path parent-path-name-count path-name-count))
                    md5-checksum (if (.isDirectory (.toFile abs-path))
                                   :dir
                                   (digest/md5 file))]
                (assoc contents-map rel-path md5-checksum)))
            (sorted-map) (rest files))))

(defn diff-dirs
  "Determine if two directories have the same file structure and content. If
  differences, return list of relative filenames that are different and log
  diffs. If identical, return nil"
  [dir1 dir2]
  (let [cm1 (dir-contents-map dir1)
        cm2 (dir-contents-map dir2)
        [d1 d2 _] (data/diff cm1 cm2)
        rel->abs-path (fn [parent-dir path] (str parent-dir "/" path))
        mismatches (-> (merge d1 d2) keys)]
    (doseq [mismatch mismatches]
      (log/warn "Found mismatch:" mismatch)
      (-> (safe-sh "diff" (rel->abs-path dir1 mismatch) (rel->abs-path dir2 mismatch))
          :out
          println))
    mismatches))

Hmm, I don't think I've seen anyone suggest that (that it's an anti-pattern). I can see good reasons -- from a tooling p.o.v. -- to keep them separate, since it makes it easier to group .cljs and .cljc for FE tooling and .clj and .cljc for BE tooling. But I think it also depends on the project and why you need to separate code itself into those different extensions. For example, HoneySQL doesn't separate those files, but everything is .cljc except one test file, which is specific to some JVM/Clojure functionality. Expectations (the clojure.test version) also doesn't separate those files, but everything is .cljc except for one test file which is specific to ClojureScript.

Yeah, tooling is a major reason why I've not tried to merge them. In the codebase I'm thinking about, there's a very clear delineation between frontend and backend code with a small amount of shared code. There’s not been any reason to merge them yet, but that idea has been itching my brain for a couple years now. Glad to hear that you think it's fine to keep them separated.

You might be remembering it from some convos I've seen on #shadow-cljs (if you search src/clj ). From the docs: https://shadow-cljs.github.io/docs/UsersGuide.html#source-paths I can't recall if it was just a personal preference thing of his but I switched to using frontend/ and backend/ directories after reading it in there.

@U9J50BY4C What do you do about code shared between both ends?

I haven't gotten that far. hahaha. Maybe I would throw it in a shared/ directory.

I also separate FE and BE (call the directories ‘app’ and ‘server’) and have a ‘shared’ for .cljc. I guess that sets me free to commingle code, but in practice it doesn't happen. So in the end I'm still separating the files by type, just without using the extensions as the folder names.

@U9J50BY4C may i suggest the name "middlend" instead of "shared"?

When I started working on rewrite-clj v1 I had separate dir structures but then realized the majority of my sources were becoming .cljc so the separate dirs were more noise (to me) than helpful and I turfed them.

@U9J50BY4C good find! That is the quote I had in mind.