This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-01-19
Channels
- # announcements (2)
- # babashka (1)
- # beginners (159)
- # biff (19)
- # clj-http (2)
- # clj-kondo (14)
- # clojure (105)
- # clojure-argentina (1)
- # clojure-art (3)
- # clojure-europe (17)
- # clojure-nl (1)
- # clojure-norway (10)
- # clojure-spain (1)
- # clojure-spec (3)
- # clojure-uk (26)
- # clojurescript (15)
- # conjure (4)
- # cursive (17)
- # datomic (8)
- # gratitude (1)
- # humbleui (1)
- # hyperfiddle (30)
- # joyride (10)
- # kaocha (1)
- # lsp (41)
- # malli (11)
- # off-topic (1)
- # pedestal (1)
- # polylith (12)
- # releases (1)
- # sci (4)
- # shadow-cljs (136)
- # squint (32)
- # tools-deps (28)
My problem is that I have a tree I am walking. When I see certain nodes I need to record info in a map. When I see others I need to read from that map. AFAIK there isn't a clean way to do this in clojure without mutable state (ie the map)
If you are only traversing the tree without changing it, you can use something like tree-seq with doseq or run!
I don't know the exact algorithm here, but it sounds like you could potentially do it with reduce
You can use something like for in a recursive function to do something like tree-seq with a lot of control
If you want to update the tree you can do all the stuff mentioned when you asked here https://clojurians.slack.com/archives/C03S1KBA2/p1705542318601829?thread_ts=1705542318.601829&cid=C03S1KBA2
there are other libraries like https://github.com/noprompt/meander
I do want to update the tree. Right now I'm walking the tree using zipper. I've got a loop and update the loc from the zipper with zip/next until I get to zip/end?. I'm struggling seeing how I can also use a map while walking the tree. I could update the map for each iteration when I call recur but having to wait till the end of the loop for logic to update it doesn't seem right...
maps are immutable, so when you update you make a new map, so you can make a new map whenever you want
(loop [loc (zip/vector-zip [{:a 1} {:b 2} {:c 3}])
mem {}]
(if (z/end? loc) (z/node loc)
;; Update the tree and mem based on items seen:
(match [(z/node loc)]
[{:a x}]
(when (< x 2)
;1 (assoc mem :x x))
[{:b x}]
(when (> x 2)
;2 (assoc mem :x x))
:else loc)
(recur (z/next loc)
mem)))
The code above walks a structure with zipper. I also want to have a "mem" map that I can store data in that I can acess while walking the tree. I'm not sure how I pass this mem to the next loop
I could have all if statements at the bottom in recur but if I have lots of patterns to match that becomes a pain
if you have a recursive function, you have to pass all the arguments when you make a recursive call
you have to use if, because you want to keep processing the rest of the tree even when the condition is false
I guess what I'm trying to do is map
and reduce
a tree at the same time. I want to apply a mapping function to transform each node in the tree. I also want to have a map that gets changed when it visits each node. Maybe there is a pattern for this?
Your example code indicates a lack of understanding (the use of when, not having recur in all the tails but the base case, etc)
It might click better if you write it first as actually a recursive function (which will maybe be more familiar, but not safe for large levels of recursion) and then change it to loop/recur (which won't take much changing)
A function that calls itself. In order to easily change it to be a loop/recur it needs to be tail recursive which basically means the recursive calls need to be the last expression in each control flow branch
https://www.sicpdistilled.com/section/1.2.1/ is about scheme, and I don't think it mentions "tails" but recursive functions that result in iterative processes are tail recursive
zippers are a technique for turning tree traversal which is recursive but not iterative (a recursive call for each branch means some of those recursive calls are not in the tail position), into an iterative process
I'm trying to find a pattern for this type of problem and I'm not sure there is a good one
There are many patterns for it, zippers, abstract machines, the spectre library, the meander library, etc, etc
Let's say you have a structure. Let's say it's a list to make it simple. If I want to update each item in that list there's a good pattern for it, mapping. Just about any language you use you can tell someone you want to map a function over some data. Also, if I want to take that list and turn it into a single piece of info there's a good pattern for it, reducing. This problem I'm working with feels like a combination of mapping and reducing and I'm not sure if there's a name for it. I want to take a list and map a function over it to transform it, but at the same time I want to thread some data thats shared between each transforming (but the end result doesn't require returning that accumulated piece of info.
What makes it trickier is I'd like to seperate walking the structure (list) from applying the transformation.
map and reduce are both defined for a certain kind of structure, you have a custom structure, you'll need to define the operations on it
I can put all the the structure walking code in a seperate function and that function can handle the walking logic and call the "mapping" function for each piece of data in the structure. It would pass the mapping function two things, the piece of data and the state that's being shared. The mapping function would then return the transformed piece of data and updated state. The walking function would move to the next piece of data and transform it with the updated state, etc...
Depending on your tree you can even merge tree nodes and your context into the same structure
Finally figured out my problem. I stoped trying to seperate walking the tree from the logic of manipulating it and the code is much simpler. I figured out the pattern I was searching for is technically OOP. I wanted functions that have state that persists between calls which is really just a method that has access "this" or self" which can store state.
Follow up question to the one above for people that write lots of clojure, do you use atom
often? I still have yet to use it (although I've only written clojure for hobby projects).
My clojurescript project https://github.com/samcf/ogres uses 3 atoms... one for app state, another for connection bookkeeping, and a third as a small cache
We have 178 atoms in 140K lines of Clojure code (incl. tests). Those are nearly all caches of some sort. We don't tend to use it for local state and we try to avoid global state in general -- except for caches.
I use atoms a fair bit, but generally not just for mutable state. Usually for concurrency.
(and 56 of those are actually in tests)
@U04V70XH6 under what circumstance would you rather use an atom cache over redis
Local in-memory cache means I can use it like regular Clojure data. We do have some stuff in Redis too (some TTL, some plain storage) but that's where we specifically need data across a whole cluster.
yea, hyenas are a totally different animal to dogs, I just never petted a hyena before
I like being able to reboot without losing a cache, that's what I was wondering about
We don't reboot servers very often and in-process cache can hold all sorts of things you can't serialize to Redis...
Also, if you're caching something for speed, you probably want it closer than a network call...
Redis may have gotten disk back storage at some point, but is usually used as an in memory store as well
It's persistable now. I generally have a small co-located redis instance with my web server to do caching
Even if you don't persist redis data, you don't need to reboot it if you're rebooting your server
Anyway, caches are another instance of using atoms for managing concurrent updates
People have also written things that add durability to atoms like duratom discussed here https://clojurians.slack.com/archives/C053AK3F9/p1705198559197789?thread_ts=1705198559.197789&cid=C053AK3F9
I get the impression that atoms and atom like things are more prevalent in clojurescript code, some libraries and frameworks there have custom atom like types for state management in react style apps
To me it seems like it's less a clojurescript thing and more a frontend/UI thing to have (reactive) atom-likes. HumbleUI for example seems to have something like that as well on the jvm. It's just usually clj on the backend and cljs for GUI.
These numbers are a bit out of date, but 63% of clojure repos on github didn't have any mutable references. The average number of mutable references per repository was 1.94. https://blog.phronemophobic.com/dewey-analysis.html#Reference-type-usage
WE use atoms for google credentials that may or may not need refreshing. saves building the initial Java object every time, Just build it on first call to get it, then subsequent calls get the value from the atom, if it needs refreshing, refresh it and swap it out in the atom. No reason why it couldnt just live in the ig context we spit out either, but just seemed nicer to have a namespace that deals with it that you just call and pass some account data
I cloned a deps.edn project to contribute but can't seem to jack into the project or install the dependencies (I am an Emacs/Leiningen user). I installed babashka because the project has a bb file, what else am i missing?
if you don't have it, you'll most conventionally need the clojure CLI for a deps.edn project: <https://clojure.org/guides/install_clojure>
an alternative would be deps.clj: <https://github.com/borkdude/deps.clj> - deps.clj is essentially the scripts installed for the clojure CLI, but as an executable, so you'll still need java installed (but if you're using lein, that's presumably already done)
a third option would be using babashka's built-in clojure CLI tool (which is deps.clj, but bundled into babashka). Since you have babashka installed, it should be possible to use bb clojure ...
from the project root just like the CLI tooling
Jacking in from Emacs with a deps.edn
project should "just work" automatically these days with CIDER etc...
Ah, yes, true you will need an up-to-date clojure
CLI installed for that.
It jacks into my user namespace instead of my core namespaces (which is the behavior I am used to from lein), and when I try to evaluate I get dependency complaints
What seems to go wrong when you try to jack-in? Maybe Emacs doesn't have the same PATH as your terminal so it can't find clojure
?
lein in some configurations will do things like load all the code into the repl, where clj doesn't do that so you'll have to load what you want to use
"when I try to evaluate I get dependency complaints" can you be more specific?
Yeah, and lein
has the idea of an "init" namespace which seems pointless to me since I eval code from my source files in my editor (I don't type into a REPL)
> lein in some configurations will do things like load all the code into the repl, That's probably what I am missing, but when I try to eval namespaces, I get errors s.a.,
Is this a public project we can look at that you're trying to use?
I am attempting to squash https://github.com/wkok/openai-clojure/issues/52
(~/clojure)-(!2000)-> git clone
Cloning into 'openai-clojure'...
remote: Enumerating objects: 904, done.
remote: Counting objects: 100% (443/443), done.
remote: Compressing objects: 100% (166/166), done.
remote: Total 904 (delta 239), reused 384 (delta 209), pack-reused 461
Receiving objects: 100% (904/904), 283.42 KiB | 3.26 MiB/s, done.
Resolving deltas: 100% (459/459), done.
Thu Jan 18 18:30:59
(~/clojure)-(!2001)-> cd openai-clojure/
Thu Jan 18 18:31:01
(~/clojure/openai-clojure)-(!2002)-> clj
Downloading: com/github/oliyh/martian/0.1.24/martian-0.1.24.pom from clojars
Downloading: com/github/oliyh/martian-hato/0.1.24/martian-hato-0.1.24.pom from clojars
Downloading: lambdaisland/uri/1.12.89/uri-1.12.89.pom from clojars
Downloading: frankiesardo/tripod/0.2.0/tripod-0.2.0.pom from clojars
Downloading: com/github/oliyh/martian-hato/0.1.24/martian-hato-0.1.24.jar from clojars
Downloading: frankiesardo/tripod/0.2.0/tripod-0.2.0.jar from clojars
Downloading: com/github/oliyh/martian/0.1.24/martian-0.1.24.jar from clojars
Downloading: lambdaisland/uri/1.12.89/uri-1.12.89.jar from clojars
Clojure 1.10.3
user=> (require 'martian.core)
nil
user=>
If you do that sequence in a fresh directory, does it work?I opened that project in VS Code, jacked-in, and was able to load wkok.openai-clojure.core
just fine too...
Or just create a new directory somewhere to run those three commands and the require
(I'm trying to eliminate anything specific to your existing directory/project setup)
If it works in a fresh directory, then try clj
and the require
in your existing directory to confirm it also works there.
If that works, we can go back to Emacs and jack-in and see if we can figure out what the disconnect is there...
$ rm -rf openai-clojure
$ git clone [email protected]:nsadeh/openai-clojure.git
Cloning into 'openai-clojure'...
remote: Enumerating objects: 855, done.
remote: Counting objects: 100% (478/478), done.
remote: Compressing objects: 100% (174/174), done.
remote: Total 855 (delta 271), reused 411 (delta 236), pack-reused 377
Receiving objects: 100% (855/855), 256.34 KiB | 5.70 MiB/s, done.
Resolving deltas: 100% (429/429), done.
$ cd openai-clojure
$ clj
Clojure 1.10.3
user=> (require 'martian.core)
nil
user=>
I think it worked? although I didn't see it download anything, maybe it already didThat stuff is already downloaded because you've used the library before.
Clojure CLI
Once it connects, open the wkok.openai-clojure.core
file and do whatever CIDER invocation would "load file"...
Shouldn't matter.
But I'm glad it's working now.
> Yeah, and lein
has the idea of an "init" namespace which seems pointless to me since I eval code from my source files in my editor (I don't type into a REPL)
sometimes I need it open for stdout, I don't think that prints on the source file with comments. Also for more elaborate experiments which are common at my stage of learning
There’s not really even a complicated way, unless you include trying it :(
Does anyone have a fun idea for a project to ease me into the way Clojure works? I barely understand it, and it's very overwhelming to get set up. If anyone has some good guides/material that would be much appreciated 😄
• which parts are difficult? The language or the setup/tooling? • what kind of project are you interested in? frontend or backend? data engineering or web server?
I have been learning Clojure for the past few weeks. Writing a couple data engineering script to populate an S3 database and store vector embeddings of documents has been a really helpful study in async and concurrent programming in Clojure, for example. I then wrote a web server in Aleph to serve the data I gathered. Overall it was quite instructive
I struggle with both parts, understanding the tooling and the language itself. I have a lot of knowledge in the web, and I've made a lot of projects using PHP and Laravel.
Maybe build a web server then using Ring or Aleph? I recommend the latter personally
I tried both deps.edn and Leiningen and found the latter to be more beginnner-friendly
I would also recommend to go through https://www.braveclojure.com/clojure-for-the-brave-and-true/ and do everything there except the chapter on Emacs. Use your favorite IDE but make sure you install its Clojure tooling, especially cider, a paredit editor, and an LSP client
@U04QU97E19U I wouldn't recommend starting from web-apps, even if you are already familiar with it, since it is going to require you to wrap your head around too many things, like libraries, how http is handled in Clojure, and much more. I would recommend you to install VSCode with Calva unless you are already more familiar with Emacs or IntelliJ. Follow the Calva setup instructions so you can start a repl and evaluate things from your editor. Once you get that running, try to write very small Clojure programs, like the ones in https://4clojure.oxal.org/ or start with some book like "Clojure for the brave and true" and play with the examples on your editor. IMHO the important part first is to grab the basics on using Clojure datastructures, and basic clojure.core functions to solve simple problems and how REPL driven development works. All this doesn't require any project setup. Once you feel comfortable with all that, the next step if you are coming from PHP is maybe understand how you can start a repl with web dependencies (ring) and try to spin up a server and serve some html.
You will also need to spend some time in the beginning understanding how to edit Lisp code, so you don't get frustrated trying to balance parens by hand
First up: what is your preferred editor/IDE/flow? You don't have to maximize it straight away, but definitely get it to a point you can eval code directly. Let us know which and we can point you to useful guides. As for what first, I have two wildly diverging recommendations. One is to use babashka to build come CLI tools. For example, I had a few web sources I wanted to scrape regularly, and bb was a real boon for getting that done. Two is to try #C2X8D0EMT if you like genart. You can iterate quickly and get visual feedback. And you can just explore instead of feeling like there's some goal to reach. If you really want to start with web stuff, I'd say just build some sort of JSON or EDN based API. Don't jump directly into ClojureScript as the tooling requires more investment.
Alright. I’ve got plenty of things to try now. Thank you guys :)
http://practical.li has lots of free videos and guides you may find useful
One thing I'm noticing is that it's best to write functions in a way that they can be used with clojures built in higher order functions like map and reduce. That means that if you have a function that takes multiple arguments what you really want to do is wrap those args into a map so that your function only takes on or maybe two arguments. Is this in line with peoples experience?
Not necessarily, you can write functions with as many arguments as you want and use the partial
macro to pass it into threading/pipeline situations
As long as you keep the arguments that you want to operate on at the end of the list of arguments, and the "configuration" arguments at the beginning
I find that it really depends on the purpose of the function. If the arguments to the function can logically be grouped together into a map then I will do that sometimes. If there's no real logical grouping, then I tend to continue to use separate arguments and use something like (partial func arg1 arg2)
.
I would recommend against using maps for this reason... typically maps are used to represent an entity from your data model or as a bunch of options (notably optional)
In general, contorting your code in the name of brevity alone generally leads to less readable programs. You see this a lot with new Clojure programmers changing their code to more easily fit into long ->
and ->>
thread chains
Using anonymous functions is idiomatic Clojure so (map #(my-fn % arg1 arg2) some-data)
is "fine". Give it a local name via let
if it makes the code clearer. You can also use fn
, which has the benefit of allowing you to name the "anonymous" function:
(map (fn my-specific-fn [x] (my-fn x arg1 arg2)) some-data)
then if you get an exception, at least the stacktrace will have my_specific_fn
instead of just a number for the name.This is worth bearing in mind when defining functions and thinking about argument ordering: https://clojure.org/guides/faq#arg_order
I see, thanks everyone for the feedback. My one issue with partial
is that it doesn't combine well with reduce
since the first arg of a function is used in both reduce and partial.
I think Rich considers partial
less idiomatic than anonymous functions... I seem to recall reading that here more than once...
@U056FU0UH0F partial should play well with reduce so long as the two last arguments to the function are the result and list element reduced over
@U0552GV2X32 Thanks thats a good point
I don't think I've ever partially applied a reducing function
The same should go for map - constant arguments (for the sake of the map) would generally be placed first
I'm leaning more towards fn
. Partial works but it requires you order your args in a func in a specific way which seems like an easy source of bugs
Anonymous functions are your bread and butter in Clojure, its what you should be reaching for first. comp
and partial
are a bit harder to read by those unfamiliar with your program
(in my opinion)
> Partial works but it requires you order your args in a func in a specific way which seems like an easy source of bugs
Agreed. Picking the most "obviously correct order" for your arguments outweighs the apparent convenience of making your function suitable for partial
in other contexts -- better to design your function to be as good as possible standalone.
I'm still leaning towards throwing everything in a map since maps are open to changes. A common thing I've run into is that I have a function I want to apply over some set of data and then realizing after I write and test it that I need to including another argument to the function for some extra peice of data. This means I need to rewrite the function definition and any calling code (the code that calls it often needs to have the new arg added, and the code that calls that code needs the arg, etc...)
Yup, if you have a lot of "optional" (or named) arguments, a single hash map is a good approach. Although, as of Clojure 1.11, you can support both trailing named args and a single hash map arg -- so for map/reduce, you'd still want that "bag'o'args" as the last argument to make that work.
(defn foo [data & {:as args}] ...)
-- can be called as (foo something {:arg 1 :other "stuff"})
or (foo something :other "stuff" :arg 1)
Thats great to know, thanks for sharing. This will have heavy use in my code. Function calls with those trailing named args are much more readable. Another point for using the map approach
It's nice that it supports both the human-readable (unrolled) form and the program-created (single hash map) form.
"if you have a function that takes multiple arguments what you really want to do is wrap those args into a map so that your function only takes on or maybe two arguments." ...and later... "[realizing] after I write and test it that I need to including another argument to the function for some extra peice of data" Can this be decided in the abstract? Not knowing more, it sounds like this function itself may be doing too much work. Maybe instead of looking for a way to live with a dozen parameters, look for a little family of functions that can manage the same overall functionality, with one or three params each. The fact that we got all the way thru writing and testing the function before discovering the need for even more variability is an intriguing tell. That it is a common problem suggests a tendency towards over-busy functions worth solving with sth other than a language feature. Throwing up our hands and creating a map might be sweeping a design shortfall under the rug, and stop us from discovering the true nature of the underlying algorithm. It also makes the function opaque, since it masks the inputs. Jes thinkin out loud. :thinking_face: ps. I am getting ready to rewrite an important, resursive function that takes a map. Seemed like a good idea. Unfortunately, certain recursive calls need different key-value "parameters", so now I have to tweak the original map. So now the monolithic map is the problem.
"Throwing up our hands and creating a map might be sweeping a design shortfall under the rug, and stop us from discovering the true nature of the underlying algorithm. It also makes the function opaque, since it masks the inputs". Very well put. I was hoping someone would call out the bad design because that was my first thought as well. The monolithic map problem is a great counter point which shows how this approach could go wrong. I'm curious to hear how the rewrite goes.
"I'm curious to hear how the rewrite goes." Already I have noticed that three controlling vars should never be changed on recursive calls, so I am establishing an outer function that takes those, and handles them, so the recursive search does not see those. Nice simplification I did not notice until I thought on this issue.