This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-12-29
Channels
- # adventofcode (13)
- # announcements (2)
- # asami (59)
- # babashka (6)
- # beginners (273)
- # calva (18)
- # cider (3)
- # cljs-dev (3)
- # clojure (84)
- # clojure-estonia (1)
- # clojure-europe (2)
- # clojure-france (11)
- # clojure-nl (3)
- # clojure-taiwan (2)
- # clojure-uk (23)
- # clojurescript (7)
- # code-reviews (321)
- # conjure (4)
- # data-science (1)
- # depstar (6)
- # fulcro (37)
- # helix (20)
- # juxt (3)
- # keechma (3)
- # leiningen (3)
- # malli (7)
- # nrepl (1)
- # off-topic (20)
- # pathom (25)
- # re-frame (30)
- # reveal (5)
- # shadow-cljs (25)
- # sql (15)
- # tools-deps (4)
- # uncomplicate (2)
- # xtdb (3)
why do I get here a classcastexception on the let part :
(def work (ref (apply list (range 1e5))))
; And the sum will be a ref as well:
(def sum4 (ref 0))
; Write a function which, in a dosync transaction, removes the first number in
; work and adds it to sum. Then, in two futures, call that function over and
; over again until there's no work left. Verify that @sum is 4999950000.
; Experiment with different combinations of alter and commute–if both are
; correct, is one faster? Does using deref instead of ensure change the result?
(defn add-dosync [work sum4]
(dosync
(alter (ensure sum4) + (first @work))
(ref-set work (next @work))))
(let [worker1 (future (while @work (add-dosync work sum4)))
worker2 (future (while @work (add-dosync work sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work
(deref sum4)
@marcus.akre you can always pick up where Tim Baldridge left off here https://github.com/halgari/com.tbaldridge.hermod
@roelof (ensure sum4)
returns the current value of sum4
, which is a long. alter
takes a ref, not a long
that's incorrect - ensure is only needed for values you are not altering but only reading
you should use ensure
on the ref
you're reading the value of in the dosync body, which is not sum4
but still if I change the code to this :
(defn add-dosync [work-ref sum-ref]
(dosync
(alter sum-ref + (first @work-ref))
(ref-set work-ref (next @work-ref)))
one additional problem may be that (first @work-ref)
may be null. maybe the easiest way around that would be to check for that condition inside the dosync
body
that's even though you're checking in the future
bodies, but there's a possible race condition there; you shouldn't rely on that
the exact error message is :
; Execution error (ClassCastException) at ground-up.chapter6/add-dosync$fn (form-init16078161836259666359.clj:80).
; class java.lang.Long cannot be cast to class clojure.lang.Ref (java.lang.Long is in module java.base of loader 'bootstrap'; clojure.lang.Ref is in unnamed module of loader 'app')
that check is OK to make the future finish when it's done, but you are not guaranteed that work won't be completed between the check and the execution of add-dosync
yes, but that doesn't look like it could cause the particular error you're getting; you should get a nullpointerexception instead
of course
; Write a function which, in a dosync transaction, removes the first number in
; work and adds it to sum. Then, in two futures, call that function over and
; over again until there's no work left. Verify that @sum is 4999950000.
; Experiment with different combinations of alter and commute–if both are
; correct, is one faster? Does using deref instead of ensure change the result?
(defn add-dosync [work-ref sum-ref]
(dosync
(alter sum-ref + (first @work-ref))
(alter work-ref (next @work-ref))))
(let [worker1 (future (while @work (add-dosync work @sum4)))
worker2 (future (while @work (add-dosync work @sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work
(deref sum4)
that's causing the same problem you were getting before by calling (alter (ensure sum4) ,,,)
now I see this error :
; Execution error (ClassCastException) at ground-up.chapter6/add-dosync$fn (form-init16078161836259666359.clj:81).
; class clojure.lang.PersistentList cannot be cast to class clojure.lang.IFn (clojure.lang.PersistentList and clojure.lang.IFn are in unnamed module of loader 'app')
changed it to :
(defn add-dosync [work-ref sum-ref]
(when (seq @work-ref)
(dosync
(alter sum-ref + (first @work-ref))
(alter work-ref next))))
probably using ensure
instead of deref
/`@`(but give both a try to verify that ensure
is needed)
I tried this :
(defn add-dosync [work-ref sum-ref]
(dosync
(while @work-ref (alter sum-ref + (first @work-ref)))
(while @work-ref (alter work-ref next))))
(defn add-dosync [work-ref sum-ref]
(dosync
(when @work-ref (alter sum-ref + (first @work-ref)))
(when @work-ref (alter work-ref next))))
@roelof Please try the following
(defn add-dosync [work-ref sum-ref]
(dosync
(when-let [work (seq (ensure work-ref))]
(alter sum-ref + (first work))
(ref-set work-ref (next work)))))
— a a single check for non-empty sequence,
— single ensure
Not sure if this question was discussed before thought (can't read all those pages above), but this is also not a good case for using ref
s, maybe only as an exercise.
Because this algorithm of summing a list does not benefit from parallelization at all (all entries are processed one by one sequentially anyway).
Ok, let me fire up the repl 🙂
Would you please paste the code?
(def work (ref (apply list (range 1e5))))
; And the sum will be a ref as well:
(def sum4 (ref 0))
; Write a function which, in a dosync transaction, removes the first number in
; work and adds it to sum. Then, in two futures, call that function over and
; over again until there's no work left. Verify that @sum is 4999950000.
; Experiment with different combinations of alter and commute–if both are
; correct, is one faster? Does using deref instead of ensure change the result?
(defn add-dosync [work-ref sum-ref]
(dosync
(when-let [work (seq (ensure work-ref))]
(alter sum-ref + (first work))
(ref-set work-ref (next work)))))
(let [worker1 (future (while @work (add-dosync work sum4)))
worker2 (future (while @work (add-dosync work sum4)))]
@worker1 ;wait for worker1
@worker2 ;wait for worker2
@sum4) ;return the result of their work
@roelof from what I see, it's not running forever, it's just being REALLY slow 🙂
No idea, first time using ref
s myself. I'm not an expert, I was just curious 🙂
me too first time use it. The "Clojure from the ground up" was telling me to use a ref
We'll crack it, no worries 🙂
Overall, ref
seems like a very heavy-weight tool, that's why it's rarely used, and people use atom
s instead.
Result looks correct @roelof
I think around that, more or less.
I can time it.
Yeah, 2:30 on my macbook, but I ran it as uberjar.
Anyway, I think it’s not a real evaluation of speed of how refs work. Here the workload is so that two CPU threads are constantly fighting and re-doing each-other’s work. I would expect that on a single-core CPU the result will be calculated much faster :)
(defn add-dosync [work-ref sum-ref]
(dosync
(when-let [work (seq (commute work-ref))]
(alter sum-ref + (first work))
(ref-set work-ref (next work)))))
You should use commute instead of alter
Cool. Good to know. I am not yet proficient enough to know when commute will do, and when alter is required to get a correct result.
According to the book alter/ensure if the order is important and commute when the order is not important
so as a example 1 + 2 + 3 the order is not important because also 1 + 3 + 2 - 6 also 3 + 1 + 2 = 6
Ok, makes sense
this :
(defn add-dosync [work-ref sum-ref]
(dosync
(when-let [work (seq (ensure work-ref))]
(alter sum-ref + (first work))
(ref-set work-ref (next work)))))
and this :
(defn add-dosync [work-ref sum-ref]
(dosync
(when-let [work (seq (ensure work-ref))]
(commute sum-ref + (first work))
(ref-set work-ref (next work)))))
Hi, I’ve been trying the fox-goose-corn kata and worked out this solution - https://paste.ofcode.org/mpzmsutwdEK3GczkzjZ6Tx I guess there are still a few things wrong…
Given a configuration, calc-path
tries to generate the possible options using for
and evaluate them recursively. If a child configuration is invalid or already explored, returns [] .
If the child configuration has all the items on the right bank, returns the entire path.
Because I’m using for
, I had to apply a few quick fixes to make it work, for example I use remove empty?
at line 80 and 86.
Another one is at line 115. The path is inside a series of lists ((((path))))
so I had to find a way to extract it.
I wonder if there’s a better alternative to for
to reach the same goal and avoid these those confusing lines
Result looks correct @roelof
Hello! Can someone help me debug this error? Here is my code
(defn get-books-with-extra-data
[url]
(let [cached (cache/fetch url)]
(if (not (nil? cached))
cached
(do
(timbre/info "Fetching book: " url)
(http/get url {} (fn [{:keys [status headers body error]}]
(->>
(map :body)
(map #(hash-map :book-genres (get-book-genres %) :book-cover (get-book-cover %)))
;(cache/store url)
)))))))
Which is called here (I invoked this from the REPL)
(defn books-with-extra-data
[books]
(->> books
(map #(merge %1 (book/get-books-with-extra-data (string/trim (get-in %1 [:book :link])))))))
And I get this strange stacktrace
(pst)
clojure.lang.ExceptionInfo: null
main.clj:442 clojure.main/repl[fn]
main.clj:458 clojure.main/repl[fn]
main.clj:458 clojure.main/repl
main.clj:368 clojure.main/repl
RestFn.java:1523 clojure.lang.RestFn.invoke
interruptible_eval.clj:79 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj:55 nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj:142 nrepl.middleware.interruptible-eval/interruptible-eval[fn]
AFn.java:22 clojure.lang.AFn.run
session.clj:171 nrepl.middleware.session/session-exec[fn]
session.clj:170 nrepl.middleware.session/session-exec[fn]
AFn.java:22 clojure.lang.AFn.run
Thread.java:748 java.lang.Thread.run
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: org.httpkit.client$deadlock_guard$reify__17177
RT.java:557 clojure.lang.RT.seqFrom
RT.java:537 clojure.lang.RT.seq
APersistentMap.java:40 clojure.lang.APersistentMap.cons
RT.java:677 clojure.lang.RT.conj
core.clj:85 clojure.core/conj
core.clj:3049 clojure.core/merge[fn]
core.clj:944 clojure.core/reduce1
core.clj:934 clojure.core/reduce1
core.clj:3048 clojure.core/merge
core.clj:3041 clojure.core/merge
RestFn.java:421 clojure.lang.RestFn.invoke
stats.clj:2 goodstats.consumer.stats/eval17623[fn]
core.clj:2755 clojure.core/map[fn]
LazySeq.java:42 clojure.lang.LazySeq.sval
LazySeq.java:51 clojure.lang.LazySeq.seq
RT.java:535 clojure.lang.RT.seq
core.clj:137 clojure.core/seq
core_print.clj:53 clojure.core/print-sequential
core_print.clj:174 clojure.core/fn
core_print.clj:174 clojure.core/fn
MultiFn.java:234 clojure.lang.MultiFn.invoke
print.clj:21 nrepl.middleware.print/pr-on
print.clj:17 nrepl.middleware.print/pr-on
print.clj:148 nrepl.middleware.print/send-nonstreamed[fn]
print.clj:147 nrepl.middleware.print/send-nonstreamed[fn]
core.clj:2742 clojure.core/map[fn]
protocols.clj:49 clojure.core.protocols/iter-reduce
protocols.clj:75 clojure.core.protocols/fn
protocols.clj:75 clojure.core.protocols/fn
protocols.clj:13 clojure.core.protocols/fn[fn]
core.clj:6884 clojure.core/transduce
core.clj:6870 clojure.core/transduce
print.clj:156 nrepl.middleware.print/send-nonstreamed
print.clj:138 nrepl.middleware.print/send-nonstreamed
print.clj:174 nrepl.middleware.print/printing-transport[fn]
caught.clj:58 nrepl.middleware.caught/caught-transport[fn]
interruptible_eval.clj:114 nrepl.middleware.interruptible-eval/evaluate[fn]
=> nil
I have no idea how to proceed from here because I'm using a callback function to process the request so I think I'm correct to assume that the response has been received when the function is called (which is the only problem I could think of). Previously I was using clj-http and it was working fine, but I switched to http-kit for better performance and most of the requests work correctly but some fail with this error
I believe http/get
returns a promise, which you must deref somewhere to get the actual returned value
the Don't know how to create ISeq from: org.httpkit.client$deadlock_guard$reify__17177
error refers to the reified deadlock-guard
promise, which you haven't dereffed
But that's when you don't give it a callback function right? When I try to deref the body parameter of the callback I get
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.concurrent.Future
if I read the httpkit code and docstring correctly, it always returns a promise but depending on whether you gave a callback it will resolve to either the response as-is or pass the response to your callback and resolve the return value
I think I know the problem. Since this is asynchronous, I think it's returning immediately to the caller code, which does some processing with the expected response causing it to fail
https://github.com/http-kit/http-kit/blob/master/src/org/httpkit/client.clj#L244
Can anyne tell me how I can make it work that the repl is next to my code instead of the same part so I do not have to switch tabs
I guess if you’re asking about VSCode and Calva, better to ask in #calva
when I try this :
(parse-string (slurp "2008.json"))
I get a file not found error where im sure the 2008.json file is in the same src directory as the chapter7,clj fileIs the chapter7.clj source file in a directory like src/some-name-space/chapter7.clj
in a Leiningen project, or something like that? If so, the current directory of the running JVM process is likely to be the top level directory of that project when you run the code.
If you print out the value of this expression, which should return a string, it should show you the current directory of the running JVM process, or at least what it was when the JVM process began: (System/getProperty "user.dir")
Seeking advice on selection of full stack suits web app of ToDo: I'd like to develop an app of todo. It should support web browser as interface, and eventually maybe native mobile app. I hope the user experience should be better than typical web browser experience, For example, I'd like to support extensive auto-completion of user input, allow context menu by clicking the right mouse button, and have timer etc. For data storage, I guess a SQL database would be enough, as the storage would be mostly in table and sequential, Datomic would be an overkill. It should also support user authentication. Would you kindly advise me which full stack suits I should take? I've studied using ring/hiccup/compojure, rum, jdbc.next, datomic for database web app. I have not tried Coast framework, Fulcro, Luminus, Pedestal, etc. With so many seemingly good choices, I feel anxious. This is a hobby/prototype effort, for MVP (Minimal Viable Product). Thanks a lot for your help!
Wonder why the output is not sorted on the numbers
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
[(fips (fips-code county))
(:driving_under_influence county)]))
(into {})))
In the last step, you put the data into a Clojure map (into {})
. Clojure maps are not sorted by keys.
There is a sorted map variant of Clojure maps, which are sorted by keys. If that is something you want to try, you could change the last line to (into (sorted-map))
(there is no literal representation of sorted maps that is like {}
for the default (unsorted) maps).
Nor ordered, see https://github.com/clj-commons/ordered
{"AZ, Maricopa" 26235,
"CA, Los Angeles" 45056,
"CA, Orange" 17572,
"CA, Riverside" 10814,
"CA, Sacramento" 8589,
"CA, San Bernardino" 13983,
"CA, San Diego" 18562,
"NV, Clark" 10443,
"TX, Harris" 10432,
"WA, King" 11439}
If you don't need constant time lookup, you might want to consider representing your data as a list of maps instead of a map:
[{:county "AZ, Maricopa" :dui 26235}
{:county "CA, Los Angeles" :dui 45056}
...]
That way you don't have to treat maps (associative arrays) as something they're not and as an added bonus you get much prettier output out of the box using print-table!
https://clojuredocs.org/clojure.pprint/print-tableor do you mean I can change something here to make this output :
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
[(fips (fips-code county))
(:driving_under_influence county)]))
(into{})
(sort-by val)))
yup, I'm talking about this:
(fn [county]
[(fips (fips-code county))
(:driving_under_influence county)])
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
[{:county (fips (fips-code county)), :population (:driving_under_influence county)}]))
(into {})))
Got it working
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
[{:county (fips (fips-code county))}, {:population (:driving_under_influence county)} ]))
(into {})))
you can simplify it further
also this still creates a map
Try this out:
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
{:county (fips (fips-code county)),
:population (:driving_under_influence county)}))))
that gives this :
({:county "CA, Sacramento", :population 8589}
{:county "TX, Harris", :population 10432}
{:county "NV, Clark", :population 10443}
{:county "CA, Riverside", :population 10814}
{:county "WA, King", :population 11439}
{:county "CA, San Bernardino", :population 13983}
{:county "CA, Orange", :population 17572}
{:county "CA, San Diego", :population 18562}
{:county "AZ, Maricopa", :population 26235}
{:county "CA, Los Angeles", :population 45056})
where my old code gave this :
{{:county "AZ, Maricopa"} {:population 26235},
{:county "CA, Los Angeles"} {:population 45056},
{:county "CA, San Diego"} {:population 18562},
{:county "CA, Sacramento"} {:population 8589},
{:county "CA, Riverside"} {:population 10814},
{:county "CA, Orange"} {:population 17572},
{:county "CA, San Bernardino"} {:population 13983},
{:county "NV, Clark"} {:population 10443},
{:county "TX, Harris"} {:population 10432},
{:county "WA, King"} {:population 11439}}
yup, notice the parens instead of the curly brackets in the beginning/end?
then tomorrow look if I can make it work to calculate the prevalance so this number divided by the population
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
{:county (fips (fips-code county)),
:total_uid (:driving_under_influence county)
:prevalance ( / (:driving_under_influence county) (:county_population county))
}))))
only the prevalence is not calculated:
{:county "CA, Los Angeles",
:total_uid 45056,
:prevalance 45056/9872263}
yes that's expected: https://clojure.org/reference/data_structures#_ratio
it is the correct form but you don't see the calculated value, you can run float on top of the division to get the actual result
how many data points are there in the file?
he, I can do this :
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :prevalance)
(take-last 10)
(map (fn [county]
{:county (fips (fips-code county)),
:total_uid (:driving_under_influence county)
:prevalance (float (/ (:driving_under_influence county) (:county_population county)))
}))))
that key doesn't exist yet when you sort
you could either compute it twice or move sort-by
and take-last
to the end, after the map
I have now this :
(defn calculate_prevalance
[county]
( if (zero? (:county_population county))
0
(float (/ (:driving_under_influence county) (:county_population county)))))
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(map (fn [county]
{:county (fips (fips-code county)),
:total_uid (:driving_under_influence county)
:prevalance (calculate_prevalance county)}))
(sort-by :prevalance)
(take 10)))
sort-by sorts the data in ascending order
so take 10
takes the 10 items with the lowest prevalence
exactly
Also, you might want to turn the fn into an actual function
| :county | :total_uid | :prevalance | |---------------+------------+-------------| | NC, Hyde | 125 | 0.024447488 | | CA, Inyo | 424 | 0.024500173 | | NC, Ashe | 655 | 0.025449742 | | CO, Costilla | 85 | 0.026001835 | | CO, Conejos | 210 | 0.026122652 | | CO, Cheyenne | 45 | 0.026254376 | | TX, Kenedy | 11 | 0.028132992 | | VA, Norton | 118 | 0.031969655 | | MS, Tunica | 432 | 0.04056338 | | WI, Menominee | 189 | 0.040935673 |
`(defn calculate_prevalance
[county]
( if (zero? (:county_population county))
0
(float (/ (:driving_under_influence county) (:county_population county)))))
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(map (fn [county]
{:county (fips (fips-code county)),
:total_uid (:driving_under_influence county)
:prevalance (calculate_prevalance county)}))
(sort-by :prevalance)
(take-last 10)))
it's sorted, in ascending order
you're just seeing the last 10 counties in the list
if you want to see it in descending order you need to reverse
the result
no need to, just add reverse
in the end
the result is correct, it's just in ascending order
you have the top 10 counties, starting from the 10th
so if you reverse it you'll get the top county first
yep,, this looks good
| :county | :total_uid | :prevalance |
|---------------+------------+-------------|
| WI, Menominee | 189 | 0.040935673 |
| MS, Tunica | 432 | 0.04056338 |
| VA, Norton | 118 | 0.031969655 |
| TX, Kenedy | 11 | 0.028132992 |
| CO, Cheyenne | 45 | 0.026254376 |
| CO, Conejos | 210 | 0.026122652 |
| CO, Costilla | 85 | 0.026001835 |
| NC, Ashe | 655 | 0.025449742 |
| CA, Inyo | 424 | 0.024500173 |
| NC, Hyde | 125 | 0.024447488 |
next challenge from the "Clojure from the ground up"
| :county | :prevalance | :report-count | :population |
|---------------+-------------+---------------+-------------|
| WI, Menominee | 0.040935673 | 189 | 4617 |
| MS, Tunica | 0.04056338 | 432 | 10650 |
| VA, Norton | 0.031969655 | 118 | 3691 |
| TX, Kenedy | 0.028132992 | 11 | 391 |
| CO, Cheyenne | 0.026254376 | 45 | 1714 |
| CO, Conejos | 0.026122652 | 210 | 8039 |
| CO, Costilla | 0.026001835 | 85 | 3269 |
| NC, Ashe | 0.025449742 | 655 | 25737 |
| CA, Inyo | 0.024500173 | 424 | 17306 |
| NC, Hyde | 0.024447488 | 125 | 5113 |
hmm, I think the last challenge is nonsens
Write a test to verify that most-prevalent is correct.
Make them up. The logic should be correct no matter what the actual values of the numbers are.
correctness != truth
`
(defn most-duis
"Given a JSON filename of UCR crime data for a particular year, finds the
counties with the most DUIs."
[file]
(->> file
load-json
(sort-by :driving_under_influence)
(take-last 10)
(map (fn [county]
[(fips (fips-code county))
(:driving_under_influence county)]))
(into (sorted-map))))
`
sorted-map order keys by hash function, not alphabeticaly
this is wrong ) my bad
@roelof The result you showed last above is sorted alphabetically by the keys. What do you want it sorted by?
Oh, I see now you said earlier "on the numbers". One recommendation: write a little print function to print the data in the order you want to see it in, but let it be stored in a default Clojure map, which is not sorted in any particular order (well, it is sorted on a hash function of the keys, but that is unlikely to be an order you are interested in knowing).
Example:
(def d1 {"AZ, Maricopa" 26235,
"CA, Los Angeles" 45056,
"CA, Orange" 17572,
"CA, Riverside" 10814,
"CA, Sacramento" 8589,
"CA, San Bernardino" 13983,
"CA, San Diego" 18562,
"NV, Clark" 10443,
"TX, Harris" 10432,
"WA, King" 11439})
user=> (pprint (sort-by val d1))
(["CA, Sacramento" 8589]
["TX, Harris" 10432]
["NV, Clark" 10443]
["CA, Riverside" 10814]
["WA, King" 11439]
["CA, San Bernardino" 13983]
["CA, Orange" 17572]
["CA, San Diego" 18562]
["AZ, Maricopa" 26235]
["CA, Los Angeles" 45056])
I know that last print result does not look like a map. That is because it is a sequence of map entries, which is what you get when you take a map and use it as a parameter to a function that expects a sequence, like sort-by
does in the expression above. If all you care about is seeing it in the desired order, that shouldn't be a problem.
I have a pattern type of question for designing APIs. I come from a Java background where there are a lot of pure and impure functions mixed together, especially when reading and writing from a database. I've read Clojure guidance that says to write as much of the logic in pure functions as possible, and then use some sort of wiring layer to mix together these pure functions. It would seem that once you get past simple CRUD APIs, the coordination functions would become both very large and could potentially have a lot of duplication. Can anyone point me to either examples or tutorials that show how this works as the codebase grows?
This is a great question! I'll do my best to answer and hopefully my attempt will nerd snipe someone smarter into writing a better answer. > It would seem that once you get past simple CRUD APIs, the coordination functions would become both very large and could potentially have a lot of duplication. I think the key idea is that if you use simple constructs and generic data, the coordinating code doesn't grow at all as your code base grows. > Computer science offers a standard way to handle complexity:hierarchical structure. > - Leslie Lamport https://lamport.azurewebsites.net/pubs/proof.pdf Typically, the coordinating code relies on a tree-like structure for organizing all of the code/functionality needed for an application. Traversing the tree-like structure doesn't take very much code and it's the same amount of code regardless of the size of the tree. Trees also have the benefit that searching a large, organized tree is very efficient. An example of coordinating code that uses this principle is https://github.com/ring-clojure/ring/wiki. Returning a response to an HTTP request is done by composing pure functions. The coordinating code is the same size for both small and large web sites. Typically, routes are organized hierarchically and hopefully don't have too many interactions between separate branches of the routing tree. I can't think of any tutorials, but the Rich Hickey talks are all very good. On this topic, I think https://www.youtube.com/watch?v=ROor6_NGIWU and https://www.youtube.com/watch?v=drmNlZVkUeE might be good talks to start with.
It can be hard to completely separate out database interaction from pure logic because you typically have a lot of situations where the pure logic at some point may well rely on DB activity from earlier in the request (assuming a complex DB-backed web app). The pragmatic question is: how separate does it really need to be? You definitely want to aim for separation and composability but there are practical limits to what you can realistically achieve in general, in my opinion, without resorting to somewhat artificial code structure and monadic-looking code.
https://www.youtube.com/watch?v=yTkzNHF6rMs https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell The 2 vids may help somewhat (functional core, imperative shell). Also, personal anecdote: afaict functional architectures tend to converge into middleware pattern (ring's wrap-*, re-frame interceptors, stuff I seen in enterprises).
Years ago, I built a generic "workflow" engine library for Clojure that separated everything into "sources", pure functions, and "sinks". The idea was that sources could (only) be queried -- the sources were passed into the pure code, so you could write mock sources for testing -- and that the pure functions would return data structures that indicated what actions would need to be taken on the sinks, which could be database writes (inserts and updates) and email systems and anything else you needed to apply changes or effects on. As soon as a later part of your pure workflow needs the generated ID from an earlier DB insert, things get complex, and it also gets a bit artificial when the DB inserts you need "at the end" have to include stuff that you got from side-effecting calls to third-party systems. It's doable, but it gets pretty convoluted. And I finally gave up on the engine after using it for a while in production code because it was just too awkward (and monadic) to be readable once the actual workflow got complex. It's still on GH as an archive https://github.com/seancorfield/engine with an explanation of why I stopped using it.
Having an imperative shell drive everything is a much more straightforward approach but it can still take quite a bit of work to completely separate out pure and impure logic -- and if you have an imperative shell, you are still going to need some amount of tests to verify that the imperative combining of the pure/impure calls really does what you want. Definitely more maintainable than just arbitrarily mixing pure and impure code together tho' -- and more maintainable than going "full-monad" IMO.
I think it's interesting that many of these functional ideas can even be applied to games, https://fabiensanglard.net/quake3/index.php
Thanks for the information and the links. I'm working my way through a course on building APIs using Reitit, and once I get past whatever's blocking me right now, I'll try to build a sample API with this pattern in mind.
Old lisp expert here. New to Clojure. Writing a Swank client in ClojureScript (replacement for Emacs SLIME, for the browser). Once that’s done, I’ll likely add a cider client as well. This is a retirement project for me. I don’t have the mental energy I once did, so the Clojure startup transient is harder than it would have been 10 years ago, but I’ll get there. Clojure is, after all, just another lisp with some added syntax and a new standard library to learn (he said, not really knowing what he’s saying). https://github.com/billstclair/wilfred Not much here yet. If somebody has ideas for improvement of the basic project, I’d love the feedback. I’m just copying what I found on the web at this point. I think it will be good enough to get started, however.
@andy.fingerhut thanks, solved it
I'm trying to grasp clojure here - playing with stumps of code. How do I loop through a list of maps and rewrite the maps (in other languages filter can be used for this):
(def mylist
({:a 1 :b 2 :c 3}
{:a 1 :b 5 :c 6}
{:a 7 :b 8 :c 12}
{:a 10 :b 11 :c 12}))
and I want to modify the key 'b' so it contains the contents of key 'c'(map (fn [m] (assoc m :b (m c)) mylist)
Thanks. Trying that out however, Calva repl (visual studio code) just gives me:
println mylist #prints out mylist in repl window
(map (fn [m] (assoc m :b (m :c)) mylist)) #just prints out: #function[clojure.core/map/fn--5847]
😞so println mylist works.. map just fails.. I'm doing ctrl+shift+p and evaluate top form
It appears that it evaluated just the sub-expression map
, which I would expect to return a result like you showed.
this worked:
println (map (fn [m] (assoc m :b (m :c))) mylist)
Thank youNow I have something to play with.. trying to get the hang of this 🙂
FYI the println
at the beginning of those lines is probably not even being evaluated at all. You should be able to leave that out.
Unless the REPL you are using is quite different than ones I have used before somehow ...
you're right.. now it outputs it without the prinln @andy.fingerhut - odd it didn't at first
I would be very surprised if it actually required that -- the P in REPL is for Print, so after reading and evaluating your expression, it should always Print the return value (in addition to printing anything that the expression itself causes to print, e.g. a function call with calls to functions print
and/or println
inside of it).
This is a great question! I'll do my best to answer and hopefully my attempt will nerd snipe someone smarter into writing a better answer. > It would seem that once you get past simple CRUD APIs, the coordination functions would become both very large and could potentially have a lot of duplication. I think the key idea is that if you use simple constructs and generic data, the coordinating code doesn't grow at all as your code base grows. > Computer science offers a standard way to handle complexity:hierarchical structure. > - Leslie Lamport https://lamport.azurewebsites.net/pubs/proof.pdf Typically, the coordinating code relies on a tree-like structure for organizing all of the code/functionality needed for an application. Traversing the tree-like structure doesn't take very much code and it's the same amount of code regardless of the size of the tree. Trees also have the benefit that searching a large, organized tree is very efficient. An example of coordinating code that uses this principle is https://github.com/ring-clojure/ring/wiki. Returning a response to an HTTP request is done by composing pure functions. The coordinating code is the same size for both small and large web sites. Typically, routes are organized hierarchically and hopefully don't have too many interactions between separate branches of the routing tree. I can't think of any tutorials, but the Rich Hickey talks are all very good. On this topic, I think https://www.youtube.com/watch?v=ROor6_NGIWU and https://www.youtube.com/watch?v=drmNlZVkUeE might be good talks to start with.
I want to do some natural language processing (summarizing) and DL (one-shot-learning). I found this: https://dragan.rocks/articles/19/Deep-Learning-in-Clojure-From-Scratch-to-GPU-0-Why-Bother and Dragan recommends to use intel-mkl. He states it's 750 mb, but the only download I can find is a whooping 22gb. Are there other libraries I should use, or is this really the way to go?
It is best if you install via your package manager. Failing that, install via the Intel installer. I don't believe either of those are anywhere close to 22GB. The instructions at https://neanderthal.uncomplicate.org/articles/getting_started.html#the-native-library-used-by-neanderthals-native-engine-optional should be helpful. Which OS are you on?
Yikes! I just checked and on my MacOS machine, /opt/intel (where MKL is installed) is around 3GB. Perhaps 22GB is required during installation but not after the installation is complete?
Hi everybody, maybe this is a dumb question, but, I don't feel confortable with my solution up to now. I work with a defmulti, defined in a ns, and some defmethod, provided in their respective namespace. Now, when I want to call the method, I fall in situations where de defmethod are not found, until I explicitly require the namespace containing the defmethod. I would prefer a behavior where the calling function needs not to know its implementation, as a plugin is expected to act.
One thing you can do is expose a namespace that requires all of the defmethod-containing namespaces and make it so users require that
Right now it sounds like they are requiring the namespace with the defmulti- maybe stick the defmulti somewhere else, then in this top level NS require all of the namespaces that extend the defmulti
@caumond maybe you want a protocol? https://stackoverflow.com/questions/4509782/simple-explanation-of-clojure-protocols
it seems like there is an implicit dependency between the code calling the multi method and the namespace implementing some corresponding defmethod. it seems like the "right" answer is to make the implicit dependency an explicit dependency (ie. make it so there's no way to call the the multi-method with args you expect it to handle without requiring the namespace that provides the implementation for those args). it's hard to offer more specific advice without knowing a little more about the use case.
@U3ES97LAC this will have the same problem, I believe
Hi, @U3ES97LAC I don't see the breakthrough with protocols, maybe I missed something. @U7RJTCH6J, I guess this is my best option, I'm not sure how to do it. I will provide a simple example just to share my issue an better see the solution
What I am not happy with are the near useless require of english and french namespace in the core.ns
And here is a git repo of this example https://gitlab.com/caumond/defmulti-multi-ns
looks good to me. I think it's reasonable to expect someone that uses greeting to load the namespaces it needs to fill in which ever languages they expect
you can also override the default implementation of greeting to throw a more helpful exception
maybe an exception that says something like (str "Did you forget to load the plugin for language " x "?")
reading through Clojure Applied and it mentions something similar:
(defmethod convert :default [u1 u2 q]
(if (= u1 u2)
q
(assert false (str "Unknownunitconversionfrom "u1" to "u2))))
this chapter is available online, http://media.pragprog.com/titles/vmclojeco/validating.pdf
Glad you found a solution.
Hi everyone! Does anybody use hickory library? https://github.com/davidsantiago/hickory