This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-01
Channels
- # announcements (6)
- # atom-editor (4)
- # babashka (59)
- # beginners (51)
- # bristol-clojurians (6)
- # calva (5)
- # cider (22)
- # clara (6)
- # cljfx (28)
- # clojure (182)
- # clojure-australia (1)
- # clojure-berlin (1)
- # clojure-dev (21)
- # clojure-europe (12)
- # clojure-italy (8)
- # clojure-nl (7)
- # clojure-norway (13)
- # clojure-uk (20)
- # clojurescript (2)
- # code-reviews (24)
- # conjure (9)
- # cursive (9)
- # datomic (35)
- # defnpodcast (45)
- # fulcro (18)
- # graalvm (3)
- # graphql (1)
- # hugsql (1)
- # jobs (3)
- # jobs-discuss (2)
- # klipse (1)
- # leiningen (1)
- # malli (5)
- # membrane (4)
- # off-topic (7)
- # portal (2)
- # reitit (30)
- # remote-jobs (1)
- # rewrite-clj (5)
- # ring (2)
- # shadow-cljs (25)
- # test-check (2)
- # tools-deps (5)
- # vrac (26)
- # xtdb (17)
Looking for a defn that does the following: given n hashmaps which have deeply nested entries and may have an heterogeneous shape between them, determine a "sub hashmap", possibly nested, that all n items are a "super hashmap" of Example:
(let [corpus [{:foo {:a 1 :b 2 :c 3}}
{:foo {:a 1 :b 9}}]]
(is (= (desired-fn corpus)
;; :b is discarded because the :b values aren't equal
;; :c is discarded because it's only present in one item of the corpus
{:foo {:a 1}})))
(defn keys-in
"Returns a sequence of all key paths in a given map using DFS walk."
[m]
(letfn [(children [node]
(let [v (get-in m node)]
(if (map? v)
(map (fn [x] (conj node x)) (keys v))
[])))
(branch? [node] (-> (children node) seq boolean))]
(->> (keys m)
(map vector)
(mapcat #(tree-seq branch? children %)))))
(defn find-submap [ma mb]
(let [find-in (fn [m ks]
(find (get-in m (drop-last ks))
(last ks)))
ks (keys-in ma)]
(reduce
(fn [m k] (let [va (get-in ma k)
eb (find-in mb k)]
(if (and eb (= va (val eb)))
(assoc-in m k va)
m)))
{}
ks)))
this basically takes the first map and gets all of the vectors that would have some answer for (get-in m v)
and then builds a map where the answer to the get in is the same for both maps.
keys-in
code taken from https://dnaeon.github.io/clojure-map-ks-paths/ which is the important bit reallyVery nice one! Yes, I had thought of the same algo after posting the question. Had seen keys-in
somewhere, not sure if I could have googled it anyway.
ooh that's a really nice idea 👍 (comp last clojure.data/diff)
I guess it acts a little differently as map values will also get diffed if they are sequences or other handled things
@ghadi a while ago you posted an implementation of pipeline
that did not preserve order. Just dotting i's but wanted to be explicit if we could use that code in our codebase at work
(editscript/diff [:div {:class "Apple"}
[:p "I hate pears!"]]
[:div {:class "Apple" :id "aaa"}
[:p "I hated pears!"]])
=> [[[1 :id] :+ "aaa"] [[2 1] :r "I hated pears!"]]
(let [out (ByteArrayOutputStream. 4096)
writer (transit/writer out :json)]
(transit/write writer [[[1 :id] :+ "aaa"] [[2 1] :r "I hated pears!"]])
(.toString out))
=> "[[[1,\"~:id\"],\"~:+\",\"aaa\"],[[2,1],\"~:r\",\"I hated pears!\"]]"
(let [out (ByteArrayOutputStream. 4096)
writer (transit/writer out :json)]
(transit/write writer (editscript/diff [:div {:class "Apple"}
[:p "I hate pears!"]]
[:div {:class "Apple" :id "aaa"}
[:p "I hated pears!"]]))
(.toString out))
Execution error (NullPointerException) at com.cognitect.transit.impl.AbstractEmitter/marshalTop (AbstractEmitter.java:203).
null
If I pass it the result of calling the function directly I get a null pointer exception
(deftype ^:no-doc EditScript [^:unsynchronized-mutable ^PersistentVector edits
^boolean auto-sizing?
^:unsynchronized-mutable ^long size
^:unsynchronized-mutable ^long adds-num
^:unsynchronized-mutable ^long dels-num
^:unsynchronized-mutable ^long reps-num]
Yeah. It's printing nice because it has
#?(:clj (defmethod print-method EditScript
[x ^java.io.Writer writer]
(print-method (get-edits x) writer))
:cljs (extend-protocol IPrintWithWriter
EditScript
(-pr-writer [o writer opts]
(write-all writer (str (get-edits o))))))
get-edits returns a plain data representation that you could transfer over the wire and then turn back into a EditScript instance. I've done that before
(let [out (ByteArrayOutputStream. 4096)
writer (transit/writer out :json)]
(transit/write writer (edit/get-edits (editscript/diff [:div {:class "Apple"}
[:p "I hate pears!"]]
[:div {:class "Apple" :id "aaa"}
[:p "I hated pears!"]])))
(.toString out))
=> "[[[1,\"~:id\"],\"~:+\",\"aaa\"],[[2,1],\"~:r\",\"I hated pears!\"]]"
What’s an easy way to split a vec of n items into a vec of vecs, where the number of items in each ‘split’ is given by another vec? E.g. (get-splits (range 10) [5 2 3]) => [[0 1 2 3 4] [5 6] [7 8 9]]
The nicest solution I can come up with is
(loop [[n & ns] [5 2 3], xs (range 10), acc []]
(if n
(recur ns (drop n xs) (conj acc (vec (take n xs))))
acc))
…but I’m thinking there’s probably a more clever way :)subvec
might be worth a look.
https://clojuredocs.org/clojure.core/subvec
Your loop recur implementation is the most readable IMO.
Use of reductions is super elegant, @UA5LS8DMJ!
i don't know if it would be the best to put in production, loop-recur is more standard and anyone can understand that, but I am fond of the elegant solutions myself 🙂
Legibility and maintainability are important to me too, but so is composition. Loop/recur doesn’t compose well with other operations.
Good answers on the topic of loop/recur here for anyone who’s interested: https://stackoverflow.com/questions/32641902/clojure-loop-recur-pattern-is-it-bad-to-use
Wdyt about this (using reductions and subvec):
(let [elements (vec (range 10))
indexes [5 2 3]]
(map (fn [from to]
(subvec elements from to))
(reductions + (cons 0 indexes))
(reductions + indexes)))
I am hoping to be able to put my webserver in production by packaging it as an uberjar, running the uberjar, and then put it behind a nginx reverse proxy for SSL support.
I'm using the component framework, and I've been having some troubles with having a graceful exit.
I figured I could do something simple with Runtime/getRuntime and addShutdownHook, but it does not seem to be working and I am getting rather frustrated.
The rest of the code is here: https://github.com/schmidt73/guidescan-web/blob/master/src/guidescan_web/core.clj
my only guess is that the way you have been testing it somehow doesn't count as an "orderly shutdown" for the jvm
Also, if the O/S gives a SIGKILL () signal (kill -9 in Unix/Linux) or TerminateProcess (Windows), then the application is required to terminate immediately without doing even waiting for any cleanup activities. In addition to the above, it is also possible to terminate the JVM without allowing the shutdown hooks to run by calling Runime.halt() method.
where the first not here is the first "mot here" that appears when reading the snippet from top to bottom
@emccue I think I figured it out... it was occuring because I was testing with "lein run"
so if I send a sigint, it is sent to the lein process that is running my process...
some internal state per user -> ... -> rendering function -> ... -> Hiccup -> Editscript Diff that Hiccup -> Send over websocket -> Reagent render that hiccup
... -> events come from users somehow -> events fed to the "process" that maintains that state -> ... -> update state -> ... -> GOTO START
Encode that process model as data, let me put it in a database, then I can be stateless, then servers and connections don't matter.
anyone have experience with having user.clj along with dependencies in a separate development directory?
i tried adding creating dev/user.clj so that user.clj wouldn't be under src/ and then modifying the :dev profile appropriately, but now i'm getting that my tests can't find any code namespaces in src when I reload everything...
reading through this there are a lot of things about the JVM that get closer to BEAM with the addition of virtual threads
what's the difference between a thread pool and virtual threads?
to run more than N tasks concurrently on N threads, you need to structure those tasks so that they yield control at specific scheduling points
http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html is a good overview if its the document i think it is
for virtual threads, the scheduling points are added by the JVM and not the programmer, so your code isn't "infected" by needing explicit async-style constructs
clojure gets as close as it can to goroutines via rewriting code around puts and takes in a macro
but fairly widespread and fundamental stuff like JDBC will never be able to participate
are virtual threads possible on the jvm without changes to the jvm?
the person who last worked on that is the person who is the public face of project loom
is the big difference between M virtual threads and M OS threads just the amount of memory consumed per thread? iirc, most OS's have improved their schedulers to work better with high numbers of threads
I'm personally not super sure about the properties of OS threads that requires a larger stack space
https://rcoh.me/posts/why-you-can-have-a-million-go-routines-but-only-1000-java-threads/
...each OS thread has its own fixed-size stack. Though the size is configurable, in a 64-bit environment, the JVM defaults to a 1MB stack per thread. You can make the default stack size smaller, but you tradeoff memory usage with increased risk of stack overflow. The more recursion in your code, the more likely you are to hit stack overflow. If you keep the default value, 1k threads will use almost 1GB of RAM!
even if the kernel scheduler is "improved" to juggle many threads, parking and unparking OS threads is not cheap
hmm, wasn't the main issue with shutdown-agents just that if you don't do it your program won't halt for 30 seconds or something?
we could mark the pool as daemon threads, which would cause the jvm to exit as soon as the main thread exits (assuming no other non-daemon threads are created)
hmm, so there are 3 possible behaviors we are talking about, 2 of which could be made true today
https://puredanger.github.io/tech.puredanger.com/2010/06/08/clojure-agent-thread-pools/
and the one we can't do because it would break expectations is, we have any running threads be marked non-daemon, but they aren't pooled so whenever their tasks finish they die
you can change those pools to whatever you want with http://clojure.github.io/clojure/clojure.core-api.html#clojure.core/set-agent-send-executor%21 and http://clojure.github.io/clojure/clojure.core-api.html#clojure.core/set-agent-send-off-executor%21
so programs can "fix" this now if they really want to
the jvm also runs at least one default threadpool now (for the completablefuture stuff) which maybe agents should be running on, but that is also a compatibility issue
for fork join stuff I guess https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html#commonPool--
yeah, the reason I am thinking of it at all is because I was trying to make my own agents
(defn process-create [initial-state update-function]
#:process{:state (atom initial-state)
:error (atom nil)
:message-queue (ArrayBlockingQueue. 10)
:update-function update-function})
(set-agent-send-executor! (Executors/newUnboundedExecutor (-> (Thread/builder)
(.daemon false)
(.virtual)
(.factory))))
(set-agent-send-off-executor! (Executors/newUnboundedExecutor (-> (Thread/builder)
(.daemon true)
(.virtual)
(.factory))))
Is there space on the craft taking you to Mars?