This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-18
Channels
- # announcements (1)
- # babashka (15)
- # beginners (152)
- # calva (28)
- # circleci (1)
- # clj-kondo (24)
- # cljsrn (8)
- # clojure (137)
- # clojure-berlin (3)
- # clojure-czech (2)
- # clojure-dev (20)
- # clojure-europe (69)
- # clojure-finland (5)
- # clojure-france (3)
- # clojure-italy (11)
- # clojure-my (1)
- # clojure-nl (4)
- # clojure-uk (15)
- # clojuredesign-podcast (1)
- # clojurescript (13)
- # conjure (15)
- # cursive (13)
- # datomic (41)
- # deps-new (50)
- # events (1)
- # fulcro (9)
- # graalvm (27)
- # joker (2)
- # kaocha (11)
- # off-topic (22)
- # pathom (48)
- # rdf (6)
- # reagent (25)
- # reitit (47)
- # reveal (10)
- # ring-swagger (1)
- # rum (4)
- # sci (27)
- # shadow-cljs (73)
- # tools-deps (49)
- # vrac (2)
- # xtdb (4)
is there any way for me to setup a development environment using clj
, on a computer that doesn't have direct access to the internet?
in the prerelease version of clj, i think there's a -P
-- you might try #tools-deps for related inquiries
I want to be able to: 1. install clojure (it's a windows machine) 2. pre-download all dependencies and transfer them from my local machine to that computer
so that I'm able to run clj
on that machine without it trying to download dependencies
setting up a local maven repo might be a solution, but I have no idea how to do that / how hard it is to do
hmm, how do add a writer for native Java arrays for Transit?
user> (require '[cognitect.transit :as transit])
(import [ ByteArrayInputStream ByteArrayOutputStream])
niljava.io.ByteArrayOutputStream
user> (def out (ByteArrayOutputStream. 4096))
#'user/out
user> (def writer (transit/writer out :json))
#'user/writer
user> (transit/write writer [1 2])
nil
user> (.toString out)
"[1,2]"
user> (transit/write writer (into-array Long [1 2]))
Execution error (NullPointerException) at com.cognitect.transit.impl.AbstractEmitter/marshalTop (AbstractEmitter.java:203).
null
So something like this:
user> (def writer (transit/writer out
:json
{:handlers {(Class/forName "[Ljava.lang.Long;")
(transit/write-handler
(fn [_] "array")
(fn [x] (vec x)))}}))
user> (transit/write writer (into-array Long [1]))
nil
user> (.toString out)
"[1]"
So, I have been reading a few things about Clojure and GraalVM. There is not much about using both for AWS lambdas, but it seems promising. Has anyone been experimenting with this? I am especially more interested in knowing the tradeoffs Vs using Cljs (performance-wise, not regarding the library ecosystem)
I don't know anything about lambda functions specifically (I'm guessing you mean AWS lambda?), but overall graalvm vs cljs the tradeoff is: • building graalvm takes a long time • startup is ~same (I think <0.01s) • graalvm faster runtime overall
I think @UG1C3AD5Z has more to say on the subject
I heard that GraalVM became quite a bit more friendly towards Clojure (eg. plays nicer with common libs), how true is that in your experience?
i think a good person to ask that question to would be borkdude. on the subject of clojure working better with graalvm, i think there was some work on the clojure side toward this end. see here for details: https://github.com/lread/clj-graal-docs
personally, I've mostly used the "talk to c" capabilities of graalvm: https://github.com/Saikyun/clobits so don't know much about clojure libs
this page lists some of the libs that are compatible with graalvm 🙂 https://github.com/BrunoBonacci/graalvm-clojure
@UG1C3AD5Z You're right, my bad
Is anyone successfully running Clojure on Big Sur, please?
I haven’t had the chance to dabble much with Clojure lately, but I’ve upgraded to Big Sur and the Clojure REPL works as expected. What are the problems you’re facing?
@U0B1SDL67 I was unable to install clojure via homebrew. solved it by installing manually.
Unable how?
sorry, missed your message. basically brew tells you this package is not compatible with big slur, so good bye
Yes. That's only for the heap size though, there are also other resources that take memory
RoamResearch is written in Clojure/Script?
From https://roamresearch.com/#/app/help/page/Vu1MmjinS: > The Roam stack: > React > Clojure and Clojurescript > Datalog
Thanks @p-himik . I just registered today on Roam after reading a few times headlines about it. After getting excited I checked for the team and found a page about Conor...
No wonder why its rocking 😆
Cool shit 💪:skin-tone-2:
I have a DAG of Model
records that I need to serialize. Each model has an :id
and some attributes that may contain another model, a collection of models, a map of keyword -> model, or any other Clojure value.
Right now I have this code:
(defrecord Ref [id])
(let [acc (atom (transient {}))]
(postwalk (fn [item]
(if (satisfies? Model item)
(let [id (:id item)]
(swap! acc assoc! id item)
(Ref. id))
item))
roots)
(persistent! @acc))
The atom
is there because assoc!
can return a new transient collection.
The question is, can I rewrite this code so it stays simple but doesn't rely on atom
anymore? I couldn't think of anything better than a number of nested loops or calls to reduce, and that's far from being simple.you often see things like this with walking/updating state while you do it and I think it's perfectly fine if you're using walk (you might find volatiles to have slightly lower overhead though and sufficient for single-threaded use case)
another option is to use zippers but I'm not sure it's any better for this
this is a pattern I've also used in a couple of places (with volatile!) (e.g. discovering used params in a function body)
actually, the entire clj-kondo analysis is basically a walk + swap! to a couple of atoms
Good example of how introducing some local state can sometimes make functions much easier to read 🙂
I'm being a bit unfair, the big difference in readability might be mostly due to the use of postwalk
compared to the custom traversal in the functional style.
Yeah, it's a useful function in this case. But on the other hand, with a custom traversal code I could do that traversal in a more efficient way.
The above use of swap!
for atoms that contain mutable things (transients in the example above) is safe for single-threaded use, but can lead to incorrect results if you try to do it from multiple threads, I believe? I am not saying "Don't do it in a single-threaded case", just trying to make sure for myself when it is unsafe.
putting mutable things in atoms is safe re visibility (atoms guarantee visibility)
atoms are an issue only if you're trying to coordinate changes in multiple mutable things (as they don't do coordination)
changes in non-volatile fields are not necessarily visible to other threads for arbitrary periods of time. Clojure largely protects you from issues around this by a) using immutable values (so changes are not a thing in the first place) and b) providing tools for safe change (atoms, refs, etc)
transients are kind of a special case - they are "local" mutability
@andy.fingerhut swap!
is safe, only e.g. deref
+ reset!
is not, I guess?
that's still "safe" in terms of well defined semantics
but it is a race condition
Perhaps that race condition might be what @andy.fingerhut meant
Doing assoc!
on a transient collection from multiple threads would be unsafe in many cases, true?
you need some way to guarantee that a) the result of the transient op is used for the next op and b) other threads see those changes
basically, serializability is the answer to both, which atoms give you
Sure, atoms give you visibility of changes, but assoc!
by multiple threads on the same transient is going to give you visibility of garbage. (edit: not garbage in the gc sense, but in the sense of "a mutable object that has been banged on by multiple threads in parallel, and they have likely messed up its internal state)
it's not the same transient if you're using assoc! in a swap!
swap! is replacing the value with the result of the assoc!
I guess I should it may or may not be the same transient, but the important thing is that you follow the same usage pattern as assoc
even though the atom usage above is scoped to a single function and doesn't escape threads... Clojure makes no promise that your swap! will be called only once
Is the transient guaranteed to be different when you call assoc!
? I thought it was just that it may return a different value
@isak it will not always be a different value - sometimes the current one is modified and returned
@ghadi that's a good point and could indeed cause problems
I recant! :)
not necessarily
I got that impression from this part of the release notes: > volatiles - there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.
thread isolation here means 1 thread at a time not only one thread so you should be careful about the difference
@ghadi I was thinking that atom+transient (or any other mutable object in general) + single-thread only might be safe, which is the context where it was originally brought up in this discussion. It is atom+transient (or any other mutable object in general) + multiple threads concurrently that I was double-checking my understanding about. But maybe I'm wrong that single-thread is safe there.
hypothetically if we used weakCompareAndSet instead of compareAndSet under the hood, it could possibly retry even in one thread
Huh, hadn't seen weakCompareAndSet before. Sounds like something only an expert implementer of concurrency mechanisms might ever use?
That is some rule-of-thumb for mutable data? Seems like a reasonable one.
Looks like it, at least with the current implementation: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Volatile.java#L25-L27
Called from here: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L2541-L2548
Is this a superfluous indent in select-keys
, or am I missing something? https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L1545
Clojure source is the result of a large number of patches so it's not particularly regular
looks like that particular one was a mass re-format from Rich, must have been something off with whatever tool was used
https://github.com/clojure/clojure/commit/703c847b1a470100d2113ae76352647400d32476
that was 11 yrs ago, so who knows
I was also wondering why there are large chunks of commented-out code still around? Ex: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java#L819 https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Numbers.java#L2014 I assumed it was just a “move fast, break things” kind of result, but perhaps there’s another reason? In any event, not being critical, just curious.
@jeffrey.wayne.evans that’s a rich comment
It is the closest one can get in a Java file to a Rich comment containing test code for the code in the same file, I think.
Ha a little bit but I thought it was outdated but didn’t realize it was a separate main for testing
more interesting is this: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java#L437-L466
It looks like the last one is in inner class ChunkedSeq, so potentially a copy-and-paste from similar inner classes, then realized that he never wanted to do assoc
or conj
to increase the size of a ChunkedSeq
object, so commented it out. Hard to tell sometimes the history/reasons, unless it is somehow clear from the git commit history, but even then not always.
i've seen these blocks before and always just assumed it was abandoned features or commented out and didn't think they were kinda debug/introspection bits
Yeah, that was sort of my assumption as well, but I figured there would be a comment just above that with a short explanation as to why. You can glean some of that by checking the history, of course.
In general, there is very little answer to 'why' in Clojure's implementation code that I have found. There are some answers to those questions in some of Rich's talks, which you can find transcripts for most of them and links to videos here: https://github.com/matthiasn/talk-transcripts. None of those answers are 'why' at the detail line-by-line code level, but at a higher level of explanation. For line-by-line kinds of questions, the best source I know of is asking here or in #clojure-dev to see if anyone knows. There might be some answers in the book Clojure: The Essential Reference, but I do not know how closely any reasons given there correspond to the reasons of the Clojure developers.
sometimes Rich keeps explorations around in comments. it's Rich's world, we just live in it.
Hello all. is there anyway to get to the underlying map type for a lazy seq? (type (doall (lazy-seq {:1 2 :3 4})))
I have a return value that might be a vector
or map
and I need to dispatch off that type
yes but its a lazy-seq
I need to realize the lazy-seq to find its type.
(type (doall (lazy-seq {:1 2 :3 4})))
=> clojure.lang.LazySeq
(type (doall (lazy-seq [1 2 3 4])))
=> clojure.lang.LazySeq
Once you've turned the map into a lazy sequence, it's a sequence (of MapEntry
items) -- it is no longer a map. Same with turning a vector into a lazy sequence -- it is no longer a vector. (and this feels more suitable for #beginners -- not sure where you are on your journey @gary001?)
i think this is a bit like the underlying type of the result of (+ (float 1) (float 1))
. its a double regardless of how you got there. lazy-seq produces a new value
@seancorfield Thanks thats what I thought, I am consuming from an external library so have to find another way.. thanks
@gary001 So you're getting the lazy sequence from the library? Which library is it? I'm curious as to how/where the map vs vector dilemma comes into such code...
@seancorfield http response from my server can return data in different shapes and I am trying to dispatch my output based off of the type so I can format the results.
@gary001 Still not following how the lazy seq aspect plays into this... can you elaborate? (and I figured we'd take it to a thread rather than clog up the main channel at this point)
Which library is returning a lazy seq where you need to find our what the lazy seq was originally built from?
seems strange to get a lazy-seq back as a response from a webserver. that doesn't really make sense to me