Fork me on GitHub
#clojure
<
2020-09-18
>
Saikyun07:09:09

is there any way for me to setup a development environment using clj, on a computer that doesn't have direct access to the internet?

sogaiu08:09:32

in the prerelease version of clj, i think there's a -P -- you might try #tools-deps for related inquiries

Saikyun08:09:22

thanks : )

👍 3
Saikyun07:09:06

I want to be able to: 1. install clojure (it's a windows machine) 2. pre-download all dependencies and transfer them from my local machine to that computer

Saikyun07:09:25

so that I'm able to run clj on that machine without it trying to download dependencies

Saikyun07:09:39

setting up a local maven repo might be a solution, but I have no idea how to do that / how hard it is to do

viesti08:09:59

hmm, how do add a writer for native Java arrays for Transit?

user> (require '[cognitect.transit :as transit])
(import [ ByteArrayInputStream ByteArrayOutputStream])
niljava.io.ByteArrayOutputStream
user> (def out (ByteArrayOutputStream. 4096))
#'user/out
user> (def writer (transit/writer out :json))
#'user/writer
user> (transit/write writer [1 2])
nil
user> (.toString out)
"[1,2]"
user> (transit/write writer (into-array Long [1 2]))
Execution error (NullPointerException) at com.cognitect.transit.impl.AbstractEmitter/marshalTop (AbstractEmitter.java:203).
null

viesti05:09:56

So something like this:

user> (def writer (transit/writer out
                                  :json
                                  {:handlers {(Class/forName "[Ljava.lang.Long;")
                                              (transit/write-handler
                                                (fn [_] "array")
                                                (fn [x] (vec x)))}}))
user> (transit/write writer (into-array Long [1]))
nil
user> (.toString out)
"[1]"

Adam Helins08:09:09

So, I have been reading a few things about Clojure and GraalVM. There is not much about using both for AWS lambdas, but it seems promising. Has anyone been experimenting with this? I am especially more interested in knowing the tradeoffs Vs using Cljs (performance-wise, not regarding the library ecosystem)

Saikyun08:09:42

I don't know anything about lambda functions specifically (I'm guessing you mean AWS lambda?), but overall graalvm vs cljs the tradeoff is: • building graalvm takes a long time • startup is ~same (I think <0.01s) • graalvm faster runtime overall

👍 3
Saikyun08:09:37

these are my experiences, don't have any hard numbers to back it up 🙂

Saikyun08:09:02

I think @UG1C3AD5Z has more to say on the subject

Adam Helins08:09:06

I heard that GraalVM became quite a bit more friendly towards Clojure (eg. plays nicer with common libs), how true is that in your experience?

sogaiu08:09:16

i think a good person to ask that question to would be borkdude. on the subject of clojure working better with graalvm, i think there was some work on the clojure side toward this end. see here for details: https://github.com/lread/clj-graal-docs

Saikyun08:09:48

personally, I've mostly used the "talk to c" capabilities of graalvm: https://github.com/Saikyun/clobits so don't know much about clojure libs

Saikyun08:09:01

this page lists some of the libs that are compatible with graalvm 🙂 https://github.com/BrunoBonacci/graalvm-clojure

sogaiu08:09:33

probably it's good to inquire at #graalvm for this type of thing

Adam Helins08:09:37

@UG1C3AD5Z You're right, my bad

Josef Richter10:09:02

Is anyone successfully running Clojure on Big Sur, please?

hkjels11:09:27

I haven’t had the chance to dabble much with Clojure lately, but I’ve upgraded to Big Sur and the Clojure REPL works as expected. What are the problems you’re facing?

Josef Richter11:09:51

@U0B1SDL67 I was unable to install clojure via homebrew. solved it by installing manually.

hkjels11:09:09

Ahh, OK. Nice that it panned out!

Josef Richter15:09:20

sorry, missed your message. basically brew tells you this package is not compatible with big slur, so good bye

fabrao11:09:02

hello all, Is there any way to limit memory usage of clojure program?

borkdude11:09:52

@fabrao -Xmx and similar Java options are one way to limit memory

fabrao12:09:05

like java -Xmx512m ?

fabrao12:09:49

limit using 512Mb?

borkdude12:09:07

Yes. That's only for the heap size though, there are also other resources that take memory

fabrao12:09:27

so, my clojure program can get more than 512 Mb?

fabrao12:09:47

what is the other resources?

Norman Eckstein13:09:20

RoamResearch is written in Clojure/Script?

p-himik13:09:29

From https://roamresearch.com/#/app/help/page/Vu1MmjinS: > The Roam stack: > React > Clojure and Clojurescript > Datalog

Norman Eckstein13:09:50

Thanks @p-himik . I just registered today on Roam after reading a few times headlines about it. After getting excited I checked for the team and found a page about Conor...

Norman Eckstein13:09:07

No wonder why its rocking 😆

Norman Eckstein13:09:30

Cool shit 💪:skin-tone-2:

p-himik13:09:03

I have a DAG of Model records that I need to serialize. Each model has an :id and some attributes that may contain another model, a collection of models, a map of keyword -> model, or any other Clojure value. Right now I have this code:

(defrecord Ref [id])

(let [acc (atom (transient {}))]
  (postwalk (fn [item]
              (if (satisfies? Model item)
                (let [id (:id item)]
                  (swap! acc assoc! id item)
                  (Ref. id))
                item))
            roots)
  (persistent! @acc))
The atom is there because assoc! can return a new transient collection. The question is, can I rewrite this code so it stays simple but doesn't rely on atom anymore? I couldn't think of anything better than a number of nested loops or calls to reduce, and that's far from being simple.

Alex Miller (Clojure team)13:09:14

you often see things like this with walking/updating state while you do it and I think it's perfectly fine if you're using walk (you might find volatiles to have slightly lower overhead though and sufficient for single-threaded use case)

Alex Miller (Clojure team)13:09:07

another option is to use zippers but I'm not sure it's any better for this

borkdude14:09:58

this is a pattern I've also used in a couple of places (with volatile!) (e.g. discovering used params in a function body)

borkdude14:09:14

actually, the entire clj-kondo analysis is basically a walk + swap! to a couple of atoms

p-himik14:09:45

Thanks! That's reassuring. :) Right, I completely forgot about volatiles.

ghadi14:09:27

recursive for or mapcat are nice functional options, too @p-himik

benoit16:09:43

Good example of how introducing some local state can sometimes make functions much easier to read 🙂

benoit16:09:46

I'm being a bit unfair, the big difference in readability might be mostly due to the use of postwalk compared to the custom traversal in the functional style.

p-himik16:09:57

Yeah, it's a useful function in this case. But on the other hand, with a custom traversal code I could do that traversal in a more efficient way.

benoit16:09:40

Yes, it is always the same trade-off in programming :)

andy.fingerhut14:09:42

The above use of swap! for atoms that contain mutable things (transients in the example above) is safe for single-threaded use, but can lead to incorrect results if you try to do it from multiple threads, I believe? I am not saying "Don't do it in a single-threaded case", just trying to make sure for myself when it is unsafe.

Alex Miller (Clojure team)14:09:26

putting mutable things in atoms is safe re visibility (atoms guarantee visibility)

Alex Miller (Clojure team)14:09:56

atoms are an issue only if you're trying to coordinate changes in multiple mutable things (as they don't do coordination)

dpsutton14:09:23

what do you mean by visibility here?

Alex Miller (Clojure team)14:09:10

changes in non-volatile fields are not necessarily visible to other threads for arbitrary periods of time. Clojure largely protects you from issues around this by a) using immutable values (so changes are not a thing in the first place) and b) providing tools for safe change (atoms, refs, etc)

👍 3
Alex Miller (Clojure team)14:09:41

transients are kind of a special case - they are "local" mutability

jsn14:09:04

@andy.fingerhut swap! is safe, only e.g. deref + reset! is not, I guess?

Alex Miller (Clojure team)14:09:27

that's still "safe" in terms of well defined semantics

Alex Miller (Clojure team)14:09:30

but it is a race condition

jsn14:09:00

Perhaps that race condition might be what @andy.fingerhut meant

andy.fingerhut14:09:15

Doing assoc! on a transient collection from multiple threads would be unsafe in many cases, true?

Alex Miller (Clojure team)14:09:12

you need some way to guarantee that a) the result of the transient op is used for the next op and b) other threads see those changes

Alex Miller (Clojure team)14:09:26

basically, serializability is the answer to both, which atoms give you

andy.fingerhut14:09:01

Sure, atoms give you visibility of changes, but assoc! by multiple threads on the same transient is going to give you visibility of garbage. (edit: not garbage in the gc sense, but in the sense of "a mutable object that has been banged on by multiple threads in parallel, and they have likely messed up its internal state)

Alex Miller (Clojure team)14:09:47

it's not the same transient if you're using assoc! in a swap!

Alex Miller (Clojure team)14:09:06

swap! is replacing the value with the result of the assoc!

Alex Miller (Clojure team)14:09:33

I guess I should it may or may not be the same transient, but the important thing is that you follow the same usage pattern as assoc

ghadi14:09:12

I find atom+transient harder to reason about

ghadi14:09:29

atom + persistent -> no brainer

ghadi14:09:24

even though the atom usage above is scoped to a single function and doesn't escape threads... Clojure makes no promise that your swap! will be called only once

isak14:09:53

Is the transient guaranteed to be different when you call assoc!? I thought it was just that it may return a different value

Alex Miller (Clojure team)14:09:22

@isak it will not always be a different value - sometimes the current one is modified and returned

Alex Miller (Clojure team)14:09:39

@ghadi that's a good point and could indeed cause problems

ghadi14:09:42

atom+transient is inappropriate, IMHO

isak14:09:51

Ok thought so

isak14:09:07

How about volatile + transient though?

isak14:09:24

I think it is ok, because then the expectactions are different

isak14:09:35

volatile is only about usage from 1 thread, right?

isak15:09:36

I got that impression from this part of the release notes: > volatiles - there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.

Alex Miller (Clojure team)15:09:28

thread isolation here means 1 thread at a time not only one thread so you should be careful about the difference

isak15:09:07

Got it, thanks

andy.fingerhut15:09:21

@ghadi I was thinking that atom+transient (or any other mutable object in general) + single-thread only might be safe, which is the context where it was originally brought up in this discussion. It is atom+transient (or any other mutable object in general) + multiple threads concurrently that I was double-checking my understanding about. But maybe I'm wrong that single-thread is safe there.

ghadi15:09:17

hypothetically if we used weakCompareAndSet instead of compareAndSet under the hood, it could possibly retry even in one thread

3
andy.fingerhut15:09:18

Huh, hadn't seen weakCompareAndSet before. Sounds like something only an expert implementer of concurrency mechanisms might ever use?

ghadi16:09:02

AtomicReference itself uses a weak CAS internally

ghadi16:09:07

yes, expert stuff

ghadi15:09:43

besides, your function should have a budget of one mutative thing :)

andy.fingerhut16:09:21

That is some rule-of-thumb for mutable data? Seems like a reasonable one.

isak15:09:03

I wonder if vswap! is guaranteed to only run once, unlike swap!

walterl16:09:36

Is this a superfluous indent in select-keys, or am I missing something? https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L1545

Alex Miller (Clojure team)16:09:44

Clojure source is the result of a large number of patches so it's not particularly regular

Alex Miller (Clojure team)16:09:35

looks like that particular one was a mass re-format from Rich, must have been something off with whatever tool was used

Alex Miller (Clojure team)16:09:26

that was 11 yrs ago, so who knows

Jeff Evans19:09:05

I was also wondering why there are large chunks of commented-out code still around? Ex: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java#L819 https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Numbers.java#L2014 I assumed it was just a “move fast, break things” kind of result, but perhaps there’s another reason? In any event, not being critical, just curious.

potetm19:09:24

@jeffrey.wayne.evans that’s a rich comment

dpsutton19:09:47

Not in a Java file

potetm19:09:52

e.g. (comment …bunch-of-helpful-forms) at the bottom of a file

dpsutton19:09:18

The linked bit is just commented out Java code

andy.fingerhut19:09:18

It is the closest one can get in a Java file to a Rich comment containing test code for the code in the same file, I think.

potetm19:09:46

@dpsutton does it make a diff?

potetm19:09:11

“here is some code you might find helpful when messing w/ this class”

dpsutton19:09:16

Ha a little bit but I thought it was outdated but didn’t realize it was a separate main for testing

potetm19:09:17

which I assume was the result of some exploration

andy.fingerhut19:09:39

It looks like the last one is in inner class ChunkedSeq, so potentially a copy-and-paste from similar inner classes, then realized that he never wanted to do assoc or conj to increase the size of a ChunkedSeq object, so commented it out. Hard to tell sometimes the history/reasons, unless it is somehow clear from the git commit history, but even then not always.

dpsutton19:09:52

i've seen these blocks before and always just assumed it was abandoned features or commented out and didn't think they were kinda debug/introspection bits

Jeff Evans19:09:49

Yeah, that was sort of my assumption as well, but I figured there would be a comment just above that with a short explanation as to why. You can glean some of that by checking the history, of course.

andy.fingerhut19:09:50

In general, there is very little answer to 'why' in Clojure's implementation code that I have found. There are some answers to those questions in some of Rich's talks, which you can find transcripts for most of them and links to videos here: https://github.com/matthiasn/talk-transcripts. None of those answers are 'why' at the detail line-by-line code level, but at a higher level of explanation. For line-by-line kinds of questions, the best source I know of is asking here or in #clojure-dev to see if anyone knows. There might be some answers in the book Clojure: The Essential Reference, but I do not know how closely any reasons given there correspond to the reasons of the Clojure developers.

ghadi19:09:25

lots of the commented out static main methods are benchmarks

Alex Miller (Clojure team)19:09:52

sometimes Rich keeps explorations around in comments. it's Rich's world, we just live in it.

🙂 3
firstclassfunc21:09:14

Hello all. is there anyway to get to the underlying map type for a lazy seq? (type (doall (lazy-seq {:1 2 :3 4})))

ghadi21:09:35

@gary001 what are you actually trying to achieve?

firstclassfunc21:09:12

I have a return value that might be a vector or map and I need to dispatch off that type

ghadi21:09:18

you can check if something is a vector? or map?

ghadi21:09:29

but that is separate from being included in a lazy-seq

firstclassfunc21:09:31

yes but its a lazy-seq

firstclassfunc21:09:00

I need to realize the lazy-seq to find its type.

ghadi21:09:15

sorry I'm not quite following

ghadi21:09:28

an element knows not what it is contained in

firstclassfunc21:09:03

(type (doall (lazy-seq {:1 2 :3 4})))
=> clojure.lang.LazySeq
(type (doall (lazy-seq [1 2 3 4])))
=> clojure.lang.LazySeq

seancorfield21:09:24

Once you've turned the map into a lazy sequence, it's a sequence (of MapEntry items) -- it is no longer a map. Same with turning a vector into a lazy sequence -- it is no longer a vector. (and this feels more suitable for #beginners -- not sure where you are on your journey @gary001?)

dpsutton21:09:52

i think this is a bit like the underlying type of the result of (+ (float 1) (float 1)). its a double regardless of how you got there. lazy-seq produces a new value

firstclassfunc22:09:17

@seancorfield Thanks thats what I thought, I am consuming from an external library so have to find another way.. thanks

seancorfield22:09:21

@gary001 So you're getting the lazy sequence from the library? Which library is it? I'm curious as to how/where the map vs vector dilemma comes into such code...

firstclassfunc22:09:30

@seancorfield http response from my server can return data in different shapes and I am trying to dispatch my output based off of the type so I can format the results.

seancorfield22:09:51

@gary001 Still not following how the lazy seq aspect plays into this... can you elaborate? (and I figured we'd take it to a thread rather than clog up the main channel at this point)

seancorfield22:09:56

Which library is returning a lazy seq where you need to find our what the lazy seq was originally built from?

dpsutton22:09:10

seems strange to get a lazy-seq back as a response from a webserver. that doesn't really make sense to me