Can someone help me understand if I have found an error in Brave Clojure? It gives an implementation of partial:

(defn my-partial
  [partialized-fn & args]
  (fn [& more-args]
    (apply partialized-fn (into args more-args))))

(defn print-args
  [& args]
  (loop [index 0]
    (println (format "arg #%d: %s" index (nth args index)))
    (if (< index (dec (count args)))
      (recur (inc index)))))

=> ((my-partial print-args "Today" "was" "a") "good" "day")
arg #0: day
arg #1: good
arg #2: Today
arg #3: was
arg #4: a
nil

=> ((partial print-args "Today" "was" "a") "good" "day")
arg #0: Today
arg #1: was
arg #2: a
arg #3: good
arg #4: day
nil

into uses the semantics of whatever collection it’s collecting into. & args is a seq

@lilactown And seqs have stuff pushed onto them?

so when elements are added to the args seq, it adds them to the head

yeah

@lilactown Hmm. So the implementation in the book is actually wrong?

could be

@lilactown Here is the usage example in the book:

(def add20 (my-partial + 20))
(add20 3) 
; => 23

also they’re not using many args

but yeah, pretty much

I’m guessing it’s a mistake

Yeah I would hope so because I’ve sank way too much time into trying to understand why I don’t get it. It’s probably a mistake.

I think the easiest fix is to change into -> concat

Yes, that does the trick!

Thanks for the help.

@lilactown I was about to submit a message to the repository of the book but it has dozens of issues and pull requests that haven’t been addressed in years, doesn’t seem worth the time. Now I’m starting to doubt the quality of this resource. But it’s been helpful so far so I guess I’ll stick with it a bit longer.

@aryyya.xyz it's a good resource, stick with it.

I’m trying to use print-stack-trace with GraalVM, but that doesn’t work since it has reflection issues (which can be resolved using type hints)

I’ll post a JIRA

https://dev.clojure.org/jira/browse/CLJ-2502

Does anybody know what is the motivation for having single-argument = arity in Clojure which always returns true? Is this a technical detail or is there a mathematical reason for it?

how to spec a hashmap, whose keys are not specs themselves? say I need to provide spec for a hashmap whose keys are string literals and not keywords? should I be using something other than spec/keys?

I am using a multimethod that dispatches based on deriving the spec of an incoming message.

should I just roll my own spec (with contains?) or is there something in the std library or a simple way of working with spec/keys that i am not aware of?

> (-> '[^int? ^some? a] first meta)
{:tag int?}

well, ^foo bar is the same as ^{:tag 'foo} bar, and ^foo bar baz is the same as ^{:tag 'foo} ^{:tag 'bar} baz, which is semantically (with-meta (with-meta baz {:tag 'bar}) {:tag 'foo})

so it's not "lost", you're overriding it

(same as doing (assoc (assoc {} :foo 1) :foo 2))

I’m trying to pass a custom deftype over a core.async channel, and it seems to coming out of the other side as a seq, losing its type information and structure.

Is this a known thing, or is there something I’m missing?

doesn't ring a bell

user=> (require '[clojure.core.async :as a])
nil
user=> (deftype Foo [x])
user.Foo
user=> (def c (a/chan 1))
#'user/c
user=> (a/>!! c (->Foo 1))
true
user=> (def c2 (a/<!! c))
#'user/c2
user=> (class c2)
user.Foo
user=> (ancestors (class c2))
#{clojure.lang.IType java.lang.Object}
user=> (.-x c2)
1

looks fine to me with a simple test

Yeah, I think I’ve worked it out

this particular deftype implements ISeq

That seems to be enough for it to treated as one — if I remove that implementation it makes it through intact

I think I know what's going on

can you give me a small repro for this?

hold on

https://gist.github.com/clarkema/48da41fb5cf720f7f5bbba2428d5299a

try that as is, and then comment out the iseq implementation

ty

yeah so this is not what it appears to be ;)

user=> (->Segment 0 :foo [1 2 3])
(1 2 3)
user=> (class (->Segment 0 :foo [1 2 3]))
user.Segment

nothing at all with core.async

it's just that print-dup gives priority to ISeq over IType

so the object is printed as if it was a seq

print-dup or whatever is responsible for printing here, I can never remember what does what

one of those multimethods anyway

hmm

which is weird because I thought there were appropriate prefer-methods

let me check

this actually cropped up because I’m trying to match on it (using core.match) on the other side of the channel

looks like the prefer-methods is just for records, not types

printing what you took out of the channel can make it look like types are being lost

but they're not

yeah, it does still have the correct type

so something else is being weird

well you've told clojure that Segment is a seq so it's printing it using the seq printer

arguably not the best behavior but that's how it works now

if the type is definitely getting lost maybe you have something that tries to pr-str/read-string the value?

it’s possible that it’s actually to do with the match treating it as a seq

likely

you should probably make that type seqable instead of a seq directly if you can

I think I can live without it being a seq; it was mostly for completness

Kind of a weird interaction though

thanks

I guess that explains why a lot of things implement a “to seq” style rather than directly implement seq

I remember a comment in core match about destructing problems since the types implemented both seq and assoc protocols

yes, the general advice is to make types ISeqable instead of ISeq, when you care about them being more than simple seqs

which doesn't really make the type less powerful since most collections operate on seqables instead of just on seqs

Did anyone ever have a problem that JVM doesn't flush SoftReferences even when the heap is 99% occupied, and everything is spinning in FullGC most of the time? I'm using hot Java recompilation heavily which inflates DynamicClassLoader's classCache, and it never clears.

the simplest explanation is your soft refs are being held strongly somewhere

Nope, they are not. Explicitly triggering OutOfMemoryError clears them out.

well DCL only clears cache when you define a new class

due to Java recomp are you never defining new classes?

and thus never triggering a clear?

> due to Java recomp are you never defining new classes? It's the opposite, I define new classes all the time with defineClass.

I'm using Virgil, here's how a class gets defined there: https://github.com/ztellman/virgil/blob/master/src/virgil/compile.clj#L129

In fact it might not even be related to Virgil/Java compilation as I'm using tools.namespace too, and the main contributors to inflated heap seem to be some big data objects that are referenced by Clojure function classes, which are in turn kept around by classCache.

I took a heap dump of the application, and MAT couldn't find any live paths to GC roots from those big objects. FullGC was doing nothing. However, as soon as manually triggered OOM, those softrefs successfully disappeared.

when you say "explicitly triggering an outofmemoryerror" what mechanism are you using to do that?

@alexmiller Interestingly enough, right now the DynamicClassLoader.rq is empty, but those SoftReferences are still there.

@hiredman (long-array 5e8)

and on what thread are you raising that exception, and what happens if you raise a different exception on that thread?

Just the REPL thread. Other exceptions won't do anything.

I learned that trick here: https://stackoverflow.com/questions/3785713/how-to-make-the-java-system-release-soft-references/3810234#3810234

it sounds like you are allocating small enough amounts of memory that the full gc recovers a very small amount about that seems to be enough so refs aren't cleared, and then you allocate again, so full gc runs again, etc

either increase or decrease your heap size

Yep, it looks exactly like what is happening.

(or fiddle with gc tuning params)

I doubt changing the heap size would work here, it's currently set to 7g for devtime, and the leak manifests itself very slowly, it takes hundreds of reloads before the heap becomes occupied enough to cause lags and hangups. And it never goes past it, I was at 100% OldGen util, and it still refused to give up those SoftRefs.

> (or fiddle with GC tuning params) That one could possibly work, yet it is still strange to me that such behavior could be the default. Maybe switching a GC would help, but I don't want to move away from ParGC during development.

if it is what I described the issue is in the relationship between the gc cycles and the mutator cycles, it isn't an issue of heap size per se, but changing the heap size will change the gc cycle and hopefully break the relationship

you may want to try g1

I prefer a predictable GC in development, but this is subjective and another story. Thanks for the help, anyway.

as of java 9 g1 is the default collector

in java 12, I know the new ZGC actually has support for parallel class unloading, which might be helpful for you

in general though, I'd say this is a level of gc tuning that I am aware of, but not an expert in :)

Interesting, I didn't link this problem to class unloading. They could indeed be related. I gonna study the GC logs, see if there's a mention of it there.

On a different note, I had a huge success running Clojure with ShenandoahGC, hope to write about it someday.

would love to read that

would make a great conj talk ;)