Since Java 9, at least Hotspot, and perhaps other JVMs, use 8-bit characters for strings that contain only code points in the range [0, 127], as a space optimization.

Yes, but the exposed interface is still utf16

Alright Thank you! (I figured out what I was looking for. Or rather I think my eyes had been playing tricks on me)

Oh maybe this is a result of for using chunk-cons ? When I change the parameters of my example code so that the sequence is 100 elements long, but the total amount of memory allocated is still the same, the for version also works. I guess if the amount of memory needed for all entries in single chunk is more than can be allocated, then it OOMs?

to be clear, for is not a loop and doesn't use looping internally - it is built out of "recursive" calls that are lifted out via lambda encapsulation

IOW exactly how you'd use lazy-seq and self calls in any other lazy function you construct

for, or a lazy-seq, only causes OOM if you are irresponsible about holding the return values and preventing GC (outside pathological cases where a single element blows the RAM)

by binding the result of for in a top level def, you ensure that the vm's GC can never reclaim any value used in the lazy seq it generates, that is the source of your OOM and the answer is to never do that apologies - clearly I'm not fully awake yet, you were using for in defn, not def, so that isn't the issue

(you'd get the same problem, but faster, if you used loop to build the equivalent non-lazy list)

This classic blog post (about concat) describes so gotchas with lazy-seqs and imperative code. I think with a slight change of perspective it can also offer some insight about using resource-intensive or side effecting functions inside lazy code https://stuartsierra.com/2015/04/26/clojure-donts-concat

Yeah the issue here is definitely that the design of my code is resulting in a reference to the head of the lazy sequence. I have some code that produces a sequence of messages representing server responses for a TCP server, and another layer that calls that code and puts each message into the stream representing the socket. To best separate these layers I was hoping to just return a lazy sequence in the one case where I have a lot of messages, but it seems like I cannot implement some code that iterates that lazy sequence without ultimately keeping the entire thing in memory.

there are ways to do it carefully, but I think the simplest thing is not to use laziness when you need precise control of resource usage

core.async is often used incorrectly or for the wrong things, but backpressure on execution to control resource utilization is something it can do well (though manifold which you are using already can arguably do that even better...)

Yeah, I think that's what I assumed was happening with the design I had previously. It was working fine until I changed something to actually wait on the final result of putting all the messages into the stream.

So my solution now involves providing a function that can be iteratively called to produce the next message to put into the stream. This seems to be working great, and I think it's an acceptable design (really I was just hoping to keep stream-related concepts out of the business logic). I am curious though, you mentioned it would possibly have been possible to use a lazy sequence if it was carefully done. Do you have any suggestions on stuff I could read to understand how that might have worked? Thanks again for the help!

https://github.com/athos/clj-check I believe

thanks, I would like to see in README what check check 🙂 It is a little mysterious 🙂

Is it checking formatting? I don’t want to fail build when formatting is not as clj-check wants it

lein help check -> Check syntax and warn on reflection.

As far as I recall, it just checks for reflection warnings (https://github.com/athos/clj-check/blob/master/src/clj_check/check.clj#L14-L15)

Though, it may check within your libraries code as well

thank you, I will try this lib

You'll want to check that you get a non-zero exit code for reflection warnings -- it may only give a non-zero exit code for compilation failures.

Ah, it does not:

(! 773)-> clojure -A:check
Compiling namespace check.example
Reflection warning, check/example.clj:7:3 - call to method append on java.lang.String can't be resolved (no such method).

Thu Jul 02 14:06:26
(sean)-(jobs:0)-(~/clojure/example)
(! 774)-> echo $?
0

it also hang forever

clj -Acheck
Jul 02, 2020 11:05:38 PM api.core invoke
INFO: app start K_REVISION: nil
Jul 02, 2020 11:05:38 PM api.core invoke
WARNING: HOST variable is not set!

Do we have other alternatives to check reflections?

It looks like it not only load namespaces but also run main functions so start the server

ring is started so process never exit

No, it only loads namespaces.

so why hang forever?

So you must be doing something in a def form that causes your server to start.

I don’t

I bet you do. There's nothing in clj-check that runs any functions. It just loads files.

so maybe server not start but check never end

Nope. check works just fine. The problem is in your code.

Wait for 60 seconds and see if it exits. You may be starting thread processes just by loading namespaces, and clj-check does not include shutdown-agents so it will wait for threads to cleanly shutdown, instead of just exiting.

Should be easy enough to add. The author was quite receptive when last contacted

:check {:extra-deps {athos/clj-check {:git/url ""
                                                 :sha "b48d4e7000586529f81c1e29069b503b57259514"}}
                   :main-opts ["-m" "api.core"]}

You should be using clj-check’s main-opts

No, the readme has this

:main-opts ["-m" "clj-check.check"]}}

with this it doesn’t work for me

YOU are running your own main function

Define "doesn't work".

Compiling namespace spec.utils
Compiling namespace spec.common
Execution error (FileNotFoundException) at clj-check.check/check-ns (check.clj:15).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.

(! 767)-> clj -A:new app check/example
Downloading: seancorfield/clj-new/maven-metadata.xml from clojars
Generating a project called example based on the 'app' template.

Thu Jul 02 14:05:28
(sean)-(jobs:0)-(~/clojure)
(! 768)-> cd example/

Thu Jul 02 14:05:30
(sean)-(jobs:0)-(~/clojure/example)
(! 769)-> clojure -A:check
Compiling namespace check.example

Thu Jul 02 14:05:39
(sean)-(jobs:0)-(~/clojure/example)
(! 770)-> vi src/check/example.clj 

Thu Jul 02 14:06:11
(sean)-(jobs:0)-(~/clojure/example)
(! 771)-> clojure -A:check
Compiling namespace check.example
Reflection warning, check/example.clj:7:3 - call to method append on java.lang.String can't be resolved (no such method).

@U0WL6FA77 Your output shows it is working perfectly.

You forgot to provide aliases to add the dependencies you need.

I expect your spec.common namespace is requiring clojure.test.check and you didn't have -A:test to add that dependency?

oh true… I tried to add test but probably I made a typo or something

I have this feeling in this month I have stupid issues here on Clojurians channel

I need holidayś 🙂

You need to be more methodical and take more careful steps when you are debugging.

I know this is bad explanation, but I am really tired, because of some private reasons. But I am going on holidays in July so I will recover 🙂

I am just half brain now or call it however you want 🙂

Enjoy your vacation! Definitely sounds like you need it! Unwind, and come back refreshed.

I truly need. Thank you.

Once you get this figured out, you could run clojure -A:check 2>&1 > check.log and if there are compilation errors, that should fail the build, then do fgrep -v Compiling check.log | fgrep -v Reflection which should have a zero status if there are no reflection warnings, and a status of 1 if there are -- which should fail the build (assuming it fails on any non-zero exit status for any command)

Reflection warning, bidi/bidi.cljc:20:31 - call to static method decode on java.net.URLDecoder can't be resolved (argument types: java.lang.String, unknown).

you can always grep for your project namespaces

this is true, but solution start to be complex and prone to errors

I will try it and will see

you can try editing the clojure mode variable clojure-defun-indents by putting (put-clojure-indent 'some-symbol :defn) in you .emacs

presumably (put-clojure-indent '>defn :defn) based on your link

Perfect, that works wonderfully. I'll just tuck that in my .dir-locals file