https://github.com/nubank/docopt.clj is another one

@U06FLL69W I'm using docopt with babashka for dev tasks for rewrite-clj. I wrapped it to be more task usage oriented than command line usage oriented.

Just wanted to point out you can do this with tools.cli too using https://github.com/clojure/tools.cli#in-order-processing-for-subcommands

Purely documentation.

Have a look at java.util.Arrays/copyOfRange

seq version will likely require more intermediate allocation, copyOfRange should be a JVM intrinsic (though not 100% on that) that just allocates the new array up front

Thank you. I'm already implemented my solution using java.util.Arrays/copyOfRange but I'm interested to compare seq vs copyOfRange versions and how to see memory consumption in JVM?

You could try visualvm https://docs.oracle.com/javase/8/docs/technotes/guides/visualvm/profiler.html or yourkit https://www.yourkit.com/docs/java/help/memory_telemetry.jsp

in either case the resultant byte array will be the same size on the heap, the only difference will be in intermediate allocations to create the copy.

the intermediate collections will be garbage collected as you go so really the impact will be felt in the time it takes to copy

Thank you. I'll take VisualVM to see it.

if I have 1 000 000 array, then Arrays/copyOfRange makes new array of the same size minus 15 bytes. so it is O(n) solution. I thought that seq may be lazy and I can use this property to consume more effectively the rest of array. e.g. apply fn to current array but starting from 15th byte and not to copy it to another almost the same size.

when you call (byte-array seq) you will realise the entire sequence and allocate an array with the same size as the seq.

it might help if we knew the use case because it may be you do not need to allocate a byte-array at all, in which case the seq version (or something else) may be better

but if you need a new array which is a slice of another one copyOfRange is the best you can get afaik

My case: I have encrypted data in bytes buffer [header, data]. I need to read header, make header analysis, then decrypt data (skip 15 bytes of header). I need to do it in a most effective way cause buffers can be large.

What options do you have to decrypt? can you take an iterator, nio buffer?

if its gotta be an array, best I can do is copyOfRange 🙂

I can take any option

what does the (presumably java?) function to decrypt look like?

(i.e signature)

it is a clojure wrapper function on Bouncycastle java function, so signature is (decrypt-bytes ^Cipher cipher ^bytes encrypted-buffer)

Or to be effective we should wait for Project Valhalla? 🙂 (to apply fn to byte-array starting from nth position)?

I changed my solution to read by 1024 bytes and decrypt step by step without memory consumption.

🙂

how about just left shifting 30 bits https://clojuredocs.org/clojure.core/bit-shift-left

I think this is not for large buffers

use bytebuffers

(ByteBuffer/wrap your-byte-array)

you can then slice the buffer and pass it to a Cipher. (Be careful using BouncyCastle, it is a very error prone library)

oh, thank you. where I can read about errors in BC ? I need some investigation for my cases.

what algorithm are you attempting to use?

only gost. I’m working on wrapper for Clojure for Gost

(the general topic here is called 'misuse resistance' -- aka use google/tink rather than a low level swiss-army knife style library, such as BC)

i think BC is your only option for GOST

Peculiarities of floating point operations. I believe this section explains it: https://en.wikipedia.org/wiki/Floating-point_arithmetic#Representable_numbers,_conversion_and_rounding

TL;DR is that decimal numbers are generally only approximate in nearly all languages. In Clojure you can get precise results using rationals (+ 1/2 1/10 3/10) or BigDecimals (+ 0.5M 0.1M 0.3M), but both of these come with some more overhead that might matter if you're doing a lot of number crunching.

Also, between those, BigDecimals are of course only precise for things with a closed decimal form (e.g., 1/3 will still have some imprecision as a BigDecimal), and both are imprecise for anything that isn't rational (e.g., the square root of 2).

congrats eelke, you just found one of the monsters at the bottom of computing! floating point maths is neither commutative nor associative!

https://www.youtube.com/watch?v=ajaHQ9S4uTA

this is a really really good video on floating point in java

(or for that matter, in any lang)

Thanks, haven't seen that one.

Here are some compensated summation algorithms you can use to limit the error accumulation if you're adding many numbers : https://github.com/sicmutils/sicmutils/blob/main/src/sicmutils/series.cljc

kahan-babushka-neumaier is the one to use usually

Thanks all, interesting stuff

Indeed, the singleton clojure.lang.PersistentArrayMap.EMPTY is used. You can double-check by calling (System/identityHashCode {}) multiple times.

Cool, thank you!