Fork me on GitHub
#clojure
<
2019-11-22
>
joefromct06:11:38

i just watched a documentary on time space and naturally i'm wondering if time travel is possible is every language mutable?

Cameron06:11:30

Mutability is just time travelling back to a value you already declared, and having to constantly deal with the butterfly effect

joefromct06:11:45

hmm... but there is a quantifiable number of butterfly effects though right, directly tied to the number of values declared ... so technically that's still vars * butterfly-paths number of immutable variables ?

joefromct06:11:56

i don't know what i'm talking about. sorry.

jaihindhreddy06:11:36

^ We can continue the disc. in #off-topic ...

👍 1
Crispin08:11:30

hi! bashing my head against trying to read the filesize of a binary file in resources/

Crispin08:11:20

everything I try either explodes, or gives me the wrong value (as it's interpreting the file as an encoded stream)

Crispin08:11:29

this is surprisingly hard

Crispin08:11:14

this works at the repl

> (.length (io/file (io/resource "libspire.so")))
12968

Crispin08:11:08

but inside a jar this gives Exception in thread "main" java.lang.IllegalArgumentException: Not a file: jar:file:/full/path/to/target/uberjar/my.jar!/libspire.so

Gerome09:11:29

Could it be that clojure is looking in the wrong place for the file?

Cameron09:11:46

Nah, unless I'm missing something, the problem should have been that io/resource is basically going to get the java URL for the file, which is going to look something like jar:file:/full/path/to/target/uberjar/my.jar!/libspire.so . io/file is going to try to wrap this URL with the java File object, which I believe is expecting a legitimate path, not a path into a jar. So, it fails and gives that error. Anyone feel free to correct me on any of those details, as there's a few parts I'm hazy on myself, although that should be the jist

noisesmith17:11:42

io/file is for files on disk and doesn't work for jar resources

noisesmith17:11:58

slurp is for text, and doesn't work for binary input

noisesmith17:11:29

try creating an InputStream from the resource and checking .available maybe?

noisesmith18:11:20

user=> (.available (io/input-stream (io/resource "clojure/core.clj")))
263653

noisesmith18:11:43

I don't have binary jar resources handy off the top of my head, but that method is binary clean

Crispin08:11:30

if I slurp it first, I get the wrong size

amarjeet08:11:04

Hi, I have a question regarding ordering of keys in map. I have a map, say my-map in the following snippet, I wanted to understand if the order of keys and vals will be preserved in the results of (keys my-map) and (vals my-map) no matter how many times I call them, as long as I supply the same my-map.

(def my-map {:name "Alice" :age 30 :place "BLR"})
(keys my-map) => (:name :age :place)
(vals my-map) => ("Alice" 30 "BLR")
Or, will the order in the results of (keys my-map) and (vals my-map) change because my-map cannot guarantee its own order while calling (keys my-map) or (vals my-map)? I am currently using https://github.com/clj-commons/ordered library to first make my-map an ordered map, then apply keys and vals operations to ensure required ordering.

schmee09:11:27

the order is the same for multiple calls to an identical map, but as soon as you add or remove an element the order is not guaranteed

schmee09:11:38

if you’re using ordered then the the maintains the insertion order of the elements, which may or may not be what you want

amarjeet09:11:52

@U3L6TFEJF thanks 🙂

👍 1
Cameron08:11:22

@retrogradeorbit I am curious, what happens if you try (.count (io/input-stream (io/resource "libspire.so"))) (I'm half sending this out blind since I don't have a repl and similar situation up on my end)

Crispin08:11:37

spire.core=> (.count (io/input-stream (io/resource "libspire.so")))
Execution error (IllegalArgumentException) at spire.core/eval8772 (form-init2733631356982990701.clj:1).
No matching field found: count for class .BufferedInputStream

Crispin08:11:07

spire.core=> (count (io/input-stream (io/resource "libspire.so")))
Execution error (UnsupportedOperationException) at spire.core/eval8774 (form-init2733631356982990701.clj:1).
count not supported on this type: BufferedInputStream

valerauko08:11:45

it gets complicated when you jar stuff

Crispin08:11:48

Yeah I tried them. StringWriter gave the wrong size (reading as an encoded stream)

valerauko08:11:26

i haven't tried but it seems you need to get a JarFile , then getEntry your resource and then you can getSize or getCompressedSize. i couldn't yet get past the hurdle of the JarFile so not sure if this works at all...

Cameron08:11:37

alright one more thing I'm just curious to see

(with-open [in (io/input-stream (io/resource "libspire.so"))
            out (.ByteArrayOutputStream.)] 
       (io/copy in out)
       (.size out))

👍 1
Cameron18:11:46

Well, one reason I avoided available is

Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream.
And specifically
Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

noisesmith18:11:53

I was gambling that the resource implementation of inputstream would know the full size (where other implementations are allowed not to)

noisesmith18:11:18

I would like to find another method that finds the full size without a linear time procedure though.

Cameron18:11:39

off the top of my head, if that information is not cached somewhere (I say 'cached' in the abstract; stored) already, I do not think that would be possible, as at some point something is going to have to read through and basically count it for the first time. As always, though, I want to think on it to make sure I'm not missing something

noisesmith18:11:53

the information is in the jar itself

noisesmith18:11:08

or the filesystem if it's read off disk

Cameron18:11:52

hmm indeed I wonder if there'd be good way to just directly read that

noisesmith18:11:45

so here https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#available() "many will not" here http://www.docjar.com/docs/api/java/io/FilterInputStream.html#available (the inputstream type we end up with via JarURLCOnnection$JarURLINputStream) there's no such concern - if I read correctly

noisesmith18:11:36

It also doesn't promise to report the full size, just the amount that it promises can be read without blocking. We know that the jar file we are reading from carries this information as a precise number, but the api doesn't expose that. The safer thing is either to O(n) copy the whole thing, or to read up to N bytes at a time, in a loop, and then calculate the total amount consumed at the end(?)

Crispin08:11:11

Nice. that one works! Byte Array the key huh.

Cameron08:11:56

ah damn, I should have thrown that out earlier ahahah I figured it would, I had this answer right after I wrote the first suggestion, and I figured the first suggestion wouldn't work as it was (as count is a member of BufferedInputStream I believe, but not a public one), but I wanted to test it on my end first as I don't like shouting out too many things blind

Cameron08:11:49

on my end, I have a workflow at times that allows me to be a bit lazy (I guess not just in programming) where instead of looking up something or thinking out the answer, I'll test a few answers that I know will allow me to narrow down the overall answer without having to think about it, but I feel self conscious when I use that method while helping someone because if I leave in the middle without solving the question, its going to look like I was just guessing ahahaha

carocad16:11:09

is there a best practice regarding dependencies for clojure projects ? I have a cross-platform project (js, jvm). For the Js side, the jvm dependencies would make no sense therefore I marked them as :provided , however this adds an extra burden to the end users of the jvm side which now have to add two dependencies instead of one to use the project 😕 . Any ideas/opinions on this topic ?

alexmiller16:11:04

Split the project?

carocad20:11:50

then the burden goes to me 😓. Ideally I am searching for a way to say “jvm dependencies” not for clojurescript 🙂

sogaiu16:11:31

am interested too 🙂

hugod16:11:53

When depending on a deps.edn project with :local-root, is there any way to get the dependency’s test paths and test dependencies (specified via the :test alias) added to the project?

alexmiller16:11:33

In short, no

👍 1
alexmiller16:11:55

There are a variety of ideas possible for this but nothing at the moment

p-himik18:11:03

I'm trying to generate a Java file and import it later in the same process, but for some reason it doesn't work:

(io/resource "hgs/platform/abc/antlr/ABCMusicLexer.java")
=> #object[.URL 0x6b11a979 "file:/.../hgs/platform/abc/antlr/ABCMusicLexer.java"]

(import '(hgs.platform.abc.antlr ABCMusicLexer))
Execution error (ClassNotFoundException) at .URLClassLoader/findClass (URLClassLoader.java:471).
hgs.platform.abc.antlr.ABCMusicLexer
Is it possible at all? Am I doing something wrong?

p-himik18:11:17

(`ABCMusicLexer` has been generated by ANTLR in the same process)

noisesmith18:11:51

1. you can't import a java file, only a class can be imported 2. the class can only be imported if the classloader can find it (usually you want it to be loadable via a classpath entry, not always an option if the file doesn't exist at startup)

noisesmith18:11:39

also the ' isn't needed by import (it doesn't hurt anything either, but it isn't needed)

noisesmith18:11:16

you could look at how clojure dynamically creates new classloaders that can find the classes it creates at runtime, I'm not sure what the best practice is for what you are doing, but what clojure does clearly works

p-himik18:11:44

Oh, duh... Now I wonder how #1 wasn't obvious to me from the get go. Thank you!

noisesmith18:11:41

ediited the thing about classpath - if you have a known destination dir (eg. how clojure can use target/ for files on disk) you could start up with that on classpath, and put classes you make there

👍 1
p-himik19:11:10

OK, seems to be a task that isn't worth it since it's a one-off one. If I need something like this for a more serious task, I will probably be able to use ztellman/virgil.

1
noisesmith19:11:03

oh yeah, I should have remembered virgil

borkdude19:11:47

It seems like there is a time between switching namespaces that symbols do not get qualified with the current ns anymore:

$ clj
Clojure 1.10.1
user=> (def x 1)
#'user/x
user=> (ns foo {:x x})
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: x in this context
foo=> (ns bar {:x user/x})
nil

borkdude19:11:37

(this is not a problem, just an observation)

noisesmith19:11:12

the whole ns form is evaluated in the fresh ns right?

hiredman19:11:37

It might be illustrative to try and capture the value of *ns* in your map

noisesmith19:11:05

(ins)user=> (create-ns 'foo)
#object[clojure.lang.Namespace 0x54d18072 "foo"]
(ins)user=> (intern 'foo 'bar 1)
#'foo/bar
(ins)user=> (ns foo {:bar bar})
nil
(ins)foo=> (meta (the-ns 'foo))
{:bar 1}

hiredman19:11:00

What you will find is the metadata is evaluated after the namespace has changed

hiredman19:11:01

How and when metadata us evaluated (or not) is not very consistent

borkdude19:11:46

@hiredman I tried that with *ns* but it got me an error:

$ clj
Clojure 1.10.1
user=> (ns bar {:current-ns *ns*})
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: *ns* in this context

borkdude19:11:35

again, not a problem, but just observing this behaviour. when linting code it becomes kind of important to know

hiredman19:11:36

Use the fully qualified name

borkdude19:11:06

aah:

Clojure 1.10.1
user=> (ns bar {:current-ns clojure.core/*ns*})
nil
bar=> (meta (the-ns 'bar)
)
{:current-ns #object[clojure.lang.Namespace 0xf72203 "bar"]}

hiredman19:11:27

The metadata is evaled after the ns has changed and before clojure.core is required

noisesmith19:11:00

added the meta check to my paste above showing what one would expect

borkdude19:11:12

I guess this behavior isn't well documented as it's not that important?

borkdude19:11:24

or if it's documented I'd like to see it

andy.fingerhut19:11:27

These kinds of details tend to be learned by experimentation and/or reading source code.

borkdude19:11:28

yeah, but it's not something that other Clojure implementation should adhere to if that's the way to discover it, that's my motivation to ask if this is documented

noisesmith19:11:33

what if your linter complained if symbols in ns metadata aren't fully qualified?

noisesmith19:11:02

seems like an acceptable ask, reduces complexity of knowing what you are reading

borkdude19:11:35

that seems like a good addition. right now it only complains about symbols it can't resolve, but it seems even clojure vars have to be fully qualified

p-himik20:11:37

Is it expected that eval doesn't see any imports in the current namespace?

java.lang.IllegalArgumentException: Unable to resolve classname: CommonTokenStream

Michael J Dorian20:11:29

I did a thing with eval and had to include the namespace in the string that was being evaluated. Not sure if that's what is happening or not

p-himik20:11:34

You mean, you had to fully qualify the used symbol?

p-himik20:11:49

I see, thanks!

hiredman21:11:56

Imports effect the state of the current namespace *ns*, which is then used to resolve classes, but while the value of *ns* is the declared namespace during compilation, it may be, and usually is, something entirely different when your code is run

sparkofreason21:11:17

I'm trying to write a user-defined function for KSQL in Clojure. KSQL wants you to give it an uberjar with an annotated java class. I've tried this with both gen-class and a hand-authored java file that interops to Clojure. In both cases, the uberjar gets loaded fine, and my class annotations are detected via reflection. But I get exceptions like "java.lang.ExceptionInInitializerError" and "java.lang.NoClassDefFoundError: Could not initialize class clojure.java.api.Clojure" as soon as anything to do with clojure is executed, like Clojure.var("clojure.core", "require") Feels like I must be missing something about how the uberjar needs to be built. My uberjar seems awfully small (7MB), and unpacking it, I don't see any evidence that Clojure is included in there anywhere.

p-himik21:11:53

I stumbled upon this issue today that looks similar: https://github.com/boot-clj/boot/issues/665 Not sure if that helps though.

noisesmith22:11:53

when you run lein uberjar it creates two jars, one contains standalone in the name by default, the other does not

noisesmith22:11:24

a common mistake is to use the wrong jar - the one that contains only your code and not your deps is an intermediate artifact that you shouldn't try to use directly

sparkofreason22:11:22

I'm using standalone. If I don't do anything with clojure, the UDF runs fine.

noisesmith22:11:10

OK, if you picked the right jar, and it doesn't contain clojure, I can't explain that.

noisesmith22:11:16

what is a UDF?

sparkofreason22:11:41

User Defined Function. It's the code being executed by KSQL.

noisesmith22:11:42

you should be able to run java -cp my-uber.jar clojure.main and get a repl

sparkofreason22:11:03

Everything works fine in the REPL.

noisesmith22:11:14

no, I'm not saying use a repl

noisesmith22:11:25

I'm saying use only java plus your uberjar to start a repl

sparkofreason22:11:52

Yes, that works. It only seems to be an issue when executed from the context of KSQL.

noisesmith22:11:50

could be KSQL does something with classloaders, that can prevent clojure from working since it also uses classloaders weirdly

sparkofreason22:11:25

That's my current guess. KSQL does some sort of classpath isolation so different UDF impls don't step on each other's deps.

noisesmith22:11:01

we are approaching the edge of what I understand here, but this thread might help https://groups.google.com/forum/#!topic/clojure/Aa04E9aJRog

sparkofreason22:11:07

Thanks, this looks promising.

p-himik21:11:30

Oh, huh. Is there a a way to print *ns* during compilation just so that I compare it with the one at runtime and educate myself a bit? Also, is there something decent I could read about all this?

noisesmith22:11:35

*ns* is a dynamic var. Usage of vars is not cached - the value is looked up again each time you access them. The value of a dynamic var is controlled by the callers further up the stack.