This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-10-03
Channels
- # babashka (5)
- # beginners (34)
- # biff (3)
- # calva (29)
- # cherry (11)
- # cider (7)
- # clojure (148)
- # clojure-brasil (1)
- # clojure-europe (16)
- # clojure-nl (1)
- # clojure-norway (6)
- # clojure-uk (6)
- # clojuredesign-podcast (8)
- # clojurescript (49)
- # cursive (1)
- # datalevin (7)
- # fulcro (1)
- # honeysql (1)
- # jobs (1)
- # matrix (7)
- # off-topic (13)
- # re-frame (12)
- # react (21)
- # reagent (42)
- # releases (6)
- # remote-jobs (2)
- # shadow-cljs (9)
- # solo-full-stack (5)
- # sql (7)
- # squint (9)
- # vim (2)
- # xtdb (11)
- # yamlscript (5)
What does this object represent? #object[clojure.lang.AFunction$1 0x28757197 "clojure.lang.AFunction$1@28757197"]
Functions have reference identity but implement value metadata interfaces (a mistake, and why you should avoid using metadata on function objects)
Is this perhaps a special case that might warrant a mention in this page somewhere? https://clojure.org/guides/equality
Maybe it does already and I have forgotten...
Why do you think metadata on functions is bad, @U0NCTKEV8 ?
The theory answer is they implement the wrong metadata interface. A contract of metadata is adding it doesn't change equality, but for functions it does
The practical answer is the way adding metadata is implemented effectively adds a layer of rest args+apply around your function which is bad for performance
Interesting that it's implemented with RestFn
. Is there a reason AFunction can't implement withMeta
the same way other objects implement it?
(https://github.com/clojure/clojure/blob/6975553804b0f8da9e196e6fb97838ea4e153564/src/jvm/clojure/lang/AFunction.java for those who want to read along)
Only my personal guess, with no inside knowledge available on choices made by those who wrote that code: As it is implemented there, there is no memory cost to add metadata to an instance of an object implementing AFunction
, unless you actually add metadata to it. If you expect the common case not to have metadata on such an object, that saves memory in the typical case.
Another Java detail: I believe abstract classes cannot have per-instance member fields, so there is no place to put such a field in the class AFunction
those two things are very much in conflict given the whole point of metadata is it is extra data about some object that doesn't change it for the purposes of equality
The issue being that if you made updating metadata to happen in place, that breaks assumptions about with-meta
not having side effects?
the point of metadata is it allows you to annotate some data, without changing that data
hiredman, what would you think of a different implementation that overrode Java equals
for Clojure function-like objects that allow metadata to be "attached", that compared equality on the underlying function reference, ignoring the metadata. Would that be "cleaner" in your view?
if you track down old videos of rich talking about clojure the kind of example he gives is things like: oh you get map {:a 1} from disk and map {:a 1} from the database, and you can tag them with where you got them from, but they are still equal
@U0CMVHBL2 clojure has a metadata api for references, it is what ref, atom, and agent use, functions should use that
I thought atoms couldn’t have metadata. What’s their metadata api?
$ clj
Clojure 1.11.1
user=> (def a1 (atom 5))
#'user/a1
user=> (meta a1)
nil
user=> (def a2 (with-meta a1 {:b 2}))
Execution error (ClassCastException) at clojure.main/main (main.java:40).
class clojure.lang.Atom cannot be cast to class clojure.lang.IObj (clojure.lang.Atom and clojure.lang.IObj are in unnamed module of loader 'app')
Oh, alter-meta!
. That makes sense. Yeah, using that with functions makes a lot more sense
Ah, I did not realize alter-meta!
was supported for atom, ref, etc. Never tried to do that before.
the weirdness if you go that route is some IFn's would do reference metadata (fns, vars already do this) and some would do value metadata (symbols, collections)
And some not at all (keywords) lol
but it seems way better than value metadata on function objects (although the current impl of value metadata on function objects could maybe get patched up to fix the performance pitfalls)
it might actually already be the case that if a fn object has compile time metadata attached it will get a metadata implementation that doesn't rely on restfn wrapping, not sure
user=> (with-meta ^{} (fn []) {})
#object[clojure.lang.AFunction$1 0x17ca8b92 "clojure.lang.AFunction$1@17ca8b92"]
user=> (with-meta ^{:foo 1} (fn []) {})
#object[user$eval160$fn__161 0x1ec7d8b3 "user$eval160$fn__161@1ec7d8b3"]
user=>
My Java knowledge is next to nil, but I am trying to interop with a lib, which involves JNI stuff, however that's not relevant, since the native library loads fine. However, when trying to instantiate a Java class, I am seeing the no matching ctor
error. Specifically, the VCFReader
class found here: https://github.com/TileDB-Inc/TileDB-VCF/blob/main/apis/java/src/main/java/io/tiledb/libvcfnative/VCFReader.java. Am I missing something obvious?
Tried different things, but, mainly trying to get to that VCFReader method on the class.
First I did a naïve (VCFReader.)
, obviously not correct. Gave the ctor error. Then I thought to get to that VCFReader method on the class:
(:import (io.tiledb.libvcfnative VCFReader))
(.. VCFReader VCFReader "ds-test-1" nil)
=> java.lang.IllegalArgumentException: Can't define method not in interfaces: create
That's a new one.The signature of the constructor:
VCFReader(String uri, String[] samples, Optional<URI> samplesURI, Optional<String> config)
So you must pass exactly 4 arguments there, and they all must be of specified types.So, something like (VCFReader. the-uri (into-array String []) (Optional/empty) (Optional/empty))
. No clue whether it'll work in terms of how reasonable the values are, but should work from the perspective of types.
com.google.cloud
libraries-bom
26.22.0
pom
import
com.google.cloud
google-cloud-pubsub
Any way to do it in deps.edn
?
the bom
set versions for dependencies. The dependencies itself has not specified version.
https://cloud.google.com/java/docs/bomNot currently supported: https://clojure.atlassian.net/browse/TDEPS-202 BTW there's #C6QH853H8
Did you get into an issue when in REPL pubsub work, but after compile it stuck forever on
(.build (.setData (PubsubMessage/newBuilder) data))
data
is a little different object
com.google.protobuf.LiteralByteString
vs com.google.protobuf.ByteString$LiteralByteString
I am trying to figure out how REPL is different from jar. Each time when I have such issues I don’t remember all this caveats.I don't use protobuf so I wouldn't know. My immediate suspect is different classpaths.
Hi, spec noob here, i have a function which runs fine on it's own, but when instrumented hangs - i assumed instrumenting a function used just my args, but I'm beginning to wonder if test.check is being called somewhere and is generating data that can't be resolved is that possible? This is on the babashka repl.
an easy way to check this is to poke it to prove/disprove if this is the case (I assume it is not the case). But can you start your repl without the generator lib? If there’s no test.check and you still have the problem, then that’s not it. Alternatively, if you can fmap the generator to have a side effect ( like a println or something) you could have feedback if that’s the case.
will let you know, need to setup up my deps.edn with all the babashka stuff i'm using - I've been babashka only so far 🙂
@U05NY1NK44W you can do this using bb print-deps > deps.edn
@U05NY1NK44W if you haven't installed clj
yet, you can run clojure with:
rlwrap bb clojure ...
@U04V15CAJ wrinkle? so i use babashka.classpath and it's not included in the bb print-deps output
this will take a bit of time, I'll try your suggestions and holler when I get it straightened out - thanks for pointers
So, I finally answered the initial question thanks to @U11BV7MTK suggestion, instrumentation indeed looks for generators, and fyi @U04V15CAJ, the same thing happens on jvm. (with-machines {:select [:role :db]} (fn [a m v] (str (:ip m) "-" v)) ) Execution error (FileNotFoundException) at concierto.core/eval4551 (REPL:1). Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
are you sure the error is the same in bb and JVM? in bb:
user=> (require '[clojure.test.check.generators])
nil
i just removed the generator deps in the jvm to determine if instrumentation somehow uses test.check, i was getting hang on jvm too, i didn't check on bb as i don't know how to remove a built in dep - however, my initial doubt was answered (+ project now runs on jvm which I hadn't tried before) so all in all a plus 🙂 thanks for your help , and what a great tool bb is.
how are you instrumenting the functions? And can you share what your function spec looks like? I am surprised that it’s invoking the generator when you are just calling the function
Has anyone tried a) WASM runtimes or b) QuickJS wrappers for the JVM? If you have a library you'd recommend I'd love to see it! Context: I want to give users a special-purpose language they can use to extend our system. To do this, I'm looking for ways to run sandboxed code inside the JVM. The best contender I've found so far is CEL-Java. I wanted to have a deeper look with wasm and quickjs though
I think you might want to re-examine your requirements, try and bound the problem some more, cel-java and wasm are very different things. I could in theory imagine them both existing some common problem, but I would be surprised to them actually doing that.
they are both sort of sandboxed execution environments, but cel-java isn't even turning complete, where wasm is very much a general purpose execution machine
For sure hiredman! The context is about introducing a permission system for a DBaaS. CEL can work, but I'd have do effectively fork it to make it more permissive. I would at the very least want to introduce variables and functions. This would be similar to Firestore's permission language With WASM, I would have to tighten it down. Have you used WASM / quickjs alongside the JVM?
I have not. I've done a little with sandboxing execution of clojure (which used different techniques but eventually just depended on running user code in a different process)
Js and wasm seem like maybe problematic if you want a strong sandbox, people have been breaking out of the sandboxes those use for a long time now
I think a very high level language, like prolog would be even easier to sandbox, and there is some prior art using logic programming for acls, but might end up very esoteric
I also used lua on the jvm in the past (on another job), it works quite well. It’s been a while but it was using luaj iirc
Google also open sourced a java cel lib recently. Last time I checked the parser was missing but was supposed to be open-sourced in 2023, might be worth checking
Thanks team! @U050SC7SV -- indeed I checked google's cel-java library -- it's nifty! Will look into luaj too. sends high five
Hmmm... I don't know if this is the best behavior...
(ns proto)
=> nil
(defprotocol Foo
(foo [this]))
=> Foo
(ns impl
(:require [proto])
(:import (proto Foo)))
=> nil
(extend-protocol Foo
Number
(foo [this] 1))
Execution error (IllegalArgumentException) at impl/eval4358 (form-init7956476065087726114.clj:1).
interface proto.Foo is not a protocol
(extend-protocol proto/Foo
String
(foo [this] "foo"))
=> nil
the correct way to bring the protocol Foo into scope without needing to namespace qualify it is using :require with :only, just like other clojure names
There's another thing regarding protocols... I mean, it makes sense. When compiling, it expects a class, but protocol isn't really a class, it got resolved to a Var
instead... Would it make sense to create some patch around this? I mean, there isn't anything to fix, really... It might just help with some debugging around this.
(meta ^Inst ())
=>
{:tag {:on clojure.core.Inst,
:on-interface clojure.core.Inst,
:sigs {:inst-ms* {:tag nil, :name inst-ms*, :arglists ([inst]), :doc nil}},
:var #'clojure.core/Inst,
:method-map {:inst-ms* :inst-ms*},
:method-builders {#'clojure.core/inst-ms* #object[clojure.core$fn__8490
0x728a9b11
"clojure.core$fn__8490@728a9b11"]},
:impls {java.util.Date {:inst-ms* #object[clojure.core$fn__8501 0x6a08ed66 "clojure.core$fn__8501@6a08ed66"]},
java.time.Instant {:inst-ms* #object[clojure.core$fn__8505 0x70e80c2e "clojure.core$fn__8505@70e80c2e"]}}},
:line 1,
:column 7}
(try (macroexpand '(defprotocol Foo
(^Inst foo [this])))
(catch Exception e
(.printStackTrace e)))
Syntax error macroexpanding defprotocol at (/private/var/folders/xt/5kdykg_n4jzcr016b4f0j5hw0000gn/T/form-init11234149643092752313.clj:1:1).
at clojure.lang.Compiler.macroexpand1(Compiler.java:7027)
at clojure.core$macroexpand_1.invokeStatic(core.clj:4025)
at clojure.core$macroexpand.invokeStatic(core.clj:4027)
at clojure.core$macroexpand.invoke(core.clj:4027)
at cljhdl.core$eval3698.invokeStatic(form-init11234149643092752313.clj:1)
at cljhdl.core$eval3698.invoke(form-init11234149643092752313.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:7194)
at clojure.lang.Compiler.eval(Compiler.java:7149)
at clojure.core$eval.invokeStatic(core.clj:3215)
at clojure.core$eval.invoke(core.clj:3211)
at nrepl.middleware.interruptible_eval$evaluate$fn__1429$fn__1430.invoke(interruptible_eval.clj:87)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at nrepl.middleware.interruptible_eval$evaluate$fn__1429.invoke(interruptible_eval.clj:87)
at clojure.main$repl$read_eval_print__9206$fn__9209.invoke(main.clj:437)
at clojure.main$repl$read_eval_print__9206.invoke(main.clj:437)
at clojure.main$repl$fn__9215.invoke(main.clj:458)
at clojure.main$repl.invokeStatic(main.clj:458)
at clojure.main$repl.doInvoke(main.clj:368)
at clojure.lang.RestFn.invoke(RestFn.java:1523)
at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:84)
at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:56)
at nrepl.middleware.interruptible_eval$interruptible_eval$fn__1462$fn__1466.invoke(interruptible_eval.clj:152)
at clojure.lang.AFn.run(AFn.java:22)
at nrepl.middleware.session$session_exec$main_loop__1532$fn__1536.invoke(session.clj:218)
at nrepl.middleware.session$session_exec$main_loop__1532.invoke(session.clj:217)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Thread.java:1589)
Caused by: java.lang.IllegalArgumentException: No matching field found: getName for class clojure.lang.Var
at clojure.lang.Reflector.getInstanceField(Reflector.java:397)
at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:440)
at clojure.core$emit_protocol$fn__8088$tag_to_class__8092.invoke(core_deftype.clj:661)
at clojure.lang.AFn.applyToHelper(AFn.java:154)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:669)
at clojure.core$update_in$up__6922.invoke(core.clj:6220)
at clojure.core$update_in.invokeStatic(core.clj:6221)
at clojure.core$update_in.doInvoke(core.clj:6207)
at clojure.lang.RestFn.invoke(RestFn.java:445)
at clojure.core$emit_protocol$fn__8088.invoke(core_deftype.clj:663)
at clojure.core$reduce1.invokeStatic(core.clj:946)
at clojure.core$emit_protocol.invokeStatic(core_deftype.clj:653)
at clojure.core$defprotocol.invokeStatic(core_deftype.clj:713)
at clojure.core$defprotocol.doInvoke(core_deftype.clj:713)
at clojure.lang.RestFn.applyTo(RestFn.java:146)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.lang.Compiler.macroexpand1(Compiler.java:7010)
... 31 more
=> nil
It works for the return type defined via definterface
, which actually is a Java interface.
Or maybe it actually is a Java interface? The generated bytecode is a Java interface for both variations...
Even if it's possible, you still shouldn't use it in a tag because implementing a protocol does not mean implementing the Java interface.
I am trying to use the BOMInputStream class using deps.edn
. When I used leiningen
, I would just add
(:import (org.apache.commons.io.input BOMInputStream))
to my namespace declaration but this doesn't seem to work in a project where I am using deps.edn
I found a https://github.com/clojure-interop/apache-commons-io interop repository and I added it to my deps.edn
file but I have no idea how to include it my namespace.
Below my deps.edn
file
{:deps
{
org.clojure/data.csv {:mvn/version "1.0.1"}
clojure-interop/apache-commons-io {:mvn/version "1.0.0"}
}
}
and my core.clj
file:
(ns core
(:import (org.apache.commons.io.input BOMInputStream)))
I looked at the .`/.m2/repository/` directory and do see a clojure-interop/apache-commons
directory that eventually leads to a jar
file where I can see the BOMInputStream
class.
I am a newbie using deps.edn
--any assistance would be greatly appreciated.
PS The task is to filter out the BOM in a CSV file--I am using clojure.data.csv
and the instructions in the README
work for me with leiningen
how are you using deps.edn is just a file with a map in it, you have to use it with some tool (typically the clj command line tool, but generally something tools.deps based)
The class is not found is the error
I see that the deps got loaded because they are in the .m2 directory
the .m2/ is just a cache where various tools stick stuff, that doesn't meaning anything about a given jvm process
2. Unhandled clojure.lang.Compiler$CompilerException Error compiling src/core.clj at (1:1) #:clojure.error{:phase :execution, :line 1, :column 1, :source "/home/afm/data/repos/blip/src/core.clj"} Compiler.java: 7665 clojure.lang.Compiler/load REPL: 1 user/eval7459 REPL: 1 user/eval7459 Compiler.java: 7194 clojure.lang.Compiler/eval Compiler.java: 7149 clojure.lang.Compiler/eval core.clj: 3215 clojure.core/eval core.clj: 3211 clojure.core/eval interruptible_eval.clj: 87 nrepl.middleware.interruptible-eval/evaluate/fn/fn AFn.java: 152 clojure.lang.AFn/applyToHelper AFn.java: 144 clojure.lang.AFn/applyTo core.clj: 667 clojure.core/apply core.clj: 1990 clojure.core/with-bindings* core.clj: 1990 clojure.core/with-bindings* RestFn.java: 425 clojure.lang.RestFn/invoke interruptible_eval.clj: 87 nrepl.middleware.interruptible-eval/evaluate/fn main.clj: 437 clojure.main/repl/read-eval-print/fn main.clj: 437 clojure.main/repl/read-eval-print main.clj: 458 clojure.main/repl/fn main.clj: 458 clojure.main/repl main.clj: 368 clojure.main/repl RestFn.java: 1523 clojure.lang.RestFn/invoke interruptible_eval.clj: 84 nrepl.middleware.interruptible-eval/evaluate interruptible_eval.clj: 56 nrepl.middleware.interruptible-eval/evaluate interruptible_eval.clj: 152 nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn AFn.java: 22 clojure.lang.AFn/run session.clj: 218 nrepl.middleware.session/session-exec/main-loop/fn session.clj: 217 nrepl.middleware.session/session-exec/main-loop AFn.java: 22 clojure.lang.AFn/run Thread.java: 829 java.lang.Thread/run 1. Caused by java.lang.ClassNotFoundException http://org.apache.commons.io.input.BOMInputStream
it looks like you are using an nrepl repl but your deps.edn doesn't include nrepl in it, so it is incomplete, or you are at the wrong repl
I am using emacs with cider
Everything else in my project is working just peachy
I am using tech.ml.dataset to do a bunch of manipulations and that works great
and core.clj is in /home/afm/dat/repos/src
Yes, I reduced the issue to its minimal config
the tech.ml.dataset doesn't handle BOM
that's why I had to add this new piece
clojure-interop/apache-commons-io
doesn't actually depend on Apache Commons so you need to add that to deps.edn
as well.
yeah, I just was going to say, no way clojure-interop/apache-commons contains a jar that contains the http://org.apache.commons.io.input.BOMInputStream class
@U04V70XH6 will try that
@U04V70XH6 I do see that class in the .m2 repository
As hiredman says, what's in m2 is irrelevant.
> clj -Sdeps '{:deps {clojure-interop/apache-commons-io {:mvn/version "1.0.0"}}}' -Stree
org.clojure/clojure 1.12.0-alpha4
. org.clojure/spec.alpha 0.3.218
. org.clojure/core.specs.alpha 0.2.62
clojure-interop/apache-commons-io 1.0.0
☝️:skin-tone-2: No dependency on Apache Commons itself. You need to add it to deps.edn
.@U0NCTKEV8 so what's the declaration that I need to add to my deps.edn?
OK thanks
likely you want to just ditch whatever clojure-interop/* is and find the apache commons io library and add that to your deps
or it might have just happened that some other dependency also happened to pull in apache commons io, since it is a commonly used library
That clojure-interop
is 4 years old so it may not work with more recent Apache Commons versions but the latest is commons-io/commons-io {:mvn/version "2.14.0"}
> clj -Sdeps '{:deps {commons-io/commons-io {:mvn/version "2.14.0"}}}'
Downloading: commons-io/commons-io/2.14.0/commons-io-2.14.0.pom from central
Downloading: org/apache/commons/commons-parent/62/commons-parent-62.pom from central
Downloading: org/apache/apache/30/apache-30.pom from central
Downloading: org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom from central
Downloading: commons-io/commons-io/2.14.0/commons-io-2.14.0.jar from central
Clojure 1.12.0-alpha4
user=> (import (org.apache.commons.io.input BOMInputStream))
org.apache.commons.io.input.BOMInputStream
user=>
Thanks--I really appreciate it!
Most wrapper libraries aren't worth using and the automatically generated ones are typically not idiomatic Clojure.
@U04V70XH6 I really appreciate your help--I have been dragging my feet on moving to deps.edn. It's really hard to start all over again with a new build system but today is the day!
I'm always happy to help "convert" folks to the official tooling 🙂
As you can see from the snippets I'm posting here, it's very straightforward to explore dependencies and libraries without needing any sort of project in place -- I'm just running those commands in a very minimal folder.
@U04V70XH6 Yes! I have known for a while that I wanted to switch because of the exploratory advantages of the new build system. Today is day 1--I'll keep on reading so that I will become as proficient as I was leiningen. It always pays off to know what one is doing with these things!
@U04V70XH6 again, thanks!
@U04V70XH6 one more question: how would I have known about commons-io? I think this was the missing piece....
Search for apache commons io
-- the library that Clojure wrapper refers to.
OK--thanks so much!
@U74KUQKFY A bit late, but I use this library for stripping out BOM for CSV imports
, e.g.,
(with-open [reader (bom/bom-reader (:tempfile csv))]
(process-csv! (csv/read-csv reader)))
Thanks! I never thought of using a library because this is such a simple use case but that definitely solves the problem!