Fork me on GitHub
#clojure
<
2019-12-08
>
Risetto00:12:14

How can I call a function from inside a map?

{:a #(println %)}

Risetto00:12:27

How can I call it with an argument passed to it?

dpsutton00:12:27

((:a your-map) arg)

Cameron00:12:35

You mean like this?

(def thingy {:a #(println %)})
((:a thingy) "blah")

Cameron00:12:37

yea that ahahah

Cameron00:12:49

beat me to it

Risetto00:12:04

Thanks! 😄

yuhan08:12:57

Why does get on non-associative objects return nil instead of throwing an error? I've spent lots of time hunting down bugs that turned out to be (get some-vector n) being passed a lazy sequence instead of a vector

andy.fingerhut18:12:27

You might want to try http://ask.clojure.org for questions like this, if you do not get an answer here. The record of the answer should remain longer there, and it is more suited for long-form answers there.

👍 4
yuhan09:12:06

(I understand nil-punning with (get nil ...) => nil, but shouldn't passing numbers etc be an illegal argument)

bfabry23:12:14

the answer to many, many questions like this where clojure's garbage-in-garbage-out comes into play is "you would pay a performance penalty for every single get call. it's a dynamic language". another answer is that maybe someday spec will check the types of these parameters at test/dev time

4
ido10:12:02

guys i need your help! In a lib (https://github.com/AppsFlyer/cloffeine) i have:

(ns cloffeine.cache
  (:refer-clojure :exclude [get])
  (:require [cloffeine.common :as common])
  (:import [com.github.benmanes.caffeine.cache Cache]))

(defn make-cache
  (^Cache []
   (make-cache {}))
  (^Cache [settings]
   (let [bldr (common/make-builder settings)]
     (.build bldr))))
but than, in the calling code:
(let [c (cache/make-cache)] (.estimatedSize c))  ;; this works, but
(def c (cache/make-cache))
;;; and later
(.estimatedSize c) ;; fails! -> Can't call public method of non-public class: public default long com.github.benmanes.caffeine.cache.LocalManualCache.estimatedSize()
ot looks like let behaves different from def regarding types. note that Cache is a java interface while .build returns a specific implementation.

ido10:12:12

this can only be resolved by a type hint near the def , which is not required inside the let anyone knows why?

dominicm14:12:55

Unless I'm misunderstanding, this looks right. Clojure thinks c is a Cache, but it's not, it's something else. You probably need to type hint for what you're actually returning. Type hinting this way means that in the future you can change the exact return to someething else, without it breaking. You also end up doing reflection otherwise, to find the estimatedSize field on something else. I imagine let works because it hasn't picked up the type hint, but not sure.

ido08:12:57

but c is a Cache. it is a specific implementation of this interface. also, make-cache is known to return one….

dominicm08:12:51

Exactly. It's a cache. You can't call estimatedSize on a cache

ido04:12:44

oh you should be able to… it is part pf the Cache interface

dominicm08:12:46

That's strange

dominicm08:12:01

Is there meta on the c var?

borkdude14:12:50

I want to search a user-defined / runtime classpath for a file. So something like (find-file "src:/Users/borkdude/.m2/repository/foo/bar/baz.jar" "foo/bar/baz/core.clj"). What do I use for this?

borkdude14:12:29

this has to be separate from the classpath of the JVM that is running this

borkdude14:12:54

This seems to work:

user=> (def cl (.URLClassLoader. (into-array .URL [(.URL. "file:///Users/borkdude/.m2/repository/com/cognitect/transit-java/0.8.337/transit-java-0.8.337.jar")])))
#'user/cl
user=> (.findResource cl "com/cognitect/transit/Reader.class")
#object[.URL 0x3e30646a "jar:file:/Users/borkdude/.m2/repository/com/cognitect/transit-java/0.8.337/transit-java-0.8.337.jar!/com/cognitect/transit/Reader.class"]

sogaiu14:12:56

interesting that there is also a .findResources -- somehow you can get more than one thing i guess

dominicm15:12:48

You can get multiple

dominicm15:12:59

This is how data readers work

sogaiu14:12:20

or just different ways to refer to the same thing?

lilactown16:12:31

does it make sense that this would make my REPL hang?

(let [c (a/chan)]
  (a/>!! c "1")
  (a/<!! c))

lilactown16:12:52

I guess buffer size is required?

lilactown16:12:01

or else it blocks until something takes

helios17:12:28

something weird i found:

on my laptop (java 8):
(Math/pow 9 17)
=> 1.667718169966657E16

on a friend laptop (java 11):
(Math/pow 9 17)
1.6677181699666568E16
wtf?

andy.fingerhut20:12:13

I do not know why there is a difference between the return value on JDK 8 vs. 11 on those two computers, but in general you are living in a fantasy land if you expect exact answers using floating point numbers: https://clojure.org/guides/equality#_floating_point_numbers_are_usually_approximations

andy.fingerhut20:12:50

They are useful for speed of calculation, with some kinds of error bounds on the results, when used appropriately.

helios07:12:04

Yes of course, that is the whole point of float and why you shouldn't use them for precise calculations. Here my surprise was that I wasn't aware of a change in precision between java8 and java11. The same exact code is returning two different values, not that it wasn't correct. If you want to get the precise result you clearly need to use something else like a BigInteger and elevate to power with that.

dpsutton17:12:59

The display is 15 digits on java 8 and 16 digits of precision on java 11. Seems the same number in both cases?

dpsutton17:12:01

.xx6657 vs .xx66568 the 68 is collapsed to 7 without the extra display

helios17:12:36

@dpsutton it's not just display:

java8: (long (Math/pow 9 17) => 16677181699666570
java11: => 16677181699666568

helios17:12:30

Math/pow makes into a double, so I guess it loses precision

littleli17:12:00

(-> 9 (BigInteger/valueOf) (.pow 17))
16677181699666569

littleli17:12:09

I like how both are wrong 😄

helios17:12:38

yeah, your result is the correct one according to wolframalpha as one would expect, with BigIntegers

p-himik17:12:01

Is there any way to make case accept Java fields that are public static final int?

littleli17:12:24

Can you please be more specific?

p-himik17:12:03

case works with compile-time literals. If you pass a reference to a public static final int Java class field (e.g. Table/HEADER = 0), you will get a case that tries to match its argument with the symbol Table/HEADER, not with its value 0. It seems to me that since such values are available during the compile time, maybe there's a convenient method to create a kind of case that works with values of such fields and not with the symbols.

lilactown17:12:10

it would be difficult to discern based on context between matching a symbol and a class field

p-himik17:12:05

I can see that. But it should be possible to create a variant of case designed specifically for such fields, right?

littleli17:12:13

I'm afraid something like condp is your friend here

lilactown18:12:24

You could maybe macro it so that you emit a case that expands to the values of whatever symbols passed to it

lilactown18:12:41

But it would not work for local bindings

lilactown18:12:16

Or anything not already available at the time and location of the macros usage

p-himik18:12:05

Yeah, this works for simple cases:

(defmacro static-case [e & clauses]
  `(case ~e
     ~@(mapcat (fn [[test then]]
                 [(eval test) then])
               (partition 2 clauses))))
@U4YGF4NGM What do you mean? Clojure macros are not hygienic, so the above code snippet works just fine if I use it in let where some bindings are also used in some thens.

tyler18:12:16

What’s the use case for the value argument of the nav protocol for datafy? The other arguments all make sense to me but I’m struggling to reason about that argument.

ro619:12:00

Has anyone played around with replikativ/konserve and been able to customize the types it serializes? I'm having trouble writing a map containing a java.time.Instant to the Postgres-backed store. (@whilo)

seancorfield19:12:45

@tyler The default implementation of nav is to simply return the value argument. Custom implementations of nav may navigate into the data coll via the key k or may additionally use v to compute an updated result.

seancorfield19:12:45

For example, in next.jdbc, nav is implemented to either navigate via the key k if it looks like a foreign key column or to return the value v unchanged if it's a normal column. In the navigation case, coll is ignored because nav depends only on k and the closed over database information (`coll` is also ignored in the non-navigation case).

tyler21:12:00

Interesting. So if I understand correctly the val argument is available to add additional behavior if necessary but most use cases can fall back to the default implementation? Thanks for the reply.

tyler21:12:37

Said another way, extension is to add an additional transformation when relevant?

seancorfield21:12:02

Yes. nav can ignore or use any of its three arguments to provide an appropriate transformation/computation that represents navigation from data to a new "original" result.

seancorfield21:12:55

The cycle is always: original -> datafy -> pure-data -> nav -> new-original -> datafy -> new-pure-data ...

seancorfield21:12:28

But datafy and nav can both be "no-op", just returning the input value.

Artur Aralin20:12:19

Does next.jdbc have option to remove table name from column name or I need to remove it “manually”? actual

[
{ :table/id 1 }
{ :table/id 2 }
...
]
expected
[
{ :id 1 }
{ :id 2 }
...
]

seancorfield20:12:57

@artur.aralin97 Have you read over the documentation for next.jdbc? It explains why it table-qualifies column names and how to get different results. I'm happy to answer any questions in #sql (I'm more likely to see them there than in #beginners because this channel is fairly high traffic).

seancorfield21:12:23

You should probably at least work your way through the Getting Started guide https://cljdoc.org/d/seancorfield/next.jdbc/1.0.11/doc/getting-started -- it specifically addresses qualified/unqualified column names. I would also recommend that you get used to qualified keywords and start using them in your programs. Clojure's Spec is all based on qualified keywords and they exist in the language for important reasons.

dominicm21:12:58

Has anyone played around with Codeq + Cluster analysis at all? I'm aware of Codemaat, but I want something at the function level.

jumar12:12:13

CodeScene is powerful if you are willing to pay and don't need open source tool

dominicm14:12:30

I'm not, I just want to fiddle around with some speculations I had about projects I've historically worked on.

dominicm14:12:46

CodeScene has code analysis support though?

jumar15:12:11

You can try it (the limited version) on an open source github repo here: https://codescene.io/ There are some good articles about its possibilities here https://empear.com/blog/ (in particular https://empear.com/blog/code-analysis-tool/)

jumar15:12:08

There's also a showcase project (<https://codescene.io/showcase>) using clojure core: <https://codescene.io/projects/175/jobs/15346/results>

Artur Aralin21:12:51

Ops… I need to be more attentive 👀

seancorfield22:12:51

No problem. I'm always happy to get feedback on how to improve the docs to make stuff like this more obvious...

Artur Aralin10:12:14

I think my inattention because of English is not native for me. Therefore it makes some problems for me 🙂

restenb21:12:14

anything to know about dealing with Iterablein Clojure? like if I want to loop over a Iterable<SomeType>, is there a preferred way?

seancorfield21:12:28

@restenb You probably want iterator-seq

restenb22:12:12

ordinarily yes, probably

seancorfield22:12:12

But read the caveats about mutable objects etc http://clojuredocs.org/clojure.core/iterator-seq

restenb22:12:44

turns out this wasn't a java.util.Iterable, it's some Microsoft class LazySegmentedIterablefrom their Azure APIs

seancorfield22:12:08

Sounds delightful 🙂

seancorfield22:12:27

@restenb Looks like that has a .iterator method that returns a regular Iterable so (iterator-seq (.iterator lsi-object)) should work...

restenb22:12:34

it's implementing Iterablebut doesn't appear possible to juse seqit either

restenb22:12:46

even that doesn't work straight up; java.util.NoSuchElementExceptionis thrown

restenb22:12:55

hmm, that's a bit strange. maybe it just means the iterator is empty?

seancorfield22:12:31

I would expect (seq (iterator-seq (.iterator lsi-object))) to return nil if it was empty...?

seancorfield22:12:25

i.e., normal Clojure sequence operators should just work. Can you share some code of how you're trying to process it?

restenb22:12:38

might be Microsoft's fault for all I know 😛 ... originating from com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:113)

restenb22:12:14

(let [itr (.iterator (.listFilesAndDirectories directory))]
  (iterator-seq itr))

andy.fingerhut22:12:50

As far as I know from the Java docs of the Iterator class, hasNext should always return a boolean, never throw an exception: https://docs.oracle.com/javase/8/docs/api/java/util/Iterator.html#hasNext--

andy.fingerhut22:12:12

The next method is documented to throw such an exception, if there are no more elements.

seancorfield22:12:29

Hmm, according to the Azure docs, .listFilesAndDirectories should return Iterable anyway, so plain old seq ought to work...

seancorfield22:12:47

seq on Iterable is lazy -- perhaps you're realizing the sequence outside of the scope in which the original LazySegmentedIterator is valid @restenb? That might explain .hasNext blowing up...

restenb22:12:55

the above Java code works just fine

restenb22:12:16

so what would the best clojure equivalent be

seancorfield22:12:56

(doseq [file (.listFilesAndDirectories (.getDirectoryReference rootDirectory "/test"))]
  (println (.. file getUri getPath)))

andy.fingerhut22:12:58

In your code snippet that had a call to iterator-seq inside of a let , if you instead did (do all (iterator-see ...)) inside of the let, it would force all elements of the sequence to be realized inside of the let , rather than returning an unrealized lazy sequence.

seancorfield22:12:21

That's probably where I would start, off the top of my head.

andy.fingerhut22:12:45

My suggestion would be bad if you did not want to allocate memory for all of that at one time. Sean's suggested code never needs to allocate all of those items in memory at once.

seancorfield22:12:30

(my code assumes rootDirectory is set up earlier -- however it is done in the Java code)

restenb22:12:02

still NoSuchElementException

restenb22:12:11

but again, somehow this iterator is coming back as com.microsoft.azure.storage.core.LazySegmentedIterable, not "regular" Java Iterable

seancorfield22:12:49

Call ancestors on that type, just to make sure it really does implement Iterable

andy.fingerhut22:12:14

Looking at this implementation code, that might be what you are running against, or maybe not, it appears they implement hasNext in a way that can throw an exception: <https://github.com/Azure/azure-storage-android/blob/master/microsoft-azure-storage/src/com/microsoft/azure/storage/core/LazySegmentedIterator.java#L104-L116>

seancorfield22:12:28

@restenb So my exact code fails with that exception too? Then it does sound like you're running into a StorageException per the code Andy just linked to 😞

restenb22:12:53

yeah it must be, somehow

restenb22:12:10

why'd they just recast it as NoSuchElementExceptionthough

andy.fingerhut23:12:42

It looks like they wrapped it in a NoSuchElementException , leaving the original exception inside. Not sure why, unless perhaps they thought something like "Well, next() is supposed to throw such an exception when there are no more elements, so maybe we should do the same for hasNext(), too"

restenb23:12:35

well that reasoning is incorrect in my book

andy.fingerhut23:12:49

You'll have to take that up with the library authors 🙂

restenb23:12:07

hasNext()should simply return the true/falseor throw the original exception, I really don't understand this choice

restenb23:12:18

as a user of the API that's what I'd expect at least

andy.fingerhut23:12:23

To verify you are actually hitting that case, you should see whether the exception you has has some other kind of storage exception class inside of it.

jumpnbrownweasel23:12:22

If StorageException is a checked exception then it can't be thrown by hasNext and would have to be wrapped in something. But it would have been better for them to have wrapped it in a different RuntimeException, not NoSuchElementException.

restenb23:12:24

still all this doesn't answer why the java code actually works ...

restenb23:12:06

basically I have to set up a chain of objects to get to this point, the actual storage account, the file share, the root directory and then the sub directory for which I'm requesting the files

restenb23:12:46

so maybe the answer is in there somewhere and the sub directory object doesn't have a valid reference to it's "parent"

restenb23:12:35

should be straightforward though, all the code up to this point is basically just

andy.fingerhut23:12:40

Perhaps Java's implementation of the forEach method is masking exceptions thrown by hasNext ?

restenb23:12:37

nah the Java code is fine, there's a file in that directory & i'm able to get it's reference

jumpnbrownweasel23:12:46

@restenb If you print the full stack trace of the NotSuchElementException you'll see the wrapped StorageException, and its message or stack trace may give you a clue.

andy.fingerhut23:12:45

Clojure's iterator-seq does chunking, trying to get up to 32 elements, so if there was exactly 1 valid element, then hasNext threw an exception, that could differ from Java forEach .

seancorfield23:12:49

@restenb I'll just note that the Java code you showed has getDirectoryReference("/test") but your Clojure code has (.getDirectoryReference "test") without the /

seancorfield23:12:32

@andy.fingerhut That should still stop when hasNext() returns false -- which should be the same in the Java code.

andy.fingerhut23:12:09

But if hasNext is throwing an exception rather than returning false ...

andy.fingerhut23:12:06

I realize I am tossing out guesses at why there is a difference in behavior here, based on possibilities rather than actual evidence, so maybe not useful guesses.

restenb23:12:10

fixed it. 😛 still, a bit confusing implementation on their side IMO

seancorfield23:12:19

Right, but he says in the Java code it works and since there is a file in the directory, calling hasNext once should return true and calling it the second time should return false -- and that should "terminate" the iterator-seq as well.

seancorfield23:12:30

So... what was it? :)

restenb23:12:06

the directory was actually empty ...

seancorfield23:12:33

Oh... so you weren't running the Java code and the Clojure code against the same set of data?

restenb23:12:20

not the same directory as it turns out, no 😞 😳

restenb23:12:52

there should be one more (.getDirectoryReference "1") in that chain up above, then it works, still think throwing Exceptionon simply empty directory is a bit fragile though

restenb23:12:21

i guess there's more that can go wrong there and it's all contained in a StorageException

andy.fingerhut23:12:46

If you are willing to try the Java forEach version on the empty directory that Clojure throws an exception for, I would be curious to know whether it also gets the exception thrown out of forEach

restenb23:12:24

regardless i need to code less and sleep more @ 00:30 am, so thanks for your time @seancorfield and @andy.fingerhut 🙂