This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-08
Channels
- # adventofcode (60)
- # announcements (3)
- # babashka (31)
- # beginners (5)
- # boot (1)
- # calva (13)
- # cider (9)
- # clj-kondo (1)
- # clojure (135)
- # clojure-italy (18)
- # clojure-nl (18)
- # clojure-spec (21)
- # clojure-uk (11)
- # clojuredesign-podcast (1)
- # clojurescript (47)
- # core-async (14)
- # emacs (7)
- # euroclojure (4)
- # fulcro (3)
- # graalvm (19)
- # off-topic (22)
- # reagent (29)
- # shadow-cljs (25)
- # vim (3)
Why does get
on non-associative objects return nil instead of throwing an error? I've spent lots of time hunting down bugs that turned out to be (get some-vector n)
being passed a lazy sequence instead of a vector
You might want to try http://ask.clojure.org for questions like this, if you do not get an answer here. The record of the answer should remain longer there, and it is more suited for long-form answers there.
(I understand nil-punning with (get nil ...) => nil
, but shouldn't passing numbers etc be an illegal argument)
the answer to many, many questions like this where clojure's garbage-in-garbage-out comes into play is "you would pay a performance penalty for every single get call. it's a dynamic language". another answer is that maybe someday spec will check the types of these parameters at test/dev time
guys i need your help! In a lib (https://github.com/AppsFlyer/cloffeine) i have:
(ns cloffeine.cache
(:refer-clojure :exclude [get])
(:require [cloffeine.common :as common])
(:import [com.github.benmanes.caffeine.cache Cache]))
(defn make-cache
(^Cache []
(make-cache {}))
(^Cache [settings]
(let [bldr (common/make-builder settings)]
(.build bldr))))
but than, in the calling code:
(let [c (cache/make-cache)] (.estimatedSize c)) ;; this works, but
(def c (cache/make-cache))
;;; and later
(.estimatedSize c) ;; fails! -> Can't call public method of non-public class: public default long com.github.benmanes.caffeine.cache.LocalManualCache.estimatedSize()
ot looks like let
behaves different from def
regarding types.
note that Cache
is a java interface while .build
returns a specific implementation.this can only be resolved by a type hint near the def
, which is not required inside the let
anyone knows why?
Unless I'm misunderstanding, this looks right. Clojure thinks c
is a Cache, but it's not, it's something else. You probably need to type hint for what you're actually returning.
Type hinting this way means that in the future you can change the exact return to someething else, without it breaking. You also end up doing reflection otherwise, to find the estimatedSize field on something else.
I imagine let
works because it hasn't picked up the type hint, but not sure.
but c
is a Cache. it is a specific implementation of this interface. also, make-cache
is known to return one….
I want to search a user-defined / runtime classpath for a file. So something like (find-file "src:/Users/borkdude/.m2/repository/foo/bar/baz.jar" "foo/bar/baz/core.clj")
. What do I use for this?
This seems to work:
user=> (def cl (.URLClassLoader. (into-array .URL [(.URL. "file:///Users/borkdude/.m2/repository/com/cognitect/transit-java/0.8.337/transit-java-0.8.337.jar")])))
#'user/cl
user=> (.findResource cl "com/cognitect/transit/Reader.class")
#object[.URL 0x3e30646a "jar:file:/Users/borkdude/.m2/repository/com/cognitect/transit-java/0.8.337/transit-java-0.8.337.jar!/com/cognitect/transit/Reader.class"]
interesting that there is also a .findResources -- somehow you can get more than one thing i guess
does it make sense that this would make my REPL hang?
(let [c (a/chan)]
(a/>!! c "1")
(a/<!! c))
something weird i found:
on my laptop (java 8):
(Math/pow 9 17)
=> 1.667718169966657E16
on a friend laptop (java 11):
(Math/pow 9 17)
1.6677181699666568E16
wtf?I do not know why there is a difference between the return value on JDK 8 vs. 11 on those two computers, but in general you are living in a fantasy land if you expect exact answers using floating point numbers: https://clojure.org/guides/equality#_floating_point_numbers_are_usually_approximations
They are useful for speed of calculation, with some kinds of error bounds on the results, when used appropriately.
Yes of course, that is the whole point of float and why you shouldn't use them for precise calculations. Here my surprise was that I wasn't aware of a change in precision between java8 and java11. The same exact code is returning two different values, not that it wasn't correct. If you want to get the precise result you clearly need to use something else like a BigInteger
and elevate to power with that.
The display is 15 digits on java 8 and 16 digits of precision on java 11. Seems the same number in both cases?
@dpsutton it's not just display:
java8: (long (Math/pow 9 17) => 16677181699666570
java11: => 16677181699666568
yeah, your result is the correct one according to wolframalpha as one would expect, with BigIntegers
case
works with compile-time literals. If you pass a reference to a public static final int
Java class field (e.g. Table/HEADER = 0
), you will get a case
that tries to match its argument with the symbol Table/HEADER
, not with its value 0
.
It seems to me that since such values are available during the compile time, maybe there's a convenient method to create a kind of case
that works with values of such fields and not with the symbols.
it would be difficult to discern based on context between matching a symbol and a class field
I can see that. But it should be possible to create a variant of case
designed specifically for such fields, right?
You could maybe macro it so that you emit a case that expands to the values of whatever symbols passed to it
Yeah, this works for simple cases:
(defmacro static-case [e & clauses]
`(case ~e
~@(mapcat (fn [[test then]]
[(eval test) then])
(partition 2 clauses))))
@U4YGF4NGM What do you mean? Clojure macros are not hygienic, so the above code snippet works just fine if I use it in let
where some bindings are also used in some then
s.What’s the use case for the value argument of the nav protocol for datafy? The other arguments all make sense to me but I’m struggling to reason about that argument.
Has anyone played around with replikativ/konserve
and been able to customize the types it serializes? I'm having trouble writing a map containing a java.time.Instant
to the Postgres-backed store. (@whilo)
@tyler The default implementation of nav
is to simply return the value argument. Custom implementations of nav
may navigate into the data coll
via the key k
or may additionally use v
to compute an updated result.
For example, in next.jdbc
, nav
is implemented to either navigate via the key k
if it looks like a foreign key column or to return the value v
unchanged if it's a normal column. In the navigation case, coll
is ignored because nav
depends only on k
and the closed over database information (`coll` is also ignored in the non-navigation case).
Interesting. So if I understand correctly the val argument is available to add additional behavior if necessary but most use cases can fall back to the default implementation? Thanks for the reply.
Yes. nav
can ignore or use any of its three arguments to provide an appropriate transformation/computation that represents navigation from data to a new "original" result.
The cycle is always: original -> datafy
-> pure-data -> nav
-> new-original -> datafy
-> new-pure-data ...
But datafy
and nav
can both be "no-op", just returning the input value.
Does next.jdbc have option to remove table name from column name or I need to remove it “manually”? actual
[
{ :table/id 1 }
{ :table/id 2 }
...
]
expected
[
{ :id 1 }
{ :id 2 }
...
]
@artur.aralin97 Have you read over the documentation for next.jdbc
? It explains why it table-qualifies column names and how to get different results. I'm happy to answer any questions in #sql (I'm more likely to see them there than in #beginners because this channel is fairly high traffic).
You should probably at least work your way through the Getting Started guide https://cljdoc.org/d/seancorfield/next.jdbc/1.0.11/doc/getting-started -- it specifically addresses qualified/unqualified column names. I would also recommend that you get used to qualified keywords and start using them in your programs. Clojure's Spec is all based on qualified keywords and they exist in the language for important reasons.
Has anyone played around with Codeq + Cluster analysis at all? I'm aware of Codemaat, but I want something at the function level.
I'm not, I just want to fiddle around with some speculations I had about projects I've historically worked on.
You can try it (the limited version) on an open source github repo here: https://codescene.io/ There are some good articles about its possibilities here https://empear.com/blog/ (in particular https://empear.com/blog/code-analysis-tool/)
There's also a showcase project (<https://codescene.io/showcase>) using clojure core: <https://codescene.io/projects/175/jobs/15346/results>
Ops… I need to be more attentive 👀
No problem. I'm always happy to get feedback on how to improve the docs to make stuff like this more obvious...
I think my inattention because of English is not native for me. Therefore it makes some problems for me 🙂
anything to know about dealing with Iterable
in Clojure? like if I want to loop over a Iterable<SomeType>
, is there a preferred way?
@restenb You probably want iterator-seq
But read the caveats about mutable objects etc http://clojuredocs.org/clojure.core/iterator-seq
turns out this wasn't a java.util.Iterable,
it's some Microsoft class LazySegmentedIterable
from their Azure APIs
Sounds delightful 🙂
@restenb Looks like that has a .iterator
method that returns a regular Iterable
so (iterator-seq (.iterator lsi-object))
should work...
I would expect (seq (iterator-seq (.iterator lsi-object)))
to return nil
if it was empty...?
i.e., normal Clojure sequence operators should just work. Can you share some code of how you're trying to process it?
might be Microsoft's fault for all I know 😛 ... originating from com.microsoft.azure.storage.core.LazySegmentedIterator.hasNext(LazySegmentedIterator.java:113)
As far as I know from the Java docs of the Iterator class, hasNext should always return a boolean, never throw an exception: https://docs.oracle.com/javase/8/docs/api/java/util/Iterator.html#hasNext--
The next
method is documented to throw such an exception, if there are no more elements.
Hmm, according to the Azure docs, .listFilesAndDirectories
should return Iterable
anyway, so plain old seq
ought to work...
seq
on Iterable
is lazy -- perhaps you're realizing the sequence outside of the scope in which the original LazySegmentedIterator
is valid @restenb? That might explain .hasNext
blowing up...
(doseq [file (.listFilesAndDirectories (.getDirectoryReference rootDirectory "/test"))]
(println (.. file getUri getPath)))
In your code snippet that had a call to iterator-seq
inside of a let
, if you instead did (do all (iterator-see ...))
inside of the let, it would force all elements of the sequence to be realized inside of the let
, rather than returning an unrealized lazy sequence.
That's probably where I would start, off the top of my head.
My suggestion would be bad if you did not want to allocate memory for all of that at one time. Sean's suggested code never needs to allocate all of those items in memory at once.
(my code assumes rootDirectory
is set up earlier -- however it is done in the Java code)
but again, somehow this iterator is coming back as com.microsoft.azure.storage.core.LazySegmentedIterable
, not "regular" Java Iterable
Call ancestors
on that type, just to make sure it really does implement Iterable
Looking at this implementation code, that might be what you are running against, or maybe not, it appears they implement hasNext
in a way that can throw an exception: <https://github.com/Azure/azure-storage-android/blob/master/microsoft-azure-storage/src/com/microsoft/azure/storage/core/LazySegmentedIterator.java#L104-L116>
@restenb So my exact code fails with that exception too? Then it does sound like you're running into a StorageException
per the code Andy just linked to 😞
It looks like they wrapped it in a NoSuchElementException
, leaving the original exception inside. Not sure why, unless perhaps they thought something like "Well, next() is supposed to throw such an exception when there are no more elements, so maybe we should do the same for hasNext(), too"
You'll have to take that up with the library authors 🙂
hasNext()
should simply return the true/false
or throw the original exception, I really don't understand this choice
To verify you are actually hitting that case, you should see whether the exception you has has some other kind of storage exception class inside of it.
If StorageException is a checked exception then it can't be thrown by hasNext and would have to be wrapped in something. But it would have been better for them to have wrapped it in a different RuntimeException, not NoSuchElementException.
basically I have to set up a chain of objects to get to this point, the actual storage account, the file share, the root directory and then the sub directory for which I'm requesting the files
so maybe the answer is in there somewhere and the sub directory object doesn't have a valid reference to it's "parent"
Perhaps Java's implementation of the forEach
method is masking exceptions thrown by hasNext
?
nah the Java code is fine, there's a file in that directory & i'm able to get it's reference
@restenb If you print the full stack trace of the NotSuchElementException you'll see the wrapped StorageException, and its message or stack trace may give you a clue.
Clojure's iterator-seq does chunking, trying to get up to 32 elements, so if there was exactly 1 valid element, then hasNext threw an exception, that could differ from Java forEach
.
@restenb I'll just note that the Java code you showed has getDirectoryReference("/test")
but your Clojure code has (.getDirectoryReference "test")
without the /
@andy.fingerhut That should still stop when hasNext()
returns false
-- which should be the same in the Java code.
But if hasNext
is throwing an exception rather than returning false
...
I realize I am tossing out guesses at why there is a difference in behavior here, based on possibilities rather than actual evidence, so maybe not useful guesses.
Right, but he says in the Java code it works and since there is a file in the directory, calling hasNext
once should return true
and calling it the second time should return false
-- and that should "terminate" the iterator-seq
as well.
So... what was it? :)
Oh... so you weren't running the Java code and the Clojure code against the same set of data?
there should be one more (.getDirectoryReference "1") in that chain up above, then it works, still think throwing Exception
on simply empty directory is a bit fragile though
i guess there's more that can go wrong there and it's all contained in a StorageException
If you are willing to try the Java forEach
version on the empty directory that Clojure throws an exception for, I would be curious to know whether it also gets the exception thrown out of forEach
regardless i need to code less and sleep more @ 00:30 am, so thanks for your time @seancorfield and @andy.fingerhut 🙂