This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-03-31
Channels
- # announcements (20)
- # asami (14)
- # aws (6)
- # babashka (15)
- # beginners (83)
- # biff (6)
- # calva (93)
- # cider (3)
- # clj-kondo (21)
- # cljdoc (106)
- # cljs-dev (32)
- # clojure (165)
- # clojure-dev (78)
- # clojure-europe (54)
- # clojure-italy (9)
- # clojure-nl (9)
- # clojure-norway (24)
- # clojure-uk (4)
- # clojurescript (6)
- # community-development (2)
- # conjure (2)
- # core-typed (14)
- # datahike (4)
- # datomic (2)
- # emacs (40)
- # events (1)
- # fulcro (11)
- # graalvm-mobile (29)
- # graphql (8)
- # honeysql (19)
- # java (1)
- # jobs (1)
- # lsp (232)
- # malli (5)
- # membrane (112)
- # nextjournal (11)
- # off-topic (63)
- # portal (12)
- # re-frame (6)
- # reagent (3)
- # reitit (4)
- # rewrite-clj (2)
- # shadow-cljs (25)
- # tools-deps (6)
I'm getting this strange message, anyone have any idea what it's on about?
2022-03-30T21:18:58.617-05:00 [main] WARN FilenoUtil : Native subprocess control requires open access to the JDK IO subsystem
Pass '--add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED' to enable.
I see this one for cljdoc on JDK17. It comes from https://github.com/jruby/jruby/issues/6721 which is used by https://github.com/asciidoctor/asciidoctorj/issues/1035.
ohhhh i see
I suspect that's code that's digging into classes in the module internals to get things like pid etc
if you are perhaps using Java 16+ you are probably seeing the results of https://openjdk.java.net/jeps/396
ahhhh ok I'm using 17 so that might be it
I upgraded to Clojure 1.11.0 and some of my code stopped working. I'm very confused as to why into []
with my custom transducer now returns clojure.lang.PersistentVector$TransientVector
instead of a Clojure vector as it does with 1.10.3. Any hints or ideas as to where to look?
In other words, I have to wrap (into [] (my-transducer) "abc123abc"))
with (persistent!)
to get the expected result.
No, it doesn't. It does use volatile!
, but I also tried a version with atoms, no difference.
I don't know — writing transducers isn't easy, and I wrote that one a long time ago. Here is the code:
(defn- digit?
"Return true if ch is a digit character, false if it is a non-digit character, and nil if ch is nil."
[ch]
(and ch (>= (compare ch \0) 0) (<= (compare ch \9) 0)))
(defn- flush-acc [digits? data]
(if (true? digits?)
;; Fall back to returning a string if we can't parse the number.
(or (parse-long data) data)
data))
(defn cluster-characters
"Given a string, split it into a sequence of strings and integers. If called with no args, returns a stateful transducer."
([] (fn [xf]
(let [last-character-digit? (volatile! nil)
acc (volatile! "")]
(fn
([] (xf))
([result] (xf result (flush-acc (true? @last-character-digit?) @acc)))
([result character]
(let [current-digit? (digit? character)
previous-digit? @last-character-digit?]
(cond
(nil? previous-digit?)
(do
(vswap! acc str character)
(vreset! last-character-digit? current-digit?)
result)
(= previous-digit? current-digit?)
(do
(vswap! acc str character)
result)
:else
(let [to-flush @acc]
(vreset! acc (str character))
(vreset! last-character-digit? current-digit?)
(xf result (flush-acc (true? previous-digit?) to-flush))))))))))
([coll] (into [] (cluster-characters) coll)))
The first test is (= ["ABC" 123 "C"] (into [] (cluster-characters) "ABC123C"))
and this started failing with 1.11.0.
https://github.com/clojure/clojure/commit/b962791cbb201e91c29003f2ae1f7cd7a908971f maybe caused by this change?
I would post this on http://ask.clojure.org.
Hmm. That change indeed does seem related. But I do not understand this code and the reasoning behind it…
Ok, so after perusing examples on https://clojuredocs.org/clojure.core/transduce (the docs on https://clojure.org/reference/transducers were unfortunately not helpful) it seems I was doing the completion step wrong without using unreduce
.
Hello, does anyone know if it is possible to use http-kit to send a POST request with "Content-Type: application/edn"
? When I did some digging, it seems like it isn't supported. Is sticking to clj-http
the way to go?
How exactly are you making the request? You should be able to just specify :headers {"Content-Type" "application/edn"}
in your request.
I am trying to re-create the http-clj call that works
(client/post "" {:body (pr-str {:query '{:find [e]
:where [[e :crux.db/id _]]
:limit 10}})
:content-type :edn})
into http-kit
@(http/post "" {:form-params (pr-str {:query '{:find [e]
:where [[e :crux.db/id _]]
:limit 10}})
:headers {"Content-Type" "application/edn"}})
but this doesn't seem to workthis endpoint only supports application/edn
https://docs.xtdb.com/clients/http/#post-query
You're setting :form-params
, and that overrides the headers since, well, form params can be of only application/x-www-form-urlencoded
content type.
Replace :form-params
with :body
.
Thank you! 😄 Managed to get it working! I happened to pass by that just now and thought that meant that there wasn't any way to set the "Content-Type" since it is assoc(-ed) in
Why would one use reify
defprotocol
and other OOP
related functions and mindset in Clojure? What’re the benefits?
Why do you think this OOP related?
Please note that many features we love about clojure are actually powered by interfaces/protocols.
(Sequences, Deref, …)
Related, Rich Hickey updated Alan Perlis quote: > It is better to have 100 functions operate on one data abstraction than 10 functions on 10 data structures. Mentioned in https://image.slidesharecdn.com/clojureforlispers-100131101216-phpapp02/95/clojure-an-introduction-for-lisp-programmers-22-728.jpg?cb=1264932750
Yes, it might not be exactly the right question (oop) but still - it feels kinda out of place in Clojure code.
https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/core.cljs#L547
> it feels kinda out of place in Clojure code And you don't use those things in 95% of Clojure code. You use them only when you need to - not when you can. So your feeling is 95% correct. :)
The link above is a good one. And you can just browse through clojure/core.clj
, other built-in Clojure namespaces, and other open-source libraries to see how they're used. It's hard to explain in a sentence. Maybe this link will be helpful (although now the set of possible things is larger due to at the very least the ability to extend protocols via metadata): https://github.com/plumatic/eng-practices/blob/master/clojure/20130926-data-representation.md
Lately I've had to work with some Azure Java libraries, and as one would expect they tend to be very imperative and side-effectful. I'm not used to writing this style in clojure.. Here's an example snippet below. What sort of clojure core functions would you reach for to solve this problem using the API seen below? Assume azureMessages
is a clojure sequence and sender
and batch
are java object instances of the azure sdk.
for (ServiceBusMessage azureMessage : azureMessages) {
// continue adding messages to the batch while
// it is not full
if (batch.tryAddMessage(azureMessage))
continue;
// now the batch is full, send them off
sender.sendMessages(batch, tx);
// create a new batch
batch = sender.createMessageBatch();
// add the current iteration's message
if (!batch.tryAddMessage(azureMessage))
throw new ServiceBusSendException(String.format("Message is too large for an empty batch. Max size: %d.", batch.getMaxSizeInBytes()));
}
It's pretty straightforward interop, something like:
(doseq [msg azure-messages]
(when-not (.tryAddMessage batch msg)
(.sendMessages sender batch tx)
(let [batch (.createMessageBatch sender)]
(when-not (.tryAddMessage batch msg)
(throw (...))))))
I don't think any built-in function/macro would help here since the example is rather heterogeneous.
But when you have a lot of repeating constructs in Java, you can often make them less verbose in Clojure by using ->
and doto
.
Ah, that is straightforward.. I had gotten something a little more involved:
(defn create-batch
"Returns a sequence of batches ready to be sent"
[client ms]
(loop [m (first ms)
remaining (rest ms)
batch (.createMessageBatch client)
batches []]
(if m
(if (.tryAddMessage batch m)
(recur (first remaining) (rest remaining) batch (conj batches batch))
(recur m remaining (.createMessageBatch client) batches))
(conj batches batch))))
However this pulls all the batches (and messages) into memory at once, which may not be desireableIn your example though you don't know how many levels of nesting you need. I was wondering if a Volatile or atom would be appropriate here
Seems that batch
is declared outside of that for
loop, so at the end of the loop the batch
variable will have the value of the last batch. Is that what you need?
batch is declared outside, and then if the batch is full inside the loop the batch is sent, and a new batch is created.
batch
is a mutable reference to some Batch object. multiple Batches could be created/gced during that loop depending on message size
first time using a volatile... here's my go
(let [batch (volatile! (.createMessageBatch client))]
(doseq [m ms]
(when-not (.tryAddMessage @batch m)
(.sendMessages client @batch tx)
(vreset! batch (.createMessageBatch client))
(when-not (.tryAddMessage @batch m)
(throw (ex-info "Message is too large for an empty batch." {:message m
:max-size-bytes (.getMaxSizeInBytes @batch)})))))
(when (> (.getCount @batch) 0)
(.sendMessages client @batch tx)))
I wouldn't use volatiles here. This should work:
(defn send-batches
"Sends messages in batches."
[client tx ms]
(loop [ms ms
batch (.createMessageBatch client)]
(when-first [m ms]
(if (.tryAddMessage batch m)
(recur (rest ms) batch)
(do (.sendMessages client batch tx)
(when-not (.tryAddMessage batch m)
(throw ...))
(recur (rest ms) batch))))))
Oh, wait, I forgot to create a new batch - that do
should be let
that creates new-batch
.
Alright, I should've slept more, but this should be the final working version:
(defn send-batches
"Sends messages in batches."
[client tx ms]
(loop [ms ms
batch (.createMessageBatch client)]
(when-first [m ms]
(if (.tryAddMessage batch m)
(recur (rest ms) batch)
(let [new-batch (.createMessageBatch client)]
(.sendMessages client batch tx)
(when-not (.tryAddMessage new-batch m)
(throw (ex-info "Message is too large for an empty batch."
{:message m
:max-size-bytes (.getMaxSizeInBytes new-batch)})))
(recur (rest ms) new-batch))))))
nice yes, it just needs a final .sendMessages..
after the when-first to catch the last partly-filled batch
using if-let
(defn send-batches!
"Sends messages in batches."
[client tx ms]
(loop [ms ms
batch (.createMessageBatch client)]
(if-let [m (first ms)]
(if (.tryAddMessage batch m)
(recur (rest ms) batch)
(let [new-batch (.createMessageBatch client)]
(.sendMessages client batch tx)
(when-not (.tryAddMessage new-batch m)
(throw (ex-info "Message is too large for an empty batch."
{:message m
:max-size-bytes (.getMaxSizeInBytes new-batch)})))
(recur (rest ms) new-batch)))
(when (> (.getCount @batch) 0)
(.sendMessages client batch tx)))))
thanks @U2FRKM4TW
Sure thing. Just in case - the above won't allow for nil
messages, but that's probably alright.
So I have a java.awt.Frame
that is a descendent of java.awt.Component
. I'm trying to get the peer field of the Component, or else call .getPeer() on it. But first, I need to get that superclass and can't figure out how.
(def f (new Frame "AWT test"))
(. f setSize 400 400)
(. f setLayout (new GridLayout 3 1))
(class f) ;;java.awt.Frame
(supers (class f)) ;; #{java.awt.Container java.io.Serializable java.awt.Window java.awt.image.ImageObserver java.awt.Component java.awt.MenuContainer java.lang.Object javax.accessibility.Accessible}
(filter #(instance? java.awt.Component %) (supers (class f))) ;; () - it's empty
you would just call .getPeers
directly on the Frame
wouldn't you?
(.getPeers f)
That's what I'd expect, but no:
No matching field found: getPeer for class java.awt.Frame
Oh looks like getPeer
was deprecated at some point https://docs.oracle.com/javase/7/docs/api/java/awt/Component.html#getPeer()
Ah. In the openjdk source it still shows as public. So yeah I guess I need to do some reflection workaround.
I put the question on StackOverflow. https://stackoverflow.com/questions/71696166/acessing-parent-object-fields-methods-in-clojure
I'll duplicate my comment here for visibility: > The method seem to have been removed - at least, it's not there in JDK 17.
So I need a workaround using reflection on the peer
field.
And FWIW that method seems to be there in the latest OpenJDK: https://github.com/openjdk/client/blob/master/src/java.desktop/share/classes/java/awt/Component.java#L931
It's not a part of the Component
class - it's an inline definition on top of AWTAccessor.ComponentAccessor
(scroll a bit above the line you linked to to see the context).
Maybe that AWTAccessor.setComponentAccessor
makes it possible for you to somehow use it without reflection, no clue.
It kind of works.
(def acc (AWTAccessor/getComponentAccessor))
(.getBounds acc f);; #object[java.awt.Rectangle 0x40af415a "java.awt.Rectangle[x=1280,y=0,width=400,height=400]"]
(.getPeer acc f);; #object[sun.awt.X11.XFramePeer 0x2fa3dc81 "sun.awt.X11.XFramePeer@2fa3dc81(7600007)"]
getPeer
return nil until I've made the frame visible. Even after hiding it, the subsequent calls still work. That's icky. I'll stubbornly plow ahead and see if it can be made to work, but I'm afraid everything will have already been initialized with the constructed window, and thus injecting a window ID might not work as hoped.Hello, can someone explain why I'm not getting the proper character codes from this decoded base64? I'm expecting the second number to be 156 (works in python / javascript), but instead it's -100. Sounds to me like it's doing 256 - 156, as if to prevent the charcode from exceeding 128? Why is this happening?
(import java.util.Base64)
(->> (.getBytes data)
(.decode (Base64/getDecoder))
(take 10))
;; => (20 -100 63 103 -48 -23 -51 89 -52 -2)
(->> (.getBytes data)
(.decode (Base64/getDecoder))
(map #(Byte/toUnsignedInt %))
(take 10))
'when' means at an event that will occur or given a fact that is true, whereas when
actually means 'do-if', i.e. it may not be done, so it's not 'when'
That's normal English grammar... > When I'm hungry, I will eat.
Or > When it is raining, I get wet.
you didn't read what I wrote did you
Not sure what your intent is with the original message, but that's definitely normal, proper, and not going to change.
I didn't ask for it to change, I only pointed it out
don't you find (when (< (rand-int 2) 0) (println "negative!"))
presumptuous ?
it's like saying "when pigs fly"
I do not find it presumptuous, or in any other way improper, precisely because of the picture I shared above.
"When pigs fly, take lots of pictures." That is perfectly reasonable.
yes, in English it's reasonable
that's my point - the English meaning doesn't align with the Clojure meaning
add a label to your code and it makes sense - (when (is-negative?) ...) reads fine to me - i think this has roots back to CL so like 50 years old now
no matter how well you name it, the conditional may not ever be true, so assuming that when it is true is presumptuous
It's of no importance whether the condition can ever be true. You can write code that will never be executed - that's fine. Just as in English you can use constructs that semantically make little sense, like "I'll do that when the time stops." That's a proper English sentence.
Or, to be more blunt, let's rewrite your Clojure code in English:
> (when (< (rand-int 2) 0) (println "negative!"))
"When a randomly generated integer in the range [0, 2) is less than zero, print out 'negative!' followed by a new line."
And if that feels icky to you - well, that's your personal perception of some of the constructs in the English language. We all have our preferences. I, for one, dislike "colour" and "licence" and prefer "color" and license", all due to how I studied the language. There are other, more complex, things that I don't like, but it's hard to deliberately recall them.
'when' and 'if' have different meanings, as do 'if and when' and 'iff'
Idiomatic Clojure is a thing -- and it's not the same as idiomatic English.
when
is also commonly found in other lisps, for example https://www.gnu.org/software/emacs/manual/html_node/elisp/Conditionals.html and http://www.lispworks.com/documentation/HyperSpec/Body/m_when_.htm#when.
interesting tidbit is that I seem to recall when
in other langs indicating side effects ahead
well, when
in Clojure is a macro which generates a single branch if
with a do
so yes it implies side effects
"Idiomatic Clojure is a thing -- and it's not the same as idiomatic English." yes, that's exactly what I said at the top of this thread
In that case, I don't think I understand what you're arguing for. Are you saying when
is a bad choice for the macro name?
What's idiomatic depends on tradition. Rich has stated that Common Lisp inspired Clojure, and Common Lisp has the same when
construct.
From https://en.wikipedia.org/wiki/Clojure:
I tend to approach language idioms by figuring out "what's established use of the language?" rather than trying to state what it should be.
I prefer when
to if
for pure computations if there's no else branch:
(when (and x y)
{:result (* x y)})
I particularly dislike that use of when
because when
implies side effects
would you write (if (and x y) (do {:result (* x y)}))
? No, because for pure computations you wouldn't use do
I wouldn't write (if ... (do ...))
, I would write it with when
! Just like the example above. I don't understand where get that "when implies side effects" from. The docstring doesn't mention side effects.
I think there's even a clj-kondo rule to suggest using when
instead of if
when there's no else-branch.
https://clojuredocs.org/clojure.core/when
read the source of when
(macroexpand '(when (even? 6) 7))
=> (if (even? 6) (do 7))
does (do 7)
make sense ? why write a single branch if
for non-side-effecty code ?
when
only makes sense for side effecty code e.g.
(when (time-to-notify? t)
(send-emails! people)
(store-emailed! people)
(invoke-aws-service! people))
Hi, I need to use a java classname with a dot in the class name, how is that possible in clojure?
Thank you!
👋 All, is anyone familiar with the new iteration
function? Unless I'm missing something, it doesn't return the "last page" of data. In the test below (full gist with output https://gist.github.com/giuliano108/43c0341d9a8e07cb87b42fd0eb0868ef), items 6
and 7
aren't returned...
(deftest test-iteration
;; paginated API
(let [api (fn [tok]
(condp = tok
0 {:ret [0 1 2] :tok 3 :more true}
3 {:ret [3 4 5] :tok 6 :more true}
6 {:ret [6 7] :more false}
nil))]
(is (= [0 1 2 3 4 5 6 7]
(into [] cat (iteration api :initk 0 :kf :tok :vf :ret :somef :more))))))
Note that the behaviour is consistent with the https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/iteration , which say: > Iff (somef ret) is true, (vf ret) will be included in the iteration, else iteration will terminate and vf/kf will not be called. I'm not sure how to handle APIs like in the example (where the last paginated response contains some data and an indication that there's no more of it).
@U050ECB92 Can you speak to this?
It seems to me that :somef
should be a function that tests whether :ret
is present (or perhaps not empty?), based on reading the docs.
(iteration api :initk 0 :kf :tok :vf :ret :somef (comp seq :ret))
The iteration terminates if (kf ret)
is true -- which it will be for that last page: no :tok
field.@U039P8YCZNY Does that help?
It would help, yes (still need to take :more
into account to avoid an extra API call that would return no results). Maybe I was led astray by the blog post about iteration
(https://www.juxt.pro/blog/new-clojure-iteration), which is about a real world example interacting with the AWS S3 API, they use the :truncated?
keyword (and nothing else) much like I use :more
in my test...
(defn api [tok]
(case tok
0 {:ret [0 1 2] :tok 3 :more true}
3 {:ret [3 4 5] :tok 6 :more true}
6 {:ret [6 7] :more false}
nil))
(into [] cat (iteration api :initk 0 :vf :ret :kf :tok))
[0 1 2 3 4 5 6 7]
is that the behavior you want?gets the whole dataset with no extra calls to api
. (you can verify by removing the nil
to the case or having it println in that case
and maybe with a bit more verbose keys:
query-processor=> (defn api [tok]
(case tok
0 {:payload [0 1 2] :next-page-token 3}
3 {:payload [3 4 5] :next-page-token 6}
6 {:payload [6 7]}))
#'metabase.query-processor/api
query-processor=> (into [] cat (iteration api :initk 0 :vf :payload :kf :next-page-token))
[0 1 2 3 4 5 6 7]
query-processor=>
in your original example you have :somef :more
This is false on when requesting with tok 6.
> Iff (somef ret) is true, (vf ret) will be included in the iteration, else iteration will terminate and vf/kf will not be called.
so you are saying it gave back no results when more is false. Which is false for your needs
Good point that :somef
is probably not needed here. @U039P8YCZNY As Dan points out, (kf ret)
-- :tok
-- determines whether to make the next call.
The issue that is perhaps not obvious from the docs is that there are two possible termination conditions: :somef
determining that there's nothing to call :vf
on -- stops on no more values present in the response -- and :kf
determining that there's no "next key" to make an API call with.
:tok
-- or :more
in your case -- are the second condition; you don't need the first condition here.
• somef: did i get something back • vf: function to give me my results from what’s back • kf: function to give me a key to call the api with to continue If somef is false, we didn’t get anything meaningful back to quit. If we did, if kf returns something, use it else we are done.
It all makes perfect sense now, thanks @U04V70XH6 and @U11BV7MTK, I really appreciate the help!
(I wonder if Juxt's use of :somef :truncated?
would yield incomplete S3 API results, but that's a different matter 🙂 )
Yeah, I think I agree with you about that example: it'll exclude the (partial) data from the last page.
somef is about "did the step you just performed yield a result? if not, no more further steps" kf is about "get the seed data for the next step (or stop)"
but since the S3 API already has a "NextToken" field, and clojure has nice truthiness, you can just make kf :next-token
(I will warn, some annoying amazon APIs return an empty string for the continuation token, which you have to check for.)
There are enough knobs on iteration
that you should never need to do an unnecessary API call
Right, so having :somef :truncated?
is not correct, yes?
Hi all, sorry I'm late on this. Thanks all for reporting my misunderstanding of somef
in the article, it was indeed preventing to retrieve the last page. I've amended the examples and clarified the explanation of the iteration
parameters. I also updated the article to the now final 1.11. I've linked this thread at the bottom for posterity. If you still see something incorrect, please don't hesitate and let me know. :thumbsup:
we have a new hire, new to functional programming, who is going through Clojure for the Brave and True but prefers a more academic bottom-up pedagogy. Any resource recommendations in that vein?
books are great. i think non-digital is preferred, i'll see about ordering dead tree versions of those
Ah yes, I have that too and like it. Buy all of them! And Joy of Clojure 2nd Ed 🙂
I loved the rhythm of Getting Clojure. you can see it in the ToC of the book
introduce several related language features explaining how they fit in with the whole of the language, explain gotchas for the features, give examples in the wild where the features are used to solve real problems, summarize, and repeat
it feels more like a curriculum, in a way
I think it's a great book, but I think you can learn a lot with JoC and a REPL. More my style of learning
Perhaps Clojure in small pieces would be a good fit - https://github.com/robleyhall/clojure-small-pieces/blob/master/clojure-in-small-pieces.pdf