This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-12
Channels
- # adventofcode (6)
- # beginners (148)
- # boot (5)
- # calva (1)
- # cider (10)
- # cljdoc (10)
- # cljs-dev (8)
- # cljsrn (10)
- # clojure (180)
- # clojure-dev (24)
- # clojure-europe (2)
- # clojure-finland (1)
- # clojure-italy (32)
- # clojure-losangeles (1)
- # clojure-nl (40)
- # clojure-spec (10)
- # clojure-uk (44)
- # clojured (4)
- # clojurescript (88)
- # community-development (33)
- # core-async (7)
- # cursive (19)
- # datomic (98)
- # duct (3)
- # events (1)
- # figwheel-main (10)
- # fulcro (62)
- # leiningen (23)
- # luminus (18)
- # off-topic (19)
- # pedestal (6)
- # re-frame (46)
- # reagent (21)
- # ring (17)
- # ring-swagger (3)
- # shadow-cljs (94)
- # slack-help (9)
- # spacemacs (14)
- # sql (1)
- # testing (4)
- # tools-deps (14)
Honestly, I mostly only remember those specifics because I reimplemented agents over web workers a while back. Turned out, atoms and immutable data solved 90% of our concurrency issues.
Those last 10% though are usually solved by seasoned java devs, so I think they're more inclined to reach for native thread tools, instead of using agents (or even STM)
Yeah, I have to say that includes me John. I can get most of this stuff done with core.async now, but I spent long enough in Java land to find reaching for java.util.concurrent a bit too easy. Nearly suggested a Semaphore for problem above.
Yeah, I think we should try to stay between the lines more often than we do there. Atoms and agents and stm all work in concert with one another wrt the snapshotting of data across a transformation. Seeing folks muddy the semantics makes me sad. But sure, sometimes you'll have to reach for a more specific tool.
+1 for agents in this use case. My project uses an agent in one particular spot. We routinely schedule a task but want only one to run at a time. Works like a charm.
yep, I can confirm that it works well in my case as well, also much easier to implement than I thought, thank you guys.
Interested to know how you solved the problem of communicating back to the caller. Agents do seem a little, "fire and forget". The agent data will get updated with the return 'at some point in the future', and you do have (await)
so I suppose if you do something like keep a vector of successful reservations you can (await)
to see if it's there, but aside from your constantly growing vector there, (await)
docs contains some concerning statements like, "Will never return if..." 🙂
Pass in a closure over a promise that gets delivered when the function executes on the agent? 🙂
I was thinking that too, but was afraid that was just the first step to a bad rube goldberg machine - but perhaps it is that simple
Hah, I have in my head a core.sync solution that looks a bit like that with channels.
I mean what if the agent blows up so the promise delivery never happens, now your thread is deadlocked, so then you put the deref of the promise in a future so you can use deref of that with a timeout and then...
the problem with people who write javascript is the tend to think that using a single threaded server and single threaded client will save them from thinking about concurrency
Indeed, but writes are single threaded in part so you don’t have to worry about isolation
anyway, my point is, programming in javascript will damage your brain when it comes to concurrency, and concurrency is a fact
at least JS in the browser lets you not have to think about problems locally without concurrency issues. but too many people try and venture farther and run into issues early on
Go to #slack-help or #community-development and alert an admin in either of those channels. Do not engage the trolls/spammers here please.
is there any way to add hint for returned value from function which is defined using fn
?
user=> (set! *unchecked-math* :warn-on-boxed)
:warn-on-boxed
user=> (defn abc ^double [] 1.0)
#'user/abc
user=> (inc (abc))
2.0
user=> (def abc* (fn ^double [] 1.0))
#'user/abc*
user=> (inc (abc*))
Boxed math warning, C:\cygwin64\tmp\form-init496459974570346569.clj:1:1 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object).
2.0
(macroexpand '(fn ^{:pre [(pos? i)]} [i]
(inc i)))
(fn* ([i] (clojure.core/assert (pos? i)) (inc i)))
user=> (defn make-me-aaa [] (fn ^double [] 1.0))
#'user/make-me-aaa
user=> (let [aaa (make-me-aaa)] (inc (aaa)))
Boxed math warning, C:\cygwin64\tmp\form-init496459974570346569.clj:1:26 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object).
2.0
clojure only knows how to use the unboxed path when using primitively tagged functions stored in Vars
What's the status of the functions in the Clojure incubator? I'm mainly interested in dissoc-in
. It looks like there's been no activity in incubator or on the issue (https://dev.clojure.org/jira/browse/CLJ-1063) in a while. I think it would be nice to have in core so I can stop putting a copy into each of my projects 🙂
create a package and then just pull that down! hah
incubator is dead
I think my comments at the end of 1063 summarizes my qualms about it
I don’t think there is a candidate impl that I would consider worth recommending right now
Does anyone else crash/hang Docker (on Mac) when running lein uberjar
? Last night it just hanged. Now it terminated with ERRO[0457] error waiting for container: EOF
.
I even bumped it to use 6 CPUs, 8 GiB RAM & 3.0 GiB swap.
but if other containers are having issues, it's good to hit the Reset button in Docker for Mac preferences
@ghadi Hmm, thanks — I believe it must be! (So the uberjar remains after the Docker dies. I didn't write this codebase, so I'll verify.) (I've reset a few times, so unfortunately that doesn't work. :/)
yeah, Docker for Mac filesystems mounts are quite slow. If you watch -n 1 du your-uberjar.jar
you may see that it progresses, but at a crawl
Hmm, in that case I wonder if it makes sense to run Docker inside VMWare/Debian (atop MacOS). 😛
@dts.siedel medley has dissoc-in: https://github.com/weavejester/medley/blob/master/src/medley/core.cljc#L20
Yes I agree about the extra dependency. Doesn't update-in + dissoc leave empty intermediates though? For my use case I would want the other behavior (as in the incubator/medley versions)
Greetings fellow clojurians! Does anyone have experience with creating a jpeg file from a base64 encoded string extracted from html? (Inline image in html) I managed to get those image strings from a html page with hickory, but after that I tried to decode them with (String. (.decode (Base64/getMimeDecoder) to-decode)) and http://clojure.java.io/copy them out to a folder with no success. When I try to open an image my viewer says: (Not a JPEG file: starts with 0x75 0xab) I am pretty new to this, so got the idea to ask it here. I hope someone could help me. Thank you in advance!
@mindenaaron Could you post the base64 encoded string in a gist?
@borkdude Sure, https://gist.github.com/aaronpowered/6e2f2de74e2de9be7da52889c5452f4d (Have to leave now, will be able to reply after 3 hours.)
I noticed there was a trailing slash in the base64 string. I recommend testing it in an online viewer first: e.g. https://codebeautify.org/base64-to-image-converter
Hi, I'm doing some interop with Java and I dont understand why (.write javax.imageio.ImageIO bi "jpg" f)
works but not (.write javax.imageio.ImageIO bi "jpg" f)
?
Before I did (.fillRect g2d 20 50 10 30)
and it worked fine
Is it because ImageIO is a class and not an instance?
My question is about the interop syntax
(.write javax.imageio.ImageIO bi "jpg" f)
(.write javax.imageio.ImageIO bi "jpg" f)
the two forms you put are identical@julien.rouse did you figure out what was going on?
@dpsutton hey sorry I was out, I meant (. javax.imageio.ImageIO write bi "jpg" f)
is the correct way and (.write javax.imageio.ImageIO bi "jpg" f)
didnt work
for a static method, (javax.imageio.ImageIO/write bi "jpg" f)
is the recommended syntax
@julien.rouse you can see from the javadoc which methods are static, https://docs.oracle.com/javase/8/docs/api/javax/imageio/ImageIO.html and the clojure docs for interop describe static method calls https://clojure.org/reference/java_interop#_the_dot_special_form
Thanks for the informations
but why would (.fillRect g2d 20 50 10 30)
works and not (.write javax.imageio.ImageIO bi "jpg" f)
? Is that because write is static?
right, fillRect is a method on g2d, write is a method on javax.imageio.ImageIO, the rules for static vs. instance methods are different
in java they have the same syntax, but on the bytecode level they are implemented differently, and clojure distinguishes them
Right, thanks for the clarification, It's an area to improve for me!
interop is a pretty small set of rules, and once you learn them you can use a lot of stuff for free, so it's rewarding :D
Yeah that's why I was getting my hands dirty with it, leveraging Java libs is a great bonus to the already nice language
Thank you again for your help ! 🙂
Hielo all. I'm doing some experiments with streams (but not touching IO yet), but getting the following error randomly in my lein repl, anyone have a clue why?
IOException Resource temporarily unavailable
java.io.FileInputStream.readBytes (FileInputStream.java:-2)
java.io.FileInputStream.read (FileInputStream.java:255)
java.io.BufferedInputStream.fill (BufferedInputStream.java:246)
java.io.BufferedInputStream.read (BufferedInputStream.java:265)
jline.internal.NonBlockingInputStream.run (NonBlockingInputStream.java:294)
java.lang.Thread.run (Thread.java:748)
does *e have the full stack trace?
did you open a file? if so how did you open it / access it?
nope, no open files. Not sure I can see what *e
has as the process crashes when it happens
you could set the top level uncaught exception handler to be more informative (Thread/setDefaultUncaughtExceptionHandler (reify Thread$UncaughtExceptionHandler (uncaughtException [this thread error] (prn "uncaught error\n" thread error))))
(fixed error, added example)
(ins)user=> (Thread/setDefaultUncaughtExceptionHandler (reify Thread$UncaughtExceptionHandler (uncaughtException [this thread error] (prn "uncaught error\n" thread error)))
)
nil
(ins)user=> (.start (Thread. #(/ 1 0)))
nil
user=> "uncaught error\n" #object[java.lang.Thread 0x2d631a09 "Thread[Thread-0,5,main]"] #error {
:cause "Divide by zero"
:via
[{:type java.lang.ArithmeticException
:message "Divide by zero"
:at [clojure.lang.Numbers divide "Numbers.java" 163]}]
:trace
[[clojure.lang.Numbers divide "Numbers.java" 163]
[clojure.lang.Numbers divide "Numbers.java" 3833]
[user$eval215$fn__216 invoke "NO_SOURCE_FILE" 40]
[clojure.lang.AFn run "AFn.java" 22]
[java.lang.Thread run "Thread.java" 748]]}
hm, might be related with autocomplete or something like that, as the error seems to happen when I write (def my-stream (s/stream)
(notice the missing ending )
)
haha, just fixed a missing paren then thought you were referring to it - noticed when I pasted into my repl -I'm proud that something freeform into slack was that close to correct honestly! :D
hm, unfortunately that doesn't make the error any clearer https://gist.github.com/victorb/304ed4cd9834ecb02ce1b37f641e7957
so this happens before you even finish writing the form? what repl implementation?
wait, just a regular lein repl in a terminal?
you can use lein upgrade $VERSION
to pick an arbitrary older version of lein - if it's a lein bug I bet going back a version or two would eliminate the error
nrepl stuff was radically changed recently
(and of course having the latest lein is just a question of using lein upgrade
again)
good idea, I'm on 2.8.1 so I'll try upgrading before downgrading. Thanks for the suggestion
also OS mgiht be relevant here too (since IO bugs in IPC code that only affect one OS are relatively common...)
thanks @noisesmith for helping me out 🙂
@borkdude trailing slash was only a mistake of copy paste. I had another mistake, had to drop the head of the string, (data:image/jpeg;base64) so i can use Base64/getDecoder instead of Base64/getMimeDecoder, but image file is still not available to open. I leave this image here as the difference between my decoded file and the one decoded by Best Online Base64
BTW What is this nonsense, Slack? "Your file was uploaded — it’s safe and sound in Slack. Unfortunately your workspace doesn’t have any storage space left. To get more space, you can upgrade to a paid account or delete some of your older files." Can you guys see my recently uploaded picture?
yes - I can see it - that difference almost looks like text encoding issues, which makes me wonder if you wrote the file using a String / text oriented function rather than a bytes / raw data function
Yes can see it
@noisesmith @julien.rouse Thank you! Yes good point
@mindenaaron OK I scrolled up - your mistake is using the String
constructor
pass the byte array to io/copy, this isn't textual data
Thank you again @noisesmith
I have to stop just copy pasting utility functions
@mindenaaron one of my loose heuristics for bugs is doing any sort of String conversion during IO - text to text just works via the binary methods, string conversion in the middle breaks things
Finally it works! You guys are always doing a great job here 🙂 I am very grateful
I just wrote a function which extracts images from the output of a wysiwyg, make sha1 hash and store every image once and change the inline base64 string to the path to that image. It works brilliant 🙂
You’ll often see it used via the reader macro #'
. Usually to alter the metadata associated with a Var, or sometimes in test namespaces to refer to private vars
@nwjsmith Thanks, I didn’t realise that metadata came with a var, not the associated object.
cross-posted from #luminus: does anyone know specifically where to put wrap-cors in a luminus app (which has swagger, if that matters)?
my various attempts in handler and middleware aren't working.
I need cors enabled locally for development on an app created with the equivalent of 'lein new luminus testapp +swagger'.
@avfonarev another usage of var is to pass the container holding a function to a function that creates a long running closure (eg. starts a web server using the function you pass in) - if you pass a function, the higher order function won't see redefs, if you pass the var it always sees the current value
the thing that makes this work is that calling a var attempts to call the value you put in the var
of the 3 Design / usage benefits of transducers:
in https://clojure.org/guides/faq#transducers_vs_seqs - don't the last two apply to reduce
or loop/recur
too?
unlike transducers, reduce and loop/recur are syntaxes rather than units of composition
so they are eager and explicit about resource usage, but aren't in the same category of abstraction
@noisesmith yeah, that is why I said 'the last two' - excluding transformation composition
both transducers and sequences can be passed to functions to get a new transducer or sequence
right - what I'm saying is they aren't even in the same category - strings and io streams are also eager and explicit about resource usage, but would be out of place in the comparison
@noitcudni Actually, it’s a cool usage. Thank you
@noisesmith Ok, I see what you mean, yes, they are not the same, was specifically curious what do they have over reduce
specifically and only for eagerness and resource control
transduce vs. reduce has two major improvements: formalizing a final "collate results" step after everything has run (very common pattern in my code to need that), and separating transducing (a transform applied across the input) from the folding operation itself
(aside from the previously mentioned fact that a transducer is a compositional abstraction in a way reduce isn't)
ok, your second point is about the 'transformation composition' in the FAQ, how often do you really use this? didn't understand your first one yet
the most common usage is defining the transducer as it's own testable unit of code
sometimes I use comp to join them also
(about unit of code) can comp and partial be used to create the same effect (without transducers)? ignoring the performance benefits of removing all intermediate sequence and cached value creation.
sure - but this gets heavy handed when you consider eg. transducing from a channel to another channel - the transducing version never needs to build a collection, the non-transducing version builds one collection of N elements per unit of composition
also, here's a contrived example of using the completion function
user=> (transduce (map inc) (completing
(fn [acc i] (update acc i (fnil * 1) i))
vals)
{}
[1 2 3 2 1])
(4 9 4)
- in real usage I often have "secondary accumulators" in a hash map that are needed while reducing, but not needed in the resultI tend to avoid transduce because we aren't doing things where streams of data are the perf bottleneck
but I do use transducers because they help modularize operations on IO streams to be testable on seqs, without needing to turn io into a seq
eg. we have a minimal kafka topology definition that takes a transducer, we can test that transducer outside kafka with seq in / seq out
yes, that is super helpful, experienced that with datomic, ex: developing and testing functions with no database
Anyone aware of any tools to identify/remove unused vars (not ns but mostly def and defn forms)? I can cook up a shell script to do so but would be nice if others have a tool to recommend.
there's a lein-yagni plugin if that's what you mean - it scans your codebase and tells you about functions without callers
by editing the code or calling ns-unmap, depending on what you mean
@noisesmith from memory
@ktuman I use it at work, it definitely checks for unused vars (and also is uptight about protocol methods and multimethods in ways that can be hard to make happy...)
ns-unmap is what removes a var from a namespace
@noisesmith Will give it a try. Thanks.