Honestly, I mostly only remember those specifics because I reimplemented agents over web workers a while back. Turned out, atoms and immutable data solved 90% of our concurrency issues.

$2^e$

I think you are looking for #perl 😉

Those last 10% though are usually solved by seasoned java devs, so I think they're more inclined to reach for native thread tools, instead of using agents (or even STM)

Yeah, I have to say that includes me John. I can get most of this stuff done with core.async now, but I spent long enough in Java land to find reaching for java.util.concurrent a bit too easy. Nearly suggested a Semaphore for problem above.

Yeah, I think we should try to stay between the lines more often than we do there. Atoms and agents and stm all work in concert with one another wrt the snapshotting of data across a transformation. Seeing folks muddy the semantics makes me sad. But sure, sometimes you'll have to reach for a more specific tool.

+1 for agents in this use case. My project uses an agent in one particular spot. We routinely schedule a task but want only one to run at a time. Works like a charm.

yep, I can confirm that it works well in my case as well, also much easier to implement than I thought, thank you guys.

Interested to know how you solved the problem of communicating back to the caller. Agents do seem a little, "fire and forget". The agent data will get updated with the return 'at some point in the future', and you do have (await) so I suppose if you do something like keep a vector of successful reservations you can (await) to see if it's there, but aside from your constantly growing vector there, (await) docs contains some concerning statements like, "Will never return if..." 🙂

Pass in a closure over a promise that gets delivered when the function executes on the agent? 🙂

I was thinking that too, but was afraid that was just the first step to a bad rube goldberg machine - but perhaps it is that simple

Hah, I have in my head a core.sync solution that looks a bit like that with channels.

(or just use an atom and compare and set)

I mean what if the agent blows up so the promise delivery never happens, now your thread is deadlocked, so then you put the deref of the promise in a future so you can use deref of that with a timeout and then...

Concurrency is hard

this is why people use Node.js 😉

Haha! Touchè!

pffft

the problem with people who write javascript is the tend to think that using a single threaded server and single threaded client will save them from thinking about concurrency

of course that is already two threads in your system operating concurrently

then you get more clients and more servers, and databases

@hiredman Datomic team also think this ;)

pfft

what makes you think they think that?

I’m partially joking, but lots said about transactional benefits of single writer.

but the system is entirely designed thinking about concurrency

and clojure's reference model

datomic is a database as a value in an atom

Indeed, but writes are single threaded in part so you don’t have to worry about isolation

Are they still actually?

Not sure with cloud

anyway, my point is, programming in javascript will damage your brain when it comes to concurrency, and concurrency is a fact

I am definitely not a JS evangelist.

At least probably not, not sure what this weeks syntax is.

“JavaScript will damage your brain” <- next tattoo

when it comes to concurrency

I agree. it was tongue-in-cheek 😛

the truth is that while your language might be single threaded, your system is not

at least JS in the browser lets you not have to think about problems locally without concurrency issues. but too many people try and venture farther and run into issues early on

I was one of them!

it gets even harder with threads, because of shared memory

(unless you have Clojure's fancy atoms/refs 😉 )

...or core.async

That was my approach after looking at my options.

Go to #slack-help or #community-development and alert an admin in either of those channels. Do not engage the trolls/spammers here please.

is there any way to add hint for returned value from function which is defined using fn?

user=> (set! *unchecked-math* :warn-on-boxed)
:warn-on-boxed
user=> (defn abc ^double [] 1.0)
#'user/abc
user=> (inc (abc))
2.0
user=> (def abc* (fn ^double [] 1.0))
#'user/abc*
user=> (inc (abc*))
Boxed math warning, C:\cygwin64\tmp\form-init496459974570346569.clj:1:1 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object).
2.0

Did you know? You can put pre-post map in meta 😱

(fn ^{:pre [(pos? i)]} [i]
  (inc i))

(macroexpand '(fn ^{:pre [(pos? i)]} [i]
  (inc i)))
(fn* ([i] (clojure.core/assert (pos? i)) (inc i)))

cool, didn’t know

ok... found a solution

(def ^{:arglists '(^double [])}
  abc* (fn ^double [] 1.0))

let's go deeper

what about such case?

user=> (defn make-me-aaa [] (fn ^double [] 1.0))
#'user/make-me-aaa
user=> (let [aaa (make-me-aaa)] (inc (aaa)))
Boxed math warning, C:\cygwin64\tmp\form-init496459974570346569.clj:1:26 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object).
2.0

you can't avoid boxed math while crossing anonymous function boundaries

clojure only knows how to use the unboxed path when using primitively tagged functions stored in Vars

so to your initial question.. no

ok...

invokedynamic could help with this if it ever lands in clojure \cc @ghadi

What's the status of the functions in the Clojure incubator? I'm mainly interested in dissoc-in. It looks like there's been no activity in incubator or on the issue (https://dev.clojure.org/jira/browse/CLJ-1063) in a while. I think it would be nice to have in core so I can stop putting a copy into each of my projects 🙂

create a package and then just pull that down! hah

incubator is dead

I think my comments at the end of 1063 summarizes my qualms about it

I don’t think there is a candidate impl that I would consider worth recommending right now

Gotcha

Does anyone else crash/hang Docker (on Mac) when running lein uberjar? Last night it just hanged. Now it terminated with ERRO[0457] error waiting for container: EOF. I even bumped it to use 6 CPUs, 8 GiB RAM & 3.0 GiB swap.

(It creates the ...-SNAPSHOT.jar then hangs/crashes.)

@tjg is your uberjar running across a mounted volume?

sometimes it's painfully slow if $HOME/.m2 or the app itself is mounted

but if other containers are having issues, it's good to hit the Reset button in Docker for Mac preferences

@ghadi Hmm, thanks — I believe it must be! (So the uberjar remains after the Docker dies. I didn't write this codebase, so I'll verify.) (I've reset a few times, so unfortunately that doesn't work. :/)

yeah, Docker for Mac filesystems mounts are quite slow. If you watch -n 1 du your-uberjar.jar you may see that it progresses, but at a crawl

Hmm, in that case I wonder if it makes sense to run Docker inside VMWare/Debian (atop MacOS). 😛

@dts.siedel medley has dissoc-in: https://github.com/weavejester/medley/blob/master/src/medley/core.cljc#L20

I usually write update-in + dissoc because I’m too lazy to use an extra dependency

Greetings fellow clojurians! Does anyone have experience with creating a jpeg file from a base64 encoded string extracted from html? (Inline image in html) I managed to get those image strings from a html page with hickory, but after that I tried to decode them with (String. (.decode (Base64/getMimeDecoder) to-decode)) and http://clojure.java.io/copy them out to a folder with no success. When I try to open an image my viewer says: (Not a JPEG file: starts with 0x75 0xab) I am pretty new to this, so got the idea to ask it here. I hope someone could help me. Thank you in advance!

@mindenaaron Could you post the base64 encoded string in a gist?

@borkdude Sure, https://gist.github.com/aaronpowered/6e2f2de74e2de9be7da52889c5452f4d (Have to leave now, will be able to reply after 3 hours.)

I noticed there was a trailing slash in the base64 string. I recommend testing it in an online viewer first: e.g. https://codebeautify.org/base64-to-image-converter

Hi, I'm doing some interop with Java and I dont understand why (.write javax.imageio.ImageIO bi "jpg" f) works but not (.write javax.imageio.ImageIO bi "jpg" f) ? Before I did (.fillRect g2d 20 50 10 30) and it worked fine

Is it because ImageIO is a class and not an instance?

My question is about the interop syntax

(.write javax.imageio.ImageIO bi "jpg" f)
(.write javax.imageio.ImageIO bi "jpg" f)

(unless i missed something?)

@julien.rouse did you figure out what was going on?

@dpsutton hey sorry I was out, I meant (. javax.imageio.ImageIO write bi "jpg" f) is the correct way and (.write javax.imageio.ImageIO bi "jpg" f) didnt work

for a static method, (javax.imageio.ImageIO/write bi "jpg" f) is the recommended syntax

@julien.rouse you can see from the javadoc which methods are static, https://docs.oracle.com/javase/8/docs/api/javax/imageio/ImageIO.html and the clojure docs for interop describe static method calls https://clojure.org/reference/java_interop#_the_dot_special_form

Thanks for the informations

but why would (.fillRect g2d 20 50 10 30) works and not (.write javax.imageio.ImageIO bi "jpg" f) ? Is that because write is static?

right, fillRect is a method on g2d, write is a method on javax.imageio.ImageIO, the rules for static vs. instance methods are different

in java they have the same syntax, but on the bytecode level they are implemented differently, and clojure distinguishes them

Right, thanks for the clarification, It's an area to improve for me!

interop is a pretty small set of rules, and once you learn them you can use a lot of stuff for free, so it's rewarding :D

Yeah that's why I was getting my hands dirty with it, leveraging Java libs is a great bonus to the already nice language

Thank you again for your help ! 🙂

Hielo all. I'm doing some experiments with streams (but not touching IO yet), but getting the following error randomly in my lein repl, anyone have a clue why?

IOException Resource temporarily unavailable
        java.io.FileInputStream.readBytes (FileInputStream.java:-2)
        java.io.FileInputStream.read (FileInputStream.java:255)
        java.io.BufferedInputStream.fill (BufferedInputStream.java:246)
        java.io.BufferedInputStream.read (BufferedInputStream.java:265)
        jline.internal.NonBlockingInputStream.run (NonBlockingInputStream.java:294)
        java.lang.Thread.run (Thread.java:748)

does *e have the full stack trace?

did you open a file? if so how did you open it / access it?

nope, no open files. Not sure I can see what *e has as the process crashes when it happens

unless aleph/manifold automatically open files, I would hope it doesn't

you could set the top level uncaught exception handler to be more informative (Thread/setDefaultUncaughtExceptionHandler (reify Thread$UncaughtExceptionHandler (uncaughtException [this thread error] (prn "uncaught error\n" thread error)))) (fixed error, added example)

(ins)user=> (Thread/setDefaultUncaughtExceptionHandler (reify Thread$UncaughtExceptionHandler (uncaughtException [this thread error] (prn "uncaught error\n" thread error)))
)
nil
(ins)user=> (.start (Thread. #(/ 1 0)))
nil
user=> "uncaught error\n" #object[java.lang.Thread 0x2d631a09 "Thread[Thread-0,5,main]"] #error {
 :cause "Divide by zero"
 :via
 [{:type java.lang.ArithmeticException
   :message "Divide by zero"
   :at [clojure.lang.Numbers divide "Numbers.java" 163]}]
 :trace
 [[clojure.lang.Numbers divide "Numbers.java" 163]
  [clojure.lang.Numbers divide "Numbers.java" 3833]
  [user$eval215$fn__216 invoke "NO_SOURCE_FILE" 40]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 748]]}

hm, might be related with autocomplete or something like that, as the error seems to happen when I write (def my-stream (s/stream) (notice the missing ending ))

oh, thanks for the snippet, I'll give that a try

hm, unfortunately that doesn't make the error any clearer https://gist.github.com/victorb/304ed4cd9834ecb02ce1b37f641e7957

so this happens before you even finish writing the form? what repl implementation?

yup, using lein

wait, just a regular lein repl in a terminal?

it crashes as soon as I finish the (s/stream) form

seems related with some autocomplete

you can use lein upgrade $VERSION to pick an arbitrary older version of lein - if it's a lein bug I bet going back a version or two would eliminate the error

nrepl stuff was radically changed recently

(and of course having the latest lein is just a question of using lein upgrade again)

good idea, I'm on 2.8.1 so I'll try upgrading before downgrading. Thanks for the suggestion

Cider in the mix?

no cider, just lein repl + one deadly form (s/stream)

also OS mgiht be relevant here too (since IO bugs in IPC code that only affect one OS are relatively common...)

upgraded to latest (2.9.0) and now it works

🎉

I'm on Ubuntu 18.04.1 LTS for posterity

thanks @noisesmith for helping me out 🙂

@borkdude trailing slash was only a mistake of copy paste. I had another mistake, had to drop the head of the string, (data:image/jpeg;base64) so i can use Base64/getDecoder instead of Base64/getMimeDecoder, but image file is still not available to open. I leave this image here as the difference between my decoded file and the one decoded by Best Online Base64

BTW What is this nonsense, Slack? "Your file was uploaded — it’s safe and sound in Slack. Unfortunately your workspace doesn’t have any storage space left. To get more space, you can upgrade to a paid account or delete some of your older files." Can you guys see my recently uploaded picture?

yes - I can see it - that difference almost looks like text encoding issues, which makes me wonder if you wrote the file using a String / text oriented function rather than a bytes / raw data function

Yes can see it

@noisesmith @julien.rouse Thank you! Yes good point

@mindenaaron OK I scrolled up - your mistake is using the String constructor

pass the byte array to io/copy, this isn't textual data

aaah

Thank you again @noisesmith

I have to stop just copy pasting utility functions

@mindenaaron one of my loose heuristics for bugs is doing any sort of String conversion during IO - text to text just works via the binary methods, string conversion in the middle breaks things

Finally it works! You guys are always doing a great job here 🙂 I am very grateful

I just wrote a function which extracts images from the output of a wysiwyg, make sha1 hash and store every image once and change the inline base64 string to the path to that image. It works brilliant 🙂

nice

Could someone explain to me what is the common use of var in Clojure?

You’ll often see it used via the reader macro #'. Usually to alter the metadata associated with a Var, or sometimes in test namespaces to refer to private vars

@nwjsmith Thanks, I didn’t realise that metadata came with a var, not the associated object.

cross-posted from #luminus: does anyone know specifically where to put wrap-cors in a luminus app (which has swagger, if that matters)?

my various attempts in handler and middleware aren't working.

I need cors enabled locally for development on an app created with the equivalent of 'lein new luminus testapp +swagger'.

@avfonarev another usage of var is to pass the container holding a function to a function that creates a long running closure (eg. starts a web server using the function you pass in) - if you pass a function, the higher order function won't see redefs, if you pass the var it always sees the current value

the thing that makes this work is that calling a var attempts to call the value you put in the var

of the 3 Design / usage benefits of transducers: in https://clojure.org/guides/faq#transducers_vs_seqs - don't the last two apply to reduce or loop/recur too?

unlike transducers, reduce and loop/recur are syntaxes rather than units of composition

so they are eager and explicit about resource usage, but aren't in the same category of abstraction

@noisesmith yeah, that is why I said 'the last two' - excluding transformation composition

both transducers and sequences can be passed to functions to get a new transducer or sequence

right - what I'm saying is they aren't even in the same category - strings and io streams are also eager and explicit about resource usage, but would be out of place in the comparison

@noitcudni Actually, it’s a cool usage. Thank you

@noisesmith Ok, I see what you mean, yes, they are not the same, was specifically curious what do they have over reduce specifically and only for eagerness and resource control

transduce vs. reduce has two major improvements: formalizing a final "collate results" step after everything has run (very common pattern in my code to need that), and separating transducing (a transform applied across the input) from the folding operation itself

(aside from the previously mentioned fact that a transducer is a compositional abstraction in a way reduce isn't)

ok, your second point is about the 'transformation composition' in the FAQ, how often do you really use this? didn't understand your first one yet

Or your two improvements seem highly related

the most common usage is defining the transducer as it's own testable unit of code

sometimes I use comp to join them also

(about unit of code) can comp and partial be used to create the same effect (without transducers)? ignoring the performance benefits of removing all intermediate sequence and cached value creation.

Anyone aware of any tools to identify/remove unused vars (not ns but mostly def and defn forms)? I can cook up a shell script to do so but would be nice if others have a tool to recommend.

there's a lein-yagni plugin if that's what you mean - it scans your codebase and tells you about functions without callers

how do you remove an unused var manually?

by editing the code or calling ns-unmap, depending on what you mean

last time I check yagni does not detect vars but ns only.

@noisesmith from memory

@ktuman I use it at work, it definitely checks for unused vars (and also is uptight about protocol methods and multimethods in ways that can be hard to make happy...)

ns-unmap is what removes a var from a namespace

@noisesmith Will give it a try. Thanks.