Fork me on GitHub
#clojure
<
2020-04-06
>
Mitch Dzugan08:04:53

Is it possible to wrap the clojure.core.match and cljs.core.match macro with a single macro in a .cljc file that will use the correct one in the correct runtime? I am currently trying this:

;; mylib.match.cljc
(ns mylib.match #?(:clj (:require [clojure.core.match :as m])
                   :cljs (:require-macros [cljs.core.match :as m]
                                          [mylib.match :refer [match]])))

#?(:clj (defmacro match [& args] `(m/match [email protected])))
;; mylib.core.cljs
(ns mylib.core
  (:require [mylib.match :refer [match]]))

(println (match 2 1 true 2 false))
which works from .clj but when compiling .cljs I get
WARNING: No such namespace: clojure.core.match, could not locate clojure/core/match.cljs, clojure/core/match.cljc, or JavaScript source providing "clojure.core.match"
on build and
TypeError: Right-hand side of 'instanceof' is not an object
on run. I assume this is because the macro compilation pass will be following along the #?(:clj reader path so it includes the clojure.core.match macro in the .cljs source.

pithyless08:04:16

One common approach is to define a helper:

(defn cljs-env?
  "Take the &env from a macro, and tell whether we are expanding into CLJS."
  [env]
  (boolean (:ns env)))

(defmacro match [& args]
  (let [cljs? (cljs-env? &env)]
    (if cljs?
      `(cljs.core.match/match [email protected])
      `(clojure.core.match/match [email protected]))))

pithyless08:04:52

And then export the macro for CLJS in the ns:

(ns mylib.match
  #?(:cljs (:require-macros [mylib.match]))
  (:require
   #?(:clj [clojure.core.match]
      :cljs [cljs.core.match])))

g7s08:04:55

If you do a lot of that I would recommend macrovich https://github.com/cgrand/macrovich it helps a lot

Mitch Dzugan08:04:07

awesome thank you both!

Mitch Dzugan09:04:21

macrovich is indeed a lifesaver

Saikyun10:04:23

is it possible tell deps.edn to always (require '[x :refer [y]]) ? e.g. if I want access to clojure.pprint/pp in the repl

teodorlu10:04:28

Does putting

(ns user
  (:require [clojure.pprint :refer [pp]]))
in your project's src/user.clj or dev/user.clj work for you?

teodorlu10:04:20

I usually copy Duct's setup[1]: • In user.clj, there's a (dev) function that swaps over to (ns dev) • In dev.clj, I require project dependencies, and create a (go) function that starts my system (eg. with Integrant) • Any extra functions like pp go in dev.clj Reason for user/dev split: I can start a REPL even if my project doesn't compile. [1]: https://github.com/duct-framework/docs/blob/master/GUIDE.rst#starting-the-repl

Saikyun10:04:25

ah thanks, that works 🙂 didn't know about the user-ns

Saikyun10:04:13

the dev-split sounds interesting as well. thanks for the info

plins12:04:29

is pmap well suited for parallel IO? like firing 100 http requests?

delaguardo12:04:50

not really. It is for computaitionally intensive tasks, that is mentioned in pmap’s docstring.

delaguardo12:04:21

https://clojure.org/about/concurrent_programming here you can find a good starting point for your solution

nick13:04:15

@U3QUAHZJ6 you may also find this video helpful on this topic https://www.youtube.com/watch?v=eRq5UBx6cbA

plins13:04:51

thanks 🙂

uosl13:04:03

Is there a way to call a specific namespace function in an uberjar, passing it arguments from the command line when invoking it with java -jar myuberjar.jar?

flaie14:04:02

java -cp myuberjar.jar clojure.main -m my-project.core ?

uosl14:04:33

Thanks, I'll give that a try!

flaie14:04:58

If you're using ubderdeps you can also create an executable JAR by giving it the --main-class

noisesmith15:04:52

@UE27GJZR8 for that usage you want -cp instead of -jar, otherwise that is correct

noisesmith15:04:29

also this doesn't work with Capsule jars (the solution is to not use Capsule)

flaie15:04:48

Yes you're right you need -cp

rovanion13:04:00

Slack's input field is not what it used to be...

seancorfield17:04:16

No, clojure.tools.cli doesn't. I don't know how widely applicable adding parsing might be for that case, since the most common type of positional parameter is likely to be a filename?

seancorfield17:04:11

@U1QMBJY01 Feel free to describe your use case here https://ask.clojure.org/index.php/activity/contrib-libs/tools-cli and we can have a discussion about it (that's the jumping off point for issues that get into JIRA for Contrib libs).

rovanion07:04:12

Will do. And my usecase is exactly that, I've got two files as positional arguments and it would be nice if I could get precisely that verified in http://tools.cl

seancorfield15:04:56

Thanks. Will take a look and have a think about it.

introom15:04:36

I don’t understand the reader time error for dups in set and map.

introom15:04:59

#{(rand) (rand)} such a reader-check provides what meaning?

hiredman15:04:30

The reader produces data structures

ghadi15:04:27

(in this case, the reader produces a set with two lists in it, each containing a symbol)

introom15:04:47

as long as at the evaluation time, two (rand) give different value then it would be fine. the reader checker seems redundant.

alexmiller15:04:49

it is redundant, but also useful in some cases. and it's not possible for the reader, pre-eval, to determine whether it's one of those cases or not

alexmiller15:04:33

it is literally non-sensical to read #{1 1} - that is not a valid literal set

alexmiller15:04:03

that accidentally ensnares cases like #{(rand) (rand)}

alexmiller15:04:43

but a) those cases are rare, b) it is not generically possible to determine the difference pre-eval, and c) if you really want to do that, use (hash-set (rand) rand)) instead

hadils19:04:43

Does anyone have any suggestions for using core.async instead of callbacks for processes that may take hours or days?

phronmophobic19:04:50

the only suggestion I can think of is you’ll probably want to create and manage your own threads. I’ve found that the default threadpool core.async used by go blocks usually doesn’t give you right control for long running applications.

noisesmith19:04:23

for things that run for days, you probably want something outside the VM to store reloadable state, nothing in clojure directly addresses this

noisesmith19:04:02

adding to what @U7RJTCH6J says - not only is the thread pool for go loops relatively inflexible, it's specifically not intended for long running tasks, and you can break core.async by using it that way (async/thread provides a relatively easy escape hatch for that though)

phronmophobic19:04:17

you can also create threads pretty easy with

(def my-thread (java.lang.Thread. (fn [] ...)))
;; start with .start
(.start my-thread)
you can then check if .isAlive and a few other things.

hadils20:04:23

do you know how many blocked threads I can have in Java 8 -- or how much memory per thread would be used up (might be a better measure).

phronmophobic20:04:25

it probably depends on the OS, but I think the number of blocked threads can be a lot. you don’t have to create a new thread for each task. you can do something like

(defn start-task-runner [ch]
  (java.lang.Thread.
   (fn []
     (loop [work (<!! ch)]
       ;; stop when channel is closed
       (when work
         (work)
         (recur (<!! ch)))))))
(def worker-ch (chan))
(def my-thread (start-task-runner worker-ch))
(>!! worker-ch (fn [] (stuff-to-do!)))

phronmophobic20:04:31

one other note about creating your own threads is that you’ll probably want to do something similar about thread bindings that the async/thread macro does

(defn thread-call
  "Executes f in another thread, returning immediately to the calling
  thread. Returns a channel which will receive the result of calling
  f when completed, then close."
  [f]
  (let [c (chan 1)]
    (let [binds (clojure.lang.Var/getThreadBindingFrame)]
      (.execute thread-macro-executor
                (fn []
                  (clojure.lang.Var/resetThreadBindingFrame binds)
                  (try
                    (let [ret (f)]
                      (when-not (nil? ret)
                        (>!! c ret)))
                    (finally
                      (close! c))))))
    c))

(defmacro thread
  "Executes the body in another thread, returning immediately to the
  calling thread. Returns a channel which will receive the result of
  the body when completed, then close."
  [& body]
  `(thread-call (^:once fn* [] [email protected])))

phronmophobic20:04:21

I think memory per thread is also OS dependent.

hadils20:04:44

Thanks for your replies @U7RJTCH6J!

noisesmith21:04:29

there's a lot of advantage to not using go blocks to execute tasks, but I don't really see the benefit to using Thread directly instead of the expanding / cached pool that async/thread uses

phronmophobic21:04:32

there may not be. having the Thread itself gives you access to .isAlive , .getStackTrace , .join and .interrupt which you could implement on top of the async/thread, but it’s useful to get those for free. you might not need those for your production environment, but they can be really useful for development and debugging. you might be able to get access to thread even if you use async/thread , but I don’t think it’s a good idea to rely on async/thread using any particular Executor.

emccue21:04:28

@UGNMGFJG3 when I had to do some long running java tasks I had some logging/recovery/alarm logic on the outside of the thread and I had everything pushed to a queue that the thread consumed from

noisesmith21:04:31

@U7RJTCH6J fair enough, but clojure's future lets you use all of those, plus conveying bindings and using a nice caching executor

noisesmith21:04:18

and you can get the Thread object from inside any context via (Thread/currentThread)

emccue21:04:49

I tend to think core/async will be superceded by jvm fibers if we are alive when they come about

emccue21:04:16

Which kinda colors my lack of usage for things I think I'll need to maintain

emccue21:04:37

To take anything I say with a grain of salt

phronmophobic21:04:22

yea, it’s probably better to start with future and async/thread and you can switch to holding onto Thread references if you need it. where i’ve found keeping a reference to the thread useful in the past is that I’ve been doing tasks in the background and at some point, one of the background tasks drops the ball and I need to figure why. if I have a list of the threads, then I can more easily inspect them and build dev tools around it.

phronmophobic21:04:06

getting access to the current thread is less useful for what I’m thinking about since it’s usually some other thread that isn’t doing what I want

phronmophobic21:04:02

holding onto Thread references is mostly about being able to build a nice dev workflow that makes dev and debugging easier

noisesmith22:04:12

I was thinking of a pattern like (let [executor (promise) thread-c (async/thread (deliver executor (Thread/currentThread)) ...)] ...)

noisesmith22:04:26

which is clumsy but not hard to add in during dev