Fork me on GitHub
#clojure
<
2020-07-30
>
fabrao02:07:50

hello all, is there any way to hook a function and insert a delay between all others functions inside it?

jdkealy02:07:22

often when i’m using core.async, after restarting some processes or after letting them run for a while, I can’t see my log output anymore in my repl and can’t tell if my core.async processes are just not running or if the logs have just disappeared, is this a common problem and if so how do people deal with it or is it a sign of an underlying problem, too many go loops perhaps? When I start a fresh REPL, whether I run (println "TEST") or (go (println "TEST")), I can see “TEST” in my repl, but after running for a while (println "TEST") will show and (go (println "TEST")) will not.

Ben Sless05:07:40

For long running processes consider using async/thread. Sean's observation seems correct, and you can validate it by connecting with visualvm.

jrychter08:07:34

With larger applications it is too easy to run into core.async thread pool exhaustion. Depending on the app, this can manifest as a complete deadlock/hang. The advice of avoiding I/O (and too much computation) is good, but when using multiple libraries, one doesn't always control (or know) what they do. One thing I also do is increase the size of the pool using -Dclojure.core.async.pool-size=128 as a command-line argument.

seancorfield02:07:39

@jdkealy core.async uses a small, fixed thread pool I believe so it's entirely possible you're overwhelming it, if I'm understanding you correctly.

Yehonathan Sharvit06:07:56

I need help with macros. I’d like to create a macro defnmy similar to defn but that creates a function whose name is transformed by a function supplied to the macro. Example of usage:

(defnmy FOO clojure.string/lower-case [x]
    (inc x))
It should create a function named foo I was able to implement it using resolve:
(defmacro defnmy [name fn args body]
  (let [n (symbol ((resolve fn) name))]
    `(defn ~n ~args ~body)))
Is there a way to avoid using resolve?

seancorfield06:07:09

@viebel Why would you want to avoid resolve?

Yehonathan Sharvit06:07:44

1. I’d like the macro to work on self-hosted cljs 2. Understanding what macros can and cannot do

seancorfield06:07:56

The macro is passed a bare symbol. You have to turn that into an actual function, in order to apply it to the symbol passed in. Not sure how else you would do that?

seancorfield06:07:05

(I don't know anything about cljs so I don't know what restrictions it places on macros)

seancorfield06:07:43

I guess I'd also ask why you'd want a macro that is passed both a symbol and the (symbol) name of a function that is applied to it? If you're passing both, why can't you just figure out what the function name should actually be? Is this part of some code generation thing?

seancorfield06:07:59

(also, if you're going to imitate defn what about docstrings and attr-maps etc?)

seancorfield06:07:12

Sorry, no idea.

seancorfield06:07:29

I guess I wish cljs didn't have these limitations?

seancorfield06:07:58

Fixing the cljs compiler seems like a better goal to me.

Yehonathan Sharvit06:07:43

Ok. Trying on #clojurescript

leonoel06:07:37

why not eval instead of resolve ? it will work with any function definition, it will resolve the var with symbols, and it should work on self-hosted

ouvasam12:07:48

Hi, This is a question i did post on crux and ring. But the problem seems more related with deploying a war (lein ring uberwar) on tomcat. I have an init function that start a crux node but tomcat freeze once deployed. It seems to be a problem with threads but i am lost here. I wonder if someone has exeprienced this or can give me some advices ? Many thanks

SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [user-settings-webapp] created a ThreadLocal with key of type [java.lang.ThreadLocal](value [java.lang.ThreadLocal@8117ba6]) and a value of type [io.netty.util.internal.InternalThreadLocalMap](value [io.netty.util.internal.InternalThreadLocalMap@6684e4af]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

wombawomba12:07:48

Is there any way to override a particular defmethod in an external library? Or will I have to fork said library?

delaguardo12:07:14

you can override it normally as you would do for your own codebase

(ns external)

(defmulti foo ...)

(defmethod foo :x ...)

;; your code
(ns my-code
  (:require [external :as e]))

(defmethod e/foo :x ...)

wombawomba12:07:36

Oh okay, thanks! I tried doing something like that but failed… I guess I just got confused somehow 🙂

Yehonathan Sharvit18:07:27

@leonoel suggested above to replace (resolve sym) by eval (something like (eval `('var sym))`` https://clojurians.slack.com/archives/C03S1KBA2/p1596091177157900 And indeed that’s the way to do it in Self hosted Clojurescript as explained by @mfikes in https://clojurians.slack.com/archives/C03S1L9DN/p1596132644159300 Now my question is: is that acceptable to use eval instead of resolve in JVM Clojure for this use case?

mfikes18:07:07

@viebel If the function being passed is public, you can indeed further simplify as Leonoel suggested, and just evaluate the function symbol (with no worries about triggering private var access analyzer warnings)

(defmacro defnmy [name fn args & body]
  (let [n (symbol ((eval fn) (str name)))]
    `(defn ~n ~args ~@body)))

Yehonathan Sharvit18:07:06

Indeed simpler. The question remains: is that acceptable to use eval in Clojure code when we could use resolve instead?

deadghost19:07:20

Does clojure.spec work with string keys? (s/conform (s/keys :req-un ["foo"]) {:bar 1}) => {:bar 1}

dpsutton19:07:56

doc for (doc s/keys)

Creates and returns a map validating spec. :req and :opt are both
  vectors of namespaced-qualified keywords. The validator will ensure
  the :req keys are present. The :opt keys serve as documentation and
  may be used by the generator.

noisesmith19:07:09

:req-un seems like something that only make sense with things that could be namespaced (and clearly strings cannot be)

noisesmith19:07:24

@deadghost I am not 100% sure, but my hunch is that to spec a data structure with non-keyword keys, your best bet is to have a conversion function that results in keyword keys, and spec that

noisesmith19:07:51

I don't see anything about specific keys that aren't keywords in the spec guide at least

noisesmith19:07:16

(there 's map-of to describe eg. map of string to int)

Alex Miller (Clojure team)19:07:17

the attribute parts of spec do not work with string keys

Alex Miller (Clojure team)19:07:10

keywordizing first is one option and probably the best if you're looking to lean on those aspects of spec

Alex Miller (Clojure team)19:07:51

in some cases, you may be able to make do with map-of or by treating a tuple as an s/coll-of an s/or of various s/tuples describing key/value pairs

kwladyka22:07:27

why this doesn’t work (s/def ::quantity (s/and number? pos? (complement (some-fn Double/isInfinite Double/isNaN)))) Unable to find static field: isInfinite in class java.lang.Double but this works (s/def ::quantity (s/and number? pos? #(not (Double/isInfinite %)) #(not (Double/isNaN %)))) How to interop this correctly with some-fn? --- PS (s/def ::quantity (s/and number? pos? (complement #{##Inf ##-Inf ##NaN}))) ^ This is probably the nicer one. Does it?

Alex Miller (Clojure team)22:07:11

some-fn takes functions

Alex Miller (Clojure team)22:07:25

you were passing it values

kwladyka23:07:18

Double/isInfinite is a Java function. How I passing value here?

andy.fingerhut23:07:19

Java methods are not Clojure functions. Java methods can be called from Clojure programs using Java interop, but not used as arguments to functions like apply or map that expect Clojure functions.

👍 3
Alex Miller (Clojure team)23:07:41

#(Double/isInfinite %) would be a fn

Alex Miller (Clojure team)22:07:14

(s/def ::quantity (s/double-of :min 0.0 :infinite? false :NaN? false))

Alex Miller (Clojure team)22:07:31

the Inf / NaN stuff are only double values so I'm assuming you want doubles. if you want multiple number types, well why? :)

kwladyka23:07:16

Because all are valid for API request. Like it can be 3 peaces or 4,06 kg or 4 kg. The quantity field should be forgiven and not care about demand 1 or 1.0 as a value. At least this was an idea 🙂

Alex Miller (Clojure team)23:07:45

but if I want ##Inf saxophones

kwladyka23:07:41

(s/def ::foo (s/with-gen (s/and string? (complement clojure.string/blank?))
                           (partial gen/fmap (fn [_] (rand-nth ["foo" "bar"])) (s/gen string?))))
What is the best way to generate spec using only specified values like “foo”, “bar”, … ?

ghadi23:07:33

#{"foo" "bar"} as your spec

kwladyka23:07:01

but this will also accept only “foo” and “bar”. This is not what I want. I want it only to generate examples for tests.

ghadi23:07:02

(s/with-gen your-predicate #(s/gen (s/spec #{1 2 3})))

👍 3
kwladyka23:07:57

it doesn’t work like that

kwladyka23:07:13

class clojure.test.check.generators.Generator cannot be cast to class clojure.lang.IFn (clojure.test.check.generators.Generator is in unnamed module of loader clojure.lang.DynamicClassLoader @470f6965; clojure.lang.IFn is in unnamed module of loader ‘app’)

ghadi23:07:52

works for me

ghadi23:07:23

you get the idea

kwladyka23:07:37

oh I made a typo, sory

kwladyka23:07:49

ah no, you edited

hoopes23:07:31

are folks generally using schema for rest apis in (for example) compojure? i thought it would be mostly spec by now, but most of the examples i find are using schema. is there a newer (well documented) way to build rest apis with clj these days? (i haven’t done it in a couple years, looking to get back into it)

kwladyka23:07:45

Personally I use spec

hoopes23:07:55

solid. do you have anything on github i could crib off of (with your permission)?

kwladyka23:07:58

hmm I think nothing public

kwladyka23:07:09

(defn ?spec-problems [spec value]
  (::s/problems (s/explain-data spec value)))

(defn spec-problems->errors [problems spec-messages]
  (not-empty
    (for [{:keys [pred in via val]} problems]
      {:type :error
       :code :validation
       :message (some spec-messages (reverse via))
       :cause {:pred pred
               :in in
               :val val}})))

(defn ?spec->response-problems [spec value spec-messages]
  (spec-problems->errors (?spec-problems spec value) spec-messages))
This can be helpful

kwladyka23:07:39

(def spec-messages
  (let [string-message "Has to be a string."]
    {::code "Has to be a string."
     ::name string-message
     ::expiry-days "Has to be a positive integer."
     ::label-template string-message
     ::ingredients string-message
     ::countries-of-origin string-message
     ::quantity "Has to be a positive number."
     ::unit string-message
     ::lang "Has to be ISO 639-1 language code."}))

(defn ?body->spec-problems-response [request-body]
  (s-utils/?spec->response-problems ::labels-orders request-body spec-messages))

kwladyka23:07:53

shortly all of this is to figure out what failed and return human readable error

kwladyka23:07:17

but also with tech hints about which value and why

hoopes23:07:48

if you have a map of data (say, an account) do you coerce to the result spec by hand? i thought compojure would coerce the output to the spec i provide

kwladyka23:07:06

while it is not strict in topic https://clojure.wladyka.eu/posts/form-validation/ it shows how to turn spec into human readable things

kwladyka23:07:47

I use ring + bidi. I do coerce in code. Let me find something,

kwladyka23:07:06

(defn ->spec [shop]
  (s-utils/map-coercion
    {:shop/uuid uuid/as-uuid
     :shop/config json/read-str}
    shop))

kwladyka23:07:50

(defn safe-coercion [f v]
  (try (f v)
       (catch java.lang.Exception e
         v)))

(defn map-coercion [m-coercion m]
  (reduce-kv
    (fn [m k f]
      (if (contains? m k)
        (update m k (partial safe-coercion f))
        m))
    m m-coercion))

kwladyka23:07:02

it can give you some ideas

kwladyka23:07:18

sorry I don’t have it organized better to share with you

hoopes23:07:27

no worries, i’m copy/pasting all this now. thanks so much for taking the time!

kwladyka23:07:52

I am going sleep. 2am for me. Goodnight ! Happy to help.

hoopes23:07:00

good night!

vemv00:07:55

I have successfully used https://github.com/wilkerlucio/spec-coerce in APIs. Note that using spec for coercion is not considered particularly idiomatic/performant. I do think it can be "good enough" for a good while. Probably the metosin guys can provide the best solutions - check their repos: reitit, malli

hoopes00:07:38

https://github.com/metosin/spec-tools/blob/master/docs/01_coercion.md I found this in the metosin tools, which seems to be ok so far…