Fork me on GitHub
#beginners
<
2020-08-31
>
vlad_poh00:08:20

Howdy y'all why would lein run and lein repl work but the jar from lein uberjar not work

dpsutton00:08:12

not much to work on but one common issue is using io/file at the repl and then not being able to find that file on the filesystem when uberjared

vlad_poh00:08:24

yep. It has been a frustrating 3 days. resolved the file not found issues. now i get a blank report but no errors. don't know know what to do next. so tempted to just use lein run in prod

seancorfield01:08:12

Using lein run in production is a really bad idea.

seancorfield01:08:46

@kbosompem You need to explain exactly what you mean by "not work". What specifically works in lein run/`lein repl` but "does not work" with an uberjar? What exactly does "not work" mean?

seancorfield01:08:09

If you can't describe failures with more detail, you won't be able to debug them.

vlad_poh01:08:28

Agreed! I am trying to embed jasper reports into my web app. This repo https://github.com/sventechie/jasper-reports-clj got me pretty far where i'm able to run my reports with lein run /lein repl. When i create a jar and run the app I get file not found errors so added them to my resource folder. the errors went away but the reports are empty. No error messages so all i've done for the last 72 hours is blind trial and error.

seancorfield02:08:18

Have you watched Stu Halloway's "Running With Scissors" or "Debugging With The Scientific Method"?

👍 3
seancorfield02:08:23

You need to be more methodical about your debugging -- and about explaining the problem and also reducing the problem to a minimal repro case so that others can help you.

👍 6
Joel04:08:37

Writing my first macro, and obviously misunderstanding when quoting is needed I think, as it's not evaluating as I expected:

(defn fn-for-pattern [pattern]
  (map
    #(if (= '_ %)
      (fn [x] true) 
      (fn [x] (= x %))) pattern))

(defn apply-fn-to-args [args fns]
  `(map #(apply % args) fns))

(defmacro matchm
  [args & rest]
  `(remove nil? 
    (->> (quote ~rest)
      ;; turn patterns into functions
      (map #(%1 %2) (cycle [fn-for-pattern identity]))
      ;; apply functions to args
      (map #(%1 %2) 
        (cycle [(partial apply-fn-to-args (quote ~args)) identity]))
      ;; decide if result in list
      (partition 2)
      (map #(if (every? true? (first %)) (second %)))
    )))

(matchm [:a :b]
  [:a _] "x"
  [_ :b] "y"
)

seancorfield04:08:25

@joel380 I would recommend figuring out how to do this entirely with functions and quoted arguments first, and then figure out what the macro should look like.

seancorfield04:08:00

Based on the code above, I think you're trying to do far too much at runtime with the symbolic forms you have.

seancorfield04:08:59

If you can construct a solution as a pure function with fully quoted arguments, you'll be a lot closer to the macro you need.

seancorfield04:08:34

(as a general rule, avoid macros -- don't use them except for a thin syntactic sugar layer over functions)

seancorfield04:08:43

In your case, your rest expression is a set of pairs of patterns and expressions, and that should be processed/expanded before the quoted form.

Joel04:08:28

here it is working using nil instead of underscore to avoid a macro (first pass)

(defn test-list [pattern]
  (map
    #(if (nil? %)
      (fn [x] true) 
      (fn [x] (= % x))) pattern))

(defn test-list-apply [test-list args]
  (->> args
    (map vector test-list)
    (map #(apply (first %1) (vector (second %1))))
    (every? true?)))

(defn match-list [args & rest]
    (->> rest
      (map #(%1 %2) (cycle [test-list identity]))
      (partition 2)
      (map #(if (test-list-apply (first %) args) (second %)))
      (remove nil?)
    ))

(match-list [:one :a]
  [:one nil] "one"
  [:one :a] "suba"
  [:one :b] "subb")

Joel04:08:33

I think the only part that can be "precomputed" is the creation of the functions.

seancorfield04:08:29

OK, good. So now you can write a macro that just changes _ to nil in the pattern parts and then invokes the function.

seancorfield04:08:35

That's your "syntactic sugar"

Joel04:08:25

not quite, right... I don't want to match on something that is actually nil.

seancorfield04:08:59

So adjust your function so that nil matches "anything"

Joel04:08:41

but need to distinguish between "anything" and actually nil.

seancorfield04:08:55

Then use a unique qualified keyword.

seancorfield04:08:12

This isn't a macro problem, it's a function problem.

seancorfield04:08:23

Macros just provide syntactic sugar.

Joel04:08:32

i see underscore used as a standard to mean "anything".

Joel04:08:42

seems odd to invent something new.

seancorfield04:08:51

::anything is fine for this purpose.

Joel04:08:04

other than i don't learn anything 😞

seancorfield04:08:25

The first rule of macro club is ... don't use macros 🙂

😄 3
sova-soars-the-sora14:09:45

quoted and immortalized! 😃

👍 3
Joel04:08:28

trying to grasp macros.

Joel04:08:58

i think most of the stuff I'm using in clojure actually is macros.

seancorfield04:08:03

Macros are "just" functions of symbolic forms.

seancorfield04:08:24

So you need to learn to write them as actual functions (of symbolic forms) first.

Joel04:08:28

i couldn't use most things in clojure, eg. clara rules, or clara.logic, or a bunch of stuff.

seancorfield04:08:56

Using macros doesn't mean you know how to write macros.

seancorfield04:08:19

(and macros don't compose, so in general they should be avoided)

Joel04:08:27

i didn't follow.... "If you can construct a solution as a pure function with fully quoted arguments"

Joel04:08:22

yeah, so looking for someone that can help me understand where my defmacro is falling short.

seancorfield05:08:29

I don't know how to guide you otherwise: don't write macros, write functions of symbolic expressions and then -- and only then -- write macros to provide syntactic sugar.

seancorfield05:08:56

A macro is a function of symbolic expression to symbolic expression. That's all a macro is.

seancorfield05:08:31

(other than saying "macros are hard: beginners should not write macros")

phronmophobic05:08:21

I think an example might help. not sure this is a good example

(defn inc-numbers [form]
  (map (fn [form]
         (if (number? form)
           (inc form)
           form))
       form))

> (inc-numbers '(+ 1 2 3))
(+ 2 3 4)

(defmacro inc-numbers-macro [form]
  (inc-numbers form))

> (inc-numbers-macro (+ 1 2 3))
9

seancorfield05:08:23

(as a sanity check, with over 100,000 lines of production Clojure, we have just 28 macros)

3
clj 3
phronmophobic05:08:39

inc-numbers is just a regular function that takes a list. you can see that when I call it with '(+ 1 2 3), I "quote" the argument by putting ' in front.

phronmophobic05:08:05

to make the macro, inc-numbers-macro I just just call the normal function with the form

phronmophobic05:08:02

even if you don't write a lot of macros, I think it's still both fun and useful to learn how they work. you read all these blog posts about how lisp is so cool because lisp has macros, but then when you try to learn them, everyone just tells you not to use them.

Joel05:08:29

but in my case, i'm attempting to use the underscore _. I'd need to do something more sophisticated i think in that case, right?

seancorfield05:08:23

Right, but you have to have something that means "match anything". Underscore is just a piece of syntax. You need to make the function work first.

phronmophobic05:08:42

here's an updated example that replaces _ with 42:

(defn inc-numbers [form]
  (map (fn [form]
         (cond

           (number? form)
           (inc form)

           (= form '_)
           42

           :else
           form))
       form))

> (inc-numbers '(+ 1 _ 3))
(+ 2 42 4)

(defmacro inc-numbers-macro [form]
  (inc-numbers form))

> (inc-numbers-macro (+ 1 _ 3))
48

Joel05:08:18

k' i need to go back then and look at mine again... u make it look easy 🙂

Joel05:08:22

@seancorfield --- do you dynamically load clojure in your runtime? eg. upload into running environment?

seancorfield05:08:14

Yes, we have Socket REPLs running in production and we sometimes eval new versions of functions into them.

seancorfield05:08:40

Locally, yes, all the time we load updated functions into our local runtime.

seancorfield05:08:42

(that has zero to do with macros tho' so I'm curious as to why you ask @joel380)

Joel14:08:43

interested in loading clojure files into production in real-time, and what issues might arise, esp. when making mods to the same file.

seancorfield17:08:14

@joel380 Not sure what you mean about "making mods to the same file"?

seancorfield17:08:02

Clojure's compile/execute model is identical everywhere, as far as compiling top-level forms (as part of loading a file).

seancorfield17:08:57

If you use direct linking when you AOT compile, you have to be aware that you'll often need to load more code to get a function update to be seen (you'll have to get Clojure to recompile all the call sites, since direct linking removes the usual indirection that occurs on calls, and you'll have to do that all the way up the call tree). But direct linking is not the default -- you have to opt in, so you know when you are doing it 🙂

Joel19:08:29

I'm looking to replace drools with clojure, and right now we store the drools files in the DB. Likewise, I want to handle clojure, with files being loaded from DB. Wasn't sure what kinds of issues that might lead to. Right now, it takes a while to load all the drools files. At the end of the day it would be ideal to ship "as compiled as possible" with a jar, and only update the ones updated dynamically.

seancorfield19:08:42

Most of the code-loading machinery in Clojure assumes you have an actual file on disk, and it has to match in name/directory structure with the namespace. However, you can certainly read and eval arbitrary chunks of code such as you might read out of a database. You'll have to be careful about ensuring definitions end up in the namespace(s) you want, e.g., wrapping them in (do (in-ns 'my.namespace) <chunk of code>) which is the sort of thing editors tend to do when evaluating individual forms into the REPL.

Joel16:09:14

so thar be dragons? Was hoping to simply load a clojure file and call a preknown/well defined function in it.

seancorfield16:09:30

If it is a file on disk, then you can easily load-file on it, regardless of path (`require` assumes a mapping between source files and namespaces). If it's code in a DB field, you can read and eval it but there are some "dragons" you might trip over.

Joel16:09:54

definitely needs to be DB.

Joel16:09:30

uploaded from web... Right now that's how we deploy drools file, the goal would be to replace those with clj.

Joel17:09:22

although, possibly some way to dynamically compile, and distribute to servers from there, eg. jar.

Joel17:09:51

not sure where to find info, as someone must have already accomplished this.

seancorfield17:09:27

Most of the compile/package (JAR) stuff is going to work from files and assume you have a "standard" project structure (with dependencies specified etc).

seancorfield17:09:24

If you're doing this in-memory only, then read/`eval` is all you need, but if you want any sort of compile/package, you will need to create a physical filesystem project.

Joel18:09:00

read/eval would just be slow the first times, i believe?

Joel18:09:12

until AOT kicks in.

seancorfield20:09:43

read/eval will parse and compile the code every time so you'd have to figure out caching (based on exactly what sort of code you're loading)

seancorfield20:09:18

AOT has nothing to do with that. That means compile before use rather than letting Clojure compile on demand.

seancorfield20:09:03

(and AOT is only going to work for file-based stuff I think, since it's going to essentially require a namespace, which expects a file on disk)

seancorfield20:09:31

Given that article is written for Clojure 1.2.1, a lot has potentially changed since then.

Aviv Kotek10:08:37

hi, what would be the "right" clojure-way for simple http-polling: check every 2s until desired-response... a simple "while-loop" could work:

(defn poll [data]
  (let [initial (t/now)]
    (loop [existing (retrieve data)]
      (cond
        (= (:status existing) "Completed") existing
        (passed-max-poll? initial) {:status "timeout"}
        (contains? #{"Canceled" "Error"} (:status existing)) {:status "error"}
        :else (do
                (Thread/sleep (* poll-interval-s 1000))
                (recur (retrieve data)))))))
1x thread should be ok, curious if core.async could fit here (state machines with go)? or any other way? other lib? as I guess thread/sleep is not a good idea..

walterl11:08:23

> curious if core.async could fit here? Seems like overkill to me, but I'm no core.async expert. > other lib? https://github.com/sunng87/diehard has some good, advanced looping/retry functionality.

🚀 3
Aviv Kotek12:08:09

@UK0810AQ2 any thoughts here ?

Ben Sless12:08:04

core async does look like overkill here. Just return a promise and run that in a thread. On success, the thread will deliver the promise. If you do a lot of these, I'd use at-at

Aviv Kotek14:08:47

@UJY23QLS1 I guess the lib is not intended for such use-case, if I don't care of blocking I can stay with sleep, else just use core.async

littleli14:08:41

Hi. Is there a way to wrap java Future into Clojure's future to have access to deref? Does this even make sense in practice? Maybe what I want to ask is, is there a way to run future body on custom threadpool?

dpsutton14:08:25

one note. there is a call to push-thread-bindings in there. probably better for you to use (let [f (bound-fn f)] ...) and invoke that f on the executor if you want to preserve those bindings across threads. or look at the source of with-bindings* and add a (finally (pop-thread-bindings)) look at the source of bound-fn, bound-fn* and with-bindings* if you're interested

💯 3
seralbdev20:08:36

Hi all. Let's say I have a namespace with a function to decode a stream, let's call it read! That function iterates over a sequence with field definitions and the stream and, depending on types, dispatches several calls...so there is a tree of calls of several nested levels. One those functions need to access to a dictionary of (id => field sequence)...I don't want to pass a function that resolves that lookup as an argument to read! and then having to send it as an argument to all nested calls until reaches (eventually) the function that needs it...I am thinking in something like a dynamic bound function that user can define at runtime...or an atom?...any good practice about this??? many thanks in advance!!

Lennart Buit20:08:32

imo, bindings and atoms are just additional arguments to your function that happen to be passed implicitly. The general advice you’ll hear around here is to not reach for them unless you absolutely need to 🙂

Lennart Buit20:08:33

Maybe you can decompose your problem in smaller steps, and only supply this dictionary to steps that need it

seralbdev20:08:45

yep, I was thinking the same. Problem is that as I am decoding the stream I reach a field that is a complex one and I need to get its definition. My first idea was to skip this and do this part in a second step but the problem is without the complex type definition I don't know where the next field starts in the stream

seralbdev20:08:22

the nature of this process is forward-only without being able to go back 😞

seralbdev20:08:15

I don't care being academically correct about having dependencies on dynamic redefinitons or atoms...whatever...but I don't want a bad design that is going to bite me aftewards...I am kind of new to Clojure and definitely, to functional programming ... 😞

Lennart Buit20:08:30

Well its not perse that its academically correct or not, global variables and bindings just make for harder reasoning as functions are no longer just a function from its arguments to their results. Makes for harder testing and such 🙂. Anyhow that doesn’t help you much

Lennart Buit20:08:04

So you seem to have some form of a parsing problem, is that correct?

seralbdev20:08:09

well, the stream contains a sequence of data that could be primitive types or UDTs (compositions of primitive types). the module is decoding it having a definition of this sequence and the stream with data. life is beautiful until the use case of a stream containing a dynamic UDT (the id of the UDT as a string and then the data)

seralbdev20:08:56

in that case I don't know in advance the sequence of fields in that UDT so I have to look it up in a kind of dictionary

seralbdev20:08:33

but this is only used in this case...having to propagate through all the nested calls looks a bad design

Lennart Buit21:08:58

But you can’t do it two-phased, looking at this user defined type (I had to look that acronym up :p), you can’t know when it ends without having this definition

seralbdev21:08:39

exact...unless I change the format of the stream to add that length

seralbdev21:08:24

that is an option...but I don't like it too much unless justified

Sean Poulter20:08:20

How do folks scan ClojureScript projects for CVEs and/or CWEs? Are ClojureScript dependencies reported? Full-disclosure, I’ve cross-posted without a solution on: • #clojurescript - https://clojurians.slack.com/archives/C03S1L9DN/p1598904791224400#web-security - https://clojurians.slack.com/archives/C3N85R7QT/p1598905246001300

Alex Miller (Clojure team)21:08:23

there are some java CVE scanners that will work for Clojure deps too

Alex Miller (Clojure team)21:08:43

and some wrappers of that stuff into leiningen plugins etc

Alex Miller (Clojure team)21:08:12

those don't cover cljs but there may be a separate thing for that

deactivateduser22:08:44

I’ve used lein-nvd for Clojure code in the past and it seems to work pretty well. Detected a load of CVEs in a dependency of a dependency of a dependency of a dependency of mine (Jackson, fwiw which as an aside seems to be a bit of a disaster of a library in terms of CVEs…).