Fork me on GitHub
#clojure
<
2022-09-26
>
Martynas Maciulevičius08:09:15

Hey. Did I do something wrong or something isn't right about my env?

Caused by: java.lang.Exception: Cyclic load dependency: [ /clojure/spec/alpha ]->/clojure/walk->[ /clojure/spec/alpha ]->/clojure/main->/clojure/core/server
Clojure 1.11.0 Java openjdk 11.0.16.1 2022-08-12

Martynas Maciulevičius08:09:14

Went to ~/.m2/repository/org/clojure/clojure and removed rm -r 1.11.* and notnow it seems to work... How could this happen even... how could these Jars get corrupted this way? :thinking_face:

borkdude09:09:58

What deps do you have on your classpath?

Martynas Maciulevičius09:09:47

I have malli, XTDB, core.async, and many more. Are there any I should look for? Also I was trying to hack clojure.walk and tried to call it using my own functions. I was interacting with malli primarily when this came up. And this is given by lein uberjar so maybe it's somehow related to some kind of recent change in the classpath. Not sure.

borkdude09:09:25

It might be best to publish your deps.edn or project.clj somewhere to reproduce the problem

Martynas Maciulevičius09:09:50

I can't reproduce it myself. If it comes up again I'll try to do it. Maybe I'll notice something.

Ed11:09:57

In Emacs/Cider you can navigate into the clj files stored in the jars on the classpath and then edit and save those files, updating the jar file on disk. If you introduce a cyclic dependency and then restart the REPL you'd be able to create this problem. I assume that things like this are also possible with other tools, but I've only done this with Emacs in the past.

Martynas Maciulevičius11:09:35

I use nvim with Conjure and it could be possible. Edit: No, it doesn't allow to save. But it could still probably be possible somehow.

Ed11:09:29

I think that Emacs by default marks the buffer as read only when you open it from a jar but it didn't used to. But you can toggle read only mode and then save it. I'm afraid I know little too nothing about vim/nvim.

FiVo11:09:04

Hey, I am debugging something where I see a clojure.lang.ArraySeq appearing. Is there another way to get an implicit conversion to ArraySeq apart from the two following

(type (seq (make-array Integer 5)))
  ;; => clojure.lang.ArraySeq

  (defn foo [& args] (type (seq args)))
  (foo 1)
  ;; => clojure.lang.ArraySeq

FiVo11:09:51

The first one is not really implicit. I am wondering if there are cases I am not aware of.

Matthew Downey13:09:39

You could try finding references to ArraySeq's constructor and its create method here: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/ArraySeq.java There are a lot, though.

FiVo16:09:23

The answer to this question was sort

(type (sort [1 2 3]))
  ;; => clojure.lang.ArraySeq

j3vs19:09:05

how would i write a macro that would allow the injection of an argument to the body for example generating a uuid and using it:

(with-uuid [uuid]
  ;;do something with the uui)
i.e.
(defmacro with-uuid [[uuid] & body]
   `(let [uuid ;; somehow assign (java.util.UUID/randomUUID) to binding above]
      ~@body))
or some such

phronmophobic19:09:17

you could do something like:

(defmacro with-uuid [uuid & body]
  `(let [~uuid  (java.util.UUID/randomUUID)]
     ~@body))

;; usage
(with-uuid foo
  (prn foo))
;; #uuid "68c03052-01ef-4284-a7f7-0fdbc63b34f9"
• is there a reason to not just type it out? • fyi, starting in clojure 1.11, there's random-uuid • why is the uuid arg wrapped in a vector in your example?

🙌 1
j3vs19:09:12

the uuid was just a simple example i came up with to try and make my question clearer, i was basically just wondering how to "assign" to a list of bindings

phronmophobic19:09:49

another similar example you can check out is clojure.core/with-open:

(defmacro with-open
  "bindings => [name init ...]

  Evaluates body in a try expression with names bound to the values
  of the inits, and a finally clause that calls (.close name) on each
  name in reverse order."
  {:added "1.0"}
  [bindings & body]
  (assert-args
     (vector? bindings) "a vector for its binding"
     (even? (count bindings)) "an even number of forms in binding vector")
  (cond
    (= (count bindings) 0) `(do ~@body)
    (symbol? (bindings 0)) `(let ~(subvec bindings 0 2)
                              (try
                                (with-open ~(subvec bindings 2) ~@body)
                                (finally
                                  (. ~(bindings 0) close))))
    :else (throw (IllegalArgumentException.
                   "with-open only allows Symbols in bindings"))))

👍 1
phronmophobic19:09:35

although assert-args is private, so you can't use that directly in your implementation

hiredman19:09:08

The easy way to do this style of macro is to write a helper function that takes a callback, and then just write to macro to expand to a call to it

hiredman19:09:23

(defn with-uuid* [f] (f (random-uuid)))

hiredman19:09:35

(defmacro with-uuid [name & body] `(with-uuid* (fn [~name] ~@body)))

🙌 1
hiredman20:09:45

The long term problem with the callback approach is it breaks recur, but it is a useful start

j3vs20:09:23

yeh thats what we had before, someone wrote a function that accepts a callback, generates an ftp connection and then passes that connection to the callback, i thought it would look cleaner as a macro where you could do

(with-ftp [ftp]
  ;;do stuff)
similar to
(clojure.java.jdbc/with-db-transaction [tx db]
  ...)

kwladyka20:09:57

Do we have a web crawler library for Single Page Application web page? I need to get data from 1 website, but the only one way is to simulate web browser, go through the DOM and extract this data. I can write everything from the beginning, but something has to exist 🙂

Annaia Danvers20:09:31

This is what I used in the past when I needed to write frontend tests: https://github.com/clj-commons/etaoin

hiredman20:09:55

I am a big fan of htmlunit for scraping, because it basically a headless browser written in java, it is just another jar. It does have some limitations to what is it can handle

hiredman20:09:03

I've messed around a little with playwright which bundles headless browser builds with it

hiredman20:09:50

Neither htmlunit nor playwright have been able to circumvent FedEx's antiscraping stuff which is a bummer

kwladyka20:09:01

Does it work with Single Page Application. Javascript is needed and clicking on website simulation. @U0NCTKEV8 @U03TPPC0R0T?

kwladyka20:09:35

> antiscraping Can you write more?

hiredman20:09:53

Htmlunit can run javascript, playwright uses real browser runtimes, so it can as well

👍 1
hiredman20:09:38

Antiscraping, I dunno, some kind of magic so when I load the page in playwright, using a real browser runtimes, it still detects something different from a real browser engine and the js renders an error page instead of the tracking information

👍 1
Annaia Danvers21:09:05

@U0WL6FA77 etaoin uses webdriver to hook to a full Firefox browser so it should work fine with SPA, it's why I chose it.

👍 1
amithgeorge07:09:06

Regarding the anti-scraping tech, I had success using puppeteer-extra-plugin-stealth (https://www.npmjs.com/package/puppeteer-extra-plugin-stealth) to get around Cloudflares bot detection page. It might help with Fedex.

👍 1
kwladyka11:09:45

so many tools to choose… couldn’t be worst hehe 😉