This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-12-17
Channels
- # adventofcode (33)
- # announcements (1)
- # beginners (32)
- # calva (12)
- # cider (1)
- # clojure (73)
- # clojure-belgium (1)
- # clojure-europe (20)
- # clojure-norway (3)
- # datomic (3)
- # emacs (2)
- # fulcro (18)
- # graphql (11)
- # honeysql (1)
- # juxt (1)
- # lsp (10)
- # membrane (13)
- # reitit (1)
- # releases (5)
- # scittle (15)
- # spacemacs (1)
- # squint (36)
- # testing (1)
- # tools-deps (20)
- # xtdb (5)
How would i read a string into clojure where part of it is valid clojure and another part isnt. E.g "{:a [///fobar]"} Like i would like to at least end up with {:a ["////foobar"]}
.
The actual goal is to read a string which has seralized stack trace in it. e.g {:msg "UncaughException" ex "#error" {:cause "blah" via ...
it complains that some parts of it can't be handled by read-string or edn/read-string telling me there is do dispatch macro No dispatch macro for: '
Which makes sense because of this part of the string: calling #'com.stuartsierra.component/start
I was sort of hoping there would be a way to make my stack traces readable in clojure so i could pull them apart using clojure. Any ideas in this direction would be interesting to me. I usually don't worry about stack traces because i so rarely make mistakes /sarcasm.
i can probably pass edn/read-string readers to handle it...
////
is not valid Clojure, but #'
is. But, it's not valid EDN.
But you say that clojure.core/read-string
didn't work for you as well. What was the error?
It works just fine for me:
user=> (read-string "#'a")
(var a)
user=> (require '[clojure.edn :as edn])
nil
user=> (edn/read-string "#'a")
Execution error at user/eval7 (REPL:1).
No dispatch macro for: '
user=>
(-> "errors/error.edn"
slurp
read-string)
;; 1. Unhandled java.lang.RuntimeException
;; Unsupported character: \"UncaughtException
Hello 🙂 I am trying to build an uber JAR using build, but I am getting the rather cryptic:
Building uberjar target/eve-standalone.jar...
Execution error (IllegalArgumentException) at clojure.tools.build.tasks.uber/explode (uber.clj:152).
/ is not a relative path
Does someone have a clue of what’s happening?Can it somehow be because my src
directory contains .clj{,s,c}
files?
No, that shouldn’t matter. Do you have any :local deps?
If you can share your deps.edn that would help
I don’t have :local deps. Here are my industrial secrets:
{:paths ["src" "resources"]
:deps {org.clojure/clojure {:mvn/version "1.11.1"}
org.clojure/clojurescript {:mvn/version "1.11.51"}
org.clojure/data.csv {:mvn/version "1.0.1"}
;; Malli is a data-driven schema library.
;;
metosin/malli {:mvn/version "0.9.2"}
;; Third-party library for dealing with SQL and Postgre.
;; Not sure we need it since we have `next.jdbc`.
;;
clj-postgresql/clj-postgresql {:mvn/version "0.7.0"}
org.postgresql/postgresql {:mvn/version "42.3.7"}
;; Third-party for dealing with JDBC.
;;
com.github.seancorfield/next.jdbc {:mvn/version "1.3.834"}
;; Deals with JSONs.
;;
cheshire/cheshire {:mvn/version "5.11.0"}
;; HTTP server.
;;
ring/ring-core {:mvn/version "1.9.6"}
ring/ring-jetty-adapter {:mvn/version "1.9.6"}
;; Routing library that integrates with ring.
metosin/reitit {:mvn/version "0.5.18"}
nrepl/nrepl {:mvn/version "1.0.0"}
;; Frontend
thheller/shadow-cljs {:mvn/version "2.20.14"}
reagent/reagent {:mvn/version "1.1.1"}
re-frame/re-frame {:mvn/version "1.3.0"}
clj-commons/pushy {:mvn/version "0.3.10"}
cljs-ajax/cljs-ajax {:mvn/version "0.8.4"}
day8.re-frame/http-fx {:mvn/version "0.2.4"}
;; Dev dependencies
;;
;; Dev dependencies to have tracing inside the application.
;;
day8.re-frame/tracing {:mvn/version "0.6.2"}
day8.re-frame/re-frame-10x {:mvn/version "1.5.0"}
binaryage/devtools {:mvn/version "1.0.6"}
;; Not sure
com.cognitect/transit-cljs {:mvn/version "0.8.280"}
com.lambdaisland/dom-types {:mvn/version "0.5.37"}
;; Since 2.20.11 guava is not explicitly required by
;; shadowcljs anymore but apparently it's still needed
;; because it doesn't start without this pinned version.
com.google.guava/guava {:mvn/version "31.0.1-jre"}}
:aliases
{;; Builds the backend as an uberjar. Unused for now.
:build {:deps {io.github.seancorfield/build-clj
{:git/tag "v0.8.5" :git/sha "de693d0"
;; since we're building an app uberjar, we do not
;; need deps-deploy for deployment:
:deps/root "slim"}}
:ns-default build}
;; Runs the tests. No tests for now. #yolo
:test {:extra-paths ["test"]
:main-opts ["-m" "cognitect.test-runner"]
:exec-fn cognitect.test-runner.api/test
:extra-deps {org.clojure/test.check {:mvn/version "1.1.1"}
io.github.cognitect-labs/test-runner {:git/tag "v0.5.0" :git/sha "48c3c67"}}}}}
Maybe some deps are cljs
only and need to be moved in an alias?
Nah, that doesn’t matter
Yeah, turns out if I comment out the frontend deps it builds fine.
It looks to me like that’s probably a jar that has a / path in it, which is probably legal but weird
I mean if you can exclude the dep that’s weird, then do so, might be good to figure out which one it is
I’ll start the hunt
You could clone tools.build locally, add a debug print statement to the uber task, then include the local dep for tools.build on your build alias
I’ll comment them out one by one until it breaks 😄
Looks like it’s shadow-cljs
Thanks for your time and feedback. :man-bowing:
Huh, well would be good to report that
FWIW it is not shadow-cljs, it is the closure compiler. they switched their entire jar creation to bazel and ever since it has been weird. either basically releasing uberjars or apparently including a /
entry.
Clojure 1.11.1
(require ')
nil
( "/")
#object[java.net.URL 0x5c09d180 "jar:file:/home/thheller/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20221102/closure-compiler-unshaded-v20221102.jar!/"]
but for what its worth. none of your CLJS dependencies (including shadow-cljs) should be part of an uberjar build. these are development dependencies that just create unnecessary bloat (and conflicts) in uberjars
ah, I was wondering if it was something transitive. maybe I'll exclude this specific / in uber
Yes, that’s what I figure. This is a project I just created and I am learning Clojure(Script). It’s not always easy to navigate all the different pieces and how they fit together, for the language(s) and the tooling most of all.
So I'm reading through this: https://death.andgravity.com/pwned -- and out of curiosity, I'd like to see if the results can be replicated in Clojure/JVM. I've started with the naive linear approach: • First using a RandomAccessFile, but that seems to be quite slow, as there's no buffering going on • Then using a FileReader (setting file encoding to ascii), wrapped in a BufferedReader. The times I get for around 8000 iterations are: • Python: ~2ms • Clojure RAF: ~380ms • Clojure Buffered/FileReader: ~10ms Are there any other IO optimisations I can look into to bring the Java implementation closer to python?
im confused about the goal before i try to read the whole thing. > Instead, we'll do it the hard way – we'll check passwords offline. That seems fairly useless, one of the key features in any good security system is that it throttles attempts.
@U050ECB92 here's the whole thing. I haven't profiled yet but from online reading it seems that RAF does no buffering whatsoever.
(ns powned
(:require [ :as io]
[clojure.string :as str]))
(set! *warn-on-reflection* true)
#_
(def random-file (java.io.RandomAccessFile. "resources/passwords-sample.txt" "r"))
(def random-file ^java.io.RandomAccessFile (java.io.RandomAccessFile. "resources/pwned-passwords-sha1-ordered-by-hash-v8.txt" "r"))
(defn get-hash [type data]
(.digest (java.security.MessageDigest/getInstance type) (.getBytes data)))
(def hex (-> (java.util.HexFormat/of) (.withUpperCase)))
(defn sha1-hash [^String data]
(let [digest
(get-hash "sha1" data)]
(.formatHex hex digest)))
(defn find-line-linear [^String prefix]
(with-open [reader (io/reader (java.io.FileReader. "resources/pwned-passwords-sha1-ordered-by-hash-v8.txt" (java.nio.charset.Charset/forName "US-ASCII")))]
(loop [line (.readLine reader)
counter 0]
(tap> ["line2" line])
(if false #_(> counter 10000)
(println "giving up")
(when (some? line)
(cond
(str/starts-with? line prefix)
(do
(println "found in " counter)
line)
(pos? (compare prefix line))
(recur (.readLine reader)
(inc counter))))))))
(defn find-line-linear-random [^String prefix]
(.seek random-file 0)
(loop [line (.readLine random-file)
counter 0]
(tap> ["line" line])
(if false #_ (> counter 10000)
(println "giving up")
(when (some? line)
(cond
(str/starts-with? line prefix)
(do
(println "found in " counter)
line)
(pos? (compare prefix line))
(recur (.readLine random-file)
(inc counter)))))))
(defn find-password [password]
(let [sha1 (sha1-hash password)
_ (println "looking for sha" sha1)
found (find-line-linear sha1)]
(if found
(-> found
(str/split #":")
(second)
(parse-long))
0)))
(comment
(add-tap println)
(remove-tap println)
(time
(sha1-hash "blocking")
)
(time
(find-line-linear (sha1-hash "blocking")))
(time
(find-line-linear-random (sha1-hash "blocking")))
(find-line-linear "00000DBE43074309BC753F183A067ED42BC0DEA0")
(time
(find-password "blocking"))
)
@U0DJ4T5U1 this is about finding a password in a list of compromised passwords, nothing to do with security or throttling etc. In more detail, it's just an excuse to explore the boundaries of I/O performance, for academic curiosity on my part.
The Random File bit is to support a later attempt of doing a binary search by jumping around in the file. I can see obvious optimisations like applying a custom buffering layer on top of the native RAF, and trying to move away from strings entirely, but I'm looking to stay with whatever the JVM provides natively without heaps of custom code just yet.
what is best to use for a clojure mixed java spring-boot project? leiningen,maven or gradle? does anyone has experience or a sample project in like github?
Gradle with Clojurephant works well for that. https://github.com/clojurephant/clojurephant
thank you, i tried it in past and i had some problems i dont remember exactly, if you have a sample project example it would help alot, thank you anyways i will retry it
java is so popular and spring has became functional and reactive with spring 5, webflux etc, so i am thinking to have do functional java and clojure when possible
you think is bad idea? i was thinking to do functional/reactive java, and where possible to use clojure
It's the kind of "synthesis" that you can only really think about after you understand both kinds of systems
yes thats what i am thinking to go java/clojure java for safety, and clojure where possible, and if possible only clojure
Clojure calling into Java is fine, but Java calling into clojure is a relatively hard thing
And if you want to be learning spring, that would be Java calling into clojure, and that's not worth your time
I do understand the appeal of polyglot projects, believe me, I've tried and succeeded occasionally, but it's water and oil
its ok i will see what i can do, thank you from your experience, worst case learn the jvm using java, and sometime move completly to clojure
I will disagree with @U3JH98J4R here. Lots of people, including me, use Clojure with a Java web or API framework. It's not as nice as going full Clojure, but it's a great way to do Clojure and remain working inside a Java shop. Spring Boot doesn't just do DI, it'll handle the server, the routes, the authentication, the wire protocol, any interceptor chain processing you need on the request and response, etc. And it also does DI in a way that also manages initializing and maintaining the state, setting up your logger, your metrics publisher, and injecting all that on a per request basis. Again, I'd rather be using Ring with http-kit and some Clojure router, but it's a very viable route to piggy back on a Java framework, and it's much nicer than using Java still.
At my work, we use an in-house fork of Spring boot and Spring MVC, that's used company wide, it means there's a lot of internal tooling based around it and standard interceptors for it, custom internal auth, tracing, client generators, etc. Most teams use Java with it, some use Scala, my team uses Clojure.
This is a good read as well: https://github.com/stuarthalloway/clojure-from-java Though it uses maven, which is also a viable option.
If you go that route I do recommend a Java build tool, maven or Gradle or even Ant, over a Clojure one, because building Clojure is relatively simple, but Java not so much, so you benefit more from the build tool being specialized for Java.
I am mixing Java and Clojure as I move a legacy Java application to Clojure. I don’t try to have a single build however but instead consume my Clojure code as libraries. For some, I have a dedicated Java interface so in Clojure I simply implement that interface. In other cases, I build a Java glue layer that calls invoke and wraps up types so that Clojure code is easier to call. For some libraries, I only build and install locally rather than publishing to maven central or elsewhere. But I keep build tools separate using maven for Java.
i tried maven and clojure based on the github project above in past it worked fine, i tried also the clojure way to make a java class to be call from java like normal java functions, worked very good also, keeping clojure code in leinigein