This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-04-07
Channels
- # babashka (31)
- # babashka-sci-dev (10)
- # beginners (64)
- # biff (11)
- # clerk (5)
- # cljdoc (2)
- # clojure (84)
- # clojure-boston (3)
- # clojure-conj (2)
- # clojure-europe (11)
- # data-science (19)
- # datomic (118)
- # fulcro (3)
- # graalvm (3)
- # hoplon (6)
- # inf-clojure (146)
- # instaparse (5)
- # lsp (13)
- # malli (3)
- # off-topic (13)
- # pedestal (5)
- # proletarian (5)
- # re-frame (14)
- # reitit (12)
- # releases (1)
- # ring (19)
- # scittle (2)
- # shadow-cljs (155)
- # slack-help (3)
I've run into a bit of a conundrum with clojure.core/compile
: if I call compile from the repl, it fails because it can't find the class file. but if I write a function in build.clj
, calling b/compile-clj
does work. any ideas why that might be?
$ clojure -M:dev -e "(require 'noahtheduke.splint) (compile 'noahtheduke.splint)"
Syntax error (ClassNotFoundException) compiling at (noahtheduke/splint.clj:5:1).
noahtheduke.splint
Full report at:
/var/folders/5w/z50rbwg546x94k8p8jzqfbl00000gn/T/clojure-15092840249394826547.edn
$ clojure -T:build compile-
Compiling io.github.noahtheduke/splint
with implementation:
(defn compile- [opts]
(b/delete {:path class-dir})
(let [basis (b/create-basis {:aliases [:dev]})]
(b/compile-clj {:basis basis
:class-dir class-dir}))
opts)
I just double checked and the clojure -M
call works on my linux machine but it doesn't work on my mac.
i have
Anything about directories required to exist and be on the class path jump out at you?
$ ls classes/
noah@Noahs-MacBook-Pro ~/personal/splint [main ↑2 +0 ~3 -0 !]
$ ls
bb.edn build.clj CHANGELOG.md classes deps.edn dev docs justfile LICENSE pom.xml README.md resources src target tasks test tests.edn
noah@Noahs-MacBook-Pro ~/personal/splint [main ↑2 +0 ~3 -0 !]
$ head deps.edn
{:paths ["src" "resources" "classes"]
:deps {org.clojure/clojure {:mvn/version "1.11.1"}
org.clojure/tools.cli {:mvn/version "1.0.214"}
borkdude/edamame {:mvn/version "1.3.20"}}
:aliases
{:run {:main-opts ["-m" "noahtheduke.splint"]}
:dev {:extra-paths ["dev"]
:extra-deps {org.clojure/tools.namespace {:mvn/version "1.3.0"}
criterium/criterium {:mvn/version "0.4.6"}
com.clojure-goes-fast/clj-async-profiler {:mvn/version "1.0.3"}
noah@Noahs-MacBook-Pro ~/personal/splint [main ↑2 +0 ~3 -0 !]
$ clojure -M:dev -e "(require 'noahtheduke.splint) (compile 'noahtheduke.splint)"
Syntax error (ClassNotFoundException) compiling at (noahtheduke/splint.clj:5:1).
noahtheduke.splint
Full report at:
/var/folders/5w/z50rbwg546x94k8p8jzqfbl00000gn/T/clojure-9125859686944406454.edn
the directory exists and is on the class path
the namespace declaration. i have a copyright header above it
(ns noahtheduke.splint
"Entry-point for splint. Loads all of the lints up front, delegates all work
to library functions."
(:gen-class)
(:require
And does it still work on your Linux machine if you empty out the classes directory there?
okay, i just figured it out. calling require
first is what tripped it up. i'm not sure why
$ clojure -M:dev -e "(compile 'noahtheduke.splint)"
noahtheduke.splint
noah@Noahs-MacBook-Pro ~/personal/splint [main ↑2 +0 ~3 -0 !]
$ ls classes/
clojure edamame noahtheduke
compiling forces loading the code, and there are many ways that double loading code can trip things up
oh interesting, that makes sense. thanks for the help
Designing a "theme" system in my framework, where a user can extend the render function for a set of dispatch types.
In this example, a "theme" is a namespace. The user prefers the :u
theme. If a :u/TYPE
method isn't defined, it falls back to :me/TYPE
and then :default/TYPE
.
Does this sound reasonable? I like multimethods, but maybe this reverse-engineers them too much. Is there a simpler construct I should consider? Should I try something like https://github.com/camsaul/methodical?
https://clojure.org/reference/multimethods https://clojuredocs.org/clojure.core/defmulti https://clojuredocs.org/clojure.core/defmethod
Thanks @U0WL6FA77, see the snippet for how I've used multimethods so far.
why do I get a reflection warning when accessing a mutable field on a deftype? Is there a way to avoid the reflection?
user=> (set! *warn-on-reflection* true)
true
user=> (deftype T [^:unsynchronized-mutable a b])
user.T
user=> (defn get-a [^T t] (.-a t))
Reflection warning, NO_SOURCE_PATH:1:20 - reference to field a on user.T can't be resolved.
#'user/get-a
user=> (defn get-b [^T t] (.-b t))
#'user/get-b
Not only do you get a reflection warning, but if you tried to call those functions you'd get an exception about t not having the field
hmm I see
what is the fastest way of accessing those mutable fields then?
so accessing a mutable field in a deftype will always go through an invokeinterface, am I right?
given t
of type T
reading/setting the value of t.a being a
a mutable field
if you have your deftype implement an interface and those methods use the mutable field, the methods access as a field on the object using get and put field instructions
but I mean, if you want to create a getter/setter, you will need to go through the interface instead of the obj method which is slower, or am I missing something?
(definterface TI
(getA []))
(deftype T [^:unsynchronized-mutable a b]
TI
(getA [this] a))
(defn getB [^T o]
(.-b o))
(def t (->T 5 7))
(.getA t) ;; Execution time mean : 567.532462 ns
(getB t) ;; Execution time mean : 3.407326 ns
That is what I meant, that accessing mutable fields is much slower than non mutable ones on Clojure deftype while they
should be the same on Javaoh, nvm, I measured it wrong, I should have said (.getA ^T t)
the slowness was coming from the reflector
thanks!
Quick poll, how would you prefer to organize your sources in (small/medium sized) fullstack projects? With the source-sets on project root like:
project-root
dev
src
clj
ecorp
fullstack
core.clj
cljs
ecorp
fullstack
core.cljs
test
clj
ecorp
fullstack
core_test.clj
cljs
ecorp
fullstack
core_test.cljs
Or with source-sets under src like:
project-root
src
dev
main
clj
ecorp
fullstack
core.clj
cljs
ecorp
fullstack
core.cljs
test
clj
ecorp
fullstack
core_test.clj
cljs
ecorp
fullstack
core_test.cljs
Any other recommendations?The separation arose mostly due to limitations in tooling way back when, as I recall, but is not needed now.
If you feel the need to have a single (mono)repo that builds multiple, separate artifacts for deployment, you could consider #C013B7MQHJQ but I don't think that's what you're really asking here?
No, I'm mostly searching to settle on a nice fullstack (shadow-cljs + some backend) project structure
Then just src
and test
with namespaces/folders that are well-named for concerns within the application/domain (and mixed .clj
/`.cljc`/`.cljs` files in them).
There was a similar question around a year ago or so. IIRC, in the end people who were keen on keeping file types separated by folders were doing that because it felt nice. To each their own. And since you've mentioned shadow-cljs, thheller has wrote an article about the fact that the separation is not useful. :)
@U2FRKM4TW this: https://code.thheller.com/blog/shadow-cljs/2018/02/08/problem-solved-source-paths.html or this?: https://code.thheller.com/blog/shadow-cljs/2021/05/13/paths-paths-paths.html
@U03PYN9FG77 The first one, but the second one is also worth reading.
For full-stack i normally separate the backend and frontend repositories into subprojects part of a monolithic project, so it is less likely i accidentally fuse the two codebases together. In the past I’ve used lein-monolith so I can control several projects under one root project. I would add subprojects as needed (e.g. specs shared between backend and frontend).
I also avoid separating source code by file extension. For some reason all of the lein templates (e.g. luminus) do it, but I never use such templates anyway. Always build from scratch 😄
> For full-stack i normally separate the backend and frontend repositories so it is less likely i accidentally fuse the two codebases together. That would also prevent you from benefiting from the possibility of sharing some code between the frontend and backend, or making a change that affects both the frontend and the backend in one commit. Seems like a bad idea to me. Better to just get your source paths under control (at least one path for each app, and then one that is shared).
At work, we have separate frontend / backend repos -- but the frontend is a massive React.js / Redux app and the backend is a massive Clojure monorepo building 20 separate services and processes, so there's a lot of shared code between backend processes but nothing shared with the frontend (although there are times when that would be convenient...).
> That would also prevent you from benefiting from the possibility of sharing some code between the frontend and backend Extract common things into subprojects. > making a change that affects both the frontend and the backend in one commit. How? The monolithic project is what has the .git folder, not the individual subprojects.
Neither of these issues brought up happened in a fairly large codebase I was working on with some dozen other people..
Ah, so you have a monorepo with sub-"projects" for front, back, and shared stuff?
You said "normally separate the backend and frontend repositories" so folks assumed you meant you had multiple git repos.
It’ll be laid out as such. The root project.clj
contains managed dependencies and common settings used across projects.
root-folder/
subproject-1/
...
project.clj
...
subproject-n/
...
project.clj
project.clj
You’re right, I should edit my message to say “separate subprojects” insteadDo any users of http://clojure-goes-fast.com/kb/profiling/clj-async-profiler/allocation-profiling/ know what I should do to resolve this AgentInitializationException? (it's below in the code block). It only seems to work when i want to pass the {:event :alloc}
option.
I ran sudo sysctl -w kernel.perf_event_paranoid=1\nsudo sysctl -w kernel.kptr_restrict=0
and i passed the required jvm param for allowing attach self (which i tried to verify below).
(defn foobar [x] (range x))
;; => #'centriq-web.jobs.import.fix.import-all-staging-march-19/foobar
(require '[clj-async-profiler.core :as prof])
;; => nil
(prof/profile {:event :alloc} (foobar 100))
;; => Execution error (AgentInitializationException) at sun.tools.attach.HotSpotVirtualMachine/loadAgentLibrary (HotSpotVirtualMachine.java:106).
;; Agent_OnAttach failed
(defn jvm-options []
(let [runtime-bean (ManagementFactory/getRuntimeMXBean)
input-args (.getInputArguments runtime-bean)]
input-args))
;; => #'centriq-web.jobs.import.fix.import-all-staging-march-19/jvm-options
(jvm-options)
;; => #<java.util.Collections$UnmodifiableRandomAccessList@773279f [-XX:-OmitStackTraceInFastThrow, -Djdk.attach.allowAttachSelf, -Xmx20480m, -Dclojure.basis=.cpcache/3428257497.basis]>
;; using the default of :event :CPU seems to work...
(prof/profile (foobar 1))
;; => (0)
maybe my version java matters?
➜ backend git:(etl-with-adaptions) ✗ java --version
openjdk 17.0.6 2023-01-17
OpenJDK Runtime Environment (build 17.0.6+10-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-0ubuntu122.04, mixed mode, sharing)
I remember some difficulties with alloc profiling. Do you have debug symbols installed? Do you have patience to try jvm 8 and 11?
@UK0810AQ2 ty for the reply. I didn't explicitly install any debug symbols. Should i have? I can probably try jvm 11... This code is currently living in a mono repo so I'm worried something might complain though.
iirc I only got alloc profiling working on jvm 11 and 8, but don't quote me on that not sure debug symbols are related
FWIW I wasn't able to make the profiler pick those symbols up, despite having that package installed.
But the error also seem to have nothing to do with debug symbols. Don't quote me on that though.
start with trying 11 and 8 (crossing fingers etc), if 11 gives you trouble maybe 15 but then you're cooking with gas
maybe debug symbols are related https://github.com/async-profiler/async-profiler#installing-debug-symbols
That looks promising. I'll install those and report back. Tyty
I had trouble where debug symbols weren't picked up when I started the process, I used colorful language but to no avail
@UK0810AQ2, installing the debug symbols did the trick. That is, running:
apt install openjdk-11-dbg
Though...the resulting flamegraph looks wrong. ill figure it out. 😆
works with java version 11.