This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-09-23
Channels
- # announcements (5)
- # babashka (22)
- # beginners (240)
- # calva (51)
- # clj-commons (1)
- # cljsrn (9)
- # clojars (12)
- # clojure (81)
- # clojure-australia (2)
- # clojure-europe (40)
- # clojure-france (10)
- # clojure-italy (1)
- # clojure-nl (2)
- # clojure-uk (37)
- # clojurescript (59)
- # clojureverse-ops (2)
- # copenhagen-clojurians (1)
- # cursive (9)
- # datomic (18)
- # emacs (12)
- # fulcro (24)
- # graalvm (48)
- # hyperfiddle (5)
- # introduce-yourself (1)
- # jackdaw (1)
- # jobs (2)
- # juxt (8)
- # lsp (25)
- # malli (8)
- # missionary (1)
- # music (3)
- # off-topic (32)
- # polylith (16)
- # quil (4)
- # re-frame (52)
- # reitit (5)
- # reveal (3)
- # rewrite-clj (26)
- # rum (1)
- # sci (1)
- # shadow-cljs (14)
- # sql (2)
- # tools-build (40)
- # tools-deps (14)
- # vrac (2)
- # xtdb (63)
I found this https://fly.io/blog/how-we-got-to-liveview/ about liveview in phoenix. I was wondering if we have a framework at similar level of abstraction in Clojure. Clojurescript as ecosystem (browser/nodejs) seems like an option, with lower level built-ins (3rd party like Re-agent/Re-frame for higher level). LiveView seems to very useful to get an idea off the ground with less resources.
the idea in that is you can develop using hiccup components (like you would in reagent) and use sources on the server
> to get an idea off the ground with less resources that's resources as in developer time, right?
@U051SS2EU Yes, developer/creator's time
Depending on what exactly you are tracking with "presence", I don't think a full crdt is required. I wrote(wrote and rewrote, many times of the course like a month) the presence system we use at work. What we had is pretty crdt like, accept we don't need to track causal history because information for a given connection/session/whatever is only ever updated by the server where that connection lives
We use http://netty-socket.io as a transport (which does long polling or websockets, and will try and upgrade from long polling to ws). Strictly as a transport, we don't use any of the room/distribution features of http://socket.io. And I kind of hate both http://netty-socket.io (java server for speaking http://socket.io) and http://socket.io (the protocol, technically a ridiculous stack of protocols)
I have at various times started writing a js library + clojure server that communicate via a custom protocol that does polling and/or websockets and a pure clojure implementation of the http://socket.io protocol
Still makes me smile to have figured out this bug, and to have been able to reflectively patch it https://github.com/mrniko/netty-socketio/issues/754
Something to keep in mind is websockets are inherently more fragile than polling, they are more easily disrupted by weird middle boxes, flakey mobile internet, roving between networks, etc
One of my problems with the http://socket.io protocol is the "upgrade" from polling to websockets is one way, if the upgrade succeeds, but then the websockets goes away for some reason, it doesnt try to keep the "connection" going by reverting to polling
@hiredman Will check out http://netty-socket.io. But I referring to the availability of such reliable tools baked into one framework. Think Python/Clojure built-in libraries, but at a higher abstraction (diff patching, presence etc). Using these common tools, you then build interactive apps/websites
This might also be worth looking at: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
It’s back online! If you can’t see it yet, add any query param to break the cache 😊
Great!!! Thanks 😄
What are required steps to make a function discoverable by clojure.repl/source
?
I have it saved in a file and loaded in REPL.
Function has :file
metadata.
But (.getResourceAsStream (clojure.lang.RT/baseLoader) (:file (meta (resolve my-fn))))
, which is used by clojure.repl/source-fn
, returns nil
.
I'm using Leiningen
.
Btw source
works for clojure core and libs and for dependencies.
When I do lein classpath
I see src
directory there.
I think the file has to be on the classpath - it can't be a random file that you load in your REPL. But if you're loading a file from src
that's on classpath, then it's something else.
check source-fn here? https://github.com/clojure/clojure/blob/b1b88dd25373a86e41310a525a21b497799dbbf2/src/clj/clojure/repl.clj#L147
Hmm now it's working, not sure what was wrong. Thanks anyway!
If you evaluate forms through the repl they won’t have the required metadata. How can a form sent through the repl have a line number
I wrote function in a file and loaded that file in the repl
C-x C-e
Sorry, C-c C-k is for file
from Emacs Cider
I guess it calls clojure.core/load-file
function, not sure
I am having trouble finding the bottlenecks in a highly concurrent application. I have tried https://visualvm.github.io/ and it's nice for monitoring how busy certain threads are but for example the profiler just dies on me when trying to profile cpu usage. I currently have always around 20% cpu usage but would like to push that towards the 100%. How do people usually go about this these days?
do you have reason to believe your application can be compute-bound? if not, you are unlikely to get high cpu usage
no not necessarily, could also be io or network bandwidth (although the later is very unlikely as I only use a fraction of what is available)
if so, you should be looking for concurrency bottlenecks - are those threads contending for locks/queues/access to resources/etc
YourKit has some pretty interesting locking monitors that can help
but really the most important thing is having a clear mental model of how threads are being managed and how they communicate
no not necessarily, could also be io or network bandwidth (although the later is very unlikely as I only use a fraction of what is available)
you could try collecting metrics on retries or blockage of agents
also, the amount of time from an incoming input to a calculated result is likely more interesting than percentage of CPU used - most apps I've worked on spend nearly all their time waiting for work to do
I don’t know if it is good recommendation, but I always find the best solution to fix such kind of issue is to experiment with the code, add more logs, make code simpler, a little refactoring etc. I am afraid there is no gold solution. PS It can take long days to figure this out or even more depends on your use case.
Hi! I'm trying to understand extension of protocol with metadata. So metadata are associated with values. I don't see any direct link in the doc between the protocol and the value with metadata. Do I understand it correctly it means in order to decide if a value "extends" a protocol that in addition Clojure is now looking if all the fully qualified protocol function have a definition in the metadata? What happens if only some of them are defined?
And I think even if you fully satisfy a protocol via metadata, satisfies?
will still return false
.
I get that part, but what is the behavior then in the case of let's say a multimethod using the protocol but a value only implementing some of it via metadata ?
What are the results of satisfies? Or extends? And such when only partially satisfied?
I guess I should just give it a try. It was just unclear to me when I was reading the doc
satisfies?
will return false. And you can't use extends?
because it requires a type - metadata extension works only on values.
It's a relatively new addition to the language so the core docs are pretty minimal and I don't think there's much out there in terms of community tutorials/blog posts. FWIW, we use metadata for protocols quite a lot at work (can't share that, unfortunately) but I can point you at next.jdbc.connection
which uses metadata to partially satisfy Component's Lifecycle
protocol: https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/connection.clj#L319-L328
Perfectly reasonable Q given how recent that feature is.
Hello! How to translate Character.UnicodeBlock.of(xxx)
to Clojure? I thought (Character$UnicodeBlock/of xxx)
but that does not work... 🙏
(import java.lang.Character$UnicodeBlock)
=> java.lang.Character$UnicodeBlock
(Character$UnicodeBlock/of 32)
=> #object[java.lang.Character$UnicodeBlock 0x797cc02e "BASIC_LATIN"
What do you see instead?Yeah, while the java.lang
top-level classes are auto-imported, none of their nested classes are.
Ah, apparently Thread$State
and Thread$UncaughtExceptionHandler
are auto-imported!
Huh, cool. How did you check? I just found them in RT.java
so at least now I know where to look. :)
I datafy
'd a namespace in Portal which showed me all the :imports
🙂
You can do it in a bare REPL:
user=> (require '[clojure.datafy :as d])
nil
user=> (-> 'user (find-ns) (d/datafy) :imports (keys) (->> (filter #(re-find #"\$" (name %)))))
(Thread$State Thread$UncaughtExceptionHandler)
user=>
Folks! I could probably use some help about how to push data into a Kafka Topic.
I am downloading some JSON garbage from the internet and, once processed, produce a Kafka value. Since the JSON payload is an array (and with a lot of elements) I thought of leveraging the JSON Streaming capabilities of cheshire
and then put a transformation pipeline (through a transducer xf
)
My doubts is about the f
that reduces the elements together; because my ultimate aim is to push to Kafka without loading the entire dataset in memory, I came up with something like this (see code below)
I am essentially creating a step function that returns the producer at the first step, and in the successive ones pushes the value and returns the producer again. While I think theoretically this achieves my aim (avoid loading the entire dataset in memory) I am not that confident if this is the right approach.
Did anybody found themselves in the same situation?
Did you try the #apache-kafka channel? (I've no idea how active it is)
Ah, ok.
they have a reducing function that is something like (fn [channel item] (add channel item) channel)
and then transducers are applied to that
Assuming you’re talking about the core-async, then it kinda looks similar to what I am doing in my code, isn’t it?
@hiredman I took another look and it seems like this is the way to go. The only other change I did was to use completing
instead of defining my own function override. Thank you!