This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-06-15
Channels
- # ai (1)
- # announcements (1)
- # aws (4)
- # babashka (9)
- # biff (1)
- # calva (1)
- # clerk (15)
- # clj-kondo (24)
- # clojure (23)
- # clojure-austin (7)
- # clojure-europe (19)
- # clojure-nl (2)
- # clojure-norway (33)
- # clojurescript (43)
- # conjure (4)
- # data-science (2)
- # datahike (5)
- # datomic (14)
- # defnpodcast (27)
- # domino-clj (1)
- # events (1)
- # honeysql (13)
- # hyperfiddle (44)
- # introduce-yourself (1)
- # java (4)
- # jobs (1)
- # jobs-discuss (11)
- # lsp (3)
- # malli (14)
- # missionary (5)
- # off-topic (44)
- # pedestal (2)
- # podcasts-discuss (1)
- # releases (8)
- # remote-jobs (2)
- # shadow-cljs (3)
I'm trying to write a script that uses some 3rd party API, and I've never tried to use OAuth 2, but it looks like you need a redirect URI to get the Oauth 2 response, but how would I provide that in a script? Especially, the API I'm using forces an accept-list for redirect URIs, meaning I need to provide them when I register for access. Do I really need to open up a web server, say have http-kit listen to http request and provide that as an OAuth 2 redirect? And if the API wants to accept list the URI, do I need to have a public DNS setup for it with an endpoint and all that?
local web server is one way. You can sometimes/usually configure a localhost URL for the redirect, at least for dev purposes. Mobile apps often register a unique URI scheme/handler and redirect to <app://callback> or whatever. You can also see if your OAuth provider supports the device authorization flow which allows the user to complete the auth flow in a browser while your script polls for a token.
What are some open source projects, in development stage, that I can work in? (Preferably, a project suitable for a startup, or something Industry-like).
There's #C01AZBR8HHT but it hasn't had any activity for more than a year... I think a good starting point is to choose a project you use personally (assuming you're not a beginner in Clojure), go through its open issues and offer to work on it and either get the go ahead or move on to another project (or even fork it and do your own thing 🤷).
Web development; I could do with a lot of experimenting, as far as we are using cutting-edge technology and trying the optimal solution.
I could work with mathematical/engineering-related problems as well, since I have the background. So, scientific computing and the like. I'm open, just want to learn more, and build practical applications, at the same time that I collaborate.
Would you prefer to work on productivity tools? Developer tools? Business-style application?
I am trying to integrate splint linter into my clojure project. I added the dependency to my deps.edn file: io.github.noahtheduke/splint {:mvn/version "1.9.0"}}. I am unsure how to run the command to lint. These instructions seem to be for clojure CLI
probably worth discussing in #C04SCGV2ATX
I'm trying to profile/time a specific component of a large Clojure system. I was able to use the https://github.com/clojure-goes-fast/clj-async-profiler to do some initial profiling, which was very useful, but now isn't meeting my needs because of two factors:
1. The code I'm profiling produces nested lazy sequences in multiple places, using functions like map
and filter
. I found that the time spent computing the values in these sequences was attributed to the place where they were eventually calculated, not in the code that created them in the first place, which made it difficult to tell how much running time was due to the component I'm interested in. I tried to stamp out all of this laziness using doall
, and was able to get good enough results to make some optimizations, but this was a frustrating process searching for and eliminating laziness -- even if I call doall
on a laze sequence, if the sequence contains other lazy sequences, they aren't computed until needed.
2. At this point, I want to go run the code hundreds of times and measure how long each of two different implementations take on average. So, a flame graph from a single run isn't sufficient, and I'd prefer to be able to just log a bunch of running times and gather statistics on them later.
I considered manually just tracking time using something like (. System (nanoTime)
, but laziness makes this tricky -- if I record the time before and after the function call I'm interested in, but the function call has laziness in it, the time for its computation won't be tracked properly.
I'm wondering if anyone has any recommendations for this type of profiling that would assign computational time to the function that calls the lazy function, not where the computations are eventually realized.
I don't have an authoritative answer, but personally I try to eliminate laziness in performance critical code wherever I can. I use various strategies to do that. Very often I switch to using transducers via (into [] f source)
, which is both lower overhead and better for tracing. Occasionally I use transient
directly together with reduce, especially when I am concatenating a sequence or building a set from multiple sources/transforms.
Fundamentally I don't think there is a way around cleaning up laziness in your program if it's performance critical and you want to be able to profile it accurately. At least not that I know of.
Hello Tom! Laziness is indeed a big hurdle to overcome when profiling and reasoning about the performance. Unfortunately, there are no shortcuts to make it magically go away. All I can suggest is use more of vector-producing functions like mapv
, filterv
, and the like. Like @U01D37REZHP said, if it's the part of the program where you care about the performance, it's most often a good idea to get rid of laziness anyway.
Regarding something like doall
for nested collections: there is a hack for that, you can call pr-str
on the top-level collection and throw away the result. Of course, that itself would spend quite some time on serializing to string, but if your intermediate goal is to measure, not to make everything faster immediately, then it could be useful to you.
I think the pr-str
thing that I've seen around is a bad idea. I've seen this trick find its way into at least one library. Instead of that, why not something like this?
(defn realize-recursive [data]
(doall (tree-seq coll? seq data))
data)
Sample with and without:
(let [a (atom 0)
x (map (fn [i] (map (fn [j] (swap! a inc)) (range (* i 2)))) (range 100))]
[(count x) @a])
;; => [100 0]
(let [a (atom 0)
x (map (fn [i] (map (fn [j] (swap! a inc)) (range (* i 2)))) (range 100))]
[(count (realize-recursive x)) @a])
;; => [100 9900]
Maybe a better name would be doall!
... umm I could probably be more diplomatic! I didn't mean to be rude, I hope it wasn't taken that way! 😅
It's OK, don't worry:) I suggested pr-str
as a crutch to facilitate profiling, surely it shouldn't be left around in the working code. It's main benefit is that it's already there, nothing new has to be written:)
Thanks for some ideas! Perhaps I should just go the non-lazy route everywhere I care about