To answer specifically, about getting rid of the thread pool: What is get-status-message-match doing underneath? If it is an http call for example, you could use non-blocking http client. If you use a non-blocking http client, then you add a core-async wrapper around it, and get rid of the thread-pool. But more generally: But I am not 100% sure I understand what your goal is. Why do you want to get of of the concurrency, and what exception are you getting?

there are limited number of threads available, during performance tests we realized if every user hop on to this page the server may run out of threads. the get-status-message-match function is doing lot of computations underneath (calling multiple api, db calls etc.)

I am not able to figure out what data structure are best suited for these type of cases.

Where is thread-pool-executor defined? You could use a FixedThreadPool and shared it across requests. This way, you know you won't create more than the N threads you choose. For a clojure-friendly interface to threadpools, you may like https://github.com/clj-commons/claypoole --- > I am not able to figure out what data structure are best suited for these type of cases. Afaik you have two options. You can try to make get-status-message-match use non-blocking io, and get rid of the thread pool. You can use a non-blocking http client for the api calls. For db calls, you would need a thread-pool though. You could also do what I suggested above -- just make sure that whatever threadpool you use is constrained.

thanks much for your response - here is the threadpool -

(defonce thread-pool-executor
         (delay
           (Executors/newFixedThreadPool config/offer-tracking-threads-max-count
                                         (counted-thread-factory "status-thread-%d" false))))

In this case, since the thread-pool is fixed and is defined outside of the request, I don't think the server could run out of threads. What error were you getting?

I am getting error when I am trying to get rid of this threading stuff completely, removing .execute / channel etc. I wanted to convert it into plain code first and see if any optimization is possible with data structures

What does your code look like without the execute / channel, and what error do you see? It should look something like:

(defn get-all-offers-attached [context filtered-company-data offers-study-data
                               actual-discount-map]
  (let [split-list (reduce #(let [client-study-id (or (-> (get actual-discount-map
                                                               [(:dbid %2) (:sid %2)])
                                                          :client_study_id)
                                                      "NA")]
                              (if (contains? offers-study-data [(:dbid %2) (:sid %2)])
                                (assoc %1 :with-offers
                                       (conj (:with-offers %1)
                                             (let [data (assoc %2 :clientStudyId client-study-id)
                                                   log-context (or (clog/get-context) {})]
                                               (wrap-with-logging-context
                                                log-context
                                                (let [status (try (get-status-message-match context (:id data)
                                                                                            (:dbid data) (:sid data))
                                                                  (catch Exception e
                                                                    (log/error (Throwable->map e))
                                                                    "ERROR"))]
                                                  (assoc data :discountvsoffers status))))))
                                (assoc %1 :without-offers (conj (:without-offers %1) (assoc %2 :discountvsoffers "NA" :clientStudyId client-study-id)))))
                           {:without-offers [] :with-offers []} filtered-company-data)]
    (concat (:without-offers split-list) (->> (:with-offers split-list)))))

this works, thanks a ton.

FYI, cross-posting across #C053AK3F9 and #C03S1KBA2 (also #C03S1L9DN) is strongly frowned upon. Best move is probably to just stick with posting in #C053AK3F9 until it doesn't feel right anymore, and then move to #C03S1KBA2 and/or #C03S1L9DN. Personally, I feel somewhere in between. So I post mostly to #C053AK3F9 with questions that feel like beginner naiveté, and sometimes the others with questions that don't seem very beginnner-friendly.

Noted, thanks for the guidelines.

https://www.learndatalogtoday.org/ is pretty good

https://max-datom.com/ https://nextjournal.com/try/learn-xtdb-datalog-today/learn-xtdb-datalog-today is quite nice too

If you like videos then this is a good one https://www.youtube.com/watch?v=oo-7mN9WXTw there used to be a site called datascript 101 but it seems to have vanished 😕

🙏

The https://cljdoc.org/d/datascript/datascript/0.17.1/doc/readme have a link to DataScript 101, but I don't suggest following the link. It was blocked by UBlock Origin, so I tested it in a Private tab, and it redirects to a spammy sales page.

I'm a little late here but one thing that I skipped reading docs (switching between lot of materials) is thehttps://docs.datomic.com/pro/overview/introduction.html#datomic-does-this-by. At least for me was a game changing to understand the basics and start understanding by myself the https://docs.datomic.com/pro/query/query.html.

the code in the example given was not the reason for the fault. The reason was a malformed defonce in the dependencies that started the webserver everytime when the dependency was loaded:

(defonce server
  (jetty/run-jetty #'app {:port 8000 :join? false}))

(comment (.start server))
(comment (.stop server))

(defonce server (atom nil))

(defn start-webserver []
  (when (nil? @server)
    (reset! server (jetty/run-jetty #'app {:port 8000 :join? false}))
    (println "Web server started.")))

(defn stop-webserver []
  (when-not (nil? @server)
    (.stop @server)
    (reset! server nil)
    (println "Web server stopped.")))

(comment (start-webserver))
(comment (stop-webserver))

It's certeinly possible with Clojure2d or Quil. But what do you want to achieve precisely as the result? Path points?

yes i need the path points, at least a certain intervals, in order to calculate the area under sections of the curve

So there are two steps for that, one is a visual tool (Clojure2d or Quil can be useful here) to create the gui (adding/removing/moving points around), the second is to build interpolation between points. You can use fastmath.interpolation namespace for example to build a function which can be easily integrated.

Thanksss !!!

For a full-blown graph-based visual approach to Clojure computation, you may be interested in: https://www.datarabbit.com/

@U04RG9F8UJZ Hi. Can you make a repro (a single file with all I need to know to lint and see the problem?)

Thanks @U04V15CAJ You'll need a project with hiccup as a dependency, and then you can reproduce the linting error as shown here: https://gist.github.com/jbullers/4db8ce3f4ff84ad47c43c6267f7a5c01 Let me know if I should provide something more

@U04RG9F8UJZ Can you also provide the deps.edn with the exact version of hiccup you were using?

I'm now seeing this: and I'm not sure if that's the same you're seeing

@U04V15CAJ I added a deps.edn to the gist. I was also getting all of those functions highlighted (unknown var), but after restarting vs code it worked itself out and I'm left with the arity error described earlier

I'm still getting unresolved-var

which is weird, I expect also this to work, I'll take a closer look

oh sigh.. the config.edn should be in .clj-kondo

I recommend going with this config:

{:lint-as
 ;; This helped remove other linting errors when I used the defelem and defhtml macros
 {hiccup.def/defelem clj-kondo.lint-as/def-catch-all
  hiccup.def/defhtml clj-kondo.lint-as/def-catch-all}}

I also opened https://github.com/weavejester/hiccup/issues/201

Wow 😮 why does that fix it?

Oh, hmm. It seems like it's basically an ignore. If I add more args, it doesn't flag that as wrong

true, but it will still see that the var exists

if you want more sophisticated linting, I recommend writing a hook, I think a macroexpand hook will be the easiest: https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md#macroexpand

welcome... a) please use a thread b) you can reduce or apply with dissoc to supply a list of keys (otherwise dissoc tries to remove the entry with that list as the key) c) you could also use select-keys

(reduce dissoc {:a 1 :b 2 :c 3} [:a :c])
=> {:b 2}
(apply dissoc {:a 1 :b 2 :c 3} [:a :c])
=> {:b 2}
(select-keys {:a 1 :b 2 :c 3} [:a :b])
=> {:a 1, :b 2}

(sorry saw you already put that)

OMG! So many single-line posts here. I didn't even see the thread already started, and tried to thread off another one. --- dissoc takes multiple keys as inputs, not a collection of keys. So, you need to use apply to spread the collection into the arguments list. FYI ,`(remove f coll)` is the same as (filter (complement f) coll).

(def keepers #{:def :ghi})

(def foos {:abc 1
           :def 2
           :ghi 3
           :jkl 4})

(defn keep-keys [ks m]
  (apply dissoc m (remove ks (keys m))))

(keep-keys keepers foos)

; =>
{:def 2, :ghi 3}