Fork me on GitHub
#beginners
<
2020-09-23
>
joao.galrito02:09:50

what is the best practice to keep a thread running inside a with-open block

seancorfield02:09:24

You'd have to wait for it to complete inside the with-open block.

seancorfield02:09:54

Otherwise, the resource the thread is using could be closed before it was done.

joao.galrito02:09:21

I'm aware of that, the question was more on which mechanism to use for the waiting

joao.galrito02:09:31

i'm spawning 2 threads inside the with-open

joao.galrito02:09:48

future + deref?

seancorfield02:09:28

There are lots of ways to do it. Hard to say what's best without knowing a bit more about exactly what you're doing.

joao.galrito02:09:10

i'm opening a stream, reading from it into a buffer on one thread, and the other is locked waiting for a signal to start processing from the buffer

seancorfield02:09:52

future/`deref` is a simple but fairly uncontrolled way to do it. Using Java threads directly or an executor or... all sorts of possibilities depending on how much control you wanted over this.

joao.galrito02:09:47

control in terms of?

joao.galrito02:09:45

these threads should be running in the background

joao.galrito02:09:09

with no interaction other than the data they're receiving and processing

joao.galrito02:09:18

(defn watch-collection [collection-name signal]
  (println (str "Waiting for changes in " collection-name))
  (with-open [watch-cursor (.watch (get-collection collection-name))]
    (let [buffered-channel (async/chan (async/sliding-buffer 100))]
      (map deref [(future (-> watch-cursor
                              .iterator
                              iterator-seq
                              (map #(async/>! buffered-channel %))))
                  (future (do (async/<! signal)
                              (loop []
                                (when-let [change (async/<! buffered-channel)]
                                  (println change)
                                  (recur)))))]))))

seancorfield02:09:35

map is lazy so that is not safe.

joao.galrito02:09:50

need a dorun after

seancorfield02:09:56

Safer to use run! there since you want the side-effects.

seancorfield02:09:04

(run! defer [...])

joao.galrito02:09:22

other than that

joao.galrito02:09:27

would something like the above work?

ghadi02:09:38

No. You cannot do <! outside a go block

joao.galrito02:09:18

i'm calling (go (watch-collection

ghadi02:09:18

doseq + !>

joao.galrito02:09:30

should the go go inside the function body?

joao.galrito02:09:49

or use another mechanism

ghadi02:09:04

go blocks only work on immediate lexical scope; the call inside #() is a new scope

ghadi02:09:54

Consider letting the caller of this function supply a channel onto which changes are published

seancorfield02:09:31

(I always cringe a bit when I see beginners trying to use core.async 👀 -- it is really not easy to learn/use and there are so many ways to shoot yourself in the foot with it...)

seancorfield02:09:11

It's always felt like one of those "Here Be Dragons!" things in Clojure to me...

joao.galrito02:09:48

@ghadi but both the pushing and pulling from the channel are happening inside this function

ghadi02:09:28

I am reading on my phone and missed the context... sorry! What is it you are doing?

joao.galrito02:09:43

@ghadi i'm opening a stream, reading from it into a buffer on one thread, and the other is locked waiting for a signal to start processing from the buffer

joao.galrito02:09:21

and I need to keep the threads alive inside the with-open block

joao.galrito02:09:59

@seancorfield well I'm not one to shy away from a new challenge 🙂 that's why I came to clojure

joao.galrito03:09:49

@ghadi my current attempt

(defn watch-collection [collection-name signal]
  (println "Start watching" collection-name)
  (with-open [watch-cursor (.watch (get-collection collection-name))]
    (let [buffered-channel (async/chan (async/sliding-buffer 1000))] ;buffer size might need tweaking
      (run! #(async/<!! %) [(async/thread
                              (->> watch-cursor
                                   .iterator
                                   iterator-seq
                                   (run! #(async/>!! buffered-channel %))))
                            (async/thread
                              (async/<!! signal)
                              (println (str "Waiting for changes in " collection-name))
                              (loop []
                                (when-let [change (async/<!! buffered-channel)]
                                  (println collection-name change)
                                  (recur))))]))))

ntrut03:09:04

I need help, For example, if my sequence is '((if a b)(if a c), and i want to find "if" and then "b" and want to return true because i found "b" with a "if", would i use some or filter?

ntrut03:09:47

(filter #(and (sequential? %) (= (first %) 'if) (= (second %) expr)) kb)
For example with this code, how would i change (second %) so it looks for the third index of a sequence?

seancorfield03:09:07

What do you mean by "third index of a sequence"?

seancorfield03:09:14

Sequences are not indexed.

ntrut03:09:19

(if a b), i want to look at b

seancorfield03:09:39

Instead of (second %) you want (nth % 2)

seancorfield03:09:02

Or (second (rest %)) would also work

ntrut03:09:30

I was doing (nth 2 %) the whole time

joao.galrito03:09:06

a lot of the functions that manipulate colls take them as the last arg

seancorfield03:09:55

@ntrut you need to get used to doing things like (doc nth)

seancorfield03:09:37

@joao.galrito No, sequence functions take the sequence as the last argument. Collection functions take the collection as the first argument.

joao.galrito03:09:37

I still mix the two a bit

seancorfield03:09:42

Most of the time, if the first argument is a collection, the function will have some specific guarantees about performance and/or return type.

seancorfield03:09:22

If the last argument is a sequence, and you pass a collection, it will be coerced to a sequence first. For example (map inc [1 2 3]) will call seq on the vector and then process it as a sequence.

ntrut03:09:33

(filter #(and (sequential? %) (= (first %) 'if) (= (second (rest %) ) 'b)) '(if a b))
returning nil, any ideas why?

ntrut03:09:27

@seancorfield thanks for the help btw

ntrut03:09:34

i owe you guys

seancorfield03:09:01

Because you're passing an expression instead of a list of expressions (again 🙂 )

seancorfield03:09:26

filter expecs ( (if a b) ) remember @ntrut?

joao.galrito03:09:43

it's calling sequential? on 'if, then 'a, then 'b

ntrut03:09:30

Oh i see my mistake now

seancorfield03:09:28

@ntrut Some suggested viewing for you https://www.youtube.com/watch?v=Qx0-pViyIDU and https://www.youtube.com/watch?v=FihU5JxmnBg -- two talks on debugging techniques by Stuart Halloway of Cognitect. I think you'll find they'll help you figure out some of the problems you are running into.

seancorfield03:09:15

(you might also want to track down Stu's "REPL-Driven Development" talk as well)

mark35412:09:13

Hi all. Any recommendations for simple websockets client and server implementations in clojure on the jvm for both client and server? Nothing fancy, no need for fallbacks etc. Needed as a way of linking two servers to exchange information where firewalls block other types of socket and bidirectional flow would be better than polling.

joao.galrito12:09:28

never used it, just saw it yesterday somewhere

joao.galrito12:09:56

not sure if that's what you need

mark35412:09:53

Thanks I have looked at it but as I’m not an expert in this, on first inspection it has more functionality than I think I need. Of course, in its favour, it does seem to be the top hit on search - and is maintained, and the API looks good!

joao.galrito12:09:47

in the README there's also a couple other alternatives he recommends

mark35412:09:43

Thanks. I’ve seen a variety of options including sente, direct http-kit usage - although the examples / docs use now deprecated methods, immutant, aleph, and I’ve looked at wrapping java libraries directly (e.g. Java-WebSocket) which actually I think was just added to sente for the java client bit. Up until then, it looks as if the main tooling was cljs for client and clj for server, because of the browser focus, but websockets now being used more and falling back to AJAX (as per sente) and starting connection using HTTP POST isn’t necessary for my use case. Hence just asking for advice. Thank you

curiouslearn17:09:37

Hi Mark, I am very new to clojure, so this is not coming from any experience in production. But I am in the same boat as you. My experiments suggest that reitit ring and http-kit can make for a fairly easy setup.

mark35418:09:51

Thanks. I have an aleph client and server talking nicely to one another - but there’s a lot of new stuff/patterns to learn for me to fully understand how it’s working! My comparison is a golang based library : http://nhooyr.io/websocket

mark35406:09:45

See https://aleph.io/examples/literate.html#aleph.examples.websocket But you can use sente as both a clj client and server as well.

frozenfire199212:09:10

Is there an any type in lacinia that I can use while defining the schema?

joao.galrito13:09:33

when using Cursive, do I only get method name resolution if I import the corresponding classes?

curiouslearn14:09:35

Are there any simple examples of using reitit, ring and http-kit where some routes are regular routes and a websocket route?

teodorlu14:09:08

Not sure if you'll find an exact match, but the reitit repository provides plenty of examples: https://github.com/metosin/reitit/tree/master/examples

curiouslearn14:09:36

May be I am not seeing properly. But none of them are with websockets. Is reitit capable of routing websocket requests? Thank you.

ikitommi14:09:38

haven't seen a public example, but we have been using reitit + ws since day1. Just mount a ws-endpoint into a route like you would do with any other routing lib, like compojure. Sente has examples of this

curiouslearn15:09:08

Thank you @. Can you please see the ws route below. I understand that the route is not set up to handle websocket connections. Also when I visit using the url using a regular browser window, I know it will fails. But I expect the httpkit/websocket? to work as described on page Would that now print out`false`. But instead I get an error saying: java.lang.IllegalArgumentException: No implementation of method: :websocket? of protocol: #'org.httpkit.server/Channel found for class: clojure.lang.PersistentHashMap. I am very new to clojure. Just trying to figure out what to use for webapps. Thank you. My code:

(ns sampleapp 
  (:require
   [reitit.ring :as ring]
   [ring.middleware.params :as params]
   [ring.middleware.multipart-params :as multparams]
   [org.httpkit.server :as httpkit]
  ;  [ring.adapter.jetty :as jetty]
  ;  [ring.adapter.jetty9 :as jetty]
    )
  (:gen-class))

(defn wrap [handler val]
  (fn [request]
    (let [currval (get request :val [])]
      (handler (assoc request :val (conj currval val))))))



(defn jerryhandler [req]
  (println (str "Wrapped value: " (:val req)))
  {:status 200
   :headers {"Content-type" "text/plain"}
   :body (str "Hello Jerry!!\nI got values " (:val req))})

(defn busterhandler [req]
  (println (str "Parameters: " (:params req)))
  {:status 200
   :headers {"Content-type" "text/plain"}
   :body "Hello Buster!!"})

(defn fileupload [req]
  (println (str "File info: " (get-in req [:params "file"])))
  (let [fileobj (get-in req [:params "file" :tempfile])
        pathtofile (.getAbsolutePath fileobj)
        contents (slurp fileobj)]
    (println contents)
    (println pathtofile)
    {:status 200
     :headers {"Content-type" "text/plain"}
     :body "File uploaded..."})
  )

(defn wshandler [req] 
  (println (httpkit/websocket? req))
  {:status 200 :headers {"Content-type" "text/plain" :body "Not valid ws request..."}})

(def router
  (ring/router 
   [["/hellojerry" {:get {:middleware [[wrap 1] [wrap 2]] :handler jerryhandler}}]
    ["/hellobuster" {:get (params/wrap-params busterhandler)}]
    ["/fileupload" {:post (multparams/wrap-multipart-params fileupload)}]
    ["/ws" wshandler]]
   ))

(def app
  (ring/ring-handler
   router
   (constantly {:status 404 :body "Not found!"}))
  )


(defn -main
  [& args]
  (httpkit/run-server app {:port 3000})
  ; (jetty/run-jetty app {:port 3000 :h2? true}))
)

  

ikitommi15:09:54

I believe you need to pull out the channel linked to the request to get web-sockets with http-kit. There seems to be an example here: https://http-kit.github.io/server.html#channel

curiouslearn15:09:47

Thank you @. I believe the linked approach is deprecated. However, the example given here: https://http-kit.github.io/http-kit/org.httpkit.server.html#var-as-channel, works well. It appears that my mistake was using (httpkit/websocket?, when, in fact, I was supposed to be using just (`:websocket?` ).

curiouslearn15:09:09

Does https://github.com/ptaoussanis/sente require using it for both server and client, or can you use it only on the server with standard Javascript websocket on the client? Thank you.

clypto17:09:41

there are examples for connecting other types of infrastructure https://github.com/theasp/sente-nodejs-example

clypto17:09:14

if you wanted a non-clojure or non-clojurescript participant, you'd have to conform to the message passing. It looks like they do have some emulation on web sockets to make up for web sockets short comings, so that may be a little tricky

ntrut19:09:35

What is a way that i can check if there is two "nots" in my sequence like this '(not (not b)), so therefore, if there is two "not" like this example.

ntrut19:09:07

(not(not a)) would be true

ntrut19:09:15

(not a) would be false

seancorfield19:09:42

I would define a predicate that returned true if an expression was (not X) and then you can do (and (has-not? expr) (has-not? (second expr))) if you make has-not? robust (check for sequential expr and then check first item is 'not)

michael74023:09:39

clojure collections are not falsey. in another dynamic language i might test for the presence or absence of a given value of an unknown type with if . is there a conventional way to do this in clojure? so far i have (or (nil? x) (and (seqable? x) (empty? x)))

smith.adriane23:09:32

I think you can just do (if (empty? x) ...)

michael74023:09:54

that throws on integers

michael74023:09:43

i'm reminded of the little schemer where the first question you always ask is atom?...

smith.adriane23:09:13

indeed. what are you trying to do?

michael74023:09:18

i'm writing a general purpose "is this var value missing?" predicate for use with values of an unknown type

michael74023:09:33

maybe i shouldn't be doing that...

michael74023:09:39

i could do additional work to bring through the type, i suppose

smith.adriane23:09:11

trying to define a universal "missing?" predicate always reminds of https://bugs.python.org/issue13936

michael74023:09:17

ok, maybe i should rework such that i'm starting with a map where absence is represented by a missing key

michael74023:09:43

that would be more clojuristic

smith.adriane23:09:28

basically, it's highly context dependent and it might be ok if you're defining the context, but it's trouble if you're defining the context for someone else and may be confusing for others to have to figure out what your definition of "missing" is

smith.adriane23:09:26

I much prefer clojure's approach compared to something like javascript: > All values are truthy unless they are defined as https://developer.mozilla.org/en-US/docs/Glossary/Falsy (i.e., except for false, 0, -0, `0n`, "", null, undefined, and NaN). :nauseated_face:

michael74023:09:42

should i use (seq x) instead of (empty? x) ?