Fork me on GitHub
#beginners
<
2020-09-23
>
João Galrito02:09:50

what is the best practice to keep a thread running inside a with-open block

seancorfield02:09:24

You'd have to wait for it to complete inside the with-open block.

seancorfield02:09:54

Otherwise, the resource the thread is using could be closed before it was done.

João Galrito02:09:21

I'm aware of that, the question was more on which mechanism to use for the waiting

João Galrito02:09:31

i'm spawning 2 threads inside the with-open

João Galrito02:09:48

future + deref?

seancorfield02:09:28

There are lots of ways to do it. Hard to say what's best without knowing a bit more about exactly what you're doing.

João Galrito02:09:10

i'm opening a stream, reading from it into a buffer on one thread, and the other is locked waiting for a signal to start processing from the buffer

seancorfield02:09:52

future/`deref` is a simple but fairly uncontrolled way to do it. Using Java threads directly or an executor or... all sorts of possibilities depending on how much control you wanted over this.

João Galrito02:09:47

control in terms of?

João Galrito02:09:45

these threads should be running in the background

João Galrito02:09:09

with no interaction other than the data they're receiving and processing

João Galrito02:09:18

(defn watch-collection [collection-name signal]
  (println (str "Waiting for changes in " collection-name))
  (with-open [watch-cursor (.watch (get-collection collection-name))]
    (let [buffered-channel (async/chan (async/sliding-buffer 100))]
      (map deref [(future (-> watch-cursor
                              .iterator
                              iterator-seq
                              (map #(async/>! buffered-channel %))))
                  (future (do (async/<! signal)
                              (loop []
                                (when-let [change (async/<! buffered-channel)]
                                  (println change)
                                  (recur)))))]))))

seancorfield02:09:35

map is lazy so that is not safe.

João Galrito02:09:50

need a dorun after

seancorfield02:09:56

Safer to use run! there since you want the side-effects.

seancorfield02:09:04

(run! defer [...])

João Galrito02:09:22

other than that

João Galrito02:09:27

would something like the above work?

ghadi02:09:38

No. You cannot do <! outside a go block

João Galrito02:09:18

i'm calling (go (watch-collection

ghadi02:09:18

doseq + !>

João Galrito02:09:30

should the go go inside the function body?

João Galrito02:09:49

or use another mechanism

ghadi02:09:04

go blocks only work on immediate lexical scope; the call inside #() is a new scope

ghadi02:09:54

Consider letting the caller of this function supply a channel onto which changes are published

seancorfield02:09:31

(I always cringe a bit when I see beginners trying to use core.async 👀 -- it is really not easy to learn/use and there are so many ways to shoot yourself in the foot with it...)

seancorfield02:09:11

It's always felt like one of those "Here Be Dragons!" things in Clojure to me...

João Galrito02:09:48

@ghadi but both the pushing and pulling from the channel are happening inside this function

ghadi02:09:28

I am reading on my phone and missed the context... sorry! What is it you are doing?

João Galrito02:09:43

@ghadi i'm opening a stream, reading from it into a buffer on one thread, and the other is locked waiting for a signal to start processing from the buffer

João Galrito02:09:21

and I need to keep the threads alive inside the with-open block

João Galrito02:09:59

@seancorfield well I'm not one to shy away from a new challenge 🙂 that's why I came to clojure

João Galrito03:09:49

@ghadi my current attempt

(defn watch-collection [collection-name signal]
  (println "Start watching" collection-name)
  (with-open [watch-cursor (.watch (get-collection collection-name))]
    (let [buffered-channel (async/chan (async/sliding-buffer 1000))] ;buffer size might need tweaking
      (run! #(async/<!! %) [(async/thread
                              (->> watch-cursor
                                   .iterator
                                   iterator-seq
                                   (run! #(async/>!! buffered-channel %))))
                            (async/thread
                              (async/<!! signal)
                              (println (str "Waiting for changes in " collection-name))
                              (loop []
                                (when-let [change (async/<!! buffered-channel)]
                                  (println collection-name change)
                                  (recur))))]))))

Nazar Trut03:09:04

I need help, For example, if my sequence is '((if a b)(if a c), and i want to find "if" and then "b" and want to return true because i found "b" with a "if", would i use some or filter?

Nazar Trut03:09:47

(filter #(and (sequential? %) (= (first %) 'if) (= (second %) expr)) kb)
For example with this code, how would i change (second %) so it looks for the third index of a sequence?

seancorfield03:09:07

What do you mean by "third index of a sequence"?

seancorfield03:09:14

Sequences are not indexed.

Nazar Trut03:09:19

(if a b), i want to look at b

seancorfield03:09:39

Instead of (second %) you want (nth % 2)

seancorfield03:09:02

Or (second (rest %)) would also work

Nazar Trut03:09:30

I was doing (nth 2 %) the whole time

João Galrito03:09:06

a lot of the functions that manipulate colls take them as the last arg

seancorfield03:09:55

@ntrut you need to get used to doing things like (doc nth)

seancorfield03:09:37

@joao.galrito No, sequence functions take the sequence as the last argument. Collection functions take the collection as the first argument.

João Galrito03:09:37

I still mix the two a bit

seancorfield03:09:42

Most of the time, if the first argument is a collection, the function will have some specific guarantees about performance and/or return type.

seancorfield03:09:22

If the last argument is a sequence, and you pass a collection, it will be coerced to a sequence first. For example (map inc [1 2 3]) will call seq on the vector and then process it as a sequence.

Nazar Trut03:09:33

(filter #(and (sequential? %) (= (first %) 'if) (= (second (rest %) ) 'b)) '(if a b))
returning nil, any ideas why?

Nazar Trut03:09:27

@seancorfield thanks for the help btw

Nazar Trut03:09:34

i owe you guys

seancorfield03:09:01

Because you're passing an expression instead of a list of expressions (again 🙂 )

seancorfield03:09:26

filter expecs ( (if a b) ) remember @ntrut?

João Galrito03:09:43

it's calling sequential? on 'if, then 'a, then 'b

Nazar Trut03:09:30

Oh i see my mistake now

seancorfield03:09:28

@ntrut Some suggested viewing for you https://www.youtube.com/watch?v=Qx0-pViyIDU and https://www.youtube.com/watch?v=FihU5JxmnBg -- two talks on debugging techniques by Stuart Halloway of Cognitect. I think you'll find they'll help you figure out some of the problems you are running into.

seancorfield03:09:15

(you might also want to track down Stu's "REPL-Driven Development" talk as well)

Mark Wardle12:09:13

Hi all. Any recommendations for simple websockets client and server implementations in clojure on the jvm for both client and server? Nothing fancy, no need for fallbacks etc. Needed as a way of linking two servers to exchange information where firewalls block other types of socket and bidirectional flow would be better than polling.

João Galrito12:09:28

never used it, just saw it yesterday somewhere

João Galrito12:09:56

not sure if that's what you need

Mark Wardle12:09:53

Thanks I have looked at it but as I’m not an expert in this, on first inspection it has more functionality than I think I need. Of course, in its favour, it does seem to be the top hit on search - and is maintained, and the API looks good!

João Galrito12:09:47

in the README there's also a couple other alternatives he recommends

Mark Wardle12:09:43

Thanks. I’ve seen a variety of options including sente, direct http-kit usage - although the examples / docs use now deprecated methods, immutant, aleph, and I’ve looked at wrapping java libraries directly (e.g. Java-WebSocket) which actually I think was just added to sente for the java client bit. Up until then, it looks as if the main tooling was cljs for client and clj for server, because of the browser focus, but websockets now being used more and falling back to AJAX (as per sente) and starting connection using HTTP POST isn’t necessary for my use case. Hence just asking for advice. Thank you

Test This17:09:37

Hi Mark, I am very new to clojure, so this is not coming from any experience in production. But I am in the same boat as you. My experiments suggest that reitit ring and http-kit can make for a fairly easy setup.

Mark Wardle18:09:51

Thanks. I have an aleph client and server talking nicely to one another - but there’s a lot of new stuff/patterns to learn for me to fully understand how it’s working! My comparison is a golang based library : http://nhooyr.io/websocket

Mark Wardle06:09:45

See https://aleph.io/examples/literate.html#aleph.examples.websocket But you can use sente as both a clj client and server as well.

jacklombard12:09:10

Is there an any type in lacinia that I can use while defining the schema?

João Galrito13:09:33

when using Cursive, do I only get method name resolution if I import the corresponding classes?

Test This14:09:35

Are there any simple examples of using reitit, ring and http-kit where some routes are regular routes and a websocket route?

teodorlu14:09:08

Not sure if you'll find an exact match, but the reitit repository provides plenty of examples: https://github.com/metosin/reitit/tree/master/examples

Test This14:09:36

May be I am not seeing properly. But none of them are with websockets. Is reitit capable of routing websocket requests? Thank you.

ikitommi14:09:38

haven't seen a public example, but we have been using reitit + ws since day1. Just mount a ws-endpoint into a route like you would do with any other routing lib, like compojure. Sente has examples of this

Test This15:09:08

Thank you @U055NJ5CC. Can you please see the ws route below. I understand that the route is not set up to handle websocket connections. Also when I visit using the url using a regular browser window, I know it will fails. But I expect the httpkit/websocket? to work as described on page Would that now print out`false`. But instead I get an error saying: java.lang.IllegalArgumentException: No implementation of method: :websocket? of protocol: #'org.httpkit.server/Channel found for class: clojure.lang.PersistentHashMap. I am very new to clojure. Just trying to figure out what to use for webapps. Thank you. My code:

(ns sampleapp 
  (:require
   [reitit.ring :as ring]
   [ring.middleware.params :as params]
   [ring.middleware.multipart-params :as multparams]
   [org.httpkit.server :as httpkit]
  ;  [ring.adapter.jetty :as jetty]
  ;  [ring.adapter.jetty9 :as jetty]
    )
  (:gen-class))

(defn wrap [handler val]
  (fn [request]
    (let [currval (get request :val [])]
      (handler (assoc request :val (conj currval val))))))



(defn jerryhandler [req]
  (println (str "Wrapped value: " (:val req)))
  {:status 200
   :headers {"Content-type" "text/plain"}
   :body (str "Hello Jerry!!\nI got values " (:val req))})

(defn busterhandler [req]
  (println (str "Parameters: " (:params req)))
  {:status 200
   :headers {"Content-type" "text/plain"}
   :body "Hello Buster!!"})

(defn fileupload [req]
  (println (str "File info: " (get-in req [:params "file"])))
  (let [fileobj (get-in req [:params "file" :tempfile])
        pathtofile (.getAbsolutePath fileobj)
        contents (slurp fileobj)]
    (println contents)
    (println pathtofile)
    {:status 200
     :headers {"Content-type" "text/plain"}
     :body "File uploaded..."})
  )

(defn wshandler [req] 
  (println (httpkit/websocket? req))
  {:status 200 :headers {"Content-type" "text/plain" :body "Not valid ws request..."}})

(def router
  (ring/router 
   [["/hellojerry" {:get {:middleware [[wrap 1] [wrap 2]] :handler jerryhandler}}]
    ["/hellobuster" {:get (params/wrap-params busterhandler)}]
    ["/fileupload" {:post (multparams/wrap-multipart-params fileupload)}]
    ["/ws" wshandler]]
   ))

(def app
  (ring/ring-handler
   router
   (constantly {:status 404 :body "Not found!"}))
  )


(defn -main
  [& args]
  (httpkit/run-server app {:port 3000})
  ; (jetty/run-jetty app {:port 3000 :h2? true}))
)

  

ikitommi15:09:54

I believe you need to pull out the channel linked to the request to get web-sockets with http-kit. There seems to be an example here: https://http-kit.github.io/server.html#channel

Test This15:09:47

Thank you @U055NJ5CC. I believe the linked approach is deprecated. However, the example given here: https://http-kit.github.io/http-kit/org.httpkit.server.html#var-as-channel, works well. It appears that my mistake was using (httpkit/websocket?, when, in fact, I was supposed to be using just (`:websocket?` ).

Test This15:09:09

Does https://github.com/ptaoussanis/sente require using it for both server and client, or can you use it only on the server with standard Javascript websocket on the client? Thank you.

Clypto17:09:41

there are examples for connecting other types of infrastructure https://github.com/theasp/sente-nodejs-example

Clypto17:09:14

if you wanted a non-clojure or non-clojurescript participant, you'd have to conform to the message passing. It looks like they do have some emulation on web sockets to make up for web sockets short comings, so that may be a little tricky

Nazar Trut19:09:35

What is a way that i can check if there is two "nots" in my sequence like this '(not (not b)), so therefore, if there is two "not" like this example.

Nazar Trut19:09:07

(not(not a)) would be true

Nazar Trut19:09:15

(not a) would be false

seancorfield19:09:42

I would define a predicate that returned true if an expression was (not X) and then you can do (and (has-not? expr) (has-not? (second expr))) if you make has-not? robust (check for sequential expr and then check first item is 'not)

Michael Stokley23:09:39

clojure collections are not falsey. in another dynamic language i might test for the presence or absence of a given value of an unknown type with if . is there a conventional way to do this in clojure? so far i have (or (nil? x) (and (seqable? x) (empty? x)))

phronmophobic23:09:32

I think you can just do (if (empty? x) ...)

Michael Stokley23:09:54

that throws on integers

Michael Stokley23:09:43

i'm reminded of the little schemer where the first question you always ask is atom?...

phronmophobic23:09:13

indeed. what are you trying to do?

Michael Stokley23:09:18

i'm writing a general purpose "is this var value missing?" predicate for use with values of an unknown type

Michael Stokley23:09:33

maybe i shouldn't be doing that...

Michael Stokley23:09:39

i could do additional work to bring through the type, i suppose

phronmophobic23:09:11

trying to define a universal "missing?" predicate always reminds of https://bugs.python.org/issue13936

Michael Stokley23:09:17

ok, maybe i should rework such that i'm starting with a map where absence is represented by a missing key

Michael Stokley23:09:43

that would be more clojuristic

phronmophobic23:09:28

basically, it's highly context dependent and it might be ok if you're defining the context, but it's trouble if you're defining the context for someone else and may be confusing for others to have to figure out what your definition of "missing" is

👍 3
phronmophobic23:09:26

I much prefer clojure's approach compared to something like javascript: > All values are truthy unless they are defined as https://developer.mozilla.org/en-US/docs/Glossary/Falsy (i.e., except for false, 0, -0, `0n`, "", null, undefined, and NaN). 🤢

Michael Stokley23:09:42

should i use (seq x) instead of (empty? x) ?