Fork me on GitHub
#beginners
<
2017-10-04
>
lvbarbosa01:10:02

I am trying to think about how to build this small system, just for fun/learning purposes, and I’d like to hear some advise from you guys. I want to build a chat server in Clojure. The idea is: clients connect (tcp socket) to the server and receive a prompt for either A: create a new room or B: connect to an existing room. After entering a room (either newly created or existing one), you can broadcast messages there and receive messages from other people on the same room. I would like to see events too, like “John disconnected” or “Paul connected”. Since the sole purpose of this is learning, I want to avoid using third party libraries. I want plain old Java ServerSocket and Socket. Would you give me architectural ideas? I guess this is a good scenario for core.async, but I am having a bad time thinking about how to structure the application

lvbarbosa01:10:11

All advise is welcome

noisesmith01:10:18

@lvbarbosa what about an atom with room names as keys and socket connections as values?

noisesmith01:10:37

then, for each message sent to a room, you iterate and send it to each socket under that value

noisesmith01:10:36

after that, it's just a question of listening on each connected socket (which is of course reverse-mapped to room also)

lvbarbosa01:10:51

How would I detect a new message? This is something I can’t figure out

lvbarbosa01:10:00

Should I have a thread listening for each socket?

lvbarbosa01:10:20

or maybe worker threads scanning each socket for a bit of time and then going to the next one

noisesmith01:10:21

the trivial thing (that doesn't scale but works for the first hundreds of clients) is a thread created with future, one for each socket connection

noisesmith01:10:47

if you need more than a thousand or so clients, then you get into nio nonblocking stuff and maybe core.async for coordination

lvbarbosa01:10:09

I guess I’ll go with the simpler solution for now

noisesmith01:10:15

you don't need to scan, each thread can do a blocking read on a socket

noisesmith01:10:44

yeah - in my experience doing the simple thing that works first is easier than trying to optimize first and then realizing you need a completely different optimization

lvbarbosa01:10:48

but wouldn’t that require a single thread per connection?

noisesmith01:10:59

right, and up to the first 1000 connections or so that just works

noisesmith02:10:02

no need to get fancy

lvbarbosa02:10:43

After that, in case I want to improve it, I should seek java.nio for async stuff?

noisesmith02:10:45

the OS knows it can't wake your thread up until input is ready, so it's not like perf is super bad (though of course nio would perform better)

noisesmith02:10:10

nio of the io part, core.async for the coordination of the async results (if needed) yeah

lvbarbosa02:10:54

awesome! I will try the simple alternative for now 😄

lvbarbosa02:10:05

I’ll share it here as soon as I have something working

lvbarbosa02:10:08

thanks a lot @noisesmith

noisesmith02:10:33

also, aleph makes a lot of this async network stuff easier in clojure

noisesmith02:10:46

(though I realize you did say you wanted to use the raw java libs)

lvbarbosa02:10:07

I will take a look into that, maybe I get some ideas

lvbarbosa02:10:21

Yeah, I also want to learn network communication

lvbarbosa02:10:36

I know the theory, but I don’t have much practice

noisesmith02:10:15

the idea with aleph (and it's support lib manifold) is to leverage ztellman's experience and provide a bunch of the little stuff you don't get by default but you will really want if performance starts to matter

lvbarbosa02:10:38

note taken! I’m sure it will be useful later

lvbarbosa02:10:42

thanks again

thaddeus.aid08:10:25

I have a question about om I want to update the app-state but I am failing to find the correct way to do it. Can someone please point me in the correct direction?

manutter5112:10:27

I would suggest looking up the tutorial for Falcor, which is part of the inspiration for Om. I had a bit of trouble understanding Om back when I first started, and found the Falcor tutorials much later, and was like, "Wow, wish I'd seen this sooner"

thaddeus.aid14:10:57

awesome, thanks mate. I ended up coding all night to get the new version ready and I think I have fallen in love with Clojurescript in the same way I hate Javascript.

genec10:10:52

any idea why this fails to match? (let [a 1] (case (class a) java.lang.String a java.lang.Integer (str a) java.lang.Long (str a) java.lang.Double (str a)))

sundarj11:10:21

it's because the test constants in case are not evaluated

sundarj11:10:33

boot.user=> (case 'java.lang.Long
       #_=>   java.lang.Long 2)
2

genec10:10:52

IllegalArgumentException No matching clause: class java.lang.Long

genec10:10:59

using (condp = (class a) ...works. was there a breaking change some time ago as I'm working thought an old example on the prag prog blog

bandresen10:10:12

I can't help you but I couldn't help smiling at that problem, giving that prag prog had a chapter on "select isn't broken" 😛

genec12:10:33

what do you mean select isn't broken?

oliv13:10:54

I’m doing a clojure application but one thing bother me. I don’t know where/when should I close the version ? I’m still with snapshot, how you guys normally does that ?

manutter5113:10:38

Trimming off the "-SNAPSHOT" is usually the last thing I do before bundling the app up as an official release. Edit the project.clj, build the app without the "-SNAPSHOT", and ship it. Then I usually edit project.clj, bump the version number, and re-add "-SNAPSHOT"

manutter5113:10:23

Caveat: my personal experience shipping one in-house app. 🙂

lilactown16:10:25

I'm trying to convert a function that takes a callback to use a channel

lilactown16:10:45

(defn doThingCh [] (let [c (async/chan)]
    (doThing 
      (fn [res]
                 (println "putting c" res)
                 (async/put! c res)
                 (println "close c")
                 (async/close! c))
    c))

lilactown16:10:14

...and as I'm typing the rest of my question I figured it out 😆

lilactown16:10:00

I was taking it off the channel and expecting a field to live on the response that doesn't exist

madstap16:10:20

Anyone know the shortcut to eval-form-at-point in nightcode? I only use nightcode occasionally and I'm pretty certain it had that shortcut last time.

rinaldi21:10:52

I am performing a bunch of isolated tasks in an async manner.

(defn google-analytics [channel]
  (go
    (Thread/sleep 1000)
    (>! channel :google-analytics)))

(defn facebook [channel]
  (go
    (>! channel :facebook)))i

(defn run []
  (let [pipeline (chan 2)]
    (google-analytics pipeline)
    (facebook pipeline)

    (go-loop []
      (let [message (<! pipeline)]
        (println "Got:" message))
      (recur))))
This code works as is but it leaves the channel "pending". I'm looking for the best way to detect whether I have collected all items that I care about from the channel so I can close! and log it. Ideas?

noisesmith21:10:02

@rinaldi read from the two channels returned by the go blocks, then close the channel?

noisesmith21:10:13

(in another go block, of course)

rinaldi21:10:24

I am not sure how to do that

noisesmith21:10:06

(let [pipeline (chan 2)
    fb (facebook pipeline)
    ga (google-analytics pipeline)]
  (go (<! fb)
      (<! ga)
      (close! pipeline))
   ...)

tkjone22:10:49

Hey all, sorry in advance, this question is long. Before I dive into a larger question, I was looking for an idiomatic way to findIndex on a list of maps.

(books ({id: 23} {id: 45}))
and noticed that findIndex is not part of the core library. This lead me to read through a section in Programming Clojure about choosing the correct data structure. To summarize, the idea seems to be that if you are going to be searching for an item, for example, using a function like findIndex (JS) against some data structure, you should be using a set. The idea is to communicate intent. This is why, to my understanding, findIndex is not part of the core library. What happens though when you are provided with a list, with a large dataset, and you need to use something like findIndex on it? Eventually, with large enough data, this becomes a performance issue point. Which goes against idiomatic clojure - do clojurists just sigh and move on, or is there a better way to handle this kind of situation? For example, you are writing clojurescript, and receive a a JS array of objects...this would become a list (as far as I know) . To loop back to the original question - what does idiomatic findIndex look like. Here is what I did to implement find like functionality:
(first (filter #(= (.. % -id) 823423) js/comments))

seancorfield22:10:51

Regardless of whether you have an array (Java/JS) or a list, a linear search for a matching item is always going to be O(n) in any language.

seancorfield22:10:15

^ @tkjone Generally, in Clojure(Script), if you want fast lookup by a unique ID, you just use a hash map instead: (books {23 {:id 23 :title "..."} 45 {:id 45 :title "..."}})

seancorfield22:10:03

You might be given an array, but if the id is unique, you can easily turn that into a hash map accessed by that value.

seancorfield22:10:14

(reduce (fn [m v] (assoc m (:id v) v)) {} book-list) for example

seancorfield22:10:19

(or, if you want to be a bit more "modern" about that: (into {} (map (juxt :id identity)) book-list) 🙂 )

tkjone22:10:40

Excellent. So the answer to the fundamental question: transform the data structure you are provided into another data structure.

tkjone22:10:36

In this vein, if you are presented with a task where the ask is to find the index of the book with id: 45 on this data structure:

const books = [
  { id: 32 },
  { id: 34 },
  { id: 45 }
  // ...
];
(apologies for the JS) The idiomatic way to handle this in clojure would be to transform the object into a hash map (given that the ID's are unique) and thus, you would never have to make a findIndex function.

seancorfield22:10:58

Right. Because findIndex is O(n) and slow if you are going to be repeating it.

seancorfield22:10:00

My question would be: what would you do with that index? Are you just trying to find the item (and you don't care about the collection after that)?

tkjone22:10:21

ha - good call. My next question was going to be whether or not our proposal of the idiomatic way would be handled differently based on what the index is being used for - our ultimate goal that is.

seancorfield22:10:07

If it really is just "find the item" (and ignore the rest), it's fairly idiomatic to say (first (filter (fn [v] (= (:id v) ??)) book-list)) -- that's not entirely efficient but...

seancorfield22:10:41

(where ?? would be whatever ID you're searching for)

seancorfield22:10:59

(remember in Clojure you don't update things in place because we're immutable 🙂 )

tkjone22:10:51

True. I imagine if I had a real world scenario I could answer this question a little better, but this is more a test problem. for example, the question was: implement findIndex in clojure.

tkjone22:10:18

But when I researched a little, this seems to be an odd ask in clojureland...where as in JS land, its a method on Array

tkjone22:10:38

so I am working on implementing like this:

(keep-indexed
     (fn [index item]
         (when (= (.. item -id) 45)
              index)) js/books)
just to show how it would be implemented, but if I was actually answering this question, I would say we would need to reframe the question.

tkjone22:10:52

Thanks, @seancorfield!

lvbarbosa23:10:18

Can someone explain to me how pmap works in plain english?

lvbarbosa23:10:27

I can’t understand the docs

lvbarbosa23:10:38

I understand the difference from it’s cousin map in that it runs in parallel, but what is running in parallel?

noisesmith23:10:02

it’s calling f in parallel, but not on all the args at once, it only runs it a specific number of threads, determined by how many CPUs you have

noisesmith23:10:10

in order to maximize throughput

noisesmith23:10:32

if you access the first item it returns, it will call a certain number of threads and realize those - but it is lazy in the sense that you need to consume all the results to ensure they all ran