Fork me on GitHub
#beginners
<
2016-10-14
>
adamkowalski04:10:54

Has anybody done event sourcing with Clojure? Ive been looking at Apache Kafka and it seems really interesting, but I am wondering if there is some pure Clojure alternative or a more idiomatic way to handle this problem

kauko04:10:23

@adamkowalski that's a question that you could certainly ask in the #clojure channel. Not really a beginner level question imo 🙂 But I think @viesti may have do ES with clojure

eslachance05:10:18

Hmm. Ok, noob on promises here. I have the following code and I can't seem to find a way to deref a specific promise inside the response.

(let [client (http.async.client/create-client)
      resp (http.async.client/POST
             client
             ""
             :body {:username "Dithcord" :content "Still Testing"}
             :headers {:content-type "application/x-www-form-urlencoded"}
             )
      ]
  (prn (@:status @resp))
  )
I've tried different variations of (:status @resp) and (@status resp) and others but I really can't get it to work.

eslachance05:10:33

I'm sure it's.... probably basic

eslachance05:10:28

ooooooh there we go. (prn @(response :status))

viesti06:10:05

@adamkowalski So I’ve been using Kafka (0.8.2) in an analytics project, piping user activity events from currently two main sources. Small http server that produces messages to Kafka and another set of apps that read and transform messages before ingesting to Redshift through S3. Both in Clojure, using clj-kafka (https://github.com/pingles/clj-kafka).

viesti06:10:09

don’t know the actual use case here, but Kafka has a lively ecosystem, we’r now looking into PipelineDB https://www.pipelinedb.com/ to satisfy our “domain experts” which are fluent in sql but not so fluent in Clojure, yet 🙂

viesti06:10:58

depends on what you need, there might be simpler alternatives (like if just a persistent queue, then https://github.com/Factual/durable-queue)

viesti06:10:29

but true, this is more an advanced question than beginner level, although some entry into this set of tools would be neat :)

viesti06:10:11

nowadays Kafka tries to orient to microservices more than just analytics (AWS Kinesis competes in this area, using "as a service product" is compelling), which makes using Kafka for ES compelling imho, although there is a learning/installation curve

viesti06:10:27

streams api has no Clojure wrapper currently, but probably not an issue

viesti06:10:16

broader topic this thing, involves data transfer whole company wise, if one does full buy-in into ES :)

eslachance06:10:12

Why does this look like pure spamming?

eslachance06:10:47

:thinking_face:

eslachance06:10:07

And no clear question

eslachance06:10:36

It actually does look like semi-random text spam with some links though, tbh.

eslachance07:10:03

However I just realized you were called here for a previous question lol

eslachance07:10:05

so nevermind me

eslachance07:10:05

It's just 3am and I'm trying to figure out this vague non-helpful error with no stack

viesti07:10:56

too fast response while jumping off the bus and walking to work “look more into these” spam of links 🙂

eslachance08:10:32

Alright. Question. I'm using http.client.async and I'm doing a post.... and I get a 411 Length Required response. How do I fix this? None of the code in the test suite seems to specify it 😞

mikeb13:10:08

@eslachance Perhaps set the content-length header?

eslachance16:10:24

@mikeb I tried that, I'm apparently not using headers properly because only the first one (Authorization) is being sent, the rest aren't part of the request if I am to believe the contents of resp:

(let [client (http/create-client :follow-redirects true)
        resp (http/POST
               client
               (str "" channel "/messages")
               :body {:content msg}
               :headers {"Authorization" (str "Bot " (get @session :token))
                        "Content-type" "application/x-www-form-urlencoded"
                        "Content-length" 13})]
    resp)

mikeb16:10:02

Ah I see what's happening. You're using an http:// address and http://discordapp.com is re-directing you to the https address.

mikeb16:10:22

So try using (str "https://discordapp.com/api/v6/channels/" channel "/messages")

eslachance16:10:20

Hmm. Oh! huh. let me see

mikeb16:10:12

What clued me in was your :follow-redirects setting. If an http post is redirected, the posted body isn't transmitted the new address. The way you can find these sorts of things is to use the chrome dev tools network tab to see the back and forth between the browser and server.

eslachance16:10:09

oh my god... wow

eslachance16:10:45

It actually works. I can't believe I just had a successful ping/pong. @mikeb you're my saviour for today 😄

mikeb16:10:43

The first of many successes we hope 🙂

eslachance17:10:15

I really wish slack had a dark theme.

lmergen20:10:44

@viesti: for what it's worth, i appreciated the info about event sourcing solutions, didnt see factual's queue before :)

practicalli-johnny22:10:08

I have a function called encode that converts each character of a string into a binary code. How could I output the result of each encoding one by one, simulating a delay between conversion of each character? I am thinking I put the output on a queue of some kind and then the function to display pulls them off one at a time with some kind of delay between pulling each converted character from the queue. This queue does not need to be performant in any way, I'm simply trying to visualise the encoding & decoding process to a human can see it.