This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-08-10
Channels
- # announcements (2)
- # beginners (37)
- # boot (1)
- # cider (76)
- # clara (14)
- # cljs-dev (132)
- # cljsjs (1)
- # cljsrn (2)
- # clojure (18)
- # clojure-colombia (5)
- # clojure-finland (1)
- # clojure-hamburg (1)
- # clojure-italy (2)
- # clojure-nl (8)
- # clojure-russia (1)
- # clojure-spec (28)
- # clojure-uk (85)
- # clojurescript (84)
- # code-reviews (25)
- # cursive (10)
- # data-science (3)
- # datomic (30)
- # editors (1)
- # emacs (3)
- # fulcro (106)
- # graphql (4)
- # hyperfiddle (26)
- # jobs (2)
- # jobs-discuss (124)
- # keechma (3)
- # leiningen (1)
- # lumo (6)
- # off-topic (5)
- # other-lisps (5)
- # reagent (5)
- # ring-swagger (4)
- # shadow-cljs (140)
- # spacemacs (22)
- # specter (2)
- # sql (48)
- # tools-deps (78)
- # vim (7)
morning
what PGP help dyu need @yogidevbear?
Morning Craig 🙂 I came right in the end. I was trying to figure out how to encrypt using a public key that someone gave me. I was trying to use PGP instead of gpg, but was showing the error of my ways by @dominicm
ah,, yeah, gpg all the way...
morning!
Bore da pawb
bore da
Morning.
The deed is done. Flights are booked, hotel is booked and talk outline and updated bio are sent off..... y'all have to chip in for an assassin to stop me speaking at ClojureX (or bribe @jr0cket quite heavily 🙂 )
@jasonbell It would have to be a bribe of biblical proportions for me to stop you from speaking at the conference. Then I would use the money to run a "Jason Bell is awesome" conference and have a whole day of you speaking :)
@jasonbell I find @jr0cket is a cheap date... I could just buy him a pint! Seriously though...looking forward to seeing the video of the talk!
what are you talking on?
or is that spoilers
@alex.lynham - Learning how to develop self training AI with Clojure, Kafka and DeepLearning4J (with a bit of possibly core.async or factual durable queues throw in for good measure). @otfrom told me about Factual and I really like using it.
what's kafka for in that piece?
(also where did you get to do this? sounds cool)
The reason for using Kafka was to show Kafka in a more real world context. But the same could be applied to core.async or Factual, so I'll keep the Kafka stuff fairly short.
running it as like a single node for integration or something?
Kafka is usually over kill for most things in terms of size. I'm processing email events with Factual now, it's a much better fit for me.
@alex.lynham <<what's kafka for in that piece?>> Sorry I didn't answer that: basically raw data transport and events.
@jasonbell maybe next year you can come talk at http://bitrconf.org for practice? Might need to dumb down the Clojure bit tho!
@jasonbell - I am reading back... What is this Factual of which you speak..? It sounds, for all the World, like a queue...
^ I was just about to ask that 🙂
I'm using kinesis for a lot of that stuff atm
works well, except that whatever your upstream/operational system is usually tends to be some messy legacy system
and then that's the pain point for getting events cleanly into the system
I thought that Kinesis is / was a sliding window, so there's a problem with persistence..?
oh yeah
it's 24hrs configurable up to 168 so fine for events
if you need kafka-style persistence you're looking at something jury-rigged with dynamodb
...or just kafka
@dominicm - I was wondering the same thing, seeing as a lot of AWS tech "talks" effortlessly to S3
@dominicm iirc there's a way of using a tool called VCR for reply yes
so you're using KCL and dynamo (+s3?) essentially
though I guess you could also bash everything into parquet on S3
or x other format
interesting
could be good to drop that into a container
I'm using it for email confirmation processing: event happens on api, triggers email. queue works a treat for it.
Oooh, this all looks very nice 🙂 thanks @jasonbell and @alex.lynham
@maleghast To be honest @otfrom told me so thank him.
@alex.lynham - I wonder if you could get Kinesis -> Athena
@otfrom - Thanks, via @jasonbell for Factual 🙂
> can kinesis stream into s3 for replay? Kind of. There is - https://aws.amazon.com/kinesis/data-firehose/ But last time I used it you had to write a connector to go from Kinesis to Kinesis firehose
in the community tools there's also https://github.com/awslabs/amazon-kinesis-archiver
@maleghast I... imagine you could? Don't know much about Athena mind you
Me neither, but the ability to query the data in-place while it's "resting" seems useful
so is athena built on top of s3 and parquet?
was looking today and AWS aurora serverless now supports MySQL and Postgres is on the way
right
yeah that's pretty cool
deffo easier than my first skunkworks thing of deploying json blobs to IPFS and then picking them up later
I think I need to reduce myself to a part time job where I only do super serious proper engineering and then spend all my spare time relentlessly pursuing my silly ideas, it'd be ace
next Glasgow meetup Wednesday next 6.30pm