Fork me on GitHub
#beginners
<
2017-10-05
>
noisesmith00:10:00

(required background knowledge here is how laziness works if you don’t get that)

oliv14:10:03

what you guys are using to communicate with aws ? s3 / cognito / sqs … ?

oliv14:10:49

was reaging that repo, but seems do not have support for aws cognito yet

iain.tatch14:10:52

We don't use Cognito here, but Amazonica is just a very thin wrapper around the official AWS java classes. If they support Cognito, then so should Amazonica

iain.tatch14:10:56

currently it seems to wrap:

iain.tatch14:10:14

com.amazonaws.services.cognitoidentity.AmazonCognitoIdentityClient, com.amazonaws.services.cognitoidp AWSCognitoIdentityProviderClient, and com.amazonaws.services.cognitosync.AmazonCognitoSyncClient

oliv15:10:17

seems they have support for cognito. was reading the tests and didnt find any test for cognito ( https://github.com/mcohen01/amazonica/tree/master/test/amazonica/test ) I will give a try

oliv15:10:25

thanks @iain.tatch

rinaldi15:10:34

@oliv From my experience, Amazonica is very hard to debug because it does way too much magic. I have been using the Amazon Java SDK for most things. At least is well documented, you mostly have to deal with interop which in Clojure is fine and also you get decent stack traces when shit breaks

rinaldi15:10:47

Any suggestions on material that could help me better understand core.async? I have read the Brave Clojure intro to it and saw some of the videos by Tim Baldridge on the topic but I still don't feel confident enough with it. More down to earth examples would probably help a lot.

skellyb15:10:54

The advice in this post helped me along: http://www.lispcast.com/core-async-code-style

skellyb15:10:27

Mostly because I was trying to use it for everything

rinaldi16:10:44

Thanks a lot. I indeed think I might be misunderstanding its usage in a few cases. Also feel like a lot of the struggle comes from not being very familiar with RX-like stuff.

skellyb16:10:26

The distinction that clicked with me is core.async is for coordinating multiple async actions, not solitary events like a callback

kevinbheda15:10:27

i have an app which basically reads from kafka topic using kakfa streams and calls a rest end point. Im writing an end to end test using embedded kafka The problem is with-redef is not working here

(defn start-stream [config]
  (let [stream (KafkaStreams. (topology config) (StreamsConfig. (properties config)))]
    (.start stream)
    (stream)))

(defn topology [config]
  (let [builder (KStreamBuilder.)]
    (->
      (.stream builder (into-array String [(.getValueAsString config "my_sample_topic")]))
      (.mapValues (value-mapper process-transaction)))
    builder))

(defn value-mapper [process]
  (reify
    ValueMapper
    (apply [_ v] (process v))))
(defn process-transaction [transaction]
"")
Im trying to with-redef function process-transaction but it does not work. Is Java interop the problem here , how to solve it ?

kevinbheda16:10:26

@ @ Please help ^^

alexmiller16:10:00

I don’t understand what you’re trying to do or what isn’t working

kevinbheda16:10:36

i’m basically trying to with-redef a clojure function which is being called from java

alexmiller16:10:03

I don’t see with-redefs anywhere?

alexmiller16:10:43

with-redefs affects dynamic scope so you would need to call it around the Java invocation (which seems challenging?)

kevinbheda16:10:49

i wrote the test with with-redef in the most outside scope but still the original function was being called

alexmiller16:10:34

you might be encountering issues with multiple threads (where with-redefs won’t work)

alexmiller16:10:35

or rather I should say, it will work, but there are timing issues that may not result in the desired effect

seancorfield16:10:52

FYI @kevinbheda It's considered a bit rude to @ people who are not already involved in a conversation in the channel.

kevinbheda16:10:42

hey sorry , will keep this mind

oliv17:10:37

thank you @rinaldi for your 2 cents ! So when you have to use aws you are interoping, right ?

rinaldi17:10:25

I have a wrapper around SQS for instance that I use all the time @oliv. It looks something like this:

(ns fetchers.sqs
  (:import com.amazonaws.services.sqs.AmazonSQSClient
           [com.amazonaws.services.sqs.model DeleteMessageRequest Message ReceiveMessageRequest SendMessageBatchRequest SendMessageBatchRequestEntry SendMessageRequest]))

(defn delete-message
  [^AmazonSQSClient client queue-url message]
  (println "Deleting message from SQS...")
  (println "URL:" queue-url)
  (println "Message:" message)
  (->> message
       (:receipt-handle)
       (DeleteMessageRequest. queue-url)
       (.deleteMessage client)))

oliv17:10:44

cool, and the aws credentials ?

rinaldi17:10:44

As you can see it's not too bad as Clojure handles interop in a very nice way

oliv17:10:13

looks good @rinaldi, what about the credentials, how are you handling that ?

rinaldi17:10:42

I use Amazon's client builders. For SQS, it looks like this:

(def ^:private sqs-client (AmazonSQSClientBuilder/defaultClient))

rinaldi17:10:03

This will get all the information needed from either env vars (for production mostly) or from your ~/.aws

oliv17:10:18

nice ! perfect !

oliv17:10:48

one last question, sorry to bother you so much. how are you doing to polling the sqs and get messages from there ?

oliv17:10:16

I mean, are you creating a thread ? some scheduler ?

rinaldi17:10:32

@oliv No problem. This is what I also have on my wrapper:

(defn receive-messages
  [^AmazonSQSClient client queue-url]
  (let [request (-> (ReceiveMessageRequest. queue-url)
                    (.withMaxNumberOfMessages (Integer/valueOf queue-limit)))
        received-messages (->> (.receiveMessage client request)
                               (.getMessages))]
    (println "Receiving messages from SQS...")
    (println "URL:" queue-url)
    (println "Number of messages received:" (count received-messages))
    (map (partial message-map queue-url) received-messages)))

oliv17:10:32

I see, but someone is calling receive-messages in background ( polling interval somewhere ), right ?

oliv17:10:00

@rinaldi

rinaldi18:10:06

Yes, I have an ad hoc polling implementation calling this function from time to time. Unfortunately the SDK doesn't provide this out of the box (as far as I know)

rinaldi18:10:50

@oliv This could potentially help you on that part: https://github.com/TheClimateCorporation/squeedo

nbardy21:10:32

Is there way to get the lein config from project.clj in an arbitrary clojure process started from lein run?

nbardy22:10:43

For example, So I could do something like (get-in config :source-paths)

rinaldi22:10:55

@nbardy Can't you expose it as an env var? Something like:

$ CONF=`cat project.clj` lein run my-task

rinaldi22:10:19

Then inside your task you could do something like:

(edn/read-string (System/getenv "CONF"))

bschrag22:10:55

Can CIDER be configured to warn in case of a lein error? With a certain bug in my source code, lein repl tells me #error {:cause Unable to resolve symbol: Foo in this context ... }, then starts a REPL anyway with my project's namespace prompt. cider-jack-in just gives me the prompt, leaving it to me to discover that something is busted, then try to figure out what. Is there a better workflow than first running lein repl to make sure your project is qualified for cider-jack-in? Running Emacs 25.2.2, just installed CIDER from Melpa-stable. [No answer on #cider after two days, so trying here...]

gamecubate22:10:10

What is one simple way to evaluate whether any item of a sequence is nil?

gamecubate22:10:57

Say sequence is

[true true true false true]
, I would want result to be true (as in, at least one nil? was found).

gamecubate22:10:27

(some? nil? seq)

gamecubate22:10:34

I may have answered my own question.

gamecubate22:10:37

Thanks! 🙂

gamecubate22:10:53

OK got it.

(some false? seq)