Fork me on GitHub
#beginners
<
2016-12-11
>
bcbradley03:12:56

What does the arity 1 use of the reducing function provided to a transducer do?

bcbradley03:12:04

Arity 0-- (f) is used for init

bcbradley03:12:16

Arity 2-- (f acc i) is used for actual reduction

bcbradley03:12:40

but it says that f should at least be a function of arity 1 and 2, (and 0 if no init is provided)

bcbradley03:12:52

it doesn't say (in the docs) what arity 1 actually is for

seancorfield03:12:04

That last link says what arity 1 is used for: Completion (arity 1) - some processes will not end, but for those that do (like transduce), the completion arity is used to produce a final value and/or flush state. This arity must call the xf completion arity exactly once.

bcbradley03:12:11

@seancorfield sorry I guess i glossed over that bit, so basically it takes the accumulated valued thus far and produces a final value?

bcbradley03:12:30

i'd assume that completing uses identity then right?

seancorfield03:12:22

Yup. I'm on my phone right now so I can't provide code. But think of processes that need to do some post-processing when the data has all been consumed.

bcbradley03:12:45

thanks! that should probably be in the transduce docs but I think i've got it now xd

bcbradley03:12:51

or atleast in completing

Shantanu Kumar06:12:30

@roelofw Higher order functions do not necessarily imply purity, but it is a great escape from the limitations of with-redefs. To understand the problems with with-redefs you may want to refer this post I blogged recently http://charsequence.blogspot.com/2016/12/mocking-with-var-redefinition.html

roelofw10:12:58

I only have to figure out when I do this :

(read-data(f args) ( f args)) 

roelofw10:12:11

or do I need to use apply

roelofw10:12:31

I do not see it

roelofw10:12:43

I have this :

(read-data (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}}))  

roelofw10:12:05

how can I make the client/get and the {} arguments to another function

roelofw10:12:37

I thought [(f args)] would work

Shantanu Kumar10:12:49

@roelofw What is args here? If it’s already known then you can create a function closing over those args. If it’s expected at runtime then f needs to accept it as argument.

roelofw10:12:15

I was thinking that args is this part : {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}}

roelofw10:12:31

and f is here client/get

Shantanu Kumar10:12:36

OK, if this is known beforehand, then you can construct a function: (fn [] (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})) — it would be an arity-0 function

Shantanu Kumar11:12:00

so, you can call it thus: (f) taking no args because all args are known beforehand

roelofw11:12:44

oke, and when I want to use it with testing I can replace it with this part : ⁠⁠⁠⁠(read-date (constantly {:id 1 :name "Roelof"}))

roelofw11:12:08

but then I need another function when I use another url

roelofw11:12:48

on the next function I use : (client/get {:as :json :query-params {:format "json" :key (env :key)}})

Shantanu Kumar11:12:31

Where is the URL passed here? As the first argument to client/get?

roelofw11:12:54

I was hoping the function could take a function which can be one of both client/get or a one of both dummy variables

roelofw11:12:17

I m using it like this

(let [art-objects (-> (str "" id "/tiles")
                        (client/get {:as :json :query-params {:format "json"  :key (env :key)}})
                        :body
                        :levels
                        )  

roelofw11:12:41

or

(let [art-objects (-> (str "" id )
                        (read-data (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}}))
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)  

Shantanu Kumar11:12:15

Whose responsibility is it to pass the URL? If it is the caller fn’s then the contract needs to be changed to (fn [url] …) If the URL is known beforehand, then you can construct an arity-0 fn using the URL and other options.

roelofw11:12:12

part of the url is known. The only thing that is not known is the id because that is a part of a map which is made by another function

Shantanu Kumar11:12:34

Since ID seems to be a runtime argument, you may have to create an arity-1 fn like this: (fn [id] (-> (str "” id) (client/get {:as json ...})))

roelofw11:12:20

oke, and make two functions for getting the different data . Right ?

Shantanu Kumar11:12:22

Your contract is arity-1 fn that accepts ID

roelofw11:12:38

oke, lets check if I understand you right. I have to change the function to this :

(let [art-objects (->  f(id)  :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name) 

roelofw11:12:08

and make another function like this :

(fn [id] (-> (str "" id) (client/get {:as json ...}))) 

Shantanu Kumar11:12:30

I think you meant (f id) in the snippet above 🙂

roelofw11:12:39

yes, sorry

roelofw11:12:57

sometimes I mixed up things

roelofw11:12:12

@kumarshantanu but the idea is the right one ?

Shantanu Kumar11:12:48

No problem! All of us who transitioned from another OOP language do this mistake.

Shantanu Kumar11:12:59

Yes, that’s correct.

roelofw11:12:06

I did a lot in ruby

roelofw11:12:20

one last question : can I change f in something like read-data ?

Shantanu Kumar11:12:52

Yes, of course. If you make your design as inside-out as possible, you will always be free to stub/mock anything you want.

roelofw11:12:41

oke, changed everything and do now a lein run to check if everything works well

roelofw11:12:16

and everythings work

roelofw11:12:30

@kumarshantanu so I can now copy this function :

(defn read-data-painting
  "Reads the title, description, date , collection, colors and url of a image"
  [id]
  (let [art-objects (->  (read-json-data id)
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)
        description (:description art-objects)
        date (get-in art-objects [:dating :year])
        collectie (first (:objectCollection art-objects))
        colors (:colors art-objects)]
    {:id id :name name :description description :date date :collectie collectie :colors colors}))  

roelofw11:12:12

and change the read-json-data function to this :

(read-json-data  (constantly {:id 1 :name "Roelof"}))  

Shantanu Kumar11:12:31

I think now you have to pass read-json-data as argument to read-data-painting?

Shantanu Kumar11:12:09

If read-json-data is not an argument to read-data-painting, then you may have to use with-redefs to mock it when testing read-data-painting

Shantanu Kumar11:12:48

@roelofw Got to be away now. I will check the questions later in case you have any.

roelofw11:12:49

oke, thanks

roelofw11:12:03

I have this test file :

(deftest test-app
  (testing "get numbers"
    (is (= [ "1234" "abcd"] (api/read-numbers {:body {:artObjects [{:objectNumber "1234"} {:objectNumber "abcd"}]}}))))

  (testing "get data "
    (let [art-objects (->  (api/read-json-data (constantly {:body {:artObjects {:id 1 :name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"}}}))
                           :body
                           :artObject)
          name        (-> art-objects
                          :principalMakers
                          first
                         :name)
          description (:description art-objects)
          date (get-in art-objects [:dating :year])
          collectie (first (:objectCollection art-objects))
          colors (:colors art-objects)]
      (is (=  {:name name :description description :date date :collectie collectie :colors colors}
              {:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"} ))))
 

roelofw11:12:36

but I see this as output :

expected: {:name nil,
           :description nil,
           :date nil,
           :collectie nil,
           :colors nil}
  actual: {:name "Roelof",
           :description "rubbish",
           :date "12-02-1980",
           :collectie "",
           :colors "yellow"} 

roelofw11:12:07

why is the expected empty ? I thought it would be the contents of the constanly function

agile_geek13:12:06

@roelofw two things. First you probably want all the code in that let to be in your production fn but the args to the production fn to be [id read-json-data] then call it using 1 and your constantly function in the test: (read-paintings 1 (constantly {:body {:artObject {:id 1 :principalMakers [{:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"}]}}}})

agile_geek13:12:54

Secondly the reason you have nil is that you are looking for artObject but your map has :artsObjects (with an s)

agile_geek13:12:06

also you are looking inside :artObject map for :principalMakers but your artObject map has no :principalMakers key

agile_geek13:12:45

then inside :principalMakers your getting the first element from a sequential structure (a vector?)

agile_geek13:12:25

and then that first element is expecting a key-value pair keyed by :name

agile_geek13:12:22

so you need a map similar to the one I've got in my (constantly snippet above ☝️

agile_geek13:12:23

Actually even that's not right as :date needs to be inside a :dating map and the value for the year inside :year for example

agile_geek13:12:38

The map would look like this:

agile_geek13:12:05

{:body {:artObject {:principalMakers [{:name "Roelof"}] 
                                :description "rubbish" 
                                :dating {:year "1980"}
                                :objectCollection ["collectie"]
                                :colors "yellow"}}}

agile_geek13:12:30

I typed that without an editor balance parens so it needs checking

roelofw13:12:30

I know that the name was not right

roelofw13:12:12

@agile_geek why jere read-paintings and not the function that mocks up the client/get one

agile_geek13:12:53

Well it depends what you are testing

roelofw13:12:54

so I can delete the whole test function except the read-paintings part

roelofw13:12:30

I want to test if the parsing of the recieved json is going allright

agile_geek13:12:19

If your original fn is calling the api and then extracting the data and you want to test the fn that parses the data then it's whatever that's called but you pass in the fn the just does the call to the api

agile_geek13:12:00

so if the fn that parses the data is called read-paintings its that you want to call in the test (or whatever you called it)

agile_geek13:12:25

but you pass it two args

roelofw13:12:34

This is the whole function :

(defn read-json-data [id] (-> (str "" id) (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}} )))

(defn read-data-painting
  "Reads the title, description, date , collection, colors and url of a image"
  [id]
  (let [art-objects (->  (read-json-data id)
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)
        description (:description art-objects)
        date (get-in art-objects [:dating :year])
        collectie (first (:objectCollection art-objects))
        colors (:colors art-objects)]
    {:id id :name name :description description :date date :collectie collectie :colors colors}))  

agile_geek13:12:40

and the fn that mocks the data

agile_geek13:12:00

right so make read-data-painting take two args

agile_geek13:12:10

the id and a function that calls the api

roelofw13:12:12

then I have to rewrite things, the read-data-painting has now 1 argument

agile_geek13:12:54

add arg call read-json-data to the args list

agile_geek13:12:22

then pass in the real read-json-data fn when you call it in the production code

agile_geek13:12:44

and pass in (constantly ...) in the test

roelofw13:12:50

oke, then I can delete the read-json-data function and have also the change the calling function here :

(defn do-both-in-parallel [ids]
  (let [paint-thread (future (pmap read-data-painting ids))
        image-thread (future (pmap read-image-url ids))]
    (map merge @paint-thread @image-thread))) 

roelofw13:12:07

oke, I will try that

roelofw13:12:42

and the test - function is then only the call to the read-data-painting with the constantly function .

agile_geek13:12:49

You still need a fn that calls the api so you still need a read-json-data function

roelofw13:12:54

Do I understand everything right ?

agile_geek13:12:11

as you need to pass it into the call to read-data-painting

agile_geek13:12:45

(defn do-both-in-parallel [ids]
  (let [paint-thread (future (pmap (comp read-data-painting read-json-data)  ids))
        image-thread (future (pmap read-image-url ids))]
    (map merge @paint-thread @image-thread))) 
if...

agile_geek13:12:18

you rewrite the args to read-data-painting to this...

agile_geek13:12:40

(defn read-data-painting
  "Reads the title, description, date , collection, colors and url of a image"
  [call-api-fn id]
  (let [art-objects (->  (call-api-fn id)
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)
        description (:description art-objects)
        date (get-in art-objects [:dating :year])
        collectie (first (:objectCollection art-objects))
        colors (:colors art-objects)]
    {:id id :name name :description description :date date :collectie collectie :colors colors}))  

agile_geek13:12:04

I renamed the fn arg passed in to make it more obvious that it's calling whatever you pass in

agile_geek13:12:27

i.e I named it call-api-fn

agile_geek13:12:15

In fact you could change the function to not take id or a fn but just take the map from calling the api

agile_geek13:12:05

and then thread the result of calling read-json-data to the read-data-painting fn

agile_geek13:12:27

(defn do-both-in-parallel [ids]
  (let [paint-thread (future (pmap #(read-data-painting (read-json-data %))  ids))
        image-thread (future (pmap read-image-url ids))]
    (map merge @paint-thread @image-thread))) 

agile_geek13:12:41

and change read-data-painting to

agile_geek13:12:21

(defn read-data-painting
  "Reads the title, description, date , collection, colors and url of a image"
  [response]
  (let [art-objects (->  response
                        :body
                        :artObject)
        name        (-> art-objects
                        :principalMakers
                        first
                        :name)
        description (:description art-objects)
        date (get-in art-objects [:dating :year])
        collectie (first (:objectCollection art-objects))
        colors (:colors art-objects)]
    {:id id :name name :description description :date date :collectie collectie :colors colors}))  

agile_geek13:12:37

Then in the test you can just pass the map

agile_geek13:12:54

without (constantly ...) wrapped around it

agile_geek13:12:32

The only bit you can't test without calling the api is the read-json-data fn

roelofw13:12:03

oke, where is the function where the json is made ? I see response but never a call to call-api-fn

roelofw13:12:27

how does clojure knows where the response is generated

roelofw13:12:57

sorry, I see it #(read-data-painting (read-json-data %))

agile_geek13:12:06

In my second example you are simply calling read-json-data that returns a map, and then passing it as the response arg to read-data-painting

roelofw13:12:16

is now has the name read-json-data

agile_geek13:12:44

(defn read-json-data [id] (-> (str "" id) (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}} )))

agile_geek13:12:07

In your code ☝️

roelofw13:12:14

oke, and the test is one thing a call to read-data-painting and compare that to the map I compare , right ?

agile_geek13:12:01

Yeah in the test you pass in a map that looks like the response from read-json-data and then compare it to what you expect

roelofw13:12:40

Thanks for the lessons . I never thougt about this solution

roelofw13:12:48

I like the last solution the most

roelofw13:12:06

and I can use it to test another function

agile_geek13:12:08

Cool glad to help

roelofw14:12:14

You are a wonderfull person who help me a lot. Together with sean and a few more Im still in the clojure world.

roelofw14:12:46

@agile_geek I have to change [response] to [response id] because otherwise I loose the id of the painting

agile_geek14:12:50

@roelofw not if you change the read-data-painting to extract id from response

agile_geek14:12:58

it's in the map

agile_geek14:12:33

You might need to check what's returned from the api call but it's in there

roelofw14:12:46

oke, maybe that is why I see this error : Wrong number of args (1) passed to: api-get/read-data-painting

agile_geek14:12:59

that's telling you that you're read-data-painting takes two args but you're calling it with only one

agile_geek14:12:32

if you pass only the map you need to work out how to destructure id from the response map

agile_geek14:12:45

can't remember what it looks like but it's in there

roelofw14:12:01

oke, I will ,. thanks

roelofw14:12:20

I delete the id of the arguments and everything seems to work well

agile_geek14:12:26

Call read-json-data from a REPL passing in a good id and look at the response map

agile_geek14:12:48

Use the REPL to explore fn's like this

agile_geek14:12:54

and poke at data

agile_geek14:12:04

it's called REPL based development

roelofw14:12:56

thanks, I will do that

iecya17:12:29

hello everyone 🙂

iecya17:12:47

I'm new in this group, joined today in hope to find some help with clojure 😄

iecya17:12:15

I am having issues with tests in my project. in the REPL If i run my function it returns data as expected, if i validate it everything is fine and if i run the test it pass fine. however, if i run lein test my tests fail 😞 it seems that s/check from plumatic schema is returning a map instead of nil. also, the function i am trying to validate get a string as input and uses the elements in the string to "build" a function (the first element is the function name, the rest is the arguments). the test return a custom error message, while if i run the function with the same input it runs as expected. can anyone help me or point me to some documentation I might have missed?

seancorfield18:12:38

@iecya: it's a little hard to tell what's going on from your description -- do you perhaps have this project up on GitHub where we can take a look and try it out for ourselves?

iecya19:12:41

many thanks @seancorfield i got all my tests working now!

roelofw19:12:05

@agile_geek can you help me one more time. My test file looks like this now :

(testing "get data "
    (let [art-objects (->  (api/read-json-data  {:body {:artObjects {:id 1 :name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"}}})
                           :body
                           :artObject)
          name        (-> art-objects
                          :principalMakers
                          first
                         :name)
          description (:description art-objects)
          date (get-in art-objects [:dating :year])
          collectie (first (:objectCollection art-objects))
          colors (:colors art-objects)]
      (is (=  {:name name :description description :date date :collectie collectie :colors colors}
              {:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"} ))))  

roelofw19:12:37

and now I see this error :

ERROR in (test-app) (support.clj:201)
Uncaught exception, not in assertion. 

agile_geek19:12:56

I'm on my phone so can't really see but is all this inside a (deftest ...)

agile_geek19:12:22

Also I don't see you actually calling your function in the test?

roelofw19:12:24

found it, I had to change read-json-data to read-data-painting

roelofw19:12:10

now the problem that all data is nill.

roelofw19:12:32

We had this discussed this afternoon . I will try to find it back

agile_geek19:12:20

Just call your read-data-painting in you test passing a map that looks like the response from read-json-data and then see if it equals :

{:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"}  

agile_geek19:12:34

So your test would be;

agile_geek19:12:47

(is (=
              {:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"} 
(read-data-painting {...})))

agile_geek19:12:58

Where ... is a map that matches what (read-json-data "id") would return

roelofw19:12:25

hmm, still this output :

expected: {:name "Roelof",
           :description "rubbish",
           :date "12-02-1980",
           :collectie "",
           :colors "yellow"}
  actual: {:name nil,
           :description nil,
           :date nil,
           :collectie "",
           :colors nil}  

roelofw19:12:55

it looks this map is not good :

(api/read-data-painting  {:body {:artObjects {:id 1 :principalMakers [{:name "Roelof" :description "rubbish" :date "12-02-1980" :collectie "" :colors "yellow"}]}}})  

agile_geek22:12:01

Look at what comes back from call to api