Fork me on GitHub
#beginners
<
2016-12-31
>
pwalsh07:12:45

@sveri yes I know about immutability in clojure, and what I mean is exactly as you point out - taking something that mutates a data structure in the Python lib, but coies immutable structures in a Clojure port. @mingp I think I’ll port this https://github.com/frictionlessdata/jsontableschema-py It is interesting for several reasons: - I wrote quite a bit of it, and I also am one of the authors of a specification that this library implements - It does IO, it does type casting, it uses classes to process data and to hold state. All things that I need to approach differently in Clojure

sveri07:12:37

@pwalsh do I understand that right that this library takes a Schema anderen validates Any given CSV against it? I would be looking into spec then.

pwalsh08:12:26

yep, it does a few things: - validates that a given Table Schema is a valid Table Schema - casts arbitrary values (as strings/text) to Table Schema types - provides a table iterator (over CSV, for example), and casts row by row based on the Table Schema

roelof09:12:49

I have a function that uses pmap for parallel execution.

roelof09:12:21

but is there a way I can stop this if one of the validation is failing.

roelof09:12:22

here is the pmap used :

(defn fetch-paintings-and-images-front-page
  [ids]
  (let [paintings (pmap #(read-data-front-page (read-json-data %)) ids)
        images (pmap #(read-image-url (read-image-data %)) ids)]
    (mapv merge paintings images))) 

roelof09:12:33

and here is a function with a validation

roelof09:12:55

(defn read-json-data [id]
  (let [id-checked (s/conform ::objectNumber id)]
   (if (s/invalid? id-checked)
      (layout/render "error.html" {:message "Something went wrong with reading the data"})
      (client/get
        (str "" id)
        {:as :json,
         :query-params
             {:key       (env :key),
              :format    "json",
              :type      "schilderij",
              :toppieces "True"}}))))  

roelof09:12:57

now if something is wrong with the paintings variable , the paintings will continue to work and crash maybe

roelof09:12:18

and another question : I have now these specs :

; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
 

roelof09:12:06

is it right I have to make copies of artObjects if I want to check for objectNumber and for example a description ?

sveri10:12:13

@pwalsh Basically I dont see anything that would be problematic in clojure. What is a a Table Schema? Is that a construct that is defined by the jsontableschema? Or is it something a user can write himself? So like, is that csv valid for my usecase? Basically what an xsd is for a xml?

pwalsh11:12:46

@sveri yes, we are in the process of renaming JSON Table Schema -> Table Schema in prep for an IETF RFC. User can write one, lib can infer one, etc. Similar to XSD yes.

pwalsh11:12:20

So we generally use clojure.test or are there other popular test frameworks for clojure?

pwalsh11:12:20

related: lein or boot or just pick any and use it (using lein right now as most intro stuff uses it)?

sveri11:12:03

@pwalsh Interesting idea.What are the primary keys good for exactly? Is there some documentation available? Apart from that I would say, start simple with clojure.test and leiningen. But thats a very subjective choice of course. It mostly depends on the time you have to evaluate the different test frameworks available. Boot vs lein is a different topic where I cannot comment much other than boot still seems to have problems with windows: https://github.com/boot-clj/boot/issues?utf8=%E2%9C%93&amp;q=is%3Aissue%20is%3Aopen%20windows and most of the templates available might be leiningen templates for now.

agile_geek12:12:36

@roelofw answering your first question about short circuiting pmapif your call to the API fails. The short answer is the easiest way to do this is probably to throw an exception and then catch the exception near the top of your application and return the error page there. This makes sense for another reason. Ask yourself, "When will the api ever return an unexpected (error) data structure?". The answer is, "Only if the API provider changes the API in a breaking way (which they should never do if they follow proper API hygiene)". So If you ever get an error it's either because your code is wrong or they'e changed their code which means your code is no longer compatible. Either way it's something you can't write your app to cope with so you'd have to stop the app and rewrite it. So an exception is appropriate as you can't sensibly recover from this error.

roelof12:12:01

@agile_geek oke, so if I understand you well. I have to change this part : (layout/render "error.html" {:message "Something went wrong with reading the data"}) to function that trows an exception

roelof12:12:23

and change this function :

(defn home-page [page]
  (let [page-num (s/conform ::page page)
        url ""
        options {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True" :p page-num :ps 10}}]
    (if (s/invalid? page-num) 1 page-num)
    (layout/render
      "home.html" {:paintings (-> (client/get url options)
                                  (api/read-numbers)
                                  (api/fetch-paintings-and-images-front-page))})))  

roelof12:12:51

so if read-numbers trow an exception the fetch function is not executed

roelof12:12:40

This is only when the api changes so ObjectNumber is not found

agile_geek12:12:13

@roelofw yes. Have a look at assert in Spec for the throw exception part as that's what it does. https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/assert

roelof12:12:37

oke, thanks for the pointers

vinnyataide14:12:12

hello!😁 does anyone know if is possible to prevent runtime exceptions like elm?

mingp15:12:42

Anyone have any recommendations for programming practice sites which use Clojure? I'm already trying out CodeWars (https://www.codewars.com/) and 4Clojure (http://www.4clojure.com/). Any others you would recommend?

mingp15:12:56

On that note, if anyone is looking for the same thing, I would recommend both CodeWars and 4Clojure pretty highly.

mingp15:12:49

CodeWars is more general algorithms/problem-solving and 4Clojure is more Clojure language construct specifics.

yogidevbear15:12:31

It's similar to CodeWars in purpose

mingp16:12:29

Thanks @yogidevbear and @roelof. I'll go try both of those.

roelof16:12:17

@mingp but I agree with the rest, If you really want to learn and practice clojure, then 4clojure is the best choice

roelof16:12:52

I do it myself and a few to go and I solved all the elementary and easy challenges

pwalsh17:12:12

@sveri thanks for your help. I’ll post more once I have some code/tests. The only docs right now are in the README, and the spec itself ( http://specs.frictionlessdata.io/json-table-schema/ ) which I’m overhauling significantly now to release v1 in Jan. Quite a number of uses/users in the wild though. Very useful for our thing, which is open data/government transparency portals etc.

roelof19:12:00

Can someone answer my question about specs ?

notanon20:12:24

we can try

roelof20:12:10

I have now these specs :

; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))  

roelof20:12:34

is it right I have to make copies of artObjects if I want to check for objectNumber and for example a description  ?

roelof20:12:58

and use s/merge to test for both

notanon20:12:54

i'm not sure i really understand your question, but can think of two things that might answer your question

notanon20:12:46

1) change ::artObjects to something like this: (s/def ::artObjects (s/keys :req-un [::objectNumber ::description ::any-other-stuff-you-want ::some-other-property])

seancorfield21:12:43

@roelof I’ve said several times you’ll need qualified names for response and body to distinguish between the art objects response and the object numbers response.

seancorfield21:12:20

:art-objects/response, :art-objects/body and :object-numbers/response, :object-numbers/response.

seancorfield21:12:45

Your functions only accept one, or the other. They don’t accept both types of response, right?

seancorfield21:12:53

(assuming you’re still talking about the two different functions you had, one processing IDs and one processing paintings?)

seancorfield21:12:14

Otherwise, like @notanon, I’m not sure I understand what you’re asking.

roelof22:12:23

Thanks both and a happy and good 2017

roelof22:12:19

@seancorfield I have 3 seperate functions. One for ids , one for data and one for images/paintings

roelof22:12:45

I have now these 5 specs for validating the ids. I have now make one for the data which has almost the same structure

roelof22:12:56

; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
 

roelof22:12:16

These I can use with the ids and also with the data

roelof22:12:18

al I need is that the ::artObjects is altered with for example name which can also be found at the artObject keyword

roelof23:12:13

Happy 2017