This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-12-31
Channels
- # beginners (57)
- # boot (25)
- # cider (86)
- # cljs-dev (6)
- # clojure (137)
- # clojure-russia (7)
- # clojure-spec (10)
- # clojure-turkiye (1)
- # clojure-uk (47)
- # clojurescript (37)
- # cursive (10)
- # datascript (2)
- # datomic (2)
- # dirac (59)
- # emacs (1)
- # hoplon (46)
- # instaparse (122)
- # om (32)
- # om-next (1)
- # onyx (3)
- # pedestal (2)
- # perun (4)
- # protorepl (6)
- # re-frame (15)
- # reagent (60)
- # rum (4)
- # specter (7)
- # sql (3)
- # untangled (3)
- # yada (4)
@sveri yes I know about immutability in clojure, and what I mean is exactly as you point out - taking something that mutates a data structure in the Python lib, but coies immutable structures in a Clojure port. @mingp I think I’ll port this https://github.com/frictionlessdata/jsontableschema-py It is interesting for several reasons: - I wrote quite a bit of it, and I also am one of the authors of a specification that this library implements - It does IO, it does type casting, it uses classes to process data and to hold state. All things that I need to approach differently in Clojure
@pwalsh do I understand that right that this library takes a Schema anderen validates Any given CSV against it? I would be looking into spec then.
yep, it does a few things: - validates that a given Table Schema is a valid Table Schema - casts arbitrary values (as strings/text) to Table Schema types - provides a table iterator (over CSV, for example), and casts row by row based on the Table Schema
here is the pmap used :
(defn fetch-paintings-and-images-front-page
[ids]
(let [paintings (pmap #(read-data-front-page (read-json-data %)) ids)
images (pmap #(read-image-url (read-image-data %)) ids)]
(mapv merge paintings images)))
(defn read-json-data [id]
(let [id-checked (s/conform ::objectNumber id)]
(if (s/invalid? id-checked)
(layout/render "error.html" {:message "Something went wrong with reading the data"})
(client/get
(str " " id)
{:as :json,
:query-params
{:key (env :key),
:format "json",
:type "schilderij",
:toppieces "True"}}))))
now if something is wrong with the paintings variable , the paintings will continue to work and crash maybe
and another question : I have now these specs :
; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
is it right I have to make copies of artObjects if I want to check for objectNumber and for example a description ?
@pwalsh Basically I dont see anything that would be problematic in clojure. What is a a Table Schema? Is that a construct that is defined by the jsontableschema? Or is it something a user can write himself? So like, is that csv valid for my usecase? Basically what an xsd is for a xml?
@sveri yes, we are in the process of renaming JSON Table Schema -> Table Schema in prep for an IETF RFC. User can write one, lib can infer one, etc. Similar to XSD yes.
So we generally use clojure.test
or are there other popular test frameworks for clojure?
related: lein
or boot
or just pick any and use it (using lein right now as most intro stuff uses it)?
@pwalsh Interesting idea.What are the primary keys good for exactly? Is there some documentation available? Apart from that I would say, start simple with clojure.test and leiningen. But thats a very subjective choice of course. It mostly depends on the time you have to evaluate the different test frameworks available. Boot vs lein is a different topic where I cannot comment much other than boot still seems to have problems with windows: https://github.com/boot-clj/boot/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20windows and most of the templates available might be leiningen templates for now.
@roelofw answering your first question about short circuiting pmap
if your call to the API fails.
The short answer is the easiest way to do this is probably to throw an exception and then catch the exception near the top of your application and return the error page there.
This makes sense for another reason. Ask yourself, "When will the api ever return an unexpected (error) data structure?". The answer is, "Only if the API provider changes the API in a breaking way (which they should never do if they follow proper API hygiene)". So If you ever get an error it's either because your code is wrong or they'e changed their code which means your code is no longer compatible. Either way it's something you can't write your app to cope with so you'd have to stop the app and rewrite it. So an exception is appropriate as you can't sensibly recover from this error.
@agile_geek oke, so if I understand you well. I have to change this part : (layout/render "error.html" {:message "Something went wrong with reading the data"})
to function that trows an exception
and change this function :
(defn home-page [page]
(let [page-num (s/conform ::page page)
url ""
options {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True" :p page-num :ps 10}}]
(if (s/invalid? page-num) 1 page-num)
(layout/render
"home.html" {:paintings (-> (client/get url options)
(api/read-numbers)
(api/fetch-paintings-and-images-front-page))})))
@roelofw yes. Have a look at assert
in Spec for the throw exception part as that's what it does. https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/assert
hello!😁 does anyone know if is possible to prevent runtime exceptions like elm?
Anyone have any recommendations for programming practice sites which use Clojure? I'm already trying out CodeWars (https://www.codewars.com/) and 4Clojure (http://www.4clojure.com/). Any others you would recommend?
On that note, if anyone is looking for the same thing, I would recommend both CodeWars and 4Clojure pretty highly.
CodeWars is more general algorithms/problem-solving and 4Clojure is more Clojure language construct specifics.
I quite enjoy http://hackerrank.com
It's similar to CodeWars in purpose
and you can try http://www.exercism.io/
or this one : https://www.codingame.com/
Thanks @yogidevbear and @roelof. I'll go try both of those.
@mingp but I agree with the rest, If you really want to learn and practice clojure, then 4clojure is the best choice
@sveri thanks for your help. I’ll post more once I have some code/tests. The only docs right now are in the README, and the spec itself ( http://specs.frictionlessdata.io/json-table-schema/ ) which I’m overhauling significantly now to release v1 in Jan. Quite a number of uses/users in the wild though. Very useful for our thing, which is open data/government transparency portals etc.
I have now these specs :
; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
is it right I have to make copies of artObjects if I want to check for objectNumber and for example a description ?
i'm not sure i really understand your question, but can think of two things that might answer your question
1) change ::artObjects to something like this: (s/def ::artObjects (s/keys :req-un [::objectNumber ::description ::any-other-stuff-you-want ::some-other-property])
2) use s/and. here's the link with more info. http://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/and
@roelof I’ve said several times you’ll need qualified names for response and body to distinguish between the art objects response and the object numbers response.
:art-objects/response
, :art-objects/body
and :object-numbers/response
, :object-numbers/response
.
Your functions only accept one, or the other. They don’t accept both types of response, right?
(assuming you’re still talking about the two different functions you had, one processing IDs and one processing paintings?)
Otherwise, like @notanon, I’m not sure I understand what you’re asking.
@seancorfield I have 3 seperate functions. One for ids , one for data and one for images/paintings
I have now these 5 specs for validating the ids. I have now make one for the data which has almost the same structure
; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))