This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-12-30
Channels
- # adventofcode (11)
- # beginners (155)
- # boot (627)
- # cider (64)
- # cljs-dev (110)
- # cljsrn (36)
- # clojure (290)
- # clojure-austin (21)
- # clojure-russia (2)
- # clojure-spec (2)
- # clojure-uk (21)
- # clojurescript (81)
- # code-reviews (2)
- # core-async (33)
- # cursive (6)
- # datomic (9)
- # emacs (1)
- # hoplon (472)
- # instaparse (1)
- # lein-figwheel (4)
- # luminus (9)
- # om (2)
- # protorepl (10)
- # re-frame (10)
- # reagent (48)
- # schema (2)
- # sql (5)
- # untangled (17)
- # vim (1)
- # yada (108)
Given that you’re looping until you have a vector of Boolean with 35,651,584 entries in it, I’m not too surprised you’re hitting the default JVM memory limit 🙂
(esp. with all the concat, rseq, and partition operations going on 🙂 )
with the 4 gig memory flag part2 takes about 5minutes to complete for me on my 2yo 12” macbook
A 35MB string vs a vector of Boolean with 35MB elements in it? The latter is going to be bigger than 35MB.
(that would be my naïve answer — I haven’t verified that!)
I get this error message : IllegalArgumentException Key must be integer clojure.lang.APersistentVector.invoke (APersistentVector.java:292)
on this repl
(s/explain ::artobject-objectNumber [{:artObjects{:objectNumber "sk-c-5"}}])
with these specs : ;; the specs files for input validation
(def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")
(s/def ::id-type (s/and string? #(re-matches id-regex %)))
(s/def ::objectNumber ::id-type)
(s/def ::objectNumber-list (s/coll-of
(s/keys :req-un [::objectNumber])
:kind vector?))
(s/def ::artobject-objectNumber (s/coll-of
(::objectNumber-list :req-un [::artObjects])
:kind vector?))
@roelof you might want to try (s/exercise ::artobject-objectNumber)
and compare what’s generated with your expectation
hm, that’s too bad. (s/exercise) wasn’t helpful since in this case since it wasn’t able to generate data satisfying the specs.
maybe this will be helpful
;; this spec matches a map containing the key :artObject
(s/def ::art-object (s/keys :req-un [::artObject]))
(s/explain ::art-object {:artObject {}})
;;=> Success!
;; this spec matches a collection of ::art-object
(s/def ::list-of-art-objects (s/coll-of ::art-object :kind vector?))
(s/explain ::list-of-art-objects [{:artObject {}}])
;;=> Success!
and here’s how you could combine specs together to create a nested spec
;; a spec that matches a map with the key :objectNumber
(s/def ::object-number (s/keys :req-un [::objectNumber]))
;; a spec that matches a map containing the key :artObject that maps keywords to values that match the ::object-number spec
(s/def ::nested-spec (s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::object-number)))
(s/explain ::nested-spec {:artObject {:objectNumber "sk-c-5"}})
;;=> Success!
and here’s how you could combine the nested spec with a collection spec
;; here we don’t specify that the collection must be a vector - it just needs to be a collection of values that satisfy the ::nested-spec
(s/def ::list-of-nested (s/coll-of ::nested-spec))
;; so it will accept a vector
(s/explain ::list-of-nested [{:artObject {:objectNumber "sk-c-5"}}])
;;=> Success!
;; and it will also accept a list
(s/explain ::list-of-nested '({:artObject {:objectNumber "sk-c-5"}}))
;;=> Success!
@seako here it do not work :
(s/explain ::objectNumber-list [{:artObject {:objectNumber "sk-c-5"}}])
Exception Unable to resolve spec: :paintings2.api-get/object-number clojure.spec/reg-resolve! (spec.clj:68)
with this code :
(def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")
(s/def ::id-type (s/and string? #(re-matches id-regex %)))
(s/def ::objectNumber ::id-type)
(s/def ::objectNumber-list (s/coll-of
(s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::object-number))
:kind vector?))
oh, that’s because i defined a new spec called ::object-number
that is not in that code
Thanks, now it works fine :
=> #object[clojure.lang.Namespace 0x1c44c16 "paintings2.api-get"]
(s/explain ::objectNumber-list [{:artObject {:objectNumber "sk-c-5"}}])
Success!
=> nil
(s/explain ::objectNumber-list [{:artObject {:objectNumber "sk-c"}}])
In: [0 :artObject 1 :objectNumber] val: "sk-c" fails spec: :paintings2.api-get/id-type at: [1 :objectNumber] predicate: (re-matches id-regex %)
=> nil
now I can do the same I think when I want to check also for artObjects as the first one
I want to make this a success : (s/explain ::objectNumber-list [{:body{:artObjects{:objectNumber "sk-c-5"}}}])
i would recommend instead defining a new spec that matches a collection of values that match your specs
the general strategy is going to be similar - write specs for the values nested inside the structure and then combine them together with the specs for collections and maps
I have this in repl : (s/explain ::objectNumber-list2 [{:body{:artObject {:objectNumber "sk-c-5"}}}])
with these specs :
(def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")
(s/def ::id-type (s/and string? #(re-matches id-regex %)))
(s/def ::objectNumber ::id-type)
(s/def ::objectNumber-check (s/keys :req-un [::objectNumber]))
(s/def ::objectNumber-list (s/coll-of
(s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::objectNumber-check))
:kind vector?))
(s/def ::objectNumber-list2 (s/coll-of
(s/and (s/keys :req-un [::body]) (s/map-of keyword? ::objectNumber-list))
:kind vector?))
now the specs fails with this message :
In: [0 :body 1] val: {:artObject {:objectNumber "sk-c-5"}} fails spec: :paintings2.api-get/objectNumber-list at: [1] predicate: vector?
=> nil
so I deleted the kind vector?
part but then it fails with this :
In: [0 :body 1 0] val: [:artObject {:objectNumber "sk-c-5"}] fails spec: :paintings2.api-get/objectNumber-list at: [1] predicate: map?
I thought it would work because of this : (s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::objectNumber-check))
See here :
(s/explain ::objectNumber-list [{:artObject {:objectNumber "sk-c-5"}}])
Success!
=> nil
(s/map-of keyword? ::objectNumber-list)
expects a map that looks like {:any-keyword [{:artObject {:objectNumber “sk-c-5”}}]
in
(s/def ::objectNumber-list2 (s/coll-of
(s/and (s/keys :req-un [::body]) (s/map-of keyword? ::objectNumber-list))
:kind vector?))
replace ::objectNumber-list
with (s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::objectNumber-check))
(s/def ::objectNumber-list2 (s/coll-of
(s/and (s/keys :req-un [::body]) (s/and (s/keys :req-un [::artObject]) (s/map-of keyword? ::objectNumber-check))
:kind vector?))
Thanks, it worked well
(s/explain ::objectNumber-list [{:body{:artObject {:objectNumber "sk-c-5"}}}])
Success!
=> nil
(s/explain ::objectNumber-list [{:body{:artObject {:objectNumber "sk-c"}}}])
In: [0 :body 1 :artObject 1 :objectNumber] val: "sk-c" fails spec: :paintings2.api-get/id-type at: [1 1 :objectNumber] predicate: (re-matches id-regex %)
=> nil
`and here :
(s/explain ::objectNumber-list [{:artObject {:objectNumber "sk-c-5"}}])
In: [0] val: {:artObject {:objectNumber "sk-c-5"}} fails spec: :paintings2.api-get/objectNumber-list predicate: (contains? % :body)
=> nil
another problem . I have this code :
(defn read-numbers
"Reads the ids of the paintings"
[response]
(if (s/conform ::objectNumber-list response)
(->> (:body response)
:artObjects
(map :objectNumber))))
a, I see the problem : ` (s/conform ::objectNumber-list [{:body{:artObject {:objectNumber "sk-c-5"}}}]) => [{:body {:artObject {:objectNumber "sk-c-5"}}}] `
(s/explain ::objectNumber-list [{:body{:artObjects {:objectNumber "sk-c-5"}}}])
Success!
=> nil
with this code :
(defn read-numbers
"Reads the ids of the paintings"
[response]
(let [response-checked (s/conform ::objectNumber-list response)]
(if (s/invalid? response-checked)
(layout/render "error.html" {:message "something was wrong with reading the ids"})
(->> (:body response)
:artObjects
(map :objectNumber)))))
Doesn't it look like this instead: {:body {:artObjects [{:objectNumber "sk-c-5"}]}}
?
@henriklundahl yep, you are right :
{:status 200,
:headers {"Date" "Fri, 30 Dec 2016 12:34:44 GMT",
"Cache-Control" "no-store, must-revalidate, no-cache",
"Pragma" "no-cache",
"Content-Type" "application/json; charset=utf-8",
"Access-Control-Allow-Origin" "*",
"X-Frame-Options" "DENY",
"Content-Length" "12707",
"Connection" "close"},
:body {:elapsedMilliseconds 43,
:count 4746,
:artObjects [{:
You should only need to change ::artObjects and the top one.
It is a map.
sorry, I do not understand , @henriklundahl
Ah, I thought you had an ::artObjects
spec.
Can you write all the specs you have currently?
@henriklundahl here you have :
`(def id-regex #"[a-z]{2}-[a-z]-\d{1,2}")
(s/def ::id-type (s/and string? #(re-matches id-regex %)))
(s/def ::objectNumber ::id-type)
(s/def ::objectNumber-check (s/keys :req-un [::objectNumber]))
(s/def ::objectNumber-list (s/coll-of
(s/and (s/keys :req-un [::body]) (s/map-of keyword?
(s/and (s/keys :req-un [::artObjects])
(s/map-of keyword? ::objectNumber-check))))
))
::artObjects
is mentioned in ::objectNumber-list
, but I can't the definition of it?
I think ::objectNumber-check
should be renamed ::artObject
.
And then the spec for ::artObjects
should be (s/def ::artObjects (s/coll-of ::artObject))
.
And then you add a spec for ::body
: (s/def ::body (s/keys :req-un [::artObjects]))
.
Finally, add a spec for ::response
: (s/def ::response (s/keys :req-un [::body]))
.
Then you validate against the ::reponse
spec.
I believe so, yes.
Still the same output :
(s/explain ::response {:body{:artObjects {:objectNumber "sk-c-5"}}})
In: [:body :artObjects 0] val: [:objectNumber "sk-c-5"] fails spec: :paintings2.api-get/artObject at: [:body :artObjects] predicate: map?
=> nil
Don't forget to change the :artObjects value to a vector.
::id-type
could be inlined into ::objectNumber
, so you would have the specs ::objectNumber
, ::artObject
, ::artObjects
, ::body
and ::reponse
.
Great!
@henriklundahl you mean it could be this
(s/def ::objectNumber (s/and string? #(re-matches id-regex %)) )
And you'll end up with five very simple specs.
That's not a spec. 🙂
That's great. Good luck!
Hey all. I’m a fairly experienced developer, fluent in Python and JavaScript. I’ve been reading up on Clojure for some time, and played with it in small doses via Overtone, and just some REPL learning sessions. However, for me the only way to really learn is to build a project, so I think I’ll port a Python library I know well to Clojure. I think the areas I need most initial guidance on are conventions around (i) grouping functions that operate on mutable data structures (something that I’d use classes for in Python), and (ii) library structure in general. Would this be an appropriate list to ask questions around that?
@pwalsh Yes, this is the right channel for beginner questions. My first comment. There are hardly any mutable data structures in clojure. Most of them are immutable and you will create a data flow that creates new data structures via structural sharing mostly.
@pwalsh Which library were you thinking of porting? I'd agree that the default mindset in Clojure is functions which accept and return immutable data, but of course it can be situational depending on what it is you're writing and what you want from it in terms of behavior/performance.
another spec question . I have these specs :
(s/def ::objectNumber (s/and string? #(re-matches id-regex %)))
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
(s/def ::objectNumbers (s/coll-of ::objectNumber))
but then I see this output : In: [0] val: "SK-C-5" fails spec: :paintings2.api-get/objectNumber predicate: (re-matches id-regex %)
@pwalsh I also come from python, curious to follow your progress, please keep us posted here
but now when I run my web application I see this error :
An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.\r\n
Hell guys, I came across an interesting issue. I am using migratus
for creating DB schemas so I have init.sql
file in src/migrations
directory. However when I do lein ring uberwar
file is not included in the war file (checking with jar tvf target/veit-is.war | fgrep init.sq
). When I include this path in project.clj
via :resource-paths ["src/migrations"]
, the file is there but directly in the WEB-INF/classes/init.sql
folder and not in migrations directory. Any ideas what is happening there?
Btw I have a very similar project with the same setup and it's working there, init.sql is correctly in WEB-INF/classes/migrations
directory.
Are you sure :resource-paths
in the working project is src/migrations
or just src
?
Resources will be copied into classes
using their path relative to the resource paths.
Actually there is none and it's still working
src/migrations
is included by defafult
@seancorfield is there a way I can see what id is when I do lein run in this code :
(defn read-image-data [id] (client/get
(str " " id "/tiles")
{:as :json, :query-params {:format "json", :key (env :key)}}))
that one or this one :
(defn read-image-data [id] (client/get
(str " " id "/tiles")
{:as :json, :query-params {:format "json", :key (env :key)}}))
(defn read-image-url
"Reads the image-url"
[response]
(let [art-objects (-> response
:body
:levels)
url (filter #(= (:name %) "z4") art-objects)
tiles (:tiles (first url))
image (get-in tiles [0 :url])
]
{:tiles image}))
(defmacro infix2
"Takes a list like (1 + 2 * 3 - 2) and transforms to appropriate clojure with op order"
[infixed]
(reduce
(fn [prev [op term]]
(if (= op *)
(concat (butlast prev) [* (last prev) term])
(concat prev [(op term)])))
(list + (first infixed))
(partition 2 (rest infixed))))
Very wierd . when I change it to
defn read-image-data [id]
(let [id-checked (s/conform ::objectNumbers id)]
(if (s/invalid? id-checked)
(layout/render "error.html" {:message "something was wrong with reading the ids"})
(client/get
(str " " id "/tiles")
{:as :json, :query-params {:format "json", :key (env :key)}}))))
I work on clojure with ISS web server. But schould adding specs and input validation not interfer when I do lein run
but why do I see here a explanation here :
;; the specs files for input validation
(def id-regex #"[A-Z]{2}-[A-Z]-\d{1,2}")
; spec whci defines how the objectNumber must look like
(s/def ::objectNumber (s/and string? #(re-matches id-regex %)))
; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
; spec to validate the id
(s/def ::objectNumbers (s/coll-of ::objectNumber))
; core functions
(defn read-numbers
"Reads the ids of the paintings"
[response]
(layout/render "error.html" {:message (s/explain ::response response)}))
@petr.mensik That’s because src
is on the classpath — so src/migrations/init.sql
is copied over as migrations/init.sql
— so remove :resource-paths
and you should see the correct behavior.