This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-08-20
Channels
- # announcements (3)
- # beginners (63)
- # calva (1)
- # cider (24)
- # clj-kondo (98)
- # cljdoc (8)
- # cljsrn (19)
- # clojure (106)
- # clojure-conj (2)
- # clojure-europe (5)
- # clojure-italy (5)
- # clojure-nl (8)
- # clojure-spec (8)
- # clojure-uk (13)
- # clojuredesign-podcast (7)
- # clojurescript (54)
- # core-async (1)
- # cursive (3)
- # data-science (1)
- # datomic (19)
- # fulcro (7)
- # hoplon (1)
- # off-topic (3)
- # re-frame (13)
- # reitit (1)
- # shadow-cljs (234)
- # test-check (10)
- # tools-deps (59)
- # unrepl (1)
- # yada (20)
hello, good morning! do you know if there is a simple way to use a function without arguments as a custom generator for a spec? something like:
(spec/def ::repository (spec/with-gen #(satisfies? AProtocol) (gen-from-function my-aprotocol-instantiator)))
No, and there are good reasons not to do this. In test.check, the generators are the only things that create novelty (/ randomness). Importantly, they do so in a repeatable way (so you can retest with a known seed) and in a shrinkable way (so a failing test can be “shrunk” to a simpler failing test)
thank you for the answer. maybe i’ll have to rethink my strategy. 🙂
the only way that i found is to use a combination of gen/return
and gen/fmap
: return
to “start” the generation and fmap
to call the function ignoring the value generated by return
; this is a very ugly hack. 🙂
@andy.fingerhut hmm yeah, I want a way to test this behavior on CI
Would redefining in work in this case?
Via binding
It seems perhaps good enough to test an interactive behavior like this interactively, and repeat such interactive tests if anyone messes with that function again?
and try as much as possible to "push I/O to the boundaries" of whatever library this is in, i.e. hopefully it is easy to separate this function and when it is called from most other functions, and most other functions can remain without I/O, or only file I/O, which can easily be tested with automation.
yeah, well, there's a PR that reads input from /dev/tty
and the problem is, I don't know how that behaves on different platforms, so I would really like to test this on CI
Maybe there is a better way, but at some point you have to let folks try it interactvely on those different platforms and see if ti behaves correctly?
WIll CI actually test it on all the platforms you care about?
Hi all, I have the following function:
(defn ->attribute-value [attribute-spec value]
(cond
(as/view? attribute-spec :user)
(->byte-buffer value)
(as/name? attribute-spec
#{:creation-time
:last-access-time
:last-modified-time})
(->file-time value)
(and
(as/view? attribute-spec :posix)
(as/name? attribute-spec :permissions))
(->posix-file-permissions value)
:default value))
In this case, attribute-spec
looks like {:view :posix :name :permissions}
, value
is any object.
I'd like to make this extensible by the consumer of the library as the number of views and attributes is not fixed. I can't use a multi method since I need to match on parts of the provided attribute spec. How would you go about making this extensible?I was thinking of using a protocol but I'd need some way of registering implementations if I took that approach
So my library calls this function internally and I want consumers of the library to be able to register new conversions for future views and names
The attribute-spec
looks like {:view <some-view> :name <some-name>}
.
So in some cases, I want to apply a conversion regardless of name, based only on view
In others based only on name, regardless of view
And others, more specifically based on a view and a name
In that case how would I match any name for specific view?
> I want to apply a conversion regardless of name, based only on view
(defmethod ->attribute-value [:my-custom-view :default] ...)
> In others based only on name, regardless of view
(defmethod ->attribute-value [:default :my-custom-name] ...)
In the view only case, the map coming in would look like {:view :user :name :some-name}
so (juxt :view :name)
would yield [:user :some-name]
.
Which would look for (defmethod ->attribute-value [:user :some-name] ...)
and wouldn't match a default.
I think I'd need pattern matching or predicate dispatch to be able to achieve that. However in that case, it would be hard for users to register new matches or predicates.
:default
is a bit of magic
this works though:
(defmulti ->attribute-value (juxt :view :name))
(defmethod ->attribute-value [::my-custom-view ::default] [_] "my custom view")
(defmethod ->attribute-value [::default ::my-custom-name] [_] "my custom name")
(derive ::blep ::default)
(->attribute-value {:view ::blep :name ::my-custom-name})
;; => "my custom name"
my mistake, :default
is a part of a multimethod, not part of a hierarchy, that's why it does not work in isa?
check
Yeah, what I mean by a bit of magic is it's hard coded into the definition of a multimethod
If I use hierarchies, the user has to derive from :default
for every possible name.
Really all I need is a :*
entry that matches anything, so that I can say (defmethod ->attribute-value [:user :*] ...)
or (defmethod ->attribute-value [:* :creation-time] ...)
It looks like for a (defmulti ...)
you can override the hierarchy
Haha, yeah
I'd need a way to say :*
is a descendent of any other keyword
Not sure how to achieve that
if you look at isa?
source to see how it uses hierarchy, you'll see this:
(contains? ((:ancestors h) child) parent)
your hierarchy can be a map with :ancestors
key, that instead of whatever is there by default is actually a function that makes this code return true
Ah yes, nice
it will screw up derive
for this particular multimethod, but you probably don't want that anyway?
Yeah that would be fine
Ok, I'll give it a go, thanks
@toby924 behold the horrors of abusing implementation details!
(defmulti ->attribute-value (juxt :view :name)
:hierarchy (atom {:ancestors (constantly #{:default})}))
(defmethod ->attribute-value [:my-custom-view :default] [_] "my custom view")
(defmethod ->attribute-value [:default :my-custom-name] [_] "my custom name")
(->attribute-value {:view :blep :name :my-custom-name})
;; => "my custom name"
Does seem a little hacky though 🙂
The docs are insistent that you use make-hierarchy
for any custom hierarchy
No, and there are good reasons not to do this. In test.check, the generators are the only things that create novelty (/ randomness). Importantly, they do so in a repeatable way (so you can retest with a known seed) and in a shrinkable way (so a failing test can be “shrunk” to a simpler failing test)
Hey Guys, I am trying to use http://clojurequartz.info/ for scheduling periodic jobs for one of my Clojure projects. I am wondering, what tools/libraries I can use to monitor (web UI) to monitor these jobs? Any sort of pointer would be very helpful. Thanks in advance. :spock-hand:
I think there are quartz dashboards, I remember one being mentioned in the documentation
Thanks. I had a look at it earlier and project doesn’t seem to be in active development and lacks documentation. Wondering, if there is something in Clojure?
@U09LZR36F Cool. Thanks. Also, what’s best way to trigger scheduled jobs in Clojure?
@U09LZR36F Cool. Thanks. So, basically, I can use Qaurtzite to schedule jobs, then put a table that maps the logs/status of the jobs and then show it up on a web UI.
I have a "what's the most beautiful way" kinda question.
I have a function get-user
that takes a user id and gets the rest from a database. I also have a looong ole list of users (currently in a vect, but I can change that if needs be). Whats the best/most beautiful way to parallelize the calls to get those?
perhaps you’d like not hammer your database and write a single database query and join the user-id’s to it.
That sounds far too easy on the DBAs. In all honesty, I'm hitting a service actually and the maintainer said they didn't have time for a batch request and told me to perform my requests in parallel. I stared in shock for a bit, shrugged, and told them they were free to be crazy. ⚰️ I'm kinda hoping they'll "ask me for an emergency fix" to batch after they see the performance logs?
if number of users/ids is below 10, might be fine using pmap
, otherwise, using SQL like id=id1 or id2
or id in [id1,id2]
should be the way to go.
I know nothing of their service/db setup. Maybe there's some arcane path where batching doesn't help much?
I know nothing I'm afraid. ¯\(ツ)/¯
Even with couch, mongo, or dynamo I'd want batching if I were them. Maybe Cassandra doesn't care? But it's enterprise software, so I'd be a bit surprised. I really oughta ask more just for my own curiosity.
I think even Cassandra would tell difference between 1000 request/response and 10 request/response batches
Right? But I'm just speculating on someone else's stack.
The only downside I know of there is that if one of them dies you end up chugging through the rest of the threads before you can move on, right?
https://clojuredocs.org/clojure.core.async/pipeline-blocking might give you a little more control over the rate at which you hit the service? Depending on the length of time an average query takes, it may be worth it to "batch" your requests within pmap - to avoid spinning up a new thread for each user: i.e.
(apply concat (pmap (partial map get-user) (partition-all 100 user-ids)))
;; ^ will fetch 100 users in a single thread, instead of 1 user per thread. This greatly reduces the amount of time spent managing thread versus:
(pmap get-user user-ids)
Why do you need the concat
?
it's to move the batches back into the same list
user=> (apply concat (pmap (partial map inc) (partition-all 3 (range 10))))
(1 2 3 4 5 6 7 8 9 10)
user=> (pmap (partial map inc) (partition-all 3 (range 10)))
((1 2 3) (4 5 6) (7 8 9) (10))
Ohhh, that's pretty. 👏
Why do you need the concat
?
@borkdude In school the way we tested stuff like that was by first factoring out the input and output into Readable and Writable "things" (I forget the exact java interfaces we used)
@emccue that's a good strategy. one issue is that the PR reads from (io/reader "/dev/tty")
and I have no clue how that behaves over different operating systems, that's why I want a test for that in CI
but maybe replacing /dev/tty by an explicit reader thing and testing that is sufficient
But there is value in separating the first one even if you can't think of a way to do that validation
Does anyone happen to have any experience using @seancorfield’s clj-new.generate
feature and has a working CLI and associated generate
function?
ahh nvm found the source code 😊