Fork me on GitHub
#clojure-spec
<
2018-09-12
>
samedhi15:09:34

Is it possible to write a spec that generates a sequence where each element in the sequence has some relationship to the previous element(s) in the sequence?

samedhi15:09:08

As an example, like one that multiplies the last element of a sequence by 2 or 3 (random).

samedhi15:09:51

so [1 2 4 8 24 48 144] and [1 3 9 18] would both be sequences it might generate.

taylor15:09:05

are you only interested in generation, or do you want a spec to validate inputs too?

samedhi15:09:17

I guess both

samedhi15:09:57

I mean I would like to use test.check with this generated sequence, so I assume that means using both (I think).

Alex Miller (Clojure team)15:09:25

the way I would handle this is to use gen/bind

Alex Miller (Clojure team)15:09:48

generate a series of random multipliers (a sequence of random 2s and 3s)

samedhi15:09:24

Ah, I see where you are going.

Alex Miller (Clojure team)15:09:30

then gen/bind to turn the sequences of multipliers into a sequence of values

Alex Miller (Clojure team)15:09:40

or rather, I guess gen/fmap is sufficient here

Alex Miller (Clojure team)15:09:52

it’s just apply a function to that sequence

samedhi15:09:00

Yeah, so what I was really hoping was to generate the initial (seed) state of my program as the first element in the sequence, and generate the rest of the sequence by constantly looking at the last element in the sequence, determining what actions can be taken at that state, and then picking one of those actions and applying it to said state.

Alex Miller (Clojure team)15:09:09

start from a gen of a tuple feeding the fmap

Alex Miller (Clojure team)15:09:54

(s/gen (s/tuple int? (s/coll-of #{2 3} :max-count 10)))

Alex Miller (Clojure team)15:09:40

then gen/fmap a function over that that takes the first element in the tuple and repeatedly applies the multipliers in the second element to produce a sequence

samedhi15:09:56

Ok, the only wrinkle I see there is that only a subset of all of the actions are valid for some of the states.

samedhi15:09:02

But that is an idea.

samedhi15:09:25

Maybe, I can just make it so that if the action taken is NOT a valid action for that state, it just returns the original state (identity).

Alex Miller (Clojure team)15:09:27

if you need to randomly choose an action per state, you will need to move to gen/bind

Alex Miller (Clojure team)15:09:01

but I think you could make that work too

samedhi15:09:54

Interesting, interesting. I mean I tried to read through the "rose tree" thing for test.check generators. And... well, it looks like it is generating some sort of tree using something like a recursive algorithm. But, yeah, got lost there. 🙂

Alex Miller (Clojure team)15:09:56

in that case, I’d probably gen the initial state and the number of actions to gen

Alex Miller (Clojure team)15:09:18

yes, but you don’t need to worry about that - the key is just to capture all randomness via generators

Alex Miller (Clojure team)15:09:25

test.check will do the rest

samedhi15:09:48

Ok, thank you for your help, I'll look at it some more this evening.

Alex Miller (Clojure team)15:09:14

this is advanced generating, but it’s where things get interesting

Alex Miller (Clojure team)15:09:33

a lot of interesting problems in simulation testing look like this

samedhi15:09:59

Really, it is just the fantasy I have of letting spec generate every possible way that my program could run and being like "yep, that one thing encompasses all the test".

samedhi15:09:33

You still get a great deal of benefit out of using the normal fdef and test.check per function imo, but the idea of actually generating all states of the program would be really cool.

gfredericks15:09:33

@samedhi @alexmiller this is exactly what the stateful-check library tries to automate, I believe. I haven't used it

samedhi15:09:21

@gfredericks Awesome :thumbsup:

taylor15:09:32

would there be a way to spec this kind of sequence without a custom predicate?

roklenarcic15:09:45

Is there some way to specialize a key in s/keys? Like I have a (s/keys :req [.... ::type]) where type is #{"ORG", "USER"}, but now I'd like that same spec but where type is limited to `#{"ORG"}

Alex Miller (Clojure team)15:09:08

@taylor do you mean custom generator?

taylor15:09:00

no I mean if you wanted to conform/validate a sequence like that, asserting that each element is a multiple of the previous

taylor15:09:25

I was trying to imagine how you might use the spec primitives to do that but I can't imagine how

Alex Miller (Clojure team)15:09:05

not as a pred in the coll spec, but you could s/and with a function that verified that

👍 4
Alex Miller (Clojure team)15:09:37

or, maybe you could put that in :kind if you had a seq-of-mults? pred

Alex Miller (Clojure team)15:09:37

@roklenarcic no, but you can s/and it with a more restrictive predicate

roklenarcic15:09:42

It tried to s/merge it with another keys spec which has key that is more restrictive, with different keyword ns and same name

roklenarcic15:09:14

it kinda works, in that it will check both

Alex Miller (Clojure team)15:09:12

there are pros/cons to those approaches wrt conform and gen

roklenarcic15:09:30

ah, haven't thought about that

Alex Miller (Clojure team)15:09:03

I think s/and is the most direct way to say it: “I have this map AND it must meet this extra constraint”

Alex Miller (Clojure team)15:09:23

s/multi-spec can also be handy for some cases like this but it’s a lot of weight to add if it’s only this

roklenarcic15:09:43

I'll need to check out multi-spec

roklenarcic15:09:28

Using spec-tools's data-spec solves a couple problems, but sadly data-spec has a bug where they don't name their specs

misha19:09:02

@arohner well, that's embarrassing opieop (re: or in s/keys)

taylor19:09:07

what's opieop all about

misha20:09:59

face expression

taylor20:09:30

it's so small I can't make out his expression

taylor20:09:14

I only see it used on this Slack and I feel like I'm missing... something

misha20:09:26

it is from twitch .tv chats originally

👌 4
lilactown22:09:29

clojure.spec has the ability to transform data from one form into another, right?

lilactown22:09:05

are there any ground-up guides around data transformation?

seancorfield22:09:52

The basic rule of thumb is "don't do that"

😵 4
seancorfield22:09:45

Spec isn't designed as a data transformation system and when folks ask about doing it, @alexmiller usually pops up and says that's not a good idea 🙂

seancorfield22:09:01

That said, I think there are some use cases where limited coercion is acceptable. For example, we use spec for API / form input validation so data is mostly strings but we want strings, longs, doubles, Booleans, and sometimes dates out of it. I think that sort of spec is "OK" if you're careful (in particular, think about generation of data using such specs, as well as applicability of the spec elsewhere in your system -- just because you accept strings at the API/form level doesn't mean you want to accept strings elsewhere in your domain model).

seancorfield22:09:26

What sort of data transformation were you thinking about @lilactown?

lilactown22:09:03

I'm creating an API for creating content schemas in a CMS

lilactown22:09:11

i have a general purpose edn format I'd like to use, and then coerce it into the JSON representation for the particular CMS we're using

Alex Miller (Clojure team)22:09:31

to be specific, transforming data use conformers in spec is imo not a good idea. using spec as a target for driving transformations is potentially a good idea. I think https://github.com/wilkerlucio/spec-coerce is pretty good.

lilactown22:09:50

thanks! I'll take a look