Fork me on GitHub
#clojure
<
2016-11-23
>
baptiste-from-paris10:11:43

hello guys, little question about clojure.spec! I have been playing with it and I find it awesome! But I am quite confused on how to use it at runtime

baptiste-from-paris10:11:35

which is the best approach, :pre/:post or using conform/valid? in my function for example. I know it really depends on the level of error you want to get but I would be glad to have feedbacks of people using it in prod

mpenet11:11:37

yep, that's one way

mpenet11:11:34

s/assert is probably better for deep code since you can toggle it

baptiste-from-paris11:11:02

is there any public repo where spec is massively used ?

mpenet11:11:53

pre/post are toggle'able too, since it expands to assert

mpenet11:11:04

never used it tho tbh

baptiste-from-paris11:11:24

I have played with it and it's awesome, really

baptiste-from-paris11:11:49

But I still look for an example of how well it can be integrated in a robust project

baptiste-from-paris11:11:53

for example, ring maintainer made a different lib for validating request response handler

mpenet11:11:56

"massively", none that I know of

mpenet11:11:11

I played with ring as well https://github.com/mpenet/ring-spec, but I am not sure this qualifies as "massively".

mpenet11:11:52

there's clojure.jdbc that has specs and I have some for 2 projects, but they aren't not covering generation on one of them

baptiste-from-paris11:11:12

oh ok, let me see it then

mpenet11:11:47

the latter is not 100% complete

baptiste-from-paris11:11:31

you earned +1 follower on twitter 😉

mpenet11:11:36

ultimately I'd like to be able to generate a valid cql schema and queries for that schema and run arbitrary queries against cassandra

mpenet11:11:45

🙂 hey merci

baptiste-from-paris11:11:29

for alia - (haven't checked everything) - you define specs and use it in core or it's up to the user ?

mpenet11:11:14

I use instrumentation in the tests, but otherwise it's totally optional yes, up to the user to add them to their dependencies and use instrumentation at dev time or not

mpenet11:11:56

same for the other project, it's both to generate random schema/queries and tests cassandra itself and to validate query generation via the dsl or the ast in hayt

mpenet11:11:23

up to the end user. Then ultimately, once things settle I might integrate this more deeply, but it's all still alpha

baptiste-from-paris11:11:40

I really have to go but I'll look at it this afternoon!

Aron11:11:00

Anyone knows a good way to visually explain immutability the concept? (not the implementation with tries)

gfredericks12:11:47

@ashnur do you mean patterns of usage? The raw concept is pretty basic: you can't mutate them

Aron12:11:29

That's kind of circular, isn't it? Immutable that I can't mutate.

Aron12:11:24

Also immovable is that I can't move, irresistible is that i can't resist...

gfredericks12:11:31

it's a definition

gfredericks12:11:39

how do you explain immovability the concept?

Aron12:11:54

I don't, never did, why?

gfredericks12:11:12

I'm saying there's nothing left to explain

gfredericks12:11:21

so I'm thinking you want something else explained

Aron12:11:33

oh, well, if someone already understand what they are talking about, certainly there is nothing to be explained

gfredericks12:11:34

like "how to use immutable data structures to do normal things"

Aron12:11:44

no, i just want to say what immutability is

gfredericks12:11:03

it means the data structure is frozen; you can read from it but you can't make changes to it

gfredericks12:11:23

⇑ I'm just using more words to say "you can't mutate it"; I can't tell what else you want

Aron12:11:00

right, i am not trying to explain immutability to people who already understand that what they are doing is mutating things, even though if i asked they would probably realize

Aron12:11:38

i am trying to explain it to people who do not think about these things, they just write their neat java code all day long, everything works fine, what do you mean "mutate"? like a database?

gfredericks12:11:55

okay, so you need to get into things like when two objects are different vs the same?

Aron12:11:56

so when you say, the definition says everything, i have my doubts

gfredericks12:11:13

references and values?

Aron12:11:43

my problem is that i am not pitching clojure this time, but it's something from the blockchain 😞

Aron12:11:55

so maybe the way clojure people approach this is not applicable here

Aron12:11:05

i just didn't know anyone who uses it 🙂

gfredericks12:11:12

the blockchain itself or some component of it?

Aron12:11:58

not sure what you mean by "itself". and of course some people imagine mutable blockchains.

gfredericks12:11:49

I was just curious what aspect of a blockchain you were trying to describe as immutable

Aron12:11:55

nah, I just think of the distributed datastructure formed from linked blocks that consist of list of transactions.

Aron12:11:23

well, normally it should be immutable, that's one of the promises, that you can check all transactions from the genesis block until today

gfredericks12:11:02

I think blockchains have a lot of conceptual overlap with git in this respect

Aron12:11:26

git is not immutable at all though, you can rewrite history at will

Aron12:11:47

in a blockchain the consensus algorithm doesn't allow you to do that

niwinz12:11:51

rewriting history does not delete the old objects

gfredericks12:11:11

didn't the ethereum community just rewrite history at will? :)

niwinz12:11:12

only a special gc operation will remove that objects, and when they are disappear from reflog

gfredericks12:11:28

@ashnur "can in theory" and "can in practice" are different in this case

gfredericks12:11:44

blockchains are rewritable in theory the same with a git repo is in practice

Aron12:11:45

gfredericks I don't think it's fair to compare. one requires one person to want something, the other required a lot of people to agree

gfredericks12:11:10

sure but that's a property that's a level or two above the data structures involved

gfredericks12:11:20

they both use hash-addressable content to drive immutability

Aron12:11:20

niwinz but I can change the history. I can rewrite it, and force push it, and if you pull it might happen that you lose things you don't want to lose, doesn't it?

Aron12:11:56

hmm, i believe you, but then i have to go and learn this again

Aron12:11:28

gfredericks right, the blockchain term might mean just the data structure, or the technology that includes the consensus algorithm and a bunch of other things.

niwinz12:11:31

when you rewrite history in git is like transoform inmutable hash-map, some objects become invisible/unreacheable by normal code

niwinz12:11:39

but they are still there

niwinz12:11:53

and can be reached using specific git commands

Aron12:11:03

thanks for the clear up

gfredericks12:11:10

@ashnur clojure's distinction between values and identities would probably help here

Aron12:11:38

yeah, i was thinking about that too

gfredericks12:11:43

git and blockchains have a very similar approach to representing values

gfredericks12:11:58

(objects with cryptographic hashes as their IDs)

gfredericks12:11:10

they also both have identities

gfredericks12:11:19

git has a bunch of them, you can create them anytime

gfredericks12:11:38

a blockchain generally has one, and a fancy algorithm for everybody to agree about the value of the identity at any given time

gfredericks12:11:10

so when you talk/ask about immutability on a blockchain, I'm thinking only about values, not identities

Aron12:11:39

value of the identity?

Aron12:11:44

i am confused now

dev-hartmann12:11:59

crosspost from beginners: hopefully some of you fellow clojurians can help me out i have a map that has an event keyword as key and a fn as a value in some of these pairs the i get this result :user.handler/confirm-invite #object[clojure.lang.Var$Unbound 0x31e50ab5 Unbound: #'users/confirm-invitation-handler] other pairs in the same map, coming from the same namespace do not have these issues

dev-hartmann12:11:49

issue meaning fn's beiung unbound

gfredericks12:11:48

@ashnur a value is immutable; an identity is a reference to a single value, that can potentially change over time

Aron12:11:14

yes, i just got confused a bit 🙂, but I understand now

Aron12:11:22

thank you

gfredericks13:11:08

in git, the values are commits, trees, files; the identities are branches and tags; in blockchains the values are blocks and their contents; the identity is the most recent block

gfredericks13:11:35

it's like a repo with just one branch that only moves forward

Aron13:11:06

so, what is a path to a file. when I do git checkout x-branch -- path-to-y-file?

baptiste-from-paris13:11:39

@mpenet nice lib 🙂 (`alia`)

baptiste-from-paris13:11:13

do you plan to generate test based on you specs ?

mpenet13:11:35

Thanks. Kinda once hayt specs are complete this should be usable with alia, there shouldn't be much to do in alia directly (generate a cql schema from hayt specs, and then derive cql statements from there to be used with alia and run tests against these).

mpenet13:11:33

the specs in alia are just guards for devs (instrumentation), or pieces to build validation for libs using it (ex with cluster options, some libs expose them in their conf, for instance cyanite/pithos could build validation on top of these)

baptiste-from-paris13:11:30

it does mean that you have to force the user to use clojure 1.9.* ?

mpenet13:11:56

ultimately if hayt gen work well enough this could be used by other cassandra related projects

mpenet13:11:15

no it's all separate libs atm, you can use alia without all this, even without hayt

mpenet13:11:36

it requires clj 1.7 since it depends on IReduceInit, but that's it

gfredericks15:11:32

@ashnur every commit contains a directory structure (a "tree"), so a path is a way to identify a file object in a tree object

Aron15:11:57

so it's an identity?

gfredericks15:11:02

Just like a sequence of keys you could pass to get-in

gfredericks15:11:20

No, the branch is the identity

gfredericks15:11:27

An identity is something you can dereference. So something like (-> branch deref :tree (get-in [...path...]))

Aron15:11:33

i see, i mixed up changing the content of a place with the value a reference points to

Aron15:11:39

thanks again for explaining

mikerod15:11:56

here's a shot in the dark. has anyone read some OpenJDK source code and know where to find this doc/server_compiler/checktype.txt referred to in http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/87ee5ee27509/src/share/vm/oops/klass.hpp#l132

mikerod15:11:02

I don't see that anywhere in the repo

gfredericks15:11:02

@mikerod have you looked through the whole history?

mikerod15:11:16

gfredericks I don’t think so?

mikerod15:11:25

I’m not sure what that means exactly I guess

mikerod15:11:18

gfredericks are you just suggesting that the file existed at some point

mikerod15:11:28

and now is gone

gfredericks15:11:03

Or has been renamed

mikerod15:11:27

Ok. Luckily, I know very little about navigating Mercurial - so I guess I should figure that out

mikerod15:11:49

I think I’m too new-age when it comes to source control and just think everything should be like git

mikerod15:11:54

version control*

gfredericks15:11:03

You can convert the repo

gfredericks15:11:17

There are some mirrors on github

mikerod15:11:43

interesting

mikerod15:11:48

I’ve never looked for it ther

pupeno17:11:35

I’m using Sente for communication between my web server and the clients. If I have more than one web worker, what method can I use to synchronize them so that when a message is sent to a client, it reaches it no matter which web worker it connected to?

slipset18:11:55

So, I have a ns foo with (defmulti baz :bar) in another ns boo, I have the following

slipset18:11:39

(ns boo
  (:require [foo]))
  
  (defmethod boo/baz :lol [] ...)

slipset18:11:16

This works as expected when I run it from within the repl, but when I run it from an uberjar, I get java.lang.IllegalArgumentException: No method in multimethod ‘baz’ for dispatch value :lol

slipset18:11:07

the uberjar task in lein contains

slipset18:11:13

:omit-source true
                       :aot :all

slipset18:11:08

I’m guessing this has to do with the compiler doesn’t include the boo ns since noone is referring to it, but I’m not sure how to fix this.

fellshard18:11:56

Your multimethod definition (defmethod boo/baz :lol ...) needs an argument. In effect, it determines :lol by applying your dispatching function - :bar - to all the arguments in your implementation.

fellshard18:11:14

Unless that's just a detail that slipped through your snippet.

slipset18:11:46

That’s just a detail that slipped through my snippet.

slipset18:11:53

sorry ‘bout that.

fellshard18:11:43

No worries. 🙂

baptiste-from-paris18:11:45

hello guys does someone know if it's possible to use nested defmulti/defmethod in clojure.spec ?

sinistral18:11:24

@slipset Shouldn't your defmethod in boo refer to foo/baz, not boo/baz, since that's the multifn to which the method is being attached?

slipset18:11:36

@sinistral: correct, but that's another snippet-slip

baptiste-from-paris19:11:14

another problem is I can"t figure out how removing namespace with spec in this example

baptiste-from-paris19:11:13

(gen/sample (s/gen :button/button))

baptiste-from-paris19:11:19

I got =>

({:title "", :button/type "type-round"}
 {:title "4", :payload "2", :button/type "postback"}
 {:title "", :payload "+16505551234", :button/type "phone_number"}
 {:title "47cfSAK", :payload "+16505551234", :button/type "phone_number"})

baptiste-from-paris19:11:57

but I don't want the namespace :button

baptiste-from-paris19:11:16

any idea guys ? (sorry for the length of the message)

slipset19:11:35

oh, and here is a minimal repro for my multimethod problem

alexmiller20:11:10

@baptiste-from-paris the :button/type is being added based on your retag in s/multi-spec

baptiste-from-paris20:11:31

yes, but I need a mutli-spec

baptiste-from-paris20:11:31

I think I miss something here

baptiste-from-paris20:11:21

its exactly my use case

One common occurrence in Clojure is to use maps as tagged entities and a special field that indicates the "type" of the map where type indicates a potentially open set of types, often with shared attributes across the types.

alexmiller20:11:34

kind of interesting - doesn’t look like that will work in tandem with multi-spec using the defaults

alexmiller20:11:49

but I think it can be made to work, give me a sec

alexmiller20:11:09

actually the doc string gives an example with unqualified keys

baptiste-from-paris20:11:00

ok, give me 5min and I'll try to be more understandable with the doc example

alexmiller20:11:59

I’m not sure how general this problem is

alexmiller20:11:51

but what’s happening is that the generator is generating random :type’s which create instances that don’t conform to the spec

alexmiller20:11:28

so the multi-spec gen fails

alexmiller20:11:44

you don’t want to constrain the :button/type spec though (since it’s an open spec)

alexmiller20:11:56

(that’s even warned against in s/multi-spec doc string)

baptiste-from-paris20:11:55

if we take the guide example :

(s/def :event/type keyword?)
(s/def :event/timestamp int?)
(s/def :search/url string?)
(s/def :error/message string?)
(s/def :error/code int?)

(defmulti event-type :event/type)
(defmethod event-type :event/search [_]
  (s/keys :req [:event/type :event/timestamp :search/url]))
(defmethod event-type :event/error [_]
  (s/keys :req [:event/type :event/timestamp :error/message :error/code]))

(s/def :event/event (s/multi-spec event-type :event/type))

baptiste-from-paris20:11:20

we do constrain the :event/type right ?

alexmiller20:11:22

the :event/type type doesn’t constrain it to just :event/search or :event/error

alexmiller20:11:27

it’s spec’ed as keyword?

alexmiller20:11:32

so open to add new keywords

alexmiller20:11:37

that example gen's

alexmiller20:11:50

but if you had for example

alexmiller20:11:53

(s/def :event/type keyword?)
(s/def :event/timestamp int?)
(s/def :search/url string?)
(s/def :error/message string?)
(s/def :error/code int?)

(defmulti event-type :type)
(defmethod event-type :event/search [_]
  (s/keys :req [:event/timestamp :search/url] :req-un [:type]))
(defmethod event-type :event/error [_]
  (s/keys :req [:event/timestamp :error/message :error/code] :req-un [:type]))

(s/def :event/event (s/multi-spec event-type :type))

alexmiller20:11:23

or I guess that :req-un should be [:event/type]

baptiste-from-paris20:11:02

I did not know we could use both :req and req-un

alexmiller20:11:40

those s/keys specs will gen fine

alexmiller20:11:51

but they will generate random keywords for :type

baptiste-from-paris20:11:53

oh ok, got it, not what I want though ^^

alexmiller20:11:36

well the generator for the multi-spec deals with this by only generating the registered dispatch values

alexmiller20:11:56

but there must be an assumption in there that is breaking with the unqualified keys

alexmiller20:11:06

so, atm I’m saying it looks like a bug

baptiste-from-paris20:11:26

atm means ? (not english native ^^)

alexmiller20:11:50

sorry, “at the moment"

alexmiller20:11:01

what, are you from paris or something?

baptiste-from-paris20:11:23

yes, surprising giving my username ^^

baptiste-from-paris20:11:48

but I thought it was a good way to find other clojurians from paris

alexmiller20:11:08

I thought this could be addressed via the retag function but doesn’t seem to work

baptiste-from-paris20:11:56

I am trying little hack here, I let you know where it goes but I am not sure I can resolve a clojure bug even after I read your book 😉

fellshard20:11:49

Check out #clojure-france, might be some folks there as well. 🙂

baptiste-from-paris20:11:07

ok, if you take this code

(ns specs.core
  (:require [clojure.spec :as s]
            [clojure.spec.gen :as gen]))

(s/def ::type keyword?)
(s/def :event/timestamp int?)
(s/def :search/url string?)
(s/def :error/message string?)
(s/def :error/code int?)

(defmulti event-type ::type)
(defmethod event-type :event/search [_]
  (s/keys :req-un [:event/timestamp :search/url]))
(defmethod event-type :event/error [_]
  (s/keys :req-un [:event/timestamp :error/message :error/code]))

(s/def :event/event (s/multi-spec event-type ::type))

(s/valid? :event/event
          {:type :event/search
           :timestamp 1463970123000
           :url ""})

(gen/sample (s/gen :event/event))

baptiste-from-paris20:11:38

it will give you something like this:

({:timestamp -1, :url "", :specs.core/type :event/search}
 {:timestamp -1, :message "v", :code 0, :specs.core/type :event/error}
 {:timestamp 1, :message "kw", :code -2, :specs.core/type :event/error}
 {:timestamp 0, :url "w40", :specs.core/type :event/search}
 {:timestamp -5, :message "Ek", :code 1, :specs.core/type :event/error}
 {:timestamp -5, :url "", :specs.core/type :event/search})

baptiste-from-paris20:11:10

so it means I can't have unqualified key with multimethod i guess

baptiste-from-paris20:11:50

@fellshard I did not know when I subscribed but yes, there is a user group here, which is really cool

alexmiller20:11:21

I can’t think of a good reason that this shouldn’t work with unqualified keys so I’d call it a bug and I’d be happy to log it and take a longer look at it later

baptiste-from-paris20:11:29

ok then, if I have spare time to look at it, have you any guess in the code-base ?

nfisher21:11:19

Quick question any ideas on the release of 1.9.0? I’d like to start picking up spec but it looks like it’s not production ready yet?

slipset21:11:00

@nfisher: check out future-spec by @tonsky

alexmiller21:11:10

@nfisher we’ll prob have some more stuff to say about that next month

alexmiller21:11:22

@baptiste-from-paris this seems to work for me if I switch to a clean repl (multimethod was holding onto state)

alexmiller21:11:26

(defmulti button-type :type)
(defmethod button-type "type-round" [_]
  (s/keys :req-un [:button/title :button/type]))
(defmethod button-type "type-square" [_]
  (s/keys :req-un [:button/title :button/type]))
(s/def :button/button (s/multi-spec button-type :type))
(gen/sample (s/gen :button/button))

alexmiller21:11:19

({:title "", :type "type-round"} {:title "", :type "type-square"} {:title "R", :type "type-square"} {:title "", :type "type-square"} {:title "ww", :type "type-square"} {:title "NcyuB", :type "type-round"} {:title "9UjE", :type "type-round"} {:title "4Q37a", :type "type-round"} {:title "3dA89v", :type "type-round"} {:title "7", :type “type-round”})

alexmiller21:11:58

so I’m going to say it’s fine :)

baptiste-from-paris22:11:26

nice ! I'll look at it right now

baptiste-from-paris22:11:40

working for me, thanks a lot for you time 🙂!