is it possible to use custom clojure.spec conformers in generators?

I’d like Yada to NOT catch my exceptions when developing, so that my IDE can start the debugger. I cannot find a way to do this, seems exceptions are always handled

Anyone having had the same problem?

@andrea.crotti swagger seems to be de facto now, all/most web libs have support for it (using plumatic schema). and there is both openapi3 & spec support soon.

I'm having trouble creating specs for nested vectors

For example, creating a spec for [[n]], where n is an integer

(s/cat :bar (s/cat :foo int?)) doesn't seem to work

Anyone know what I'm doing wrong?

you have to explicitly indicate the nesting with s/spec, e.g. (s/cat :bar (s/spec (s/cat :foo int?)))

Thanks, that works - I missed that part in the docs

in the future i would run (sg/sample (s/gen (s/cat :bar (s/cat :foo int?)))) so you can see what it actually does

which in this case gives you ((-1) (-1) (0) (1) (-1) (4) (-1) (-1) (-8) (0))

so what’s the reasoning behind s/cat not nesting without s/spec? is the idea that you are still talking about the same regexy spec?

adambros, if cat would respect the nesting you would need something else than cat to concatenate two regexes, clojure just chose the way it is and i personally prefer it that way

anyone else find that for storing data, maps work much better than vectors becuase you can do nested updates via update-in, whereas with vectors, it's constant destructuring + rebuilding ?

if you know the index of the thing you want to modify update-in works just the same

but maps are generally more useful, true

yeah, do you like (update-in ... [ 3 1 4 1 5 9 2 6] inc) or (update-in .... [:bank-account :checking 2345232] inc)

welcome @actionshrimp !

hello there

Do iterators like map pmap doseq exert "backpressure"? If i have a chain of iterators where (A -> B) are both iterators and A iterates faster than B does my seq bundle up between A and B or does B only ask for As output at roughly the speed it can use it?

Maybe It cant be generalised for them all but I'm talking the clojure.core iterators for now.

Not sure about pmap (it may be lazy or eager) but map and friends like filter are single-threaded. There's no implicit paralelism with collection functions. map and filter produce lazy sequences that are realized in chunks of 32 on demand. So (range) returns an infinite lazy sequence but (take 20 (range)) will only realize some initial elements. Since lazy sequences chain nicely, (take 20 (map inc (range))) also won't blow up your memory.

and doseq is a macro, like a for i in sth: loop in Python.

Yeah, sorry i maybe tried to oversimplify my question

I guess pmap is the tricky one since it uses futures

But yeah, now i've thought about it, your answer is correct for things that consume/produce lazy seqs

Thanks 🙂

fits with what i got from my simple (and probably naive) test (doseq [i (map (fn [x] (Thread/sleep 100) (println "in" x) x) (range 1000))] (Thread/sleep 1000) (println "out" i))

pmap runs ahead by availalbProcessors +2

but is otherwise lazy

what’s the reasoning behind having lists as the default datastructure? (vs vectors?) is it lisp compatability, or is there more technical reasons for it?

@jooivind lists, which append to the left side of the datastructure fit really well with code transformations. A lot of macros will manipulate the head of the expression. Also it matches the evaluation order of arguments. Lisp evaluates left-to-right, top-to-bottom.

This happens to fit perfectly with lists, which do the same if you use first/next.

but yes, you could write a lisp with vectors.

obviously- ring buffers

for efficient front insertion

I suppose, but why go to all that trouble when you can do it with a Cons cell

but yeah that makes sense

id say ring buffers are simpler than lists

but thanks for your thorough explanation 🙂

So...I have to challenge that assertion, what's simpler than: (deftype Cons [head tail])

the datastructure is java, no?

(->Cons 'println (->Cons 42 nil))

ring buffers just cannot be simpler than lists

isn't a ring buffer a list with rules about pointing to itself and a complicated dance when removing the last item, etc

it’s an array

just that you can wrap around

no it's a contract on top of an underlying type

the list has no contract

it's two pointers

And complex logic on what to do when you have to resize the array

Yeah, a ring buffer is (deftype RingBuffer [array insert-pos end-pos])

I think i’m coming from this from a different direction than you are arguing.

and then it will need to implement some interfaces for inserting and other stuff

well, if your ring buffer doesn't wrap around and allow resizing, etc, we aren't talking about the same thing

@jooivind you're right, I don't think I understand.

interesting answers though

and I think i understand your points

So what do you mean by a ring buffer being simpler than a list?

Data is data

not pointers

in memory

but this is a bit moot

considering we’re running on the JVM anyways

right, but that's the same in any language, an array of two items in C is exactly the same as a struct with two item pointers.

no

or more correctly an array of pointers to items in C

you can have intrusive datastructures

and arrays can be in memory defined objects

True, but it's really hard to have intrusive data pointers that are fully polymorphic.

can you give me an example of an intrusive data structure?

looked it up

@jooivind one of the draws of Clojure is that you can arbitrarily nest data: {[1 {2, 3}] :some/key 11 {:my :map}}

had never heard of that term

That is next to impossible in C++ without everything inheriting from a common class, in which case intrusive data structures become really inefficient.

boost

and it’s a very specific implementation

But still an array in C

will in effect be a memory area with actual stuff

not pointers to other stuff

Right, but you really don't want that. Hashmaps require a overhead of close to 100bytes, while an integer may take 4-8bytes. You don't want "Object" to be the size of the largest object in your runtime.

which is why int is not an object in c/c++

anyways, thanks for all the responses

and yes i like clojure

@jooivind think about that though...that's the entire value proposition behind Clojure, arbitrarily nested data. It's impossible to have both. Boost or no, it's not possible without a ton of JIT magic.

Yeah i’m not saying i want that

i was just giving examples

And if that was possible Cons cells would still be simpler than arrays, since most arrays in safe languages contain stuff like a length count.

and arrays in Clojure are quite different from just a simple area in storage

but lists are as well ?

to keep them immutable

?

ah i think i see you're point

as far as i understand it arrays are shallow trees with buckets

i think we were arguing that a list is inherently simpler than a ring buffer

but you are arguing that Clojure's implmentations are not necessarily so

is that correct?

sure

ah

that was my first point

Lists in Clojure are just a bog-standard single-linked-list like in any other language.

i'm with you now

(defn cons [h t] (fn [c] (if c h t))) (defn first [c] (when c (c true))) (defn rest [c] (when c (c false))) I always do this to my co-workers as a revenge for office pranks

@tbaldridge is the primary difference between the nREPL and your preferred network repl the time at which the supporting functions/libraries are loaded into the host?

@futuro No, it's more that there isn't a reason to load anything on the remote end at all

99% of what a remote repl does can be done by just sending forms over. For example, to get the docstring for a var first you could just send: (:doc (meta #'first))

I hadn't thought about that. Cool, thank you.

@futuro I just looked at refactor-nrepl, apparently that's middleware that runs on the server side of the connection. There's no reason for that, why should my refactor logic be inside my repl server?

Perhaps this has been talked about before, but are there any plans to provide a way in Clojure to easily create a subnamespace without a corresponding file? I know you can do it now with a combination of in-ns and *ns*, but thought that a more pleasantly looking, bult-in mechanism would be useful. I am asking because of the increasing use of qualified keywords as a means of defining entities and their attributes.

@patrkris Yes and no. Recognize that what you're asking is a possible solution, for the real problem: You want to use ::foo/bar without foo existing as a file. That is something I've seen some discussion around.

Personally I doubt the outcome of that will be something that creates a namespace just to be able to alias it. That seems like a work-around for the root problem of wanting a way to alias namespaces that don't exist.

@tbaldridge looking at the code for refactor-nrepl, it seems that sans all that code living on the server, I'm either recreating most of the logic in my editor's scripting language, passing every form to the repl anyway, or perhaps a mix of the two. With the code living on the server I only need to handle message creation, sending, and reception.

@tbaldridge I see what you mean. Could you maybe provide anecdotes on what you've done in your own work?

Something like fn documentation is pretty straightforward, but they're doing much more in refactor-nrepl

Of course, in some cases it makes sense to have a file for the subnamespace for various reasons

@futuro right, but the problem is that if nrepl-baz uses [foo 1.2.3] and I need [foo 3.4.4], I'm sunk, I can only one version of a library, if I use the newer library, it may break nrepl-baz. Not to mention that there's tons of stuff that can conflict. Basically I have to blindly include any deps that are in use by the middleware.

So the crusty-old-programmer side of me wants to tell editor developers: "Go use a real language in your editor...something like Java where you can run Clojure". But I know it's not that easy.

what's the point of "sequence" xform signature? I am running some quick benches and it seems to always be slower than normal (->> coll (map ..) (filter ..) (remove ...)) etc

(other than reuse of the xform)

@mpenet can you post your code? A true apples-to-apples comparison would require you to consume all the results of the seqs

(def x (comp (map inc) (filter odd?) (remove nil?)))


(dotimes [_ 5 ]
  (time (dotimes [i 10000]
          (doall (sequence x (range 1000))))))

(dotimes [_ 5 ]
  (time (dotimes [i 10000]
          (doall
              (->> (range 1000) (map inc) (filter odd?) (remove nil?))))))

I know I should be using criterium, but it just a quick dirty run

1800 ish msec for the first, 1500ish for the second (on average), tried different seq sizes as well

or is there something magic happening with range maybe (I recall it might have specialized path for iteration)

yeah, sequence is a strange beast.

that might be it (take 100 (repeat 10)) turns it around

in that case sequence is 30% ish faster

a lot more actually

yeah, and 1000 items is rather small, that might play into it as well.

there is some spinup time every time you use a transducer

I tried various sizes, same result

true

Sizes does matter indead 100k iterations changes the results too (I guess jit warmup takes time).

mpenet try using Criterium, it tries to compensate for stuff like GC and JIT warmup

I kinda knew I was heading into strange results land by not using it

and it will also warn you when lein mucks with your JVM JIT settings (as it does by default)

anyway, sequence is actually quite nice

personally I really like eduction you can use like like a sequence (append stuff to it), but at the end if you use reduce or transduce it's just as fast as a transduce pipeline.

eduction is nice too yep, using it in a few places, but I somehow discarded sequence early and I am now rediscovering it

so for my own edification, when should I use sequence vs eduction? I never took the time to read-up on `sequence.

at a glance my guess is the difference is caching

sequence will return a lazy seq

sequence gives you a lazy seq that you can read extra times for free, but have to keep memory in mind if you do that

eduction a reducible/iterable

eduction gives you a reducible so it takes the same amount of work to process it repeatedly, but might be easier to reason about memory implications

also obvious differences if the source data isn't immutable

@tbaldridge eduction has some head holding implications

it will keep a reference to the original sequence

so beware of that

@bronsa the original might not be a sequence though, right?

whatever coll is

it might be a reducible thing that queries a database each time you reduce it, for example

yeah sure

which I guess is what people usually use with eduction

sequence is apparently a good replacement for the very common (->> xs (map f1) (remove f2) (filter f3))

also IIRC http://dev.clojure.org/jira/browse/CLJ-1793 fixes that head retention thing in eduction and it's marked for 1.9 so hopefully not an issue in the future

well when hotspot can do its magic at least (apparently)

at one point I was trying to figure out how to remove intermediate data structures in test.check's shrink trees, and I think I concluded that sequence was the only way to do it

yes exactly, that's why I was using it in the first place

it's a bit the sidekick of transduce for lazy seqs results

@bronsa it should still hold onto the head if the caller is holding onto the return value of eduction, right?

yes

okay cool

but not in when you're e.g.`(into [] (eduction .. coll))`

right

(i'm going off of memory about this tho, I might be recalling the issue wrong. It's been a while)

ah look at that I added a comment with an example http://dev.clojure.org/jira/browse/CLJ-1793?focusedCommentId=42558&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-42558

any frequent users of cider here?

I use it every day :]

I found a channel for it, so i asked my question there 🙂

is keeping a list of atoms in an atom a very bad idea? how else I could model a “mutable” collection of “mutable” records?

yeah @bronsa @gfredericks eduction will hang on to head while reducing

I don’t want to lock the whole thing when swapping individual items but I still want the list to grow and shrink.

@nooga is it a huge performance problem just to use one atom?

If that did become a problem I might just use a indexing atom or something to keep track of some sort of Ids for the others and store the others in their own atoms but not nest them themselves.

I’ve never tried though and a basic google turned up this so take it with a grain of salt http://stackoverflow.com/questions/8117404/clojure-states-within-states-within-states

Slight, the records are holding state associated with open websockets - I could easily get away with making this state completely local to the websocket handlers but then I will not be able to access it in any way if I’d like, for example, to write some kind of admin UI that will allow me to modify the data, kill sockets etc.

So basically, what I have is just a bunch of atoms holding records and they’re local. What I want is to keep track of them globally.

Unless you’re bashing each of those states constantly you can probably just store them all in one atom.

but then I will have to do lookups every time I want to access/change a record, instead of just touching an atom that’s local to my handler

So you can use assoc-in, get-in, update-in, etc. to deeply modify the atom

If you just don’t want to refactor your current handler that’s different

(defn make-handler [socket] (let [state (atom {}))] (fn [msg] … (swap! state assoc :foo 1))) vs (defn make-handler [socket] (let [state (new-global-state))] (fn [msg] … (swap! global-state assoc-in [state :foo] 1)))

and then suddenly the handlers must care about global state

@nooga Use cursors to provide partial view. And deref cursors in components.

oh, I thought that cursors are only in reagent

I have a question. In JavaScript, you can place variables directly in object literals, with no key, and they're added with the variable name as the key. Is there a shortcut like this in Clojure, for maps? (I'm actually making a JS object, but the syntax uses a map.)

and I’m writing in Clojure, this is server-side

@nooga can’t you just make a function that transforms the individual handlers state in the atom and pass that to the handler.

Like a partial that just does the assoc-in so it’s transparent to the handler

hm, good idea, but I would take it further and write some kind of universal proxy for swap!, which is essentially a cursor 😉

Basically, what I'm asking is I'm making a map with keywords mapped to local variables (made with let) that are the same as the keywords. Is there any shortcut for this?

@tech_hutch macros can do it, but there's no way to do it with curly braces, which I've found ruins it for me

Okay, thanks.

unless

I'll just do it by hand.

you made a data reader like #local-map {:_ foo bar baz}

where you'd need some dumb placeholder like :_ to get an even number of entries

which would drive me bananas

Well, you could use a vector for the syntax instead, I suppose.

That's more work than it's worth, I think, since this is only one time I'd use it.

@tech_hutch here's the macro approach: https://github.com/plumatic/plumbing/blob/master/src/plumbing/map.cljx#L104

Oh, someone's already done it?

That’s nifty

That's not in CLJS core, is it?

I mean, it's a library?

correct it's a library

just steal borrow the macro 😛

it's one line

How little code can you steal without violating licensing

> Distributed under the Eclipse Public License, the same as Clojure.

Violating licensing? It's under the EPL.

. That ^

Oh is the EPL commercial useable?

like apache?

"Public"

It's an open source license.

Oh, commercially? I don't know.

I would assume so.

Yeah, GPL is a public license but you can’t use the code unless you opensource your code.

Or don’t redistribute it

Quick question: if I would use a macro like keyword-map, could I put #js before it to make it a JavaScript object? (In CLJS, of course)

I think so, but not sure.

Okay. I'll try it.

I can’t get the macro to work with http://clojurescript.net

I was just trying that, myself.

Same.

It returns an empty map.

I would consider posting in #clojurescript

they might know what’s going on

Okay.

how come I can set! *warn-on-reflection* but I can’t do the same for my own dynamic var?

set! requires that the var have a thread-local binding

and it only changes that

(in contrast to alter-var-root)

there are some special vars, like *warn-on-reflection*, that the compiler and/or the repl set up thread-local bindings for

oh ok, thanks @gfredericks

np

How can we build better "first contact" experiences for new programmers and Clojurians? Our open source learning group is hosting the free online webinar Essential Klipse this Saturday at 7 pm UTC to address that question. Everyone is welcome! http://discuss.thevalueoflearning.org/t/webinar-discussion-2-essential-klipse/39?u=jay

I just found out that clojurescript interfaces are totally different from clojure interfaces :S

do you mean protocols?

hmm… I think that there are interfaces defined in clojure.lang

( I’m trying to port this to clojure: https://github.com/reagent-project/reagent-cursor/blob/master/src/reagent/cursor.cljs )

I think I just got deref,`reset!` and swap! working but I still need the others 😮

yeah, they're all named slightly differently,

a few pointers: IMeta -> IObj

IPrintWithWriter -> is the clojure.core/print-method multimethod (extend that instead of using a protocol)

and IWatchable -> clojure.lang.IRef

protocols on the clojurescript side and interfaces on the clojure side

@tbaldridge @hiredman thanks!

Has anyone here worked with alumbra (graphql for clojure)?

@tbaldridge looks like it actually works! I’m surprised with how straightforward that was given that I just deftyped something that looks and quacks like one of the core types 😄

yep, that's the awesome thing about clojure almost everything is backed by really simple interfaces. These interfaces were broken up even more in CLJS, but CLJ is still really good.

are there any libraries out there that help in turning xml to edn, something higher level than the data.xml elements. so the resulting structure is just a recursive clojure map of tag name to the content of the element?

@tbaldridge Yeah, I hear that. It's personally a price I haven't had to pay yet, but I can definitely see that as a possibility. I've wanted a way to load multiple versions of the same dependency into a project -- specifically to avoid dep collision between my code and libraries -- but haven't spent the time to learn about dependency resolution/namespacing to do anything meaningful about that desire.

hi please can you help me?? how I can make a spec case insensitive:

(s/def :vtex/channel #{"KUSHKIPAGOS"})

you could replace that with #(= (clojure.string/upper-case %) “KUSHKIPAGOS”)

the spec part that is

I love your username, @zerocooljs

@alexmiller thank you that works!!

@dpsutton 👍

lol. zerocoolclj

(into {} [[1 2] [3 4]]) <-- works (into [] ((1 2) (3 4))) <-- does not work now, I have a list of even length, and I do (into {} (partition 2 lst)) <-- does not work how do I make this work?

qqq, can you post the error that you are getting on teh second one? ((12) (34))

@qqq the map entries specifically have to be vectors i believe

(into [] `((1 2) (3 4)))

ah ok, i was just making sure you were quoting the list

so (into {} (map vec (partition 2 lst))) should work I believe but I think there might be a better way to accomplish what you're trying

@jjttjj: confirmed that works

hello everyone, im new to clojure and the JVM in general... i have an existing database, which is, unfortunately full of triggers .. i have a field (phone region prefix) which isnt optional ( you have to provide a value ), but there are triggers that calculate those values and input them for me if i try to make a insert statement on the mysql CLI, everything goes fine, but when i try to run it via JDBC I get an java exception java.sql.SQLException: field 'ddd' doesn't have a default value [6:10] is there a way to mimic the CLI behaviour?

(defn create-usuer
  [^String email ^String phone ^String username ^String cpf]
  (jdbc/insert! db :tb_usuario {:tipo "p"
                                :logado 0
                                :ativo 1
                                :idPaisTelefone1 1
                                :telefone1 phone
                                :usuario email
                                :email email
                                :senha "hash the cpf"
                                :celular phone
                                :cpf cpf
                                :dataCriacao "NOW()"
                                :dataAtualizacao "NOW()"
                                :nome username}))

"NOW()" like that is going to attempt to insert the literal string NOW(), not call the mysql function

@hiredman thanks, but i getting errors prior to this 😕

inserting a record like that escapes all the values

what is field ddd?

isnt defined in the query, its a field which the pre-insert trigger is supposed to take care of everything is fine when i try to insert it via CLI

https://forums.mysql.com/read.php?39,365762,365956

will take a look

@hiredman, thank you very much, this looks like the problem!

https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc.clj#L1111 this looks like the code that makes the query in jdbc. You could perhaps see what code it is generating and compare to what you are running from the cli

@dpsutton According to the link @hiredman gave, the difference is “strict” mode — JDBC is stricter than the CLI.

whoops. my bad

I saw a mailing list post somewhere that it has changed in newer versions of mysql

As I said in #sql I was surprised that CLI even allowed it!

Ah, so CLI is strict by default now?

I think they did the reverse of course 🙂

https://bugs.mysql.com/bug.php?id=72568

Doh!

Ah, so a more recent JDBC driver might have a different behavior too?

or a more recent mysql-server

howscome nobody uses slack threads here? not complaining, juswondering. not useful?

threads are a rather low level primitive

@mobileink: it's easier to reason about stms + queues than locks/mutual exclusion

no, i mean slack threads.

is there anything like https://github.com/elixir-lang/flow for Clojure?

@mobileink: the ui is weird and often it's easier to just DM people

this channel is a firehose. i go away for a day and there are 9000 messages waiting. threads would help with that, in principle. but maybe they're not so great in practice?

chat is something you consume more or less live

sure, but dm is private, no?

why do you want to read the channel log?

@hiredman except when you can't.

all the important stuff goes into blog posts anyway

FOMO

the other reason: on the main channel everything gets mixed up. not a tragedy, but it does make it harder to follow the conversation.

is there a variant of (let[{:keys [foo bar dog cat]} ... ]) that also says assert that foo, bar, dog, cat are NOT nil ?

mm this doesn't look right

At src/cljs/scrabble/views.cljs:86:
Consider using:
  :div
instead of:
  (fn [v] [:div v])

one of the suggestions from kibit

should probably file a an issue unless I'm missing something obvious

@andrea.crotti no, pretty sure that’s a bug 🙂

Can anyone give me a clue as to how I could create a spec for a collection with frequency limits for distinct items? An example might be a vector that contains at most two 0's and one 1, where [0 0 1] would be valid, but [0 0 0 1 1] wouldn't.

Assuming that's something that can be done

since spec is a predicative system, your step #1 is to make a predicate that identifies such a collection

after you do that, clojure.spec/and is your friend

@mike706574 you could do (s/and <your-coll-spec> (fn [coll] (let [freqs (frequencies coll)] <verify-freqs>)))

Ah, that makes sense