Fork me on GitHub
#clojure
<
2016-08-28
>
didibus00:08:34

Could allow the person using the macro to choose expanssion order, maybe if you prefixed the macro name with a ^ it should expand inside out

erlis00:08:03

@didibus that will be a book I'll buy for sure DDD in clojure

hiredman00:08:13

I think you would more likely see the introduction of a hygenic macro system (which actually restricts what macros can do) over something that expands it

hiredman00:08:23

(not that I think that is super likely either)

didibus00:08:27

@erlis Ha, not sure if I'm up to the challenge quite yet

hiredman00:08:38

gfredricks: have you looked at using tools.analyzer similar to what core.async does?

gfredericks00:08:13

@hiredman: yeah, I assume that would work fine if you don't mind having that dependency

didibus00:08:19

I thought Clojue's guaranteed unique binding names gave you all the benefits of hygene

hiredman00:08:13

clojure's macro system is unhygenic, but the syntax quoting stuff is like an assist to help make macros hygenic

hiredman00:08:43

but you can write macros without using syntax quote, or using any kind of quoting at all

hiredman00:08:06

(but don't do that)

didibus00:08:08

Right, I see what you mean. The guarantees aren't there in Clojure, you just need to hope the designer of the macro made it hygenic

didibus00:08:35

Alright, thanks both of you. I learned a few things today

hiredman00:08:37

@gfredericks: two paths, one is to get that functionality or something like it in to the compiler, so you don't need a dependency, the other is to reduce the drag of the dependency, so you don't mind having it. the main way to reduce the drag of dependency seems like it would be some kind of isolating dependency module system, because the big down side is if everyone depends on and uses tools.analyzer, but everyone is using a different version, that is going to be painful

gfredericks00:08:19

@hiredman bronsa said he could split it up into separate libs, but also alex thinks my macro idea is dumb, so maybe it's a wash

hiredman00:08:40

I think the map syntax makes sense

gfredericks00:08:52

well I guess that's unanimous then

gfredericks00:08:13

except I haven't checked if @alexmiller hates it

hiredman00:08:06

I think the map syntax is tied to an intuitive understanding of a reader returning datastructures, which people can struggle with, but on the other hand, other lisps have different forms of let for parallel and sequential binding, so having the map has just a syntactic marker of using a different kind of let without worrying about why it makes sense because it is a map out of the reader seems ok

hiredman00:08:12

let* would be sort of an obvious candidate for the name of an alternative form of let, except that the relationship of test.check's let to let* would be more like common lisp's than clojure's; but maybe that would be fine, because who cares about clojure's let*?

gfredericks00:08:50

Alex said he didn't like a variant name

gfredericks00:08:04

I like :parallel because it lets you mix things

hiredman00:08:58

Hard to say, I would even consider just leaving it, which is easy for me to say since I haven't felt the pain.

hiredman00:08:26

Have you looked at fixing gen/bind to recover the lost shrinking?

gfredericks01:08:16

that's super hard

gfredericks01:08:52

if you can come up with a good general solution to that problem I will give you twenty seven genuine north american dollars

hiredman01:08:21

Tempting, I haven't been paid to write clojure for 9 months now

dpsutton01:08:51

do you miss it?

yonatanel12:08:58

What are people using today as an event store?

lmergen13:08:05

totally depends upon the way it is interacted with

lmergen13:08:32

which also is the mantra of many people nowadays

lmergen13:08:49

most default to something sane like postgresql, and build it out as the requirements change

lmergen13:08:17

for us, that means a hybrid of postgresql, redshift, and s3

tiensonqin13:08:00

Hi guys, happy to announce Lym is on apple store now! It's for learning different cultures, languages. Completely written by Clojure and Clojurescript, thank you so much! http://lymchat.com https://itunes.apple.com/us/app/lym/id1134985541?ls=1&amp;mt=8

yonatanel14:08:11

@lmergen: Do you serialize clojure data in one column? How did you implement the order of messages? What about idempotent message handling and rerunning some of them? Any integration with Kafka and the like?

lmergen14:08:39

right, those are a lot of different things 🙂

lmergen14:08:15

yes, we do those of those kind of things. usually, ordering is something that should be used by the message broken (such as kafka), and idempotence is something that you need to take into account in your application logic.

lmergen14:08:11

i am not a fan of serialization of cojure data into one column, and if i need to do so, i usually use json

yonatanel14:08:42

@lmergen: so you modelled an sql schema according to the various messages?

lmergen14:08:35

it’s a balance. nowadays, schema’s are non-strict, but you do want your data store to “understand” what is stored, so you can e.g. query on it

lmergen14:08:28

strict schemas are suboptimal in today’s rapid changing production environments

lmergen14:08:18

other than aleph, are there any good async clojure http client libraries ? i find aleph’s http client to be very, very buggy

lvh15:08:35

@lmergen: Buggy in what sense? I’ve had no issues.

lmergen15:08:47

well, i’ve run into a few issues so far

lmergen15:08:53

multipart post uploads are broken

lvh15:08:56

We sometimes use http-kit, but only because server-side sente anyway

lmergen15:08:08

chunked transfer responses cause it to hang indefinitely

lmergen15:08:23

and more issues like that

lmergen15:08:30

doesn’t exactly bring me a lot of confidence

lmergen15:08:08

but in other ways it’s a really nice library, so i’m trying to see what the alternatives are

lvh15:08:43

Are those being tracked in the issue tracker somewhere? The issue you mentioned appears to be submitted, but only 16 minutes ago 🙂

lvh15:08:00

Having dealt with other libraries I’ve fairly happily moved everything to aleph, so I’d much rather go bug fixing there

lmergen15:08:10

yes, that’s me 🙂

lmergen15:08:19

and the others are also being tracked, and PR’d by me

lmergen15:08:59

but it’s a lot of work fixing these issues, and i didn’t expect this from such a well-recommended library

lvh15:08:24

I suppose for most people the consequences of worse asynchrony management (i.e. not having manifold) is more serious than these issues I guess — just never had those problems in particular 🙂

lvh15:08:33

(Also, netty)

lmergen15:08:53

i agree, it’s a very good library, it just has some rough edges

lvh15:08:59

e.g. we care about things like fairly extensive TLS management where other libraries fall short from the just-go-talk-to-netty part

lvh15:08:15

not trying to convince you, just making sure we have those issues tracked so at least someone can go fix them maybe 🙂

lvh15:08:03

I wonder if Zach would consider extra maintainers; a big part of the problem seems to be the bus number and lack of time on his part 🙂

lmergen15:08:05

i think i might want to message the maintainer directly about this

lmergen15:08:13

yes, exactly

lmergen15:08:33

i think that would be a good thing, it takes him weeks/months to respond to some issues, if at all

lvh15:08:54

As much as I use and like aleph and have libraries around manifold (i.e. banach) I’ve just become the maintainer for cloverage so maybe I should say “no” more often

lmergen15:08:19

now where’s the fun in that 🙂

lvh15:08:05

I feel a big part of the problem is the fact that there’s so many, but it’s sufficiently treated like commons that its outcome is bound to be tragic

lvh15:08:16

e.g. your middleware PR

lvh15:08:28

that really shouldn’t be so specific (same goes for ring middleware)

lvh15:08:36

it gets tricky when you want asynchrony I guess 🙂

lmergen15:08:02

you mean, there should be a better abstraction ?

lmergen15:08:47

aleph’s http client middleware is pretty much copy/pasted from clj-http, but aleph uses 3 different types of middleware, which are not compatible

lmergen15:08:56

but they’re all the same “type” of middleware

lmergen15:08:25

it’s a bit of a mess, but i didn’t want to do a complete overhaul

lvh15:08:39

Yes; I think there should be a better abstraction

lvh15:08:45

right; not a criticism of your PR

lvh15:08:10

also not really sure how to do any better without some crazy macros that rewrite the fn a-la go / <!

lvh15:08:29

since ring middleware can act both on the request and the response that way

lmergen15:08:49

yeah, that’s exactly’s aleph’s problem

lmergen15:08:59

the middleware either has to act on the request or on the response

lmergen15:08:05

and needs to be invoked on the correct place 🙂

lvh15:08:06

also not super sure that I want call-cc in my programs

lvh15:08:28

I think the biggest issue is person-power — we could solve e.g. the ring middleware problem a lot better upstream by manually splitting up the pre- and post- work, separating out juuuust the let vs let-flow bit

lmergen15:08:45

yep, agreed

lmergen15:08:06

ah, looks like i have contact with zach! probably a coincidence, or is he lurking...

amacdougall15:08:13

Anyone here familiar with the inner workings of test.check? I have some clojure.spec definitions working, but test.check is having a very difficult time testing them. Specifically, I'm playing with some maze generation and pathfinding algorithms. A core construct is a "grid", a two-dimensional vector of cells. Here's a spec and a function:

(spec/def ::x integer?)
(spec/def ::y integer?)
(spec/def ::direction #{::n ::ne ::e ::se ::s ::sw ::w ::nw})
(spec/def ::exits (spec/coll-of ::direction :kind set?))

(spec/def ::cell (spec/keys :req [::x ::y ::exits]))
(spec/def ::row (spec/coll-of ::cell :kind vector?))
(spec/def ::grid (spec/coll-of ::row :kind vector?))

(defn grid-contains?
  "True if the supplied x and y coordinates fall within the grid boundaries."
  [grid x y]
  (and (<= 0 y)
       (<= 0 x)
       (< y (count grid))
       (< x (count (grid y)))))
(spec/fdef grid-contains?
  :args (spec/cat :grid ::grid :x ::x :y ::y)
  :ret boolean?)
I can verify, using spec/valid? and spec/exercise, that the ::grid spec is correct, and that clojure.spec knows how to generate examples of it. But when I run clojure.spec.test/check on the grid-contains? function, Java maxes all cores and then runs out of memory.
#'user/grid-contains?
user=> (stest/check `grid-contains? {:num-tests 1})
"grid-contains? grid 0 0"

OutOfMemoryError GC overhead limit exceeded  clojure.lang.PersistentVector$TransientVector.persistent (PersistentVector.java:568)
Am I doing something wrong here?

amacdougall16:08:00

My first guess is that test.check is generating vectors so large that they crash Java, but when I altered the specs such that the grid vectors could not be larger than 1000 items, I got the same results.

lvh16:08:11

@lmergen FYI Zach is a really nice and very reasonable guy; so I think if you’re interested in doing some work he’ll be receptive to your suggestions 🙂 I can volunteer some time to weed through the issue tracker, but I definitely can’t take on another maintenance project

lmergen16:08:22

yeah, i figured i would focus in first getting patches accepted the regular way

lvh16:08:05

Software is hell 🙂

lmergen16:08:37

apparently you haven’t met humans yet, they are even worse

meta-meta17:08:15

what's an elegant way to avoid evaluating or calling things like reverse or last on an infinite lazy seq?

lmergen19:08:43

@meta-meta: interesting question, but I can't see an obvious solution either. what problem are you trying to solve ? I assume you're exposing some logic as a lazy seq ?

brandoff20:08:45

Hi All -- Of the folks here who have used Onyx, has anyone written a k-means (or ideally, a k-means||) implementation? Was considering writing my own but don't want to reinvent the wheel 🙂

gfredericks20:08:11

I am familiar with the inner workings of test.check

gfredericks20:08:12

I can't tell what your problem is based on your description though

amacdougall20:08:04

Oh hey, author of test.chuck! Hi! I'm afraid I can't give a lot more context, because, well, I don't know where to start. If you feed my above code into a REPL, you may get the same result, though. Just be ready to kill the java process. And if you don't, well, the mystery deepens.

gfredericks20:08:39

Especially OOM on a single test suggests it has nothing to do with sizing subtleties

amacdougall20:08:41

I'm dipping my toe in the waters of generative testing for the first time, so I don't really know what I should expect.

gfredericks20:08:15

I don't have a repl on hand

gfredericks20:08:25

I can try it when I get home

gfredericks20:08:34

You're on alpha11?

amacdougall20:08:10

alpha10, but let me see what happens if I bump it.

gfredericks20:08:53

You could also debug by creating the generators for each of the arg types and sampling them to see if anything is weird

amacdougall20:08:41

I'll try that out. spec/exercise generated reasonable output for the largest composite type (the grid)—is that the same thing?

amacdougall20:08:47

...interesting. I put print statements so I could see what values the system is generating for the arguments, and it seems to be doing reasonable stuff now. Maybe something just got fixed in the latest alpha.

amacdougall20:08:59

"grid-contains? 12 x 10 grid; -611476993305, -240027313508" ...reasonable enough for me!

amacdougall21:08:59

Not sure how to restrict the number of tests, though. The code seems to say that clojure.spec.test/check takes an options hash, of which one key can be :clojure.spec.test/opts, whose values can be a hash which is passed through to test.check... but {:num-tests 1} and {:clojure.spec.test/opts {:num-tests 1}} both have my system running... well, who knows how many tests? It's been going for a while now. (Update: fixed it. I needed (stest/check 'grid-contains? {:clojure.spec.test.check/opts {:num-tests 1}}).) I wonder if my code is having a hard time testing the boundary status of the coordinates -72927184749070415, -21020661, or if that's just a totally normal number to have trouble with.

brandoff21:08:31

@amacdougall you're looking at a matrix of (x,y) pairs?

amacdougall21:08:02

A two-dimensional vector of maps, if that makes a difference.

amacdougall21:08:05

That should mean that (count coll) is O(1), so I can't see why the grid-contains check would run slow even when given an extremely large input value.

amacdougall21:08:50

But then test.check is calling the function with (according to my print statement) "grid-contains? 7 x 6 grid; -490, 7232120168103"...

amacdougall21:08:07

And it's taking many seconds for my computer to come up with the answer.

amacdougall21:08:19

Whoa, and it just ran out of memory again! What gives?

amacdougall21:08:40

Anyway, at the REPL, it returns instantly, as I would expect.

mazes.grid=> (grid-contains? (create-grid 8 8) -490 7232120168103)
false

amacdougall21:08:04

So somewhere at the interface of test.check and clojure.spec.test, something is going very wrong.

amacdougall21:08:50

This stuff is in alpha, after all; I'm not going to sweat it for now.

brandoff21:08:43

@amacdougall I don't know how you are generating matrices, but I'm wondering if the memory issues are there. Are you using an optimized implementation?

amacdougall21:08:51

If the issue were in the implementation, I would expect to see it every time I call the function with the same values, regardless of whether test.check is running it, right?

amacdougall21:08:37

Got to head out; don't worry too much about me! I'm sidestepping the test.check thing and just doing stuff the old-fashioned way.

brandoff21:08:29

Not sure, but may be worth playing around with something like https://github.com/mikera/vectorz-clj.

amacdougall21:08:08

I'll take a look! I have a feeling it won't do the trick for me, though. My grid is going to eventually be a game board, so instead of doing math on it, I'm going to be carving rooms out of it, placing entities on it, etc.

tom23:08:54

I'm looking to do some batch inserts. YesQL seems to have slowed or stopped development. HugSQL seems to be in? If so, has anyone done a migration to HugSQL from Yesql?