Fork me on GitHub
#clojure
<
2016-07-25
>
dpsutton11:07:46

I'm working on a project where there are different test paths under different profiles. Is this a common paradigm? Right now cider can't run these tests in the project when you say run all tests in project, and i'm wondering if this would be a common request for people, or if tests under different profiles isn't that common of a setup

josh_tackett12:07:37

Anyone know the best posgres lib for Clojure?

cezar12:07:04

just look for a decent JDBC/SQL library. What postgres specific stuff would you need in a Clojure lib?

lmergen12:07:47

@dpsutton: this is something you can do pretty well, but yes, you'll need to write some logic that handles this

surreal.analysis13:07:01

Is there a more Clojure-y version of TimeLimiter - https://google.github.io/guava/releases/snapshot/api/docs/com/google/common/util/concurrent/TimeLimiter.html If you’re unfamiliar with the library, the concept is limiting the amount of time functions can run and giving them, essentially, a timeout value. So I’m interested in something along the lines of:

(limit my-function 5000 “default-value”)
Which I’d expect to run my-function, and if it doesn’t return in under 5 seconds, returns the string “default-value” instead.

mpenet13:07:23

seems like a promise/future with a call to deref with a timeout would do it

surreal.analysis13:07:34

Ah, thanks! I wasn’t aware of the deref w/ timeout option, that looks pretty perfect.

mpenet13:07:18

just be carefull with future, it uses an cachedthreadpool (basically an unbounded thread pool)

mpenet13:07:42

you might want to roll you own executor and go a bit lower level for this stuff

scriptor14:07:26

let's say you had a java method that took java lambdas as an argument, how would you pass it clojure functions instead?

mpenet14:07:48

I think you can just reify the Function interface

mpenet14:07:02

(in java.util.function)

mpenet14:07:16

you could write a tiny wrapper to do this

scriptor14:07:04

@mpenet: got it, looks like it just needs to implement apply, thanks!

mpenet14:07:16

I think I recall somebody wrote a lib to ease java8 interop

lumengxi15:07:36

anyone use expectationsfor testing? have this dummy test

(let [x 10]
  (expect 20 x))

(let [x 20]
  (expect 20 x))

lumengxi15:07:45

it actually passed bc it only ran the latter one

lumengxi15:07:49

wonder why is that?

seancorfield16:07:16

expect expands to a defn so having it inside a let may well confuse the symbol generation it uses for the name.

urbanslug19:07:59

Hey, how can I achieve something like (reduce and [true false true])

urbanslug19:07:20

Like trying something like a haskell fold

dg19:07:16

and is a macro, so you may need to do something like (reduce #(and %1 %2) [true false true])

dg19:07:55

Someone else may have a more clever way though

dg19:07:02

Simpler would be (every? identity [true false true])

urbanslug19:07:04

hmmm let me try that

urbanslug19:07:39

I want something like (not (contains? [false false true false] true))

urbanslug19:07:02

Basically I want to check whether a true is contained in a coll

urbanslug19:07:08

If it is return false

dg19:07:50

(some identity '(false false true))

jr19:07:57

(not (some true? coll))

stuartsierra19:07:33

contains? is only for sets and maps.

stuartsierra19:07:25

(contains? [:a :b :c :d] 2) ;=> true) because vectors are associative collections with integer indices as keys.

jamesnvc19:07:29

Hello; I’m seeing some very strange things with transit-clj decoding msgpacked data

jamesnvc19:07:58

The minimal case I can reduce it to is if there’s a string of a certain length (only happing to me when it’s around 140+ bytes long) transit-clj fails to decode msgpacked data if the string is stored like D9 <len>, but works if it’s encoded as DA 00 <len>,

jamesnvc19:07:27

msgpack can read the data though, so it seems to be something specific to transit parsing

jamesnvc19:07:43

does anyone have an idea where I should look? Or do I just need to report a bug to transit-clj?

dg19:07:18

Maybe @alexmiller can point in the right direction if he's around

Alex Miller (Clojure team)19:07:25

Best to file an issue on transit-clj

Alex Miller (Clojure team)19:07:32

Then everyone gets notified

Alex Miller (Clojure team)19:07:36

If you have a simple repro that would be ideal

jamesnvc19:07:15

alexmiller: I will do that then, working on making the smallest repro I can now

bcbradley19:07:54

I'm trying to use a transducer to do some work on some meshes. I have something like (transduce (map first) min (:collision-mesh player))

bcbradley19:07:10

the issue is that min will be asked for a value to begin with, but has no 0 arity

bcbradley19:07:04

i could provide an init value, but it would look ugly. I would need to select the highest number possible, to ensure that it is immediately replaced in the reduction

bcbradley19:07:49

another approach would be to let the init be the first (or any) value of the collision-mesh, which feels a little better but still feels kind of wonky

bcbradley19:07:05

is there a non-wonky way to do this?

Alex Miller (Clojure team)19:07:04

In the initial value case you could use Long/MAX_VALUE or double or whatever is appropriate

Alex Miller (Clojure team)19:07:51

There is a actually a monoid function in the reducers namespace that can also be used to patch in the no arg case iirc

bcbradley20:07:12

i think you might be talking about completing

bcbradley20:07:15

and it patches the 1 arity case

bcbradley20:07:03

its a shame that min doesn't have a 0 arity that defaults to the maximum value representable

Alex Miller (Clojure team)20:07:56

I'm aware of completing but I thought we were talking about 0 arity

bcbradley20:07:17

is there a different function in reducers that completes for 0 arity?

bcbradley20:07:47

that looks like a handy utilty

Alex Miller (Clojure team)20:07:58

It's for this need - adding the 0 arity to a reducing function

bcbradley20:07:05

i'd still have the ugliness of "X/MAX-VALE

bcbradley20:07:08

where X isn't known

bcbradley20:07:53

i had considered taking the first value as the init, and letting the collection that the transducer operates on be the "rest"

bcbradley20:07:09

that feels a little better, but i was just wondering if you guys have ever used transducers this way

bcbradley20:07:12

and if there is a better way

dg20:07:54

Just hardcode initial value as 100 and assume it's good enough 🙂

bcbradley20:07:05

ecks dee nice maymay

bcbradley20:07:56

so right now i'm considering (transduce (map first) min (first (:collision-mesh player)) (rest (:collision-mesh player)))

bcbradley20:07:00

i like being able to just tell what code does without thinking about what i'm looking at, and the transducer here is better than the hard-coded init type, but it isn't quite as good as i'd like

Alex Miller (Clojure team)20:07:10

Do you need transduce at all?

Alex Miller (Clojure team)20:07:27

You could just min of map first?

bcbradley20:07:31

i suppose i could do a reduction on a map

bcbradley20:07:37

but thats the definition of transduce

bcbradley20:07:52

well, i guess transduction is more general

bcbradley20:07:02

its a reduction on an arbitrary series of stuff

bcbradley20:07:42

what did you have in mind? what would i gain from doing a (reduce fn init (map first (:collision-mesh player)))

Alex Miller (Clojure team)20:07:37

(apply min (map first (:collision-mesh player))) ?

bcbradley20:07:01

that is much more in line with what i'd like to do thank you very much alex

Alex Miller (Clojure team)20:07:50

(reduce min (map first (:collision-mesh player))) ?

Alex Miller (Clojure team)20:07:08

that one’s probably more efficient

bcbradley20:07:20

i don't really know if thats the case

bcbradley20:07:18

the reason i figured on using transducers in the first place was trying to come up with something more efficient than reduce

Alex Miller (Clojure team)20:07:19

apply is going to build an intermediate sequence before applying

bcbradley20:07:32

the reduce method will involve two sequences

bcbradley20:07:57

the one you feed it and the one generated from the map

Alex Miller (Clojure team)20:07:10

oh, well apply will be one more

Alex Miller (Clojure team)20:07:26

transduce would be better, but you just need to cover the initial case

bcbradley20:07:36

i think i'll just put it in a let and do [some-point (first (:collision-mesh player)) other-points (rest (:collision-mesh player))

bcbradley20:07:43

then feed those into the transduction

Alex Miller (Clojure team)20:07:03

that first/rest can be destructured

Alex Miller (Clojure team)20:07:32

(let [[f & r] (:collision-mesh player)] (transduce (map first) min f r))

bcbradley20:07:00

is that more efficient or just more compact?

Alex Miller (Clojure team)20:07:40

more compact, should be same efficiency-wise

bcbradley20:07:21

i'd like to ask a different question, do you guys think this is an abuse of transduce?

bcbradley20:07:31

if you saw it in code you were collaborating on, would you call it hacky?

Alex Miller (Clojure team)20:07:08

for the reasons you mentioned above - you want to make a single pass through the input, map out the first of each, and then use min to decide which to keep

Alex Miller (Clojure team)20:07:24

so it’s more efficient than either the reduce or apply versions above

Alex Miller (Clojure team)20:07:36

so, I’d say: not hacky, but good use

Alex Miller (Clojure team)20:07:52

the non-0 arity of min was the only sticking point

bcbradley20:07:04

its regrettable that i have to work around the non-0 arity of min

Alex Miller (Clojure team)20:07:14

but that’s not something that can be done (as min is polymorphic across all number types)

dg20:07:37

If you wanted to you could wrap min' in a function that answers for the 0-arity case, but that's the hacky version compared to what's been suggested

bcbradley20:07:49

so it isn't possible to establish a notion of "replace me with anything-ness"

bcbradley20:07:25

the only thing about the transducer version i don't like

bcbradley20:07:36

is the fact that we arbitrarily chose the first of the sequence

bcbradley20:07:45

we could have chosen any point whatsoever

bcbradley20:07:26

imagine we chose the last as the init

Alex Miller (Clojure team)20:07:28

then the first seems equally good :)

bcbradley20:07:47

in fact, based on the way min works

bcbradley20:07:57

we don't necessarily have to choose "the rest" as the collection

bcbradley20:07:04

we could have just fed the entire collection

bcbradley20:07:36

but that is specific to min perhaps, and the fact we chose "first" or "last" is an arbitrary choice

bcbradley20:07:41

the specificity is something that irks me

Alex Miller (Clojure team)20:07:41

that is the apply version

bcbradley20:07:40

i think the point of transducers, besides efficiency, is decomplecting process from data

bcbradley20:07:07

i feel its a good policy to make your algorithms no more specific than they have to be

Alex Miller (Clojure team)20:07:10

how many of these things things are there?

bcbradley20:07:17

they are all over

bcbradley20:07:23

i have just 8 in this one function

Alex Miller (Clojure team)20:07:42

and have you actually considered reducers?

bcbradley20:07:57

if i use reducers i will double my memory footprint (at least)

Alex Miller (Clojure team)20:07:12

b/c they can fold in parallel, giving you additional gains if you have a lot of values

bcbradley20:07:22

ah you mean those reducers

Alex Miller (Clojure team)20:07:23

where “lot” is probably at least 1k

bcbradley20:07:46

yeah i'm considering that, but given the fact that at the "highest level" most of my transducers are composed entirely of maps

bcbradley20:07:51

i could just use a pipeline

dg20:07:10

It seems like you may need to make a decision between the most performant solution and the most philosophically correct

bcbradley20:07:36

in that case (first ...) (last ...)

bcbradley20:07:43

or the destructered version would be best

Alex Miller (Clojure team)20:07:50

I like the transduce personally :)

bcbradley20:07:23

i'll just make sure to communicate the essence in the name

bcbradley20:07:34

some-player-point, and other-player-points for instance

em20:07:13

Silly question, but isn't there a name/concept in clojure for an item that's not a collection? Similar to Lisp atoms?

dg20:07:19

values, or symbols, depending on the context?

em20:07:11

Well, independent of context, I think. The set includes numbers, strings(?), symbols, keywords, characters, (and what else?)

em20:07:42

Oh, sorry. I now see what you meant.

dg20:07:20

It's fair to just use the lisp terminology, but of course most people will think clojure atoms are something else

em20:07:58

Symbols can't be it. For example (symbol? :keyword) is false.

em20:07:26

values is more like it? But collections are not values?

richiardiandrea20:07:05

mmm I don't think there is something analogous to atom? you can maybe fabricate your own with symbol? keyword?...

dg20:07:10

Things that are not collections are typically called either symbols (+, defn, and) or values/literals (7, "foo", :keyword). I'm not sure there's an official universal term for everything, and if there is it's not widely used.

dg20:07:07

But there would be no reason for a predicate to exist to test it, because any possible parameter would return true.

richiardiandrea20:07:46

@em also in Clojure these checks are based a lot on the underlying Java Interface (not saying this is good, but it is what it is) -> http://clojuredocs.org/clojure.core/coll_q

em20:07:54

thanks. I'll roll my own atom?. (I'll call it something else to avoid confusion)

bcbradley22:07:05

hrmph. apparently (reduce min (map first coll)) isn't the same as (transduce (map first) min (first coll) (rest coll))

bcbradley22:07:31

for instance, if i use [[0 0] [0 1] [1 0] [1 1]] as the coll, i'll get a class cast exception

bcbradley22:07:07

it should be (transduce (map first) min (first (first coll)) (rest coll))

bcbradley22:07:37

the extra first is required because the "init" isn't passed through the transducer (the map to first in this case)

dg22:07:49

btw, (ffirst x) is (first (first x))

bcbradley22:07:12

thanks. i feel like i'll have to approach the problem differently though

bcbradley22:07:29

(map first) is a simple transducer

bcbradley22:07:43

i have more complex ones, and I don't want to have to push a single value through it all the time

scriptor22:07:08

@bcbradley: that makes sense, the transducer will only be called on a collection, and init isn't always going to be the collection

scriptor22:07:47

actually, never mind, I think I see your point

bcbradley22:07:22

here is a way to do what I want (apply min (map fun coll))

bcbradley22:07:31

another way is (reduce min (map fun coll))

bcbradley22:07:41

in order to improve performance, you can make one pass instead of two

bcbradley22:07:57

(transduce fun min coll) comes to mind

bcbradley22:07:13

however, min has no 0 arity, so an initial value must be provided

bcbradley22:07:22

(transduce fun min init coll)

bcbradley22:07:40

it turns out that init isn't pushed through fun

bcbradley22:07:52

so imagine fun is (map first)

bcbradley22:07:07

and coll is a mesh-- [[0 0] [0 1] [1 0] [1 1]]

bcbradley22:07:53

you'd have to say (transduce (map first) min 0 [[0 0] [0 1] [1 0] [1 1]])

bcbradley22:07:00

but how did you get that "zero"

bcbradley22:07:15

you got it by choosing "any" point in the mesh

bcbradley22:07:19

could be the first one for instance,

bcbradley22:07:26

and then using (map first) on it

bcbradley22:07:43

in other words, you really want the init to be pushed through the fun, but transduce isn't doing that.

bcbradley22:07:12

i'd like to say (transduce (map first) min (first coll) (rest coll)) for instance

bcbradley22:07:57

so how do you express performing a transducer on a single value?

bcbradley22:07:06

another transduction.

ghadi22:07:51

transducers are functions of reducing functions, not individual values. this has less to do with transduce and more to do that you can't give the reducing function (min) a sensible init value

ghadi22:07:21

have you tried

(apply min-key first coll)

bcbradley22:07:54

(apply min (map fun coll)) is what i've tried with apply

ghadi22:07:11

min-key and max-key are in core

ghadi22:07:21

if you really want to use transduce with this case:

(transduce (map first) min Long/MIN_VALUE coll)

ghadi22:07:52

that way there is something for the first element of coll to be compared to, and it will always win

bcbradley22:07:06

that has been suggested

bcbradley22:07:22

i don't like it because i may not necessarily know the type of the number in the mesh

dg22:07:04

It could be larger than 64 bits?

bcbradley22:07:37

i suppose i could create my own transduce that wraps the standard transduce, converts the init into a single element collection, performs the transduction on it to generate an single element sequence that contains the init, and use first to pass it to the standard transduce

bcbradley22:07:43

seems a bit roundabout

ghadi22:07:57

clone the first element of the collection as init?

ghadi22:07:52

(btw, why might you not necessarily know the type of the number?)

bcbradley22:07:08

because i use fractions liberally in the code

bcbradley22:07:19

i mix with floats, integers

ghadi22:07:24

gotcha. polymorphic math costs will dominate anything you're worried about wrt transduce

ghadi22:07:35

personally i wouldn't worry about it

ghadi22:07:45

are you looking for perf as #1 priority?

bcbradley22:07:32

i'm using this specific pattern 8 times alone in just this one function. I'd say probably about 12% of all the lines of code i'm writing is this exact pattern

bcbradley22:07:45

its important that it doesn't consume more memory than required

ghadi22:07:01

then I'd enable the boxed math warnings and loop/recur

bcbradley22:07:01

chains of filters, maps, and reductions are out

ghadi22:07:39

actually nm, if you're doing polymorphic math you'll always box

dg22:07:59

Is it not feasible to standardize on one type of number?

ghadi23:07:04

E_EXHAUSTED_LUCK

bcbradley23:07:25

i know it seems silly to have proper fractions in code that processes meshes

bcbradley23:07:32

in this case its not graphics code, its collisions code

bcbradley23:07:58

most of the collisions code i've seen written doesn't aim for strict reproducability

bcbradley23:07:18

the tiny errors that occur in floating point ops generally doesn't matter in most applications because they don't accumulate

bcbradley23:07:26

in video games specifically, they do

bcbradley23:07:50

there is a concept in math called chaos, it means that tiny changes in the initial conditions result in enormous changes in final conditions

bcbradley23:07:12

and that it isn't possible to ascertain the difference in the final based on the difference in the initial

bcbradley23:07:40

i'm experimenting with exact math to make it possible to have an exact simulation

bcbradley23:07:59

which means that it would be reproducible on any computer

bcbradley23:07:18

the "gotcha" of floating point ops isn't that they error, its that the error between two computers may be different

bcbradley23:07:28

so the simulation would diverge

bcbradley23:07:44

authoritative servers aren't just for competitive integrity

bcbradley23:07:03

they serve as an authority on the state of the game, so that diverging simulations know what the "true" state ought to be

bcbradley23:07:26

if you've ever played a game and had a brief glitch where your avatar sort of tunneled through a wall , moved all wonky and then finally started acting normal

bcbradley23:07:37

thats what happens when the client simulation and server simulation get vastly out of sync

bcbradley23:07:45

and in that case, the server's simulation dominates

bcbradley23:07:16

i want to avoid that so that I can have a wide peer to peer network infrastructure with no central server

bcbradley23:07:39

i figured i could avoid the floating point chaos at first by asking for a majority consensus on the state

bcbradley23:07:50

but that won't work because they diverge uniquely

bcbradley23:07:03

so i am exerimenting with exact math

dg23:07:09

So if you use Long/MAX_VALUE, you can freely mix rationals and floats and the result will be correct.

bcbradley23:07:01

i hate that i have to do it that way, but you are probably right