Fork me on GitHub
#clojure
<
2015-06-30
>
wilkerlucio03:06:49

hello guys, I'm looking up for a Clojure function that does something like this: (FN [[1 2 3] [:a 😛 :c]]) and returns -> ([1 :a] [1 :b] [1 :c] [2 :a] [2 :b] [2 :c] [3 :a] [3 :b] [3 :c])

wilkerlucio03:06:06

it's like having a (for), but with a dynamic number of iterables in the middle

wilkerlucio03:06:11

there a function in core to do that?

roberto03:06:51

; CARTESIAN PRODUCT
; all the ways to take one item from each passed-in sequence
=> (combo/cartesian-product [1 2] [3 4])
((1 3) (1 4) (2 3) (2 4))

wilkerlucio03:06:49

yes it is, although I was hoping to find that on core (since it's kind of provided with for), also I'm using ClojureScript, so I may end up just copying part of that source

wilkerlucio03:06:23

just fyi: just copied the cartesian-product and it worked jsut fine, it doesn't depends on any other function there simple_smile

meow03:06:32

@wilkerlucio: sometimes juxt, which is part of the standard library, comes in handy for similar situations or if you can change your code to make it work https://clojuredocs.org/clojure.core/juxt

wilkerlucio03:06:27

@meow: hi, I got curious now, how would you use juxt to generate a cartesian product of a list of lists?

meow03:06:33

@wilkerlucio: I'm half asleep so I'm just going to wave a white flag and surrender because I remember needing a Cartesian product a couple of weeks ago and instead my situation lent itself to some other approach that I thought used juxt but I just looked through my sandbox of code and can't find anything so I'm not sure what I'm half-remembering. Anyhow, juxt is just a cool function but cartesian-product is obviously a good match when you need something like, you know, a cartesian product... 😉

wilkerlucio03:06:29

hehe, yeah. I agree that juxt is great, I always try to find uses for it as well, but I end up using it mostly when sorting/grouping things

wilkerlucio03:06:06

on my current case I really need a cartesian product

meow04:06:34

the first time I saw juxt was somebody (can't remember who) that used it to calc the neighbors of a cell in a grid for conway's game of life:

(defn neighbors [[x y]]
  (map vector ((juxt inc      inc identity dec dec      dec identity inc) x)
              ((juxt identity inc inc      inc identity dec dec      dec) y)))
I thought that was pretty clever and so juxt stuck with me.

meow04:06:16

And now I'm going to go dream about core.async channels and what's the ideal set of them for capturing all the events that take place in a cljs app...

danielcompton05:06:39

I’m a bit confused about defrecord methods. [This](https://github.com/apa512/clj-rethinkdb/blob/1792b7ca90fe0b5396329c36cfa1c53429b2fa6c/src/rethinkdb/core.clj#L37-L41) defrecord has a close method, which calls core/close. I assumed that calling (.close connection) would be equivalent to (core/close connection) but it isn’t. What am I calling when I do .close?

danielcompton05:06:19

.close returns nil, while core/close returns :closed

jonas06:06:41

danielcompton: I guess you’re calling the function above

danielcompton06:06:21

@jonas: sorry I don’t understand?

danielcompton06:06:05

ah yes, but isn’t .close just calling this function?

jonas06:06:32

hmm.. true

danielcompton06:06:46

I think this might be related to defrecords creating functions for their implemented method names

danielcompton06:06:40

Is it possible that because Closable is a void method, that it would return nil even if something inside it returned a value?

danielcompton06:06:51

I’m hoping the answer is yes or I need to question everything I know

agile_geek06:06:32

@danielcompton: I am guessing but probably something to do with method dispatch. It’s calling the Closeable close method rather than the core/close function? (function overloading?)

danielcompton06:06:03

yeah, that’s pretty much it

andrea.crotti07:06:03

any idea why the new lein doesn't run tests for the code-maat project guys?

malcolmsparks08:06:16

@donaldball @noisesmith thanks for your insights last night on my form keyword params question

malcolmsparks08:06:23

I also discovered that I've been using clojure.walk/keywordize-keys

malcolmsparks08:06:34

I think the main reason keywords are used (rather than strings as map keys) is for ease of extraction (they satisfy IFn) whereas strings don't, I wish sometimes they would

ska09:06:08

Hmm... Given two vecs (def x [1 5 4 2 1 4]) (def choices [9 7 2 3]) I want to return the first item from choices that I find in x (`found`) and the remainder of x with found removed (only one removed if multiple in x). x will always contain at least one of choices, order of x never matters, all vecs are small (less than 20 items), speed matters a bit. So far I've only come up with loop-based solutions. Any neat ideas?

slipset09:06:06

There is a trick where you let choices be a set, probably someting a la (first (filter #{ 9 7 2 3} x)

goodwind8909:06:01

@ska: if the ordering of x does not matter, it may be more efficient to store as a frequency count map instead (something you get from (def y (frequencies x)). Then you can do (some #(if (pos? (get y % 0) (update-in y [%] dec))) choices)

cfleming09:06:03

@tcrayford: Actually I had another question for you after your talk - do you use definline at all?

cfleming09:06:25

@tcrayford: I have some namespaces which pretty much just wrap Java APIs, so most of the fns simply wrap one interop call. It seems like definline should work well there since you save a stack frame and I suspect the bytecode for an interop call is smaller.

zoldar09:06:34

I have problems with a particular setup of core.async channels. Here's a minimum example showing the intent and exposing the problem: https://www.refheap.com/104584 My intent is being able to send a message through one channel and wait until arbitrary number of messages come back from another channel - for this I'm using a blocking version of take operation (<!! - here it's actually alts!! to handle the timeout). What I observe is only the first message comes out of 'result' channel - and only on first try after running the top level go-loop. After that what I get are only timeouts. I suspect I'm doing something terribly wrong.

stijn10:06:07

I think the problem is that you immediately untap the mult after starting the go-loop

stijn10:06:09

@zoldar and you can use alts! inside a go block

stijn10:06:03

(instead of alts!!)

zoldar10:06:02

@stijn: oops I've botched my example, just a minute

zoldar10:06:02

it should be loop instead of go-loop so that it will be blocking and function will terminate only after receiving messages or timeout

zoldar10:06:14

... and this way it seems to work. heh, nevermind then, the problem has to be somewhere else

tcrayford10:06:33

@cfleming: nope. Too worried about the inliner budget and it exploding

tcrayford10:06:53

except that's prolly fewer bytecodes

cfleming10:06:02

Yeah, that was what I thought

tcrayford10:06:08

I don't have any one line functions in my hot code path 😉

cfleming10:06:20

Since a fn invoke does the getRawRoot() dance

tcrayford10:06:40

I guess I've manually inlined the only java api I use there (`java.util.HashMap`)

zoldar10:06:06

ok, here's the example closer to the problem I'm experiencing along with the output of example repl session: https://www.refheap.com/104589

tcrayford10:06:09

but that technique makes sense to me

cfleming10:06:22

Ok - the APIs I wrap are pretty huge. I wrap them partly for prettiness, but mostly to hide the type hints.

tcrayford10:06:26

guess the real thing is to try it out via some benchmarks 😉

cfleming10:06:47

Type hinting definline is a PITA, but thinking about it it’s probably worth it.

zoldar10:06:09

I'm only accepting some messages from the source channel ignoring others. First 2-3 invocations work alright - after that, timeouts happen

cfleming10:06:16

Yeah, I’m planning to try it. I actually AOT everything so counting should be simple using ASM.

tcrayford10:06:34

ah, makes sense

cfleming10:06:08

Limit is 350 bytecodes, right?

tcrayford10:06:21

on current oracle jvms

tcrayford10:06:54

then just look for FreqInlineSize

tcrayford10:06:10

though it's actually a lot more complicated than that

cfleming10:06:18

Ok, is that bytes or bytecodes? IIRC most but not all are 1 byte, right?

tcrayford10:06:32

FreqInlineSize is only for "hot" codepaths, there are other flags for less hot paths

tcrayford10:06:39

bytecodes I think

cfleming10:06:52

Of course there are, it would be too simple otherwise.

tcrayford10:06:07

see also InlineFrequencyCount, MaxInlineSize, MaxRecursiveInlineLevel, InlineSmallCode, InlineFrequencyRatio, InlineThrowCount, InlineThrowMaxSize

tcrayford10:06:35

know a jvm 😞

cfleming10:06:48

I think I’ll start with 325 and go from there simple_smile

cfleming10:06:06

I have to figure out how to benchmark the parts I care about too.

cfleming10:06:22

This is sounding like a month-long project.

tcrayford10:06:31

yeah haha 😞

tcrayford10:06:58

as always, feel free to reach out to me. <mailto:[email protected]|[email protected]> if you have anything sensitive or whatever

cfleming10:06:26

Sure, thanks - I’ll let you know how I get on when I get to this.

cfleming10:06:54

I actually have to target JDK6, but I can use a later one for running benchmarks once I get that figured out.

cfleming10:06:16

So I should be able to use Flight Recorder.

tcrayford10:06:47

perf is trickier for a product like cursive compared to Yeller - Yeller has like 3 code paths where it needs to be really really fast etc, but cursive needs to be fast everywhere

cfleming10:06:13

Yeah, but there are a couple of critical parts - indexing and symbol resolution.

cfleming10:06:29

If I can get some decent wins on those, everything else will be faster.

cfleming10:06:57

The main problem is that the infrastructure is huge, opaque and very complex, especially the indexing stuff.

tcrayford10:06:04

yeah 😞

cfleming10:06:06

I have no idea how to try to isolate that.

tcrayford10:06:50

yeah. Perf doesn't compose either, so isolation is prolly not a good idea 😞

cfleming10:06:45

Right. I think JetBrains tend to go for quite coarse grained perf work (i.e. the IDE usually takes from 3-4 seconds to start up), but they must have to optimise some hot(ter) paths too. I’ll ask them.

thomasdeutsch11:06:55

Hi everyone. Is there a nice way to return only the first value from a list-comprehension? I do not want to (first) after i have the results, i would like to break the execution after :when returns the first element.

tcrayford11:06:42

@thomasdeutsch: not that I know of. But laziness should save you here!

pesterhazy11:06:54

@thomasdeutsch: probably won't work, as clojure chunks its sequences, so it'll probably calculate the 32 first values

thomasdeutsch11:06:18

Thanks for the info, i will look into it.

pesterhazy11:06:39

@thomasdeutsch: are you concerend about performance, side effects, or something else?

pesterhazy11:06:08

as far as I know there's currently no easy way to prevent eager pre-loading of elements of a lazy seq

pesterhazy11:06:58

it would be very useful actually, when the computation is expensive or has other side effects

Alex Miller (Clojure team)11:06:23

why does it need to be a list-comprehension?

Alex Miller (Clojure team)11:06:46

reduce with reduced, or a loop-recur, or take transducer all offer early exit opportunities

pesterhazy12:06:27

@alexmiller: true, but there are scenarios where you don't want to change the code

pesterhazy12:06:00

e.g. if you're testing a function built on for or map, and you only want to evaluate the first element

Alex Miller (Clojure team)12:06:47

map chunks, but I don't think for does?

Alex Miller (Clojure team)12:06:42

nah, I'm wrong, it does

annapawlicka12:06:03

I’ve been looking at the source code of the Pepa app that 2 Moritzes presented at EuroClojure and noticed they made config a component https://github.com/bevuta/pepa/blob/master/src/pepa/config.clj Any of you know what are the benefits of doing that? The most common way is to pass different parts of a map to initialise components and not make the config itself a component.

pesterhazy12:06:57

I always assumed that map and for are pretty much equivalent for simple cases

zoldar12:06:47

filtering the channel fixed the problem: https://www.refheap.com/104594 . However I'm still not sure why the ignored messages clog up the channel pipeline - code docs for mult state that messages sent when there's no tap are discarded

pyr12:06:41

@annapawlicka: it's unclear to me as well. I'd say the equivalent holds true for componetizing logging.

pesterhazy12:06:26

I'd say, use components when you must, not when you can

annapawlicka12:06:55

it does seem like an overkill, but maybe i’m just not seeing something

tcrayford12:06:28

it does mean that config is like, trivially inspectable at runtime, which is nice

tcrayford12:06:52

(yeller has an admin endpoint available on each server that dumps a json blob of all the components)

tcrayford12:06:15

(then I just use one of the chrome json prettifier plugins to view it in more depth)

Alex Miller (Clojure team)12:06:24

pesterhazy: yeah, the primary loop for a single-source is about the same (although for does a lot more work in setup to deal with all the special clauses). regarding the original question, the takeaway is really that if you want to control the rate of evaluation of a source, lazy seqs and seq ops do not give you fine-grained control. there are better options (esp now with transducers) if that is what you need.

ordnungswidrig12:06:28

@annapawlicka: I don’t recomment making config a component because this prevents the composition of the system to depend on configuration. In other words you cannot create components selectively.

ordnungswidrig12:06:01

@annapawlicka: additionally configuration does not carry state and thus is not required to be a component.

annapawlicka12:06:18

yeah, that was my reasoning as well

pyr12:06:07

@tcrayford: it's an interesting property, but not one you need components for

tcrayford12:06:54

@pyr: for sure. I don't actually use component at all (yeller predates that library), just saying it makes that thing trivial.

ordnungswidrig12:06:58

I try to go with something like lein-env and a global set of immutable values. They’re used only at system construction time and passed into the component where necessary

ska12:06:17

@goodwind89: Neat idea with the frequencies. Will consider that.

ordnungswidrig12:06:31

@tcrayford: nothing prevents you from that. Injecting a „configuration component“ into your components is a different story.

ordnungswidrig12:06:33

@annapawlicka: another point, when you look at https://github.com/bevuta/pepa/blob/master/src/pepa/zeroconf.clj#L32 you see that this component has some hidden coupling to the two config keys.

ordnungswidrig12:06:40

I like my stuff explicit.

andrea.crotti12:06:52

if I open 2/3 repls on different projects with 8GB of Ram I already start to get out of memory errrors

andrea.crotti12:06:03

each of them take > 700 MB is that normal?

jwm12:06:17

openjdk or oracle

pesterhazy12:06:55

@andrea.crotti: you can assign each jvm less ram of course

Alex Miller (Clojure team)12:06:17

annapawlicka: if you haven't looked at immuconf, I like it a lot and have used a variant of it on some internal projects https://github.com/levand/immuconf

annapawlicka12:06:45

ohhh, haven’t seen that, thx!

Alex Miller (Clojure team)12:06:58

it's not as well known as it should be imho

andrea.crotti12:06:18

@pesterhazy: but that's the hard limit for that repl I guess right?

andrea.crotti12:06:32

so if the project said it needed X and I give less it might explode after?

Alex Miller (Clojure team)12:06:39

@annapawlicka: I cover it some in Clojure Applied

tcrayford12:06:43

@alexmiller: that looks like exactly what I should switch yeller to. Thanks 😄

tcrayford12:06:08

(currently using yaml files, but for no real good reason)

Alex Miller (Clojure team)12:06:22

oh yeah, edn maps ftw over that

tcrayford12:06:02

yeah, just a case of 13k lines of app code and basically no time. Lots of things like that in Yeller these days

Alex Miller (Clojure team)12:06:35

hey my main project is 60k lines of Java and 20k lines of Clojure ;)

Alex Miller (Clojure team)12:06:04

(that's Clojure itself of course :) )

tcrayford12:06:09

(I guessed 😉 )

tcrayford12:06:05

nice thing about yeller there: there is one developer, so like, if I wanna change something large, there's no discussion or anything, I just do it

tcrayford12:06:17

(on the other hand, there's nobody to say "WTF, why is this like this")

pesterhazy12:06:40

I should get that book, though I'll have to wait for the paper version.

pyr12:06:17

I always end up with YAML because of the easy interaction with config mgmt systems

pesterhazy12:06:43

I also like yaml, because it reminds me of ansible

pyr12:06:47

you do need a bit of mangling to get it into proper CLJ shape

pyr12:06:05

which is unfortunate, but oh well

shinych12:06:09

hi all, i have a beginner question: what does it mean to apply a keyword to a something, which is not a map? in the documentation (http://clojure.org/data_structures#toc8) there is a statement, that the keyword accepts a map as a parameter. but somehow things like this work: (:bla "blups") => nil (:bla \b) => nil I assume this "unwraps" into smth like (get obj keyword), but have no idea how to find it, neither in CLJ source, nor in the documentation.

Alex Miller (Clojure team)12:06:27

pesterhazy: print version will be out in late August/early Sept probably

pesterhazy12:06:12

shinych: keywords actually implement IFn, so they are actual functions

Alex Miller (Clojure team)12:06:24

shinych: yes, just so - keywords can be invoked as a function with an associative data structure and it turns into a get on the data structure with the keyword itself

pesterhazy12:06:57

@alexmiller: excellent, should make for great beach reading

Alex Miller (Clojure team)12:06:26

shinych: get itself is similarly tolerant about receiving non-associative inputs - there is actually a jira to change this behavior

Alex Miller (Clojure team)12:06:04

Stuart Sierra and I have been talking to Rich about that for a long time. I don't think we've convinced him yet. :)

shinych12:06:09

@alexmiller: thanks! Can you pls give me a hint where can I find the special handling of keywords in CLJ source code? Since it is neither a function nor a macro I have no idea even where to start to look...

Alex Miller (Clojure team)12:06:28

Keyword.java is the keyword implementation

pesterhazy12:06:56

oh, @alexmiller got there first

Alex Miller (Clojure team)12:06:08

Keyword implements IFn and the invoke() method is what will get called

Alex Miller (Clojure team)12:06:41

similarly, maps sets and vectors also implement IFn as an associative lookup

shinych12:06:49

thank you guys! getting closer to enlightenment simple_smile

tcrayford12:06:49

@alexmiller: I had a case like that in my talk: https://vimeo.com/132088218 from about 28:55 here

tcrayford12:06:09

had a typo: (:or (:fingerprint error) (compute-fingerprint))

tcrayford12:06:14

(a fingerprint is just a string)

shinych12:06:47

@tcrayford: the question was indeed inspired by your talk, I've been there

tcrayford12:06:22

@alexmiller: got a 5x perf speedup on a thing after fixing the typo

Alex Miller (Clojure team)12:06:02

I think the biggest question is how much existing code relies on the behavior for reals and would be broken

tcrayford12:06:18

yeah, that's a real gnarly one 😞

Alex Miller (Clojure team)12:06:30

I have not tried to evaluate that

pesterhazy12:06:12

should be easy to check by making Keyword raise an exception if called with a non PersistentHashMap argument

pesterhazy12:06:25

as a test, I mean

Busy escaping the comfort zone13:06:58

you have assoc-in update-in but no dissoc-in

avery13:06:15

@narkisr: There is an inactive JIRA card for this. You should see if you can re-visit it: http://dev.clojure.org/jira/browse/CLJ-1063

pesterhazy13:06:14

the problem as I recall is that it's unclear if it should remove empty paths along the way or not

Busy escaping the comfort zone13:06:06

What do you mean by empty paths?

Busy escaping the comfort zone13:06:17

Oh never mind I think I figured what you meant

Lambda/Sierra13:06:47

There is a dissoc-in in core.incubator as well I think.

Busy escaping the comfort zone14:06:23

Yeah the issue mentions it, since I already use incubator is not an issue for me, many folks won't find it tough

andrewhr14:06:36

there is a process for evaluating things in incubator? #lazyweb

Lambda/Sierra14:06:13

@andrewhr: Not really. It was created a long time ago but hasn't had a clearly-defined purpose or a regular maintainer.

Lambda/Sierra14:06:58

@annapawlicka: The only reason I can think of for making "Config" a component is to incorporate loading of config files into the Lifecycle/start process. I usually load configuration as a separate step before constructing the system.

andrewhr14:06:01

congrats people

Lambda/Sierra14:06:58

In other URL-related news, Clojure turns Seventeen!

donaldball14:06:20

Congrats to the core team, a lot of hard, thoughtful work went into 1.7

donaldball14:06:25

I am mildly amused that Rich Hickey is not listed among the contributors to 1.7

goodwind8914:06:34

any pointer to transducer? I'd like to learn about it

annapawlicka14:06:00

@stuartsierra: thanks. I usually load config file at the very beginning too. Config is static and I always use components for things that have some sort of a state.

goodwind8914:06:04

sth practical, I'm not much into academic stuff

hlship15:06:29

@stuartsierra: We went the route of a config component in our system. In our case, we have so many ways of building the system graph based on lots of factors (including testing vs. production), and we define what's acceptable in the configuration in terms of meta-data on the components ... a Prismatic Schema of what that component expects to find in the merged configuration. So we need to know what the components are so we can build the combined configuration Schema.

Alex Miller (Clojure team)15:06:26

donaldball: he didn't contribute any patches (just commits to master :)

hlship15:06:49

Making that configuration map a component was a better alternative than scattering the startup logic around to widely, or having to define components in terms of a callback that gets the configuration, rather than a component that has the configuration "injected".

Alex Miller (Clojure team)15:06:43

goodwind89: well http://clojure.org/transducers is one place to start with links to other things

hlship15:06:36

@alexmiller: I can never seem to find the link to the full release notes ... would be nice if that was linked to from the announcement.

hlship15:06:47

Want to see if there were any final changes since RC-1.

jonas15:06:52

@alexmiller: Congratulations on the release!

pesterhazy15:06:00

Congratulation! It's a pity we didn't properly celebrate the release in Barcelona.

Alex Miller (Clojure team)15:06:07

hlship: it's linked in the first paragraph

hlship15:06:04

Well that's egg on my face. Very subtle color difference for the link ... and I did look first before I complained here. Not sure how I missed it though.

Alex Miller (Clojure team)15:06:31

well you've probably visited it before so it's the visited color

noisesmith15:06:44

alexmiller: if only I had read that blog post

markmandel16:06:20

Congrats on the release! Time to port one of my libs from cljx to reader conditionals!

abtv17:06:11

Congratulations!

abtv17:06:09

Clojure has been released. I wonder when core.async will have released?

Alex Miller (Clojure team)18:06:15

abtv: I'd like to get a core.async release out at some point. it's supported transducers since last fall so that's not an issue.

Lambda/Sierra18:06:02

It's getting released asynchronously. 😛

cristian18:06:41

ba dum tss troll

abtv18:06:58

alexmiller, it's pretty unusual to see a mature library with an alpha status. There were at least two presentations about core.async in 2013. So long ago! Maybe you should mark it as beta or release candidate? Just for newcomers 😀 Yes, I know that alpha status for Clojure project is sometimes release candidate in projects of other languages. But nevertheless 😌

robert-stuttaford18:06:08

it is quite interesting that transducers is in a final release, given that it was made in response to not wanting to redo everything for core.async, and core.async is still alpha

robert-stuttaford18:06:40

i guess now that transducers are baked, the fine folks who work on this can pay attention to core.async again

robert-stuttaford18:06:52

that said, it doesn’t really matter from a practical standpoint. it works just fine!

arrdem18:06:37

whelp I guess it's time to build the 1.7 docs 😛

Alex Miller (Clojure team)18:06:42

Alpha is just a state of mind

Alex Miller (Clojure team)19:06:38

@arrdem: don't I say enough dumb things already?

andrewhr19:06:26

@wagjo: a nice idea for next Conj t-shirt