Fork me on GitHub
#clojure
<
2019-04-14
>
yuhan04:04:47

Is this guaranteed to match up elements in a set s with their mapped ones? (zipmap s (map f s))

seancorfield04:04:48

@qythium Yes, both zipmap and map call seq on their argument -- and seq returns the same ordering on repeated calls for the same object (collection).

dmarjenburgh11:04:18

The following probably happens because of something in with-in-str. What would be the best workaround?

(with-in-str (str (char 13))
  (.read *in*))
; => 10

dmarjenburgh11:04:59

Got it by binding *in* to a StringReader manually.

dazld12:04:57

what are the implications of using clojure values as keys in a java.util.WeakHashMap?

dazld12:04:42

actually, think i just answered my own question - guess they will be GC’d as usual, with no particular guarantees

alexmiller13:04:54

In general, Clojure values are just Java objects, so nothing special. One possible exception are keywords, which are interned and shared across the runtime

Nico14:04:01

what is the current recommended way to do desktop GUI in clojure? Still seeaw? I would like to avoid the clojurescript/electron route as much as possible. Also, are there any GUI designer tools that work with either seesaw/whatever is more recommended now?

rakyi14:04:23

there are couple other viable options: - https://github.com/cljfx/cljfx - https://github.com/fn-fx/fn-fx but I haven’t done any Clojure GUI work, so can’t compare

martinklepsch15:04:58

hey there 🙂 Anyone know a small real-world code snippet that demonstrates why Clojure can't be statically analysed? (e.g. something defing things based on a data file or similar)

vlaaad15:04:03

There is a library that generates defs from files with sql queries, don't remember what's it's name

martinklepsch15:04:11

oh right, that's a good one

Nate Sutton19:04:04

is it expensive to convert a vector to a sequence?

Nate Sutton19:04:45

it seems like (hashmap|vector|set) -> sequence happens a lot and I don't have a sense for how expensive that is

Nate Sutton19:04:57

and not sure how to get a sense of that

dpsutton19:04:35

public Seq(IPersistentVector v, int i){
		this.v = v;
		this.i = i;
	}
seems to be about it

dpsutton19:04:41

two pointer assignments

john19:04:39

I'm not sure if a conversion takes place. IIUC, 'sequence' is an abstraction that collections implement. So certain functions will treat it sequentially

Nate Sutton19:04:15

so first and rest and next just index into the vector?

john19:04:49

Well, it's a trie underneath, so there's a sort of conversion for any operation (depending on the size)

john19:04:49

But, yeah, IMO it's not like a "conversion" to an different thing... Moreso a particular view over it.

hipster coder19:04:56

The docs say that the vector is contiguous… does that mean they are placed side by side in memory?

Nate Sutton19:04:19

I was hoping it was something like that

Nate Sutton19:04:36

I couldn't imagine how expensive it would be without that being the case

john19:04:41

@nathantech2005 persistent collections are rich collections of 32 element arrays

hipster coder19:04:30

is 32 element the same as 32 bits?

john19:04:32

So it's more likely the elements with one of those arrays are contiguous in mem, than in other parts

john19:04:55

No, in java and js, those arrays are not typed

hipster coder19:04:46

It says the sequence is based on the ISeq, IPersistentList

john19:04:53

Perhaps I'm wrong and there is a significant cost to using a collection as a sequence. But I didn't think it was any heavier than than its most efficient ability to iterate across the collection.

hipster coder19:04:46

My guess is you are correct. Because I’d guess that the sequence is just the edge nodes of the trie data tree structure in memory

john19:04:13

Like, a seq is printed as a list, but that's not to say the vector is being converted into a list first.

hipster coder19:04:26

I am reading the docs. How lists and vectors are FIFO or LIFO

hipster coder19:04:03

1 notable different though in the docs… conj adds to the front of a sequence. but adds to the end of a vector

hipster coder19:04:55

I am highly interested in this… let’s thread the conversation

alexmiller19:04:53

Seqs on vectors are just views into the vector and are highly efficient

hipster coder19:04:10

@alexmiller so the sequence just walks down the edge of the trie?

alexmiller19:04:39

Effectively, yes. I mean you can read the code, it’s pretty straightforward I think

alexmiller19:04:31

Normally seqs have some overhead in caching the resulting sequence (effectively a linked or chunked linked list)

alexmiller19:04:52

Vectors seq is more efficient as it just piggybacks the vector

hipster coder19:04:35

but Clojure doesn’t use any linked lists underneath? Right?

alexmiller19:04:48

Clojure lists are linked lists

hipster coder19:04:06

wow… I thought they were array based

alexmiller19:04:06

Sequences are usually linked lists

alexmiller19:04:21

Chunked seqs are links of arrays

alexmiller19:04:34

So more memory efficient

hipster coder19:04:41

ahhh ok. I must be reading conflicting info from the web

hipster coder19:04:02

because some docs said that Clojure lists were array based, next to each other in memory, touching

john19:04:20

Are Java or js arrays really guaranteed to be contiguous in physical memory?

alexmiller19:04:32

Java arrays are

hipster coder19:04:39

wait a second… why don’t I see any middle insertion methods on the Clojure ISeq (lists) if they are linked lists?

hipster coder19:04:11

there is only conj, adding at begining and end… which would point to using an array structure

alexmiller19:04:37

In general Clojure provides only “efficient” operations

alexmiller19:04:55

The efficient place to add elements to a linked list is at the head

hipster coder19:04:17

I thought a linked list can have insertion done at any point, in constant time?

hipster coder19:04:24

because it just looks up the memory address

dpsutton19:04:53

if you have a linked list you have a node and a pointer to the rest

dpsutton19:04:19

so to insert in the middle you have to start following the links

hipster coder19:04:43

I can’t just insert based on the nth term?

dpsutton19:04:51

sure. but first ya gotta get there

Lennart Buit19:04:01

a linked list is really just a node pointing to another node

hipster coder19:04:11

so I have to walk the linked list to find the nth term?

dpsutton19:04:23

its like a scavenger hunt

hipster coder19:04:32

wow. I was really confused about that.

Lennart Buit19:04:58

its a tree without branching 😉

john19:04:21

It's a vine ;)

Lennart Buit19:04:42

I am coining “unary tree”

hipster coder19:04:45

why would anyone want to use one?

hipster coder19:04:02

if the insertion writes are linear complexity… makes no sense to me

john19:04:06

For function application ;)

hipster coder19:04:49

ahh, so you use them to chain functions together?

john19:04:50

Really, vectors are almost always preferred

john20:04:15

No, I just mean that because Clojure is a lisp we always use lists to apply functions, with the function in the operator position of the list

Lennart Buit20:04:35

well unless you want to prepend, thats O(n) in an vec and O(1) in a list. The moral of the story, different datastructures have different trade offs

hipster coder20:04:49

I must have read some erroroneus docs… that said Clojure uses arrays, contiguous, for sequences… when I was comparing it to Elixir

hipster coder20:04:27

I think Lennart was the one who pointed out linked lists don’t do well on the CPU l1, l2 cache

hipster coder20:04:45

because they can be spread all over RAM and can’t be loaded in 1 step like arrays can, contigous

Lennart Buit20:04:56

I did not, but I have read it sometimes

dpsutton20:04:10

that's true. and Clojure does some tricks mentioned earlier (chunking) to bridge this gap. The gain you get is that lazy sequences can be used interchangeably with standard collection types when this interface is pervasive.

Lennart Buit20:04:26

if you care about cache misses, you are fairly deep into optimising tho, first write code, then make it fast 😉

hipster coder20:04:57

Elixir uses linked lists as their basic type, for lists… and because the CPU can’t grab the data in 1 step… it makes threading less efficient… when the CPU has to lock memory… and do several steps to get the linked lists from all over RAM

hipster coder20:04:17

I think this is a reason why Clojure is faster and has lower level threading models

hipster coder20:04:28

@lennart.buit but you posted that info on Java Fibers… Which is like Actors… but it sounds lower level too. If clojure takes advantage of Fibers… I think that will be fantastic

john20:04:51

Does Elixir have shared memory between processes?

hipster coder20:04:10

the only way to share is off-site

hipster coder20:04:30

or through message passing

hipster coder20:04:08

this is a big reason I love clojure. We can thread stuff with shared memory… and we can also setup the actor model

hipster coder20:04:03

This is the reason Clojure is better at number crunching… But can also handle horizontal scaling with Actors. And Fibers can be a game changer for us.

john20:04:57

Aye, I don't think elixir is designed for high performance, local number crunching

john20:04:07

Which is fine

hipster coder20:04:20

correct. Elixir is setup for distributed computing

hipster coder20:04:05

I am not knocking it. It’s great for its purpose. But I do think Clojure is more robust for number crunching, and horitzontal and vertical scale

john20:04:38

Fibers / project loom will be interesting

hipster coder20:04:16

ya, it could make Clojure scale horizontally from day 1, without extra setup… just like Elixir

dpsutton20:04:20

i remember tim baldridge recently laying out why Clojure is not great for number crunching. I think it was on a reddit post about game design in Clojure?

hipster coder20:04:27

@dpsutton Clojure isn’t good for number crunching?

dpsutton20:04:38

i'm looking for his post

dpsutton20:04:14

i've never done heavy number crunching in clojure. so i like to listen to arguments and experiences from people much smarter than me 🙂

lilactown20:04:15

immutable data structures will always be at a disadvantage when it comes to raw efficiency

hipster coder20:04:21

@dpsutton he is talking about game programming (C, C++)

john20:04:55

Well, the flexibility of persistent data structures is never going to be as fast as typed arrays

hipster coder20:04:01

Games can’t handle the pauses in Garbage Collectors

sobel20:04:27

:sideeye:

john20:04:40

What lilactown said

hipster coder20:04:36

Clojure uses tricks though… like lazy-seq

sobel20:04:37

Games aren't unique in realtime programming. Any allocation strategy is going to fail if you abuse it.

hipster coder20:04:07

I think pauses in the garbage collector is ok for non user facing stuff, e.g. machine learning

hipster coder20:04:27

I’d glady make the trade off if it means I don’t code it in C

sobel20:04:33

sure, obviously batch processing is just a race to the finish line.

hipster coder20:04:57

I benchmarked a small test. Clojure lazy seq is as fast as C.

john20:04:29

Yeah, despite being super flexible, Clojure is pretty fast

dpsutton20:04:39

i don't believe you. (re: lazy seq is as fast as C)

dpsutton20:04:05

i'm assuming your terminal printing was the same speed in a C program as with clojure 🙂

✔️ 1
hipster coder20:04:21

my clojure lazy seq on the nth term of fibonacci was as fast as C

hipster coder20:04:41

around 1 second to find the 40th nth term

hipster coder20:04:14

well, I did use cython, from python recursion script

hipster coder20:04:18

I didn’t write it in raw C

Lennart Buit20:04:25

also, comparisons of language A is as fast as language B usually highly depends on your implementations in A & B. I can write number crunching Python thats faster than C if I butcher the C impl.

sobel20:04:41

i heard PyPy is faster than cython

sobel20:04:53

(implementations matter)

dpsutton20:04:57

i don't see any C in there

hipster coder20:04:05

I am going to write it in Raw C. then compare it to clojure.

dpsutton20:04:44

the python version is the naive recursive solution

hipster coder20:04:06

but Clojure optimizes recursion with a lazy-seq. So I cheated a little.

john20:04:20

I wonder if/how clojure's concurrency abstractions will fit with loom/fibers

hipster coder20:04:09

let me write some more today… and try a naive recurssion in Clojure compared to C. I am sure C is faster in that case though

dpsutton20:04:10

it's a totally different algo. you did the count up version in clojure and the naive exponential version in python

hipster coder20:04:27

ya, it’s not accurate enough. I cheated.

dpsutton20:04:58

lol. well lets not claim Clojure is as fast as C if there is 1) no C and 2) different algos used in the python versus clojure verions 🙂

dpsutton20:04:17

but i like your enthusiasm and willingness to actually benchmark

hipster coder20:04:24

I think I should restate it as… Clojure can use tricks to speed up to C level performance

hipster coder20:04:06

I’ll rewrite it today, with a more accurate comparison… and share it

dpsutton20:04:28

i look forward to it.

hipster coder20:04:34

I am biased towards Clojure… you can see my bias

dpsutton20:04:38

i am too 🙂

john20:04:29

Like, could a fiber backed atom be as simple as a cljs atom? Or does interruptibility on the jvm complicate that?

alexmiller20:04:40

In general, Clojure should be approximately as fast as Java. jit optimized Java is generally fast enough for the majority of programming uses. Where it’s not, there are options to use faster libs as a consumer.

alexmiller20:04:49

99% of users never need that though

✔️ 1
sobel20:04:46

My very sloppy Clojure use is 80-99% as fast as Java. Once in a while I tune it up with just a tiny bit of effort. Mostly it is sufficient.

hipster coder20:04:51

omg. I mis-spoke. Clojure Binet Formula is as fast as C recursion

😆 1
dpsutton20:04:18

ha. can you find the point where clojure's single computation is faster than c's recursion?

john20:04:26

There may be certain kinds of problems where persistent data structures are already the most performant abstraction. Like an editor that can edit files larger than what's in memory. I think have to implement some shared structure thing anyway.

hipster coder20:04:50

@dpsutton hahahaha this is an interesting comparison.. working on it now

dpsutton20:04:03

you were using 40 earlier i think.

sobel20:04:28

persistent data structures are a convenience for programmers to ensure certain types of design correctness

john20:04:32

Or if you were trying simulate a universe of stars that don't fit in memory all at once

sobel20:04:59

that convenience is overhead

sobel20:04:14

it's not going to pay off except in time correctness

sobel20:04:21

time to correctness

john20:04:00

Unless the problem itself looks embarrassingly like a persistent data structure problem

sobel20:04:17

there are lower overhead ways of ensuring correctness but they cost developers more at design/review time

sobel20:04:48

what is a persistent data structure problem?

john21:04:16

It doesn't have to be only about correctness. the shared structure of persistent data structures allows for efficiencies that are harder to achieve without that shared structure.

john21:04:23

Especially over infinite data sets for instance

mg21:04:37

a "convenience for programmers" - doesn't this describe every single tool?

sobel21:04:42

harder how? in developer effort? that's what i've been saying.

sobel21:04:56

mg: as in the opposite of fundamental to computation

mg21:04:03

compilers are "convenience for programmers", you should be writing assembler yourself

sobel21:04:22

that's asinine

john21:04:53

Harder to reason about, sure. But in terms of polynomial difficulty,

john21:04:09

persistent data structures may be the minimal polynomial complexity for certain kinds of problems that have shared structure

sobel21:04:40

Can you be little more concrete about that?

mg21:04:48

well I think the premise of the question fundamentally kind of silly

sobel21:04:03

Polynomial complexity as I usually hear it refers to computational complexity

john21:04:26

Navigating a procedurally generated universe of stars that don't fit in memory

sobel21:04:46

that sounds like an application not a computation problem

john21:04:04

Hmm, perhaps

john21:04:10

There's this YouTube video of a guy that built an editor using persistent data structures implemented in c++. They call them postmodern data structures lol

john21:04:47

It'd be pretty interesting to see that implemented in clojure

john21:04:16

Def check out that video sometime! It's worth a watch

john21:04:52

another efficiency perk of Clojure's persistent data structures: snapshots. So if you have some problem that needs a consistent view of snapshots of data across some threads, mid-computation, then persistent data structures already solved the problem for you. People rarely use the snapshot capability though AFAIAA

aryyya22:04:10

Why would a function have a * at the end of its name? My guess is that it’s a convention indicating that the function generates a lazy sequence. For example evens*.

seancorfield22:04:48

It's often used to indicate an "implementation" function. You'll often see it as the name of a function behind a macro.

seancorfield22:04:03

You'll also see it for the implementation of some special forms in Clojure itself. Both fn and let are technically "Special Forms" but they are implemented as macros that expand to "calls" to fn* and let* respectively (which are implemented directly in the compiler).

seancorfield22:04:51

Where I've also seen it used a lot is for functions that implement a cache: if foo is the cached version of the function, it will often be implemented in terms of foo* which will be the uncached version of the function.

aryyya22:04:47

@seancorfield Hmm. I think I understand. In other languages I would expose the modules function with a friendly name and then have another function behind the scenes that does the actual work (because it needs extra parameters for example). So this is a convention for that sort of pattern?

seancorfield22:04:41

Yes, it's a fairly common pattern for implementation detail functions.

aryyya22:04:18

@seancorfield In JavaScript I would usually pre- or post-fix an _ for this same purpose.

seancorfield22:04:46

Yeah, I've seen that convention in C/C++ as well.

aryyya22:04:33

Side note, something I love about Lisps is using a dash as-a-separator for names. It’s by far the most natural way to do this in my view and it’s a shame most other languages don’t do the same.