Clojurians
#lumo
<
2017-04-09
>

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

dominicm06:04:32

@anmonteiro Are you aware of the planck feature whereby fipp is somehow convinced to break inbetween printing and check if you've hit Ctrl-C? That would be pretty great for infinite sequences.

anmonteiro06:04:46

@dominicm I am

anmonteiro06:04:55

I wanna get that implemented in Lumo

anmonteiro06:04:05

@dominicm but we have something that Planck doesn't now :slightly_smiling_face:

anmonteiro06:04:22

just landed that yesterday

anmonteiro06:04:55

unfortunately that doesn't work with (range), I'm still investigating why

anmonteiro06:04:20

but you can now kill stuff that would block Lumo before, things like (while true)

anmonteiro06:04:52

I'm also about to land some perf improvements that bring Lumo down to the startup speeds we had in first releases

dominicm06:04:20

Great. I had noticed a little bit of a lag when I started lumo. Wasn't sure if I was just remembering how fast it had been.

anmonteiro06:04:57

perf was definitely worse when I landed the Closure Compiler stuff

anmonteiro06:04:02

next release will be fast again :slightly_smiling_face:

dominicm06:04:21

@anmonteiro So, I have some cljs which has :target :nodejs, after startup, it should be of comparable speed to lumo though yes?

anmonteiro06:04:09

@dominicm I don't understand the question

anmonteiro06:04:44

if you're asking if speed of compiled JS executed in Node vs CLJS in Lumo is comparable

anmonteiro06:04:48

the answer is probably "no"

anmonteiro06:04:07

because Lumo will still have the overhead of compiling the CLJS

dominicm06:04:41

That was my intended question, yep. Although, mostly for post-startup time.

anmonteiro06:04:09

startup time should be comparable

anmonteiro06:04:09

anyway, almost midnight here, gonna get some sleep. these are the perf improvements if you're curious https://github.com/anmonteiro/lumo/pull/122

dominicm06:04:12

Mostly curious as this particular script is pretty slow for me, and I was curious about whether I could just port it to lumo and it would be faster. I wondered if your cached functions would be better optimized

dominicm06:04:32

@anmonteiro I'll see how much I can understand. G'night :slightly_smiling_face:

anmonteiro06:04:50

I'd be curious to see benchmarks of that

dominicm06:04:26

Not a quick thing to port, as it's tied to the vim remote plugin api. So I'd need to write some glue code for rplugin's & lumo. Not that I don't want to do that anyway :smile:

jonpither08:04:40

Hi, is it possible to do an eval of some code and supply some bindings to take effect at the same time?

dominicm09:04:39

@jonpither what do you mean "bindings" ?

pesterhazy09:04:49

@anmonteiro have you seen how unravel handles infinite ranges?

jonpither09:04:17

var bindings, like in the bindings macro

jonpither09:04:06

I could wrap an arbitrary expr with a bindings form prior to passing it to eval.. wondering if Im missing anything that exists already

dominicm09:04:58

@jonpither I doubt there's anything built in, I think you are looking to wrap code in the bindings macro.

jonpither10:04:46

K, thanks @dominicm

pesterhazy10:04:44

I'm also interested in executing cljs code from lumo

pesterhazy10:04:38

Is manually calling (cljsjs.js/eval (cljs/empty-state) ..) the best option?

pesterhazy10:04:16

Or is it possible to re-use lumo's compiler state? I'd also like to have access to vars defined in my lumo code.

richiardiandrea16:04:09

@pesterhazy you can surely reuse the state in Lumo if you need to, just pass it to eval

richiardiandrea16:04:10

I think the only gotcha is that lumo's init function must be called, but if you use eval from within an already launched Lumo Repl that's already good

anmonteiro16:04:55

The function to call would be lumo.repl/execute

anmonteiro16:04:20

@pesterhazy I have not. Can you link me to it?

anmonteiro16:04:47

I think it may be hard in any case, given we're also fighting the Node.js battle

pesterhazy17:04:33

only works with Clojure's socket server at the moment, but we'll add support for lumo's Socket Server in the future

anmonteiro17:04:56

I've definitely looked around Unravel, I was asking about handling infinite ranges

pesterhazy17:04:02

Essentially unrepl implements a custom printer which a print-length (in this case 10) and a continuation, which allows the user to go back to an earlier incomplete sequence and request the next 10 elements

dominicm17:04:47

I think @cgrand's web browser demo was pretty great too

pesterhazy17:04:09

IMO it's a good user experience when the user is faced with a ton of output -- at least better than the REPL crashing because of pretty-printing overhead, or the user being overwhelmed by too much output

pesterhazy17:04:18

@dominicm, that's planned for unravel as well! Adding a small web server that opens a web browser for interactive exploration and expansion of (potentially incomplete) EDN

cgrand17:04:34

@pesterhazy s/unrepl/unravel

dominicm17:04:55

Maybe lumo shouldn't solve this, and leave it just to unrepl + clients

dominicm17:04:03

In fact, it wouldn't be a bad idea to only launch a socket server in lumo, but auto start unravel for convenience...

anmonteiro17:04:27

That's interesting, but I also agree that Lumo shouldn't handle this

anmonteiro17:04:42

It's definitely an interesting idea though

pesterhazy17:04:53

@cgrand right :slightly_smiling_face:

anmonteiro17:04:01

@pesterhazy so what are you using Lumo for in Unravel?

anmonteiro17:04:41

Does Lumo act as the client to Clojure's socket server?

pesterhazy17:04:46

the client that talks to the socket server is written in lumo, using node's readline lib (like lumo's repl)

anmonteiro17:04:12

That's amazing

pesterhazy17:04:30

lumo is great for this because it understands EDN and it has fast startup time (an essential feature for a repl client)

anmonteiro17:04:04

It's going to be even faster in the next release, as I was telling Dominic yesterday

pesterhazy17:04:06

frankly the development experience so far has been amazing

pesterhazy17:04:19

didn't run into any issues with lumo really

pesterhazy17:04:55

I discovered a nice dev pattern on the way

anmonteiro17:04:01

I hope my Euroclojure talk gets accepted so I can mention all these cool projects

pesterhazy17:04:31

which basically runs the project

pesterhazy17:04:03

which watches src for changes and kills (restarts) the loop instance

pesterhazy17:04:16

because lumo is so fast, this works really well

pesterhazy17:04:12

and I have to say that node is a good environment for network programming and even for cli tools

pesterhazy17:04:46

I'm planning to submit a talk for euroclojure about unravel as well

anmonteiro17:04:51

The fact that JS engines are optimized for latency helps a lot with that

anmonteiro17:04:32

I just wish readline were a little more pluggable

pesterhazy17:04:48

yeah but... it's javascript so you can just go in and override any methods

anmonteiro17:04:25

Sure, another concern is the event based API

pesterhazy17:04:26

node's readline module lacks a ^R for reverse-history-search, I'm planning to add that as well

anmonteiro17:04:39

Yeah, Planck has that

anmonteiro17:04:52

There's an issue to add it to Lumo too

pesterhazy17:04:13

I saw that the readline work is implemented in js in lumo

pesterhazy17:04:22

is there a particular reason for that?

pesterhazy17:04:32

or just a historical artifact

anmonteiro17:04:46

It's because of the custom V8 snapshots

pesterhazy17:04:07

but some parts of lumo are implemented in cljs, correct?

anmonteiro17:04:32

The ones that don't require Node.js stuff to be there

pesterhazy17:04:57

do you mean they don't call js/require?

anmonteiro17:04:57

Or that aren't in the critical path and can lazy-load them

anmonteiro17:04:28

require, module, console, process, all that stuff

anmonteiro17:04:57

Because the snapshot is loaded into V8 way before it knows about Node's existence

pesterhazy17:04:25

wouldn't it be possible to pass those primitives in somehow to the cljs code?

pesterhazy17:04:41

(n.b. I'm in the dark about how lumo bootstraps itself)

anmonteiro17:04:29

Yeah we do that but it requires lazy-loading of those modules via indirection and such

pesterhazy17:04:32

in my experience I use js/require and js/process in very few placs

anmonteiro17:04:03

So I choose to keep the critical path in JS for perf but also for readability

pesterhazy17:04:54

although perhaps the readline UI wouldn't be critical?

pesterhazy17:04:56

re: readability, that's a good point, I haven't found a good way to write callback-heavy code in cljs without very long lines: https://github.com/pesterhazy/unravel/blob/master/src/unravel/loop.cljs#L211

cgrand17:04:59

Before I spent time on it: is there an event-based reader out there?

pesterhazy17:04:24

not sure if that's what you mean?

anmonteiro17:04:42

@pesterhazy you could probably make the code more readable with core.async, although I don't know if it's worth it taking on the dependency

pesterhazy17:04:21

yeah I'm reluctant to use core.async in clojurescript

pesterhazy17:04:48

it may be better to use promises in some way

dominicm18:04:16

@pesterhazy no promises in cljs (unless you mean es6 ones)

pesterhazy18:04:29

yeah es6/node promises

dominicm18:04:46

A lot of things don't really support it. So you get more code around wrapping promises.

dominicm18:04:54

(or use bluebird with… interesting results)

richiardiandrea18:04:56

what about goog promises?

richiardiandrea18:04:16

I think I have used them in replumb actually

pesterhazy18:04:41

isn't promises where javascript-land in general, and node-land in particular, is going? and generators?

pesterhazy18:04:24

>>> NOTE: This class was created in anticipation of the built-in Promise type being standardized and implemented across browsers. Now that Promise is available in modern browsers, and is automatically polyfilled by the Closure Compiler, by default, most new code should use native Promise instead of goog.Promise. However, goog.Promise has the concept of cancellation which native Promises do not yet have. So code needing cancellation may still want to use goog.Promise.

dominicm18:04:38

> going Slowly. Not everything uses it though.

dominicm18:04:44

A lot of things don't infact.

dominicm18:04:47

Wrapping is hard.

pesterhazy18:04:21

@dominicm can you give an example of what is hard about it?

dominicm18:04:34

Hard is the wrong word. Tedious is the right one.

dominicm18:04:21

Maybe cljs can make it super easy (in a similar way to how bluebird does) with a macro or something.

dominicm18:04:35

But then, maybe we can do that with callbacks anyway…

pesterhazy18:04:33

This works in lumo out of the box:

(-> (js/Promise. (fn [resolve reject] (resolve "foo"))) (.then prn))

pesterhazy18:04:04

@dominicm I'm not familiar with bluebird, what magic does it provide?

dominicm18:04:43

@pesterhazy take the fs module for example, for each function you want to use you have to create a "promisified" version which does (js/Promise. (fn [resolve reject] (fs.readFile "/tmp/log" (fn [val err] (if err (reject err) (resolve val))))))

dominicm18:04:19

@pesterhazy bluebird assumes that all callbacks are the form val, err, and wraps them automatically.

dominicm18:04:30

If you give it an object to wrap anyway

pesterhazy18:04:11

I guess we could just use bluebird?

pesterhazy18:04:28

for node apis like fs

dominicm18:04:44

Could do. Yes. It works, most of the time, but it feels… weird to use. Because it's automatic.

pesterhazy18:04:07

thanks this is super valuable info for me as a node-newb

anmonteiro21:04:47

both Linux and Windows CIs are running out of memory with the PR to make Lumo start faster

anmonteiro21:04:09

I don't know what to do... does anyone know CIs that provide VMs with lots of memory?

futuro23:04:54

@anmonteiro I wonder if that's a request you could make? How much memory are they eating up?

anmonteiro23:04:15

I don't know exactly

anmonteiro23:04:37

definitely more than 3GB (in Appveyor's case, I think that's their limit)

anmonteiro23:04:49

Travis's might be 4GB, not sure

anmonteiro23:04:06

it's the C++ compilation

anmonteiro23:04:39

I added more stuff to the startup snapshot, and V8's mksnapshot occupies a whole bunch of memory