Fork me on GitHub
#clojurescript
<
2017-03-25
>
john01:03:26

So I figured out how to send clojure forms without their sources (only js under the var) over to a remote repl finally. For instance to wrap inc just put it in a var, with a js* inside, like so (str "(def avar (js* \"" (str inc) "\"))"). Apply that on the other side and apply the result to either remote state or state you sent over with it.

john01:03:35

might fail under advanced compilation though

john01:03:04

what can I call on a parameter at runtime to determine if it is a non literal form, like #object[... ???

john01:03:26

I guess just #(not (coll? %))?

noisesmith01:03:39

there are plenty of things that are readable but not collections

noisesmith01:03:49

dates, uuids, records...

noisesmith01:03:17

but maybe (fn? %) would help in the case you care about?

mfikes01:03:19

@john Take a look at the JavaScript you see for multi-arity functions—it refers to other variables that don't show up in the string form.

john01:03:19

specifically, I want to know if the var is js underneath, than can be executed by js*, or not

john01:03:13

@mfikes are you referring to a stricter mode of printing strings?

john01:03:52

I read something about strict mode

mfikes01:03:21

Take a looks at what you get for (str f) when f is defined by:

(defn f
  ([x] :single-argument)
  ([x y] :two-arguments))

mfikes01:03:41

Specifically, the keywords in the above definition don't appear. The first ends up under cljs.user.f.cljs$core$IFn$_invoke$arity$1

john01:03:25

are keywords stored on a table?

mfikes01:03:38

Well, the example above isn't specifically about keywords. It is to illustrate that the string form of a function value isn't all of the JavaScript that defines the function.

john01:03:24

where are those keywords stored? the js form will still return those keywords, right?

mfikes01:03:14

Of course, you run into a similar thing if a function refers to another var, but the multi-arity example shows that the generated code includes more than what you see when apply str to the value.

john01:03:14

Yeah, I was noticing other things disappearing too

john01:03:28

#anything

mfikes01:03:50

To answer your question about the keywords in the example above, the auxiliary JavaScript referenced by the "primary" function constructs the keywords (at least in :none mode)

mfikes01:03:42

cljs.user=> cljs.user.f.cljs$core$IFn$_invoke$arity$1
#object[Function "function (x){
return new cljs.core.Keyword(null,"single-argument","single-argument",-784978075);
}"]

john01:03:59

so single clojure functions are literally being broken down into multiple functions. Not all of which are returned by the var of the function itself, when returning it as a parameter to something.

noisesmith01:03:18

printed and returned are different

mfikes01:03:35

@john The current implementation of the compiler emits multiple JavaScript assignment statements for multi-arity functions.

noisesmith01:03:36

it prints something, but that's not the object, it's a string that doesn't reflect every property of the object

mfikes01:03:19

An easy way to experiment with this and see what is going on is to put your ClojureScript REPL into verbose mode and evaluate some defns and see what code gets emitted.

john01:03:51

understood

john01:03:20

yeah, shuffling "compiled" cljs code between environments sounds pretty hard.

mfikes02:03:32

In some sense, that's how many ClojureScript REPLs work. With the difference being that they easily have access to the emitted JavaScript because they just produced it.

john02:03:58

right, staying synced up?

mfikes02:03:20

(When you evaluate a form in a ClojureScript REPL, it compiles it to JavaScript, and then evaluates that JavaScript in the target JavaScript environment, which is usually remote.)

john02:03:44

so the guts to make it work might be there. But you get into messy business when you want to differentiate the two environments and run some code on one and some on another.

john02:03:29

There's also the overhead question of synchronizing state between multiple webworkers or remote repls

mfikes02:03:35

Do you have the ClojureScript source for the code you'd like to ship to the remote environment?

john02:03:09

so i've got a self-hosted repl up in a webworker and I'm shuttling datastructures back and forth via butler. I'm experimenting with some concurrency ideas. Currently working on an agent implementation.

mfikes02:03:33

The most sophisticated example I've seen along those lines is crepl

john02:03:53

so I'm experimenting with the different ways of sending code over to the webworker

john02:03:10

yeah, I noticed that. Didn't realize they were solving that problem too. I'll definitely have to check it out.

mfikes02:03:30

crepl is also self-hosted, and it is syncing code between two environments remote from each other, as well as atom state

john02:03:53

do you think it's a good idea to go after that compiled code sharing goal, for the purposes of an agent? Sending pure cljs forms over the wire to be evaluated on the remote end is working at the moment.

john02:03:19

though you could probably be a lot lighter weight on the webworker end if you didn't need a compiler on that end.

mfikes02:03:01

In my opinion, attempting to extract and sync the compiled JavaScript, by tracing through vars and references seems like would be challenging.

mfikes02:03:44

Yes, the self-hosted compiler has a big footprint

john02:03:58

Well it's pretty fun to experiment with 🙂 thanks for all your advice!

mfikes02:03:32

Yeah, not claiming it is impossible. Experimenting with this stuff is truly fascinating.

noisesmith02:03:55

what about patching the self-hosting compiler so it has the option to compile to localstorage? then you could get the string to send from there? maybe that's crazy

noisesmith02:03:20

with localstorage, it would also be immediately accessible within the same browser if the same site

john02:03:51

@noisesmith I think the cljs functions are still decomposed into separate functions in the source files

noisesmith02:03:08

right but you tell the other js to load the whole localstorage document

noisesmith02:03:15

(maybe that doesn't fit your usage...)

john02:03:20

when passing a compiled function, the caller then calls the others from the environment

john02:03:38

short of sending the whole environment, you'd have to do diffs. or trace out js function dependencies and send just the dependencies

noisesmith02:03:08

yeah, that's tricky

john02:03:34

and if you have two different threads banging on one datastructure that is synchronized between locations, most threads will be waiting on synchronization

noisesmith02:03:00

well, localstorage is keyed - they can each claim their "output key" without conflict

noisesmith02:03:40

I mean - I'd assume the browser could handle that since it's keyed, we could find out it would still conflict I guess

noisesmith02:03:16

but probably best to use the communications mechanisms that webworkers already have I'm sure

john02:03:41

the replikativ folks, with their CRDTs and Konserve sounds similar

noisesmith02:03:01

I was just thinking that if you are generating the functions at runtime, you have the self hosted compiler (or some subset of it) so with some patching you should be able to output to a usable textual location in the js vm

john02:03:13

they're working on datascript on localstorage that syncronizes p2p style over CRDTs

noisesmith02:03:06

yeah- I could imagine a crdt implementation where each producer claims a key, guaranteeing that order of writes are not important between producers (as long as you get each individual data producer's order right)

john03:03:28

Yeah, I'm using keys to synchronize agent actions. It's pretty hairy

john03:03:04

@mfikes You were correct ^^

john07:03:21

How long does it take yall to run this?

mfikes12:03:37

@john for this code V8 appears to be an order of magnitude faster than JavaScriptCore

mfikes13:03:19

(V8 takes about a minute)

john15:03:46

@mfikes Yup, that's about what I'm seeing. I'm creating MetaFns to carry around the function's source, but they're going roughly 8 times slower. Once they compile down on the remote side, they compile to regular functions. So I thought my webworker was going 8 times faster than my main thread 😂

john15:03:12

Turns out the main thread is going about 20% faster

john15:03:11

and there's about a 50 ms round trip overhead for sending and receiving to the worker.

john15:03:19

for small data

john15:03:07

I've implemented a round robin worker pool now though, so I can queue up 20 (fib 40)s and all four of my cores get pinged 🙂 goes 25% to 50% faster than it would have taken on one thread - by my back of the napkin measurements

john15:03:15

and that's about the most naive agent pool implementation you could make. Could probably be improved a lot.

dvingo16:03:50

klipse has a nice interface to quickly see what JS comes out: http://app.klipse.tech/?js_only=1

tagore18:03:33

@danbunea Going back a bit for this, but... I'm also just dipping my toes into the waters of generative/fuzzy testing, but I've been inspired by some examples I've seen (even if some of them are a bit toylike, they seem to point in an interesting direction. For instance: https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-air.html and https://lcamtuf.blogspot.com/2014/10/bash-bug-how-we-finally-cracked.html (the latter not toylike, but actually useful at finding bugs in bash with real security implications.)

tagore18:03:45

I've always been dissatisfied with testing- what I want is proofs, what I have is tests. I did spend a few years working on a system where we actually did include some proofs, at varying levels of formality, in both the dev process and the documentation, but... that was only practical for certain relatively isolated pieces of the software.

tagore18:03:08

So, given that what I have is tests, and that they are expensive to write and maintain relative to their effectiveness in finding non-trivial bugs... well the issue with measuring coverage is that lines of code covered is a bad metric. It's paths through code that matters (as well as the input space.) I think the idea of testing systems that can learn how to take new paths through code is pretty interesting.

tagore18:03:35

(Though in a deep sense the distinction between paths through code and the input space may be pretty closely related.)

baptiste-from-paris19:03:09

hello guys, is there a reason keywords don’t extend IMeta & IWithMeta ?

noisesmith20:03:26

because then they wouldn't be internable

noisesmith20:03:45

or at least we would need a very different interning system

Aron20:03:12

that jpeg discovery thing looks really cool

jfntn21:03:33

Is there a way to use externs provided by the closure-compiler project without copying them? https://github.com/google/closure-compiler/tree/master/externs/browser

anmonteiro21:03:18

@jfntn they should be in the classpath

tagore21:03:17

@ashnur The learning to write valid bash scripts part seems even cooler to me (but I have always been a bit parser-oriented, I guess.)

tagore21:03:47

@ashnur But what seems cool to us is not even the cool part, I think... it's what we notice.

tagore21:03:23

@ashnur the cool part is learning to exercise as many paths through the code as possible- that has always been the fly in the ointment.

jfntn21:03:56

@anmonteiro hmm that’s strange then, I’m seeing "$svg$$inline_4211$$.$createSVGPoint$ is not a function” when invoked on #object[SVGSVGElement [object SVGSVGElement]] in an advanced build

tagore21:03:48

@ashnur leaning to generate jpegs and valid bash scripts is an epiphenomenon, albeit one we find significant.

anmonteiro21:03:04

@jfntn last time I checked Closure didn't bundle SVG externs

anmonteiro21:03:12

has that changed in the meantime?

jfntn21:03:58

oh you’re right!

jfntn21:03:08

that is quite the gotcha

tagore21:03:02

I'm inclined to think that we're less than ten years away from a limited form of generative programming, but... I think this sort of generative testing is the first step.

Aron21:03:20

fly in the ointment? epiphenomenon? i am not following you, sorry :-S

Aron21:03:50

i like the idea that there is a tool to explore specific spaces efficiently

tagore21:03:24

Hmm- well what I mean by 'fly in the ointment' is that as programmers we mostly guess at correctness. We can cover every line of code with tests, but in complicated cases we can't cover every path through our code. The best we can do is try to isolate things (the point of functional programming) but we can't do that completely, so our tests shouldn't really reassure us. There are, in many reasonably complicated modern software systems, trillions of possible paths through the code- and that is likely an understatement.

mobileink22:03:05

tagore: it won't be too terribly long before most of us are replaced by machines. better keep an eye on languages that include proof engines. they can't replace interactive code, but for the functional bits they will disemploy a lot of programmers.

tagore21:03:58

This is why programming is still more art than engineering. A structural engineer can measure the load a beam can carry, and be pretty sure that if it can easily carry 1000 lbs, and can easily carry 2000 pounds, it can easily carry any weight between the two. An analogous software system might see the beam buckle at exactly 1.436785 pounds, but at no other value.

tagore21:03:57

This is the fly in the ointment.

tagore21:03:59

When I say epiphenomenon what I mean is that while we're oohing and ahing at a simple testing system learning to do huffman encoding, or write valid bash scripts, that's just an incident of something more important happening that is harder for us to see, and thus ooh and ahh about. Code learning to exercise code is more significant, to my limited mind, and different pieces of code learning to adapt to each other seems even more significant, though I doubt I'll be able to understand anything not epiphenomal about the upshot of that.

Aron21:03:05

i see that this topic is important for you 🙂

Aron21:03:38

i find it interesting for the same reason, only difference is that i don't think this would help so much to make programming a science, mostly because programming just like politics or economics is also about values that are not necessarily quantifiable. at least not with our current understanding.

Aron21:03:56

but i do think it can help with building more general tools

mikepence21:03:02

any opinions on using vue.js with clojurescript?

Aron21:03:50

if you like to experience stuff. but i see no reason to use it in production. there are probably better tools. more mature stuff, based on datascript and everything

mikepence22:03:45

@ashnur was that in response to my question?

tagore22:03:25

@ashnur I suppose so, and I'm inclined to think that the fly in the ointment bit should be important to everyone who programs. It's just so very difficult to know how correct large systems are, and sometimes it's important that they be reasonably correct. As for the rest.. well, that's speculative, but it certainly seems like the outlines of whatever is slouching toward Bethlehem (for good, or for ill, or for both) have become a bit easier to discern lately. I wouldn't be surprised if Skynet grew out of a test framework, at this point 😉. However, this is not very specific to clojurescript, so...

Aron22:03:05

well, trustware will fix all your problems, i am sure 😄

tagore22:03:18

I'm not familiar with trustware, Sounds fishy to me though. I don't trust much, but I trust software only in strictly delimited fashion.