Fork me on GitHub
#portkey
<
2017-10-03
>
baptiste-from-paris06:10:26

do you understand the error message ?

viesti06:10:46

the java.security.SecureRandom kryo serialization issue is probably separate from clj-http/request var serialization

baptiste-from-paris06:10:49

maybe I do things wrong, but do I need to upload my #‘var everytime it changes ?

baptiste-from-paris06:10:14

It slow down my feedback loop, it’s okay but I am just asking ^^

viesti06:10:46

mount! has :live true

viesti07:10:03

but haven’t gotten to putting that into mount-ring!

viesti07:10:36

thinking about lifting common pieces from mount! for use in mount-ring

viesti07:10:52

ultimately these things are related, mount! is just a special case of proxy

baptiste-from-paris07:10:55

does it upload everything ? * :live true

viesti07:10:23

yes, but only on change of the mounted var, not on change in dependencies

cgrand07:10:21

the plan is to have pk to upload deltas on change to any var used by the published ones Batching would be achieved by having tests ran before deploying, if tests fail no deploy

viesti07:10:01

one can “batch” changes that way 🙂

baptiste-from-paris07:10:16

have you looked at my keep stuff, totally wrong ?

viesti07:10:26

haven’t yet have time

baptiste-from-paris07:10:46

I have few mini-websites that are going to massively use portkey

baptiste-from-paris07:10:53

at least I am going to try hard

baptiste-from-paris07:10:40

this tool can be life changing for prototyping

baptiste-from-paris07:10:18

it’s even better than lambda with regular java/node

viesti07:10:02

repl makes a difference

cgrand07:10:33

we will have to get rid of carbonite at some point

baptiste-from-paris07:10:42

why that ? it’s just a kryo wrapper for clojure datastructures right ?

cgrand07:10:06

because for some clojure stuff we need our own custom serializer and in such cases we clash with carb

cgrand07:10:21
replied to a thread:ok, nice

the plan is to have pk to upload deltas on change to any var used by the published ones Batching would be achieved by having tests ran before deploying, if tests fail no deploy

cgrand07:10:42

@baptiste-from-paris I’m rather bad at faces, have we ever met at the paris clojure group?

cgrand07:10:27

so the issue with clj-http.core/request is a kryo one, and trying to fix it (by working around carb) I trigger another one...

baptiste-from-paris08:10:18

nope, I discovered clojure 1 year ago

baptiste-from-paris08:10:32

but i am part of the clojure paris group

baptiste-from-paris08:10:52

we just had a brainstorming to 3 weeks ago

baptiste-from-paris08:10:10

trying to make Clojure just a little more popular in france !

baptiste-from-paris08:10:03

by the way if one of you want to come to paris to talk about portkey it would be awesome

baptiste-from-paris08:10:44

I am sure we can make it all inclusive with the Clojure user group

cgrand08:10:29

back in july (iirc) I chatted with @hmadelaine about “monter à la capitale” (going to the big town) to talk about either portkey or unrepl

baptiste-from-paris08:10:08

when ever you want, we plan to make 6 events this year

cgrand08:10:35

monthly was too much of a strain?

baptiste-from-paris08:10:38

which is quite hard, there are not a lot of people willing to talk ^^

baptiste-from-paris08:10:51

yes, not enough speakers

baptiste-from-paris08:10:21

and lot’s of people who are really new to the language

baptiste-from-paris08:10:48

I bet there are less than 5 companies using clojure in France

baptiste-from-paris08:10:57

I really want to increase this number

cgrand08:10:41

we need to put parens in tap water like they do in Finland 😉

cgrand08:10:46

parenthesis

cgrand08:10:06

the uptake of Clojure there is phenomenal

baptiste-from-paris08:10:27

sometimes it’s only a matter of few people/companies that make other people choose a tech

baptiste-from-paris08:10:55

it really amaze me how tech companies choose their stack

baptiste-from-paris08:10:28

I worked for Meetic (Match group); we always did what competition were doing

cgrand08:10:15

Imitation and belief in magic bullet/pixie dust.

cgrand08:10:53

However we (collectively as devs) don’t always make rational choices

baptiste-from-paris08:10:10

Offcourse! We are humains ^^

baptiste-from-paris08:10:15

But I think it can be great to have more clojure enthousiast in France

baptiste-from-paris08:10:31

At least just to inject more money in the clojure ecosystem

baptiste-from-paris08:10:07

So that a 28/30 young dev can work on a project like portkey because he already "master" clojure

baptiste-from-paris08:10:15

do you often come to paris @cgrand?

cgrand08:10:32

no — for 1.5 year I contracted for SACEM and was in Paris 2 days a week, this ended 2 years ago and since then I didn’t have the opportunity to go to Paris. It has been too long of a time, I should plan a visit to the user group.

cgrand08:10:27

but currently I’m looking into going further in the country side, maybe I should open a (clojure) hackers retreat 🙂

baptiste-from-paris08:10:59

saint etienne right ?

viesti09:10:39

the uptake on Clojure in Finland is probably most visible in ClojuTre, this year and last year attendees in the ~300 range

baptiste-from-paris09:10:24

And how many from Finland itself ?

cgrand09:10:49

350 this year, 2/3 Finns I guess

cgrand09:10:50

to put it in perspective: same size as EuroClojure

cgrand09:10:32

it’s a free conference

viesti10:10:16

this year there was a voluntary 50e patron fee

viesti10:10:26

but there are quite many sponsors

viesti10:10:46

which tells the story of Clojure being commercially in use

viesti10:10:27

and we have sauna 🙂

baptiste-from-paris10:10:16

we have great wine 😉 🍷

baptiste-from-paris10:10:09

UCInterpreter transform clj to .class right ?

baptiste-from-paris10:10:08

I don’t understand also why use vars & classes in here =>

(defn freeze [x]
  (let [classes (atom #{})
        vars (atom #{})
        kryo (mk-kryo)
        baos (java.io.ByteArrayOutputStream.)]
    (with-open [out (com.esotericsoftware.kryo.io.Output. baos)]
      (.writeClassAndObject kryo out x))
    (.toByteArray baos)))

cgrand11:10:31

because of the deps tracing

baptiste-from-paris11:10:13

and from what I understand, it might cause problem according to kryo doc

[com.esotericsoftware/kryo "4.0.0"]
                 [com.twitter/carbonite "1.4.0"
                 :exclusions [com.esotericsoftware.kryo/kryo]]
because carbonite use kryo 3.*

cgrand11:10:57

carbonite will be gone by the end of the week

cgrand11:10:20

Deps tracing/tree-shaking

cgrand11:10:41

One starts from a value (a closure), we serialize it with kryo.

cgrand11:10:06

To serialize, Kryo performs a traversal of the object graph rooted at our initial value.

cgrand11:10:55

As kryo walks the graph we log all classes and vars found. (that’s what you see in freeze)

cgrand11:10:28

so now with have bytes (the frozen value) and some deps (classes and vars)

cgrand11:10:48

we add the classes to the package (and we add also deps of deps, that’s UCInterpreter/analysis job, vars and resources may be found too)

cgrand11:10:55

then we add resources

cgrand11:10:18

then we deref all vars encountered, and we serialize their values

cgrand11:10:26

(with kryo)

cgrand11:10:48

serialization produces more classes and vars

cgrand11:10:59

etc. etc. until nothing new is found

baptiste-from-paris11:10:30

how did you come with this strategy ^^ ?

baptiste-from-paris11:10:36

and in which phase asm generates .class ?

cgrand11:10:13

when you eval code at the repl clojure does generate classes (in memory)

cgrand11:10:44

we use ouroboros to retrieve them (or any other) from the JVM

baptiste-from-paris11:10:02

let me summarize the problem(s) portkey is trying to solve 1) bring clojure repl to lambda 2) minify/deploy clojure code -> bytecode -> package -> aws lambda

cgrand11:10:30

that’s correct but 2) is somehow necessary to make 1) useful

baptiste-from-paris11:10:42

and I guess the hard part is serialization and handling clojure’s data structure ?

cgrand11:10:58

not that much, the hard part is class inspection