Fork me on GitHub
#portkey
<
2017-07-13
>
cgrand21:07:18

Hi Daniel!

cgrand21:07:27

@spieden As you may have noticed portkey is very very young. We share the goal of “transparent local dev -> serverless deployment”

spieden21:07:17

i’d been holding back on figuring out much of what you guys have well in progress

cgrand21:07:21

startup latency: we haven’t worked on it so it’s the usual sluggish startup

spieden21:07:40

for my use case it really doesn’t matter, as step functions state machines are long running

spieden21:07:30

i’ll try to get some experimenting in soon =)

cgrand21:07:41

we have a tree-shaker, for now it doesn’t cover clojure.jar, but with time, love and care it could be robust enough to remove unneeded stuff from clojure and yields fastest startups

cgrand21:07:34

if you wat to experiment you’ll have to dig in the code to use deploy! rather than mount (which has api gateway stuff)

spieden21:07:17

so the tree shaking only covers the clojure code i’m guessing — are all bytecode jars on the classpath included?

cgrand21:07:53

basically kryo and clojure are left untouched (plus a couple of classes and namespaces), everything else is shaken

cgrand21:07:01

@viesti loves to tree-shake hadoop parts

cgrand21:07:02

Currently the tree-shaker has a bug on genclassed classes

spieden21:07:39

hadn’t encountered kryo before. only JVM slimming i have previous experience with is a little ProGuard

viesti21:07:08

so have been tossing at @cgrand this puzzle of a parquet->avro conversion which shrinks from 90MB jar to 5MB, but as said, there are some things to cover still :)

viesti21:07:04

loading resources/classes based on runtime information is the basic dilemma, more actual use of portkey would help to see what is most important to cover

spieden21:07:50

so code’s being run out of these kryo files vs .class files?

cgrand21:07:14

Compared to ProGuard which shake classes, we shake an object graph rooted at the closure we want to deploy to Lambda

cgrand21:07:32

so we have to send two things: classes and actual instances

spieden21:07:39

hmm, your portkey.core/as-doto* is very similar to our bean-dip lib except w/ reflection =) https://github.com/uwcpdx/bean-dip/

cgrand21:07:03

spieden: it’s only called during macroexpansion and that’s true than we could certainly get away without reflection at all

spieden21:07:01

ah i see. i didn’t think of that approach for some reason — just went purely on naming convention and let *warn-on-reflection* catch missing methods

spieden21:07:33

simpler implementation but same perf i guess

spieden21:07:11

not much room for smarts in my soln though

spieden21:07:32

yeah vars are objects

cgrand23:07:22

The restriction to vars is specific to mount and temporary. deploy! can turn any function even a closure into a lambda.

spieden18:07:29

wow cool. except i suppose for stateful things like database connections, etc.

cgrand20:07:41

Indeed IO channels don't serialize well.

cgrand21:07:32

so kryo is used to serialize instances

cgrand21:07:09

starting from a live graph of objects gives us an edge over ProGuard (especially when it comes to clojure code) as we have more information

cgrand23:07:22
replied to a thread:yeah vars are objects

The restriction to vars is specific to mount and temporary. deploy! can turn any function even a closure into a lambda.