Fork me on GitHub
#clojure
<
2019-09-29
>
jetzajac01:09:47

Hi everyone, here is our work on startup time imporvements: https://github.com/zajac/clojure Basically it is Rich’s branch fastload deconflicted and applied to master + some invokedynamic to eliminate the cost of additional volatile check on every var deref. Also there is an option to eliminate creation of vars for static functions (via meta on ns decl).

(ns ^:lean-ns my-ns) 
Namespaces marked with this meta couldn’t be refered, one should use regular require for them. Vars will be created lazily on resolve @alexmiller

Alex Miller (Clojure team)03:09:38

The problem with this stuff is it only helps when you’ve aot compiled, but many of the use cases where you want startup improvements are dev-time when you typically are not doing aot

Alex Miller (Clojure team)03:09:41

There are cases where it matters (aws lambdas for example), so it is useful, just not sure this is addressing the primary problem use case

jetzajac12:09:31

Well, I’d put it another way. There is a price for fast startup. AOT is inherently a price one MUST pay to achieve that because compilation + classloading can not possibly be as fast as bare classloading. But however even doing that you receive not much of it. Fixes like this are just making AOT more effective in achieving startup performance.

datran03:09:36

Does anybody have a recommendation for sha256 hashes? Just go to java or is there a clojure wrapper somewhere?

seancorfield03:09:58

@datran At work we've never bothered with wrappers, we've just used the Java security libraries via interop. That's the "Clojure way" really.

datran04:09:36

Thanks, I'll check out both the clojure wrapper and the java.security.MessageDigest stuff

pinkfrog04:09:40

can i store large blobs inside datomic?

mpenet06:09:42

I dont think you can store blobs on cloud

kulminaator06:09:54

:thinking_face: what would be a good way of passing clojure code from application A to application B (for plain boring data processing like map-group-reduce) ... the preface is that A is sort of a master application and there's an army of B's as minions and instances of B have access to data to process

seancorfield15:09:26

@U6MHHF36J either EDN or Transit would be good formats for that I think.

kulminaator18:09:55

i am not really sure how edn would help me here ... then i'd need some clojure code to edn translator (and back...)

kulminaator18:09:47

@alexmiller you know things, if you have some good pointers i would be very thankful 🙂

Alex Miller (Clojure team)18:09:59

transit is designed for this

Alex Miller (Clojure team)18:09:35

or reading more closely, if your goal is to pass code, then that's kind of a specialized use

Alex Miller (Clojure team)18:09:48

depends a lot on what level of control you have over the nodes

Alex Miller (Clojure team)18:09:39

like, can you guarantee what's on their classpath, are the tasks static (but parameterizable) or dynamic etc

seancorfield18:09:20

@U6MHHF36J EDN is natively supported in Clojure -- clojure.edn is a built-in namespace. It's "enhanced JSON", essentially, and good for Clojure/Clojure/ClojureScript communication. Transit is good too but requires external libraries. (right, @alexmiller?)

seancorfield18:09:49

Nearly all Clojure source code can be passed via EDN?

kulminaator18:09:42

yes, i want to pass code

kulminaator18:09:10

only very simple code will be allowed, nothing from external libraries 🙂

kulminaator18:09:13

it will be safe to assume that anything that A has on it's classpath B also has

kulminaator18:09:50

technically i could send over also a string and bravely claim that the other side can just load-string it and assume it's a function declaration ... but it somehow feels wrong

kulminaator18:09:26

e.g. something like this would happen on the B side

user=> (def zz (clojure.core/load-string "(fn [i] (inc i))"))
#'user/zz
user=> (zz 3)
4

Alex Miller (Clojure team)19:09:35

As long as you understand the security implications of loading and running arbitrary strings...

kulminaator19:09:47

i can protect B in a way that it only allows execution from A (or even additionally signed by A) to be handled

kulminaator19:09:52

if you know how spark or flink clusters work, i would like to do something similar, just way simpler (and proportionally less capable). there the masters just hand out work to workers saying (do operation X to data you can load at Y, and tell me back what you got). master's work is just to create equal work partitions and launch enough workers (and make sense of the multiple responses returned in the end).

kulminaator20:09:45

poked around a little bit , came up with this , good enough for start https://pastebin.com/E173tz9v

kulminaator06:09:35

if datomic runs ontop of postgresql too then postgresql itself can definitely handle your blobs 🙂

mpenet07:09:04

I think it makes little sense, that's probably why it's not in cloud

mpenet07:09:47

But I am not a datomic user. Maybe people in #datomic can help

kulminaator07:09:49

sort of agree for very large blobs - conventional database systems are not ideal to hold them

kulminaator07:09:19

for what i use in my own back pocket for those -> big stuff goes into aws s3 and database records just point to their location

theeternalpulse08:09:41

when you alias a .core namespace do you alias :as c or :as core or :as <whatever the namespace is before .core>?

dominicm13:09:37

Stuart Sierra has a blog post on this

alexyakushev10:09:40

Hi everyone! Does anyone know if it is possible to pretty-print Clojure code with both fills (block tries to fill chunks till the end of the line) and with correct indentation (arguments are aligned to the function name)?

alexyakushev10:09:13

Maybe there is a library that can do that?

alexyakushev10:09:37

Clojure's pretty printer implementation claims to be similar to one in Common Lisp, but Lisp's printer actually does this stuff correctly.

magra10:09:47

Hi, on a relatively small server backend app I get an uberjar over 50 mb. clj -Stree on the project shows that dependencies, that support both, clj and cljs pull in clojurescript and the whole ecosystem including google compiler. The backend uses only clj on the JVM, no node stuff. Is there a way to build an uberjar on a deps.edn based project that does not include the cljs and js stuff? Any hints on reading material or docs is welcome.

seancorfield15:09:35

@magra depends on which build tool you are using but :exclusions is probably going to help you in your dependencies.

dpsutton16:09:32

Also putting deps into different profiles can help you isolate things that don’t need to be on jvm

seancorfield17:09:39

That is why depstar only uses what it finds on the classpath -- it assumes you've used whatever aliases you need to get the right stuff for your app to run in production.

creese23:09:40

Has anyone generated a list of which transducers are stateless and which as “stateful”?

creese23:09:32

I can look at the source and if I see a volatile I know it’s “stateful”. However, some of these call out to other functions so proving the reverse isn’t straightforward.