Fork me on GitHub
#clojure
<
2022-07-14
>
qqq05:07:59

Does either Lumo or Planck run in Chrome browser? I'm looking to build a website where users (without having JVM / Clojure / Figwheel installed) can load up a website, type in CLJS code, and have it (possibly compiled) and executed.

JohnJ05:07:26

Look at self-hosting clojurescript

mfikes11:07:31

An example of such a thing is https://clojurescript.io

qqq16:07:39

@U04VDQDDY: http://clojurescript.io / replumb looks great; thanks for sharing the link. Is there a benchmark somewhere of replumb vs (cljs on jvm) performance ? In particular, I'm curious if (1) this is interpreted or (2) compiled. In the case of (1) interpreter slowdown and , in the case of (2), compile time.

mfikes16:07:57

Self-hosted is compiled

mfikes16:07:09

The self-hosted compiler is slower than the JVM compiler

mfikes16:07:30

I can't recall if actual perf numbers are recorded anywhere. YMMV

qqq16:07:49

@U04VDQDDY: Follow up dumb question -- why does https://github.com/arichiardi/replumb not have updates for past 5 years? Has the CLJS community moved on to something else, or has this software somehow magically designed it's API in such a way to not need updates for the past 5 years ?

mfikes00:07:26

Well, the only thing that I would think to update with Replumb is the ClojureScript compiler... not much else to update in it. Similar thing is true for https://replete-repl.org which was one of the first (if not the first) self-hosted REPL. Once Replete was created 7 years ago (https://blog.fikesfarm.com/posts/2015-06-27-replete-a-standalone-ios-cljs-repl.html), its relatively simple UI has been stable since.

pinkfrog06:07:27

Is print-dup ever used? Looking at the source: https://github.com/clojure/clojure/search?p=1&amp;q=print-dup. there is very few usage.

p-himik06:07:52

You can still see on that page that it is indeed used. Also note that GitHub search results don't include more than 2 lines from the files where the search query is found.

pinkfrog06:07:55

Can you give me a concrete example that it is used, for real?

p-himik06:07:39

Right there in the search results.

hiredman06:07:10

The compiler has to be able to serializing some what arbitrary data, to be stored in a class file, and then reconstructed when that class file is loaded or the bytecode in it is run, it has special handling for most of the built in data types in clojure, but falls back to print-dup when it encounters something unknown

pinkfrog11:07:20

In simple words, print-dup is solely for serialization and print-method is just for human readability. However, the result of print-method might be the same as print-dup sometimes. I used to serialize edn content with (pr-str) ( where by default *print-dup* is false), is this a valid way to serialize edn?

Joshua Suskalo13:07:06

honestly the biggest difference between print-dup and print-method from usage that I've seen is that print-dup must be able to recover something equal to the original object by reading the structure assuming the code for it is loaded. print-method may produce something that requires data readers to be read.

Joshua Suskalo13:07:37

(and the occasional unreadable object like with functions and java objects)

pinkfrog14:07:53

@U5NCUG8NR, Say if I define a custom type. Should I write my own print-dup function? Will there potentially any core lib that indirectly invokes my print-dup function? I am not quite sure how the print-dup function is used. Other than I do a ( binding [print-dup true] (pr something) myself and read it back.

Joshua Suskalo14:07:59

So if you introduce your own print-dup impl the primary use for that is unsafe wire protocol (prefer clojure.edn and a data reader), and to allow people to have your new data structure be part of the source code emitted from a macro. At least to my current understanding.

Joshua Suskalo14:07:56

And a key point here is this would be like having a vector in your source code, where the data structure is actually part of the syntax, not just constructed in the resulting code.

Joshua Suskalo14:07:01

a print-dup method may also be important for your data type if you have a data-reader literal to allow you to have data structure literals like this in your source code

pinkfrog14:07:55

> a print-dup method may also be important for your data type if you have a data-reader literal to allow you to have data structure literals like this in your source code print-dup is on the producing side, while the data-reader literal is on the consuming side. For example, one can directly write #inst “some-time”, which totally bypasses print-dup. The confusion of me is “Where does print-dup gets used?“. Even clojure.edn/ is totally on the consuming side, it only contains the “read” and “read-string” function.

Joshua Suskalo14:07:58

So I'm not 100% sure about this, but my understanding is that when you have a data reader and you compile code, then you have a literal object of the type it got read in as in your code. In order for an object to be stored in the produced class file it must be printed with print-dup . This means that types which have data readers should also have print-dup.

pinkfrog11:07:54

Is there any doc on different compiler phases, e.g., read time, macro expansion time? Also, if I directly run files with clj xx.clj, will the bytecode of xx.clj file be generated?

Ben Sless11:07:47

Well, it's pretty much read then eval, where prior to eval macro expansion is performed

pinkfrog11:07:17

I came across things like #= macro.

pinkfrog11:07:04

And also in the above thread (the one right above), @U0NCTKEV8 said some compiler treatment on print-dup. I am not sure how this is related to compiler. I thought it is more related serialize data to some file on disk and read it back, which has not much to do with the compiler. The compiler might only see some reader tag function to tell how to recreate some object, that

pinkfrog11:07:07

that’s it.

Ben Sless11:07:23

right, I didn't mention reader macros, which start with # and are expanded at read time

Ben Sless11:07:51

they include stuff like #{} for sets, #_ for comments, and #= for evaluation at compile time

Ben Sless11:07:43

so you can write something like (defn foo [x] (+ x #=(+ 1 2))) and it will evaluate (defn foo [x] (+ x 3))

Ben Sless11:07:37

There's also some bootstrapping involved where the reader relies on other functions being defined to fully work, which is why macros aren't implemented with syntax quote at the beginning of clojure.core but are later

pinkfrog12:07:02

I can understand the compiler transforms #=(+ 1 2) to 3. But what does #{1 2 3} be transformed to by the compiler?

pinkfrog12:07:24

Will #{1 2 3} be turned into some list form, e.g., (hash-set 1 2 3) (I doubt not), or some concrete hash-set object? If the latter, what happens for #{a 1 2} where a is unknown during the compilation time, e.g., a might be a parameter of a function.

pinkfrog12:07:50

> right, I didn’t mention reader macros, which start with # and are expanded at read time. @UK0810AQ2 I feel like for things #{a 1 2}, the evaluation happens at run time.

Ben Sless13:07:21

It isn't, the expansion is at read time. #= expands to eval, while a set, depending if all arguments are statically known, will compile to a static set or the hashset constructor

didibus20:07:41

> {a 1 2} where a is unknown during the compilation time It becomes (hash-set a 1 2)

didibus20:07:22

You can see it by doing:

(read-string "`#{~a 1 2}")

;> (clojure.core/apply clojure.core/hash-set (clojure.core/seq (clojure.core/concat (clojure.core/list 1) (clojure.core/list a) (clojure.core/list 2))))

pinkfrog13:07:00

I wonder why “a” is printed instead of 3?

(def a 3)
  (read-string "#=(println a)")
However, if I make the following contrived example
(read-string "#=(a)")
Then clojure errs with class java.lang.Long cannot be cast to class clojure.lang.IFn . So it indeed knows the value of a.

Martin Půda13:07:44

https://clojuredocs.org/clojure.core/*read-eval* See note here, mainly the part starting with "As you can see, everything after the first symbol is left untouched."

pinkfrog14:07:04

Thanks. That’s quite mysterious.

Joshua Suskalo14:07:33

The main use for this reader syntax is for print dup, where all values are already evaluated. Doing it this way means print dup implementations don't have to be careful when they contain symbols.

Joshua Suskalo14:07:06

If you must refer to a value by a symbol like this, use #=(eval (...))

👍 1
mwolf14:07:53

clojure.core.async questions: thread creates a lightweight thread, correct? so I can create a zillion of those without the OS sh_tting the bed, right? and if I'm not reading the result from the channel returned by thread I should code (close! (thread (do-something-useful))) ... or will channels be cleaned up when garbage collected? (i.e. they're just a data structure and don't have any persistent resources that need to be cleaned up like an OS pipe/file handle.) (same would apply to go I assume.)

Alex Miller (Clojure team)14:07:49

channels are gc'ed when they are no longer referred to

Alex Miller (Clojure team)14:07:17

if you're using go blocks, they are multiplexed over a fixed pool of 8 threads (by default) so you can have a zillion go blocks

👍 1
Alex Miller (Clojure team)14:07:38

there are no lightweight threads in the jvm (yet - see Project Loom)

mwolf14:07:41

OK -- go blocks will work. I'm processing an event feed and the events can be handled in parallel, but all update the same data structure (which uses refs to handle updates) -- was worried about overloading the OS if there are thousands of events in flight.

Joshua Suskalo14:07:59

The bigger problem here is that you shouldn't block execution in a go block

Joshua Suskalo14:07:27

So the previous conversation about print-dup has sparked some interest for me in a question about the intended semantics of what's produced by print-dup. I have a custom data type called a rope, and it has a print-dup implementation (and a print-method which produces a tagged literal that can have a data reader function of just ropes.core/rope), but it seems to act significantly differently when round-tripped and passed to eval than normal data structures (see below). My question is this: how can I make it so that a rope behaves in code the same way as other data structures? From my testing it seems like I would need to make it so that there are side effects to evaluating it, like it evaluates to a new rope with all the elements evaluated. How would I go about doing that?

(-> (binding [*print-dup* true]
      (prn-str {'a 'b}))
    read-string
    eval)
;; Syntax error compiling at (*cider-repl Clojure/ropes:localhost:38147(clj)*:40:13).
;; Unable to resolve symbol: a in this context
(-> (binding [*print-dup* true]
      (prn-str (rope ['a 'b])))
    read-string
    eval)
;; => #rope [ a b ]

Joshua Suskalo14:07:59

I guess by the end of this it's more like a question about eval than about print-dup, but the question stands.

nwjsmith14:07:46

What happens when you (prn-str (rope [1 2])) instead?

Joshua Suskalo14:07:57

it gives a rope with the numbers 1 and 2. Numbers don't need evaluation to have the correct value.

nwjsmith15:07:37

Sorry, I misunderstood the problem. Digging into your example, what's the value of (binding [*print-dup* true] (prn-str (rope ['a 'b]))) ?

nwjsmith15:07:18

If it's #rope[a b] you're good (I think 🤞):

user=> (defn rope [xs] xs)
#'user/rope
user=> (binding [*data-readers* {'rope rope}] (read-string "#rope[a b]"))
[a b]

Joshua Suskalo15:07:21

Yeah, the problem is that (eval #rope [a b]) (with appropriate data readers) is #rope [a b] , it doesn't try to evaluate a or b

Sidestep14:07:33

hi lazy web

Sidestep14:07:46

I think I saw a tweet somewhere

Sidestep14:07:08

that babaska can invoke bash commands directly somehow?

nwjsmith14:07:34

probably the spot where you'll get the best help

1
teodorlu14:07:00

I commonly use this thing:

(defn bash [cmd]
  (str/trim (:out (clojure.java.shell/sh "bash" "-c" cmd))))
example invocation:
(->> (bash "ls **/play.edn | sed 's|/play.edn||g'")
       (str/split-lines)
       (map (fn [id] {:id id})))

Sidestep14:07:22

full amalgamation of bash and clojure

dpsutton16:07:39

Does anyone have any tips for how to diagnose initialization errors? > Could not initialize class com.google.cloud.ServiceOptions

qqq16:07:49

https://github.com/arichiardi/replumb , first search result for "replumb" , seems to not ahve any updates in past 5 years. Is there a more recent take on "self hosted cljs" ?

pavlosmelissinos16:07:56

Not sure what "self hosted cljs" means. What exactly are you looking for?

pavlosmelissinos16:07:14

Oh like http://replit.com but self hosted and just for clj/cljs?

shaunlebron16:07:48

Is there any way to use a local checkout of org.clojure/clojure in a lein project? (I’m trying to debug what the compiler is doing during an AOT problem). Clojure’s repo has no project.clj file so lein refuses to use it as a local checkout.

p-himik17:07:25

IIRC you can't really do it, or at least not that easy at all. Clojure JAR is AOT-compiled and that compilation is tricky because there's spec which depends on Clojure which depends on spec. FWIW, Cursive is able to debug Clojure itself just fine, I've done plenty of sleuthing that way.

👍 1
hiredman17:07:19

I hesitate to ask, but what aot problem?

shaunlebron17:07:32

@U0NCTKEV8 I ran compile on a single namespace, and curiously a non-aot'd namespace cannot refer to it anymore

hiredman17:07:24

compile basically does a force reload of the namespace you pass it

hiredman17:07:20

compilation is a side effect of code loading, so to compile a namespace compile has to load it again, and compile doesn't check the set of already loaded namespaces (like require does) which would stop it from compiling already loaded namespaces

shaunlebron17:07:24

I mean, I checked that the compiled class file exists for the namespace I ran compile on, but when restarting the clojure app the namespace depending on it cannot find it

hiredman17:07:18

ah, I assumed you were in the same repl where you can compile still

hiredman17:07:39

so you are getting a class not found error?

shaunlebron17:07:27

:cause No namespace: gambit.lib.async-buffers
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message Syntax error compiling at (gambit/common/enduro_utils.clj:1:1).
   :data #:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source gambit/common/enduro_utils.clj}
   :at [clojure.lang.Compiler load Compiler.java 7652]}
  {:type java.lang.Exception
   :message No namespace: gambit.lib.async-buffers
   :at [clojure.core$refer invokeStatic core.clj 4222]}]
 :trace
 [[clojure.core$refer invokeStatic core.clj 4222]
  [clojure.core$refer doInvoke core.clj 4205]
  [clojure.lang.RestFn applyTo RestFn.java 139]
  [clojure.core$apply invokeStatic core.clj 669]
  [clojure.core$load_lib invokeStatic core.clj 5955]
  [clojure.core$load_lib doInvoke core.clj 5917]
  [clojure.lang.RestFn applyTo RestFn.java 142]
  [clojure.core$apply invokeStatic core.clj 669]
  [clojure.core$load_libs invokeStatic core.clj 5974]
  [clojure.core$load_libs doInvoke core.clj 5958]
  [clojure.lang.RestFn applyTo RestFn.java 137]
  [clojure.core$apply invokeStatic core.clj 669]
  [clojure.core$require invokeStatic core.clj 5996]
  [clojure.core$require doInvoke core.clj 5996]
  [clojure.lang.RestFn invoke RestFn.java 805]

shaunlebron17:07:48

it's a no namespace error inside refer

hiredman17:07:05

I would check to make sure you namespaces and file names all match up

hiredman17:07:33

I believe you get an error vaguely like that if the namespace a file has the ns decl at the top doesn't match the path (from the classpath root) of the file

shaunlebron17:07:36

Yeah i think they match up because when i delete the compiled class file it works

hiredman17:07:12

how are you calling compile? are you calling the function directly?

shaunlebron17:07:03

i'm not sure how else to compile except to run e.g. (compile 'gambit.lib.async-buffers)

hiredman17:07:34

well you could be using lein, or tools.build or maven even

hiredman17:07:54

what is the state of the repl where you call compile? do you have a user.clj?

shaunlebron17:07:22

yeah I run inside a comment block inside a user.clj

shaunlebron17:07:36

there's probably a ton of questions needed answering here lol

hiredman17:07:48

get rid of the user.clj

hiredman17:07:38

like move it off to the side with a different name, start a new repl, compile again and then see what happens

shaunlebron17:07:39

this is a giant project, and a lot of initial loading of everything happens in user.clj

hiredman17:07:43

any code the user.clj causes to be loaded will be part of the compilation enviroment when you call compile, which can do funky things, even mask problems with compilation

hiredman17:07:56

you really don't want that

shaunlebron17:07:39

okay I’ll try it, thanks

hiredman17:07:16

the ideal scenario for compilation is you have a sort of main namespace, and that requires everything it needs. so you call compile on that namespace, and it and all the namespaces it loads and so on get aot compiled when you do that

shaunlebron17:07:25

alright, i put the offending require statement in the main namespace, and cut out the user.clj calls, and it worked

hiredman17:07:45

basically the loading when you do the compiling matches the regular loading of the code

shaunlebron17:07:34

our main namespace is calling out to user/reset which uses tools.namespace to load all our namespaces for some reason

hiredman17:07:39

when they differ (different code loading orders, different sets of code being loaded) you often have problems

hiredman17:07:16

oof, sounds terrible

shaunlebron17:07:34

i guess it's a mechanism for reloading all the namespaces after a new checkout

hiredman17:07:54

reloading stuff while aot compiling is going to result in oddities too

shaunlebron17:07:14

thank you for clearing up that the problem might be with the way we're dynamically loading our namespaces, and with user.clj. i was hoping for a more clear view of what the compiler is doing, and now I’m more confused than when I started I think

hiredman17:07:44

the compiler is basically banging two rocks together to make fire

parens 2
shaunlebron17:07:39

i hesitate to ask you to explain that lol

hiredman17:07:09

like, it is extremely primitive, clojure code is always compiled to java bytecode, when you aot compile it basically just sets a flag that says "while you are loading this code, generating bytecode, and then executing that bytecode, also write the bytecode to disk"

shaunlebron17:07:39

that I get, I’ve been walking through clojure's compiler.java a bit

hiredman17:07:28

the fact that the aot compilation process is also running the code as it is compiled makes it very easy to perturb

shaunlebron17:07:12

well actually I never recompiled the classfile for the namespace to make it work

shaunlebron17:07:50

so the problem is with the tools.namespace.repl/refresh or whatever not finding it

hiredman17:07:26

and because you do it from the comfort of a running clojure system, that running clojure system is part of the compilation environment for the code and will effect how it is compiled

shaunlebron17:07:34

but I thought tools.namespace was just a wrapper around require :reload

hiredman17:07:51

tools.namespace has a lot in it

shaunlebron17:07:07

I understand in theory that the compiler is stateful, but I thought that would be protected, unless all the namespace tables are fragile in a way I dont understand

hiredman17:07:31

it is all very fragile

hiredman17:07:44

I would definitely recommend your main namespace not depend on a user.clj and not call functions in that, and not call any functions that reload everything

hiredman17:07:39

part of this has to do with the transitive nature of compilation

hiredman17:07:39

so if you call compile on namespace A, the compiler will aot compile all code loaded when A is loaded, but if you have a user.clj that has already loaded everything, then A's dependencies won't be loaded again (assuming they are loaded via require) which means they won't get aot compiled, which can cause problems

👍 1
shaunlebron17:07:59

main doesnt require user.clj as normal, but detects when we're in development mode then does a hard (require 'user)

shaunlebron17:07:30

would anything be different if I just renamed user.clj to foo.clj

hiredman17:07:04

if you renamed it to foo.clj and didn't load foo.clj, then yes

shaunlebron17:07:56

okay, so it's more the fact that it's being loaded and not some intrinsic effect of this being a specially treated user namespace

shaunlebron18:07:23

oh, you’re saying compile on a namespace whose dependencies may already be loaded may cause problems

hiredman18:07:33

and even within A, if A is already loaded, then you compile A, the compiled version of A could end up with some kind of dependency on the state of the already loaded version of A, which of course won't be there when you try to load directly from the class files

shaunlebron18:07:28

so I should compile namespaces before loading them somehow

shaunlebron18:07:46

separate from the comfort of my already loaded app

hiredman18:07:40

a tool like lein or tools.build or maven can assist with this kind of thing, but you can also bypass their assistance by having a user.clj in the wrong place, since they cannot stop clojure from automatically loading your user.clj

👍 1
shaunlebron18:07:20

in theory then, a properly generated classfile based on a “clean” compiler state should be loaded fine when restarting my app, even with all the weirdness of how our namespaces are loaded via tnr/reset

hiredman18:07:35

reset likely does a scan for changed clojure files which may break if you are doing it unconditionally

shaunlebron18:07:28

well what confuses me is that trying to walk through this stacktrace, is that (require '[A :refer [foo]]) seems to try to run refer before it even gets to the point where the classfile can be loaded (in RT/load)

hiredman18:07:48

I am not sure what version of clojure you are using so I haven't tried to look at the line numbers

shaunlebron18:07:14

I’ve run out of time on this for today. Thank you so much brother for walking me through this

shaunlebron18:07:29

we’re on 1.10.3