Fork me on GitHub
#clojure
<
2023-09-25
>
Muhammad Hamza Chippa17:09:09

I'm writing a web app, where the UI (written in cljs) needs to communicate with a REPL session on the local host (for ad-hoc remote evaluations of s-expression). This results in the browser security kicking in and blocking communications to the localhost. What are some approaches to allow this?

borkdude18:09:02

I've got a similar setup for scittle + nrepl. Free free to look around in the code. https://github.com/babashka/scittle/tree/main/doc/nrepl

🙌 1
cjohansen19:09:21

Is there some API where I can get notified anytime code is evaluated in the REPL? Like some compiler APIs, tools.namespace or similar?

respatialized19:09:43

https://nrepl.org/nrepl/design/middleware.html For nREPL at least, you probably could use middleware for this purpose.

respatialized19:09:10

but I'm not sure if you need this capability to be implementation-independent or not. something using just core could potentially build on core.server also https://clojuredocs.org/clojure.core.server

cjohansen19:09:57

nREPL is probably a fine option, I'll have a look at that, thanks!

seancorfield19:09:03

I'm curious: what's the use case @U9MKYDN4Q?

cjohansen19:09:00

Developer tooling: When code is evaluated, I want to check if the currently browsed web page is affected, and if so, reload it.

cjohansen19:09:39

I have a solution that reacts to file saves, but want it to react to REPL-based workflows as well.

seancorfield19:09:51

Interesting... how do you determine that a given web page might be affected by a specific backend code evaluation?

cjohansen19:09:37

I keep the previously rendered version around in an atom. After re-evaluation, I re-render the page, and if the result is different: reload the page

cjohansen19:09:14

The web page requests an SSE endpoint with the current URL as a query parameter

cjohansen19:09:43

The pages in question are server-rendered HTML, btw

seancorfield19:09:43

Ah, I was wondering if you were trying to do dependency graph analysis to see if changing some.ns/foo might cause a page rendered by other.ns/handler to potentially change...

cjohansen19:09:23

Nope, just some wonderfully naive brute-force :face_with_cowboy_hat:

Jan Šuráň20:09:24

I was recently looking into some implementation details of Clojure's protocols and related stuff and I found a lot of legacy code that I think it should be cleaned up. Here are some examples - we can definitely let some things, use transducers,... - there is a huge amount of lazy code which is not necessary. Some of the code is used just to compile defrecord etc., which doesn't impact the performance that much (but will slow down the compilation), but some of the code is also executed at runtime, such as dispatch via protocols, which can be quite expensive.

Jan Šuráň20:09:17

Although some things are only used for compilation, I think we should keep that code clean - I mean, it doesn't necessarily always have to be at compile time - what if you wanted to compile something dynamically at runtime?

p-himik20:09:38

I can already see Alex and others asking whether you have actually profiled that stuff in a diverse set of real projects and not some toy code. :D

1
p-himik20:09:57

Transducers themselves aren't always faster than lazy seqs BTW.

Jan Šuráň20:09:11

Yeah I know it doesn't always improve the performance. But I think some of the code should definitely be improved.

seancorfield20:09:34

In general, with Clojure, we start with a real world problem statement and only make changes that solve a real world problem (just setting your expectations). This "legacy code ... should be cleaned up" is not a real world problem statement.

2
seancorfield20:09:18

Making any change is risky. Changing code that works is somewhat pointless, unless you have to make a modification to that code to solve a specific problem.

seancorfield20:09:03

There's also a risk in making a change "for performance reasons" that you'll change some current "undefined behavior" that lots of real world Clojure programs depend on. That has happened (more than once) during Clojure's history. Which makes the core team even more wary of "gratuitous" changes.

seancorfield20:09:28

(it happened in 1.12 Alpha 2 for example -- and was fixed in Alpha 4)

Jan Šuráň20:09:24

I know, but there is a lot of stuff that doesn't change any undefined behavior. For example this: (first (remove nil? (map impl (butlast (super-chain c)))))

seancorfield20:09:03

These changes are not going to be made just because someone thinks the code is ugly or perhaps nanoseconds less efficient than it could be.

Jan Šuráň20:09:16

As I said, a lot of the code itsn't going anywhere. It's just a lot of lazy operations that all get consumed in the end. And I don't think it's necessary.

seancorfield20:09:53

If you have an actual bug or a real world performance problem with plenty of analysis to back it up, the core team isn't likely to want to change working code.

seancorfield20:09:27

(which is my polite way of saying "you're wasting your time, I suspect" 🙂 )

Jan Šuráň21:09:17

It's not wasting time. It's improving the code for better performance. And it doesn't cost anything. I get it there is a lot of code which might be speculative, but there is also a lot of code that can be changed and it doesn't affect anything.

Jan Šuráň21:09:51

I mean, that it doesn't cause any harm and only makes the resulting code faster.

seancorfield21:09:01

It does cost. It costs time in analysis, testing, changing, verifying. Change is expensive.

💯 8
seancorfield21:09:19

Every change has a cost.

seancorfield21:09:02

As Eugene noted above, first off transducers aren't always faster than lazy sequence operations, second unless you have profiled this code on real world projects and profiled your changed code, and shown it passes the entire test suite, you're not going to get any traction on changing code that works.

seancorfield21:09:51

(and threads like this have a cost too, for everyone who reads it or gets involved 🙂 )

ghadi21:09:41

all that "inefficient" code is run once then cached

Jan Šuráň22:09:41

Sometimes it does, but I think in most of the cases it does not.

p-himik22:09:04

Can you give an example?

Jan Šuráň22:09:11

For example this piece of code, I think it could be improved a lot.

(-> (map #(if (var? (resolve %))
            (:on (deref (resolve %)))
            %)
         (keys impls))
    set
    (disj 'Object 'java.lang.Object)
    vec)

seancorfield22:09:55

@U02FT4WAC4U You're missing the point. Unless you have a real world problem here, the code isn't going to get changed.

p-himik22:09:00

I meant the "in most of the cases it does not [get cached]" part.

Jan Šuráň22:09:34

It's not that much about caching the result. If the code is one lazy call after other, it's just slow in general. Specifically to caching, (resolve %) doesn't get cached in my opinion. But that's a really small detail.

p-himik22:09:41

> If the code is one lazy call after other, it's just slow in general This statement is false. > (resolve %) doesn't get cached It's not relevant since the whole sequence is cached. That function never gets called again after that last call to vec.

Jan Šuráň22:09:07

@U04V70XH6 I get the point and I know it really isn't a priority. However, I think I might still be put as a low-prio task somewhere. I can make the changes, test and benchmark them and describe the impact - it just looks like it never has a chance of being applied, or maybe sometime?

seancorfield22:09:54

All changes to Clojure start with a real world problem statement. If you do comprehensive benchmarking of Clojure on a variety of real world protocol-based code and redo those benchmarks with a patched version of Clojure that also passes the entire test suite, and you can show significant performance improvements, then you have a real world problem statement.

👍 1
seancorfield22:09:05

We value stability here. Changing Clojure because some small part of its implementation "could be improved" is just not going to happen unless you can show that the proposed change actually solves a problem. This is why we can put alpha releases into production. This is why code written over a decade ago still runs unchanged. Stability is extremely important. So any change needs to be carefully evaluated and well-justified.

Jan Šuráň22:09:00

I understand that that is the greatest value of Clojure, and I love Clojure for being stable. I can create a version with the changes and then post some benchmarks.

seancorfield22:09:55

Suggestion: fork clojure, create a branch with your benchmarks and results and instructions on how to run them, create a new branch from that with your patches and the updated benchmark results. That will ensure others can verify the results on different platforms/JDK versions etc.

👍 1
lread22:09:58

@U02FT4WAC4U if you are going to do those benchmarks to learn and satisfy your curiosity, that's great, go for it! But if you are going to do them with the expectation of effecting change in the Clojure core source, you will probably be disappointed.

❤️ 1
clojure-spin 2
💯 2
seancorfield22:09:59

(and expect your benchmarks to be analyzed with a very critical eye by several people)

👍 1
Jan Šuráň22:09:05

@U04V70XH6 Sure, thanks for tip!

phill22:09:32

A great subject for an experiment! I remember Jozef Wagner wrote a thrilling series of essays about a bunch of what-if patches that he gathered in a project "Dunaj".

Noah Bogart13:09:47

https://github.com/frenchy64/clojure-local-dev is a great way to develop patches for clojure for personal testing

jaide21:09:54

Currently using Notion as a Zettelkasten-style system for permanent notes and references, going on year 3. While it works decently, it would be cool to have something more customizable, open-source, and better data-ownership. Was thinking could build a system around two directories: a notes directory and a references directory. Both would contain valid clojure files as notes, and there would be a routing layer to display an index, a search, mobile UI, graph view, and possibly an editor. Otherwise can be edited via neovim, emacs, vs code, w/e. An example note would be something like this:

(note
 (head
   (title "What is an elliptic curve?")
   (tags :question :work)
   (answered false))
 (links
  (reference 1234)
  (note      345)
  (project "20230924T003400"))
 (content
   (question "What is an elliptic curve in an encryption algorithm like ecdsa?")))
Might need a file watching service too to populate an xtdb or sqlite db for links and tags so they're easily searched from a web UI or cli without having to parse every note all the time. Goal would make the routing layer hit the db for the index view, then in a single note page, would either parse or provide defs for note, head, title, tags, etc... with dynamic loading. Images and attachments could also be included, but I think a web editor will make it a bit easier but still it would be nice if it could also be consumed in text format too. Expecting the routing layer to also parse the links as actual hyperlinks. Think there is any merit to this concept?

jaide21:09:05

Was also thinking it could be built incrementally, perhaps rendering raw text if a function doesn't exist. For example, ideally in the UI (question "some question") would get a pleasant HTML view, bigger text size, maybe a bit of decoration to visually denote questions from other types of content. This would let me mark structure, but treat it as text as a default so I can grow it with me but not always have to implement it immediately.

Samuel Ludwig21:09:27

I think this is definitely a project that would be great to work on; have you heard of Logseq? It's been my note-taking daily-driver for a while now, also an all-Clojure project!

shiyi.gu21:09:19

Same using Logseq here.

jaide22:09:18

Yeah I looked at it, wasn't quite what I wanted though

Nundrum21:09:23

What is it you want? I tried doing a zettelkasten setup for myself, but didn't keep it updated and eventually abandoned it. I think I don't have enough notes and such to warrant using Logseq or Notion. But sometimes the desire is there.

jaide21:09:06

I setup a zettelkasten inspired system: a notes, and references database with relationship fields between them and have been using it fairly regularly since 2020. The reason for wanting to create something comes down to: 1. It would be cool for this to exist as an open-source project vs relying on a closed-source product 2. I would like more ownership of my data. Especially adding more mileage to it, I'd be very saddened if I were to lose all that data. 3. Notion was a solid choice for quickly setting something up, trying it out, refining it plus really good mobile accessibility (great for those late-night ideas), but now I've got a better sense of the type of content I want to store and it would be nice to have more flexibility. For example, it would be cool to have a network graph kind of view to browse the notes, as well as the ability to share a cluster of notes with people via unique URLs. That could be built on-top of Notion, but given the above, something more custom might be worth it. 4. The big dream would be a system that supports taking loose, sketch notes I can take on a tablet, then open it on desktop to start turning them into permanent notes connected to other ideas and projects. Notion supports mermaid diagrams, but I don't find that as quick as just doodling them on my tablet

Nundrum21:09:31

I've been loosely doing this in Nextcloud. It's imperfect, but has been really helpful. It at least meets point 1 and 2.