Fork me on GitHub
#off-topic
<
2021-09-12
>
Max20:09:07

Not sure if this is off-topic enough for #off-topic, but it didn’t seem to fit well in other channels either. ¯\(ツ)/¯ Is there any prior art for running a clj (backend) and cljs (frontend) project out of the same repl? How would you approach doing that? It seems like an attractive option if you have shared code between both projects, but it doesn’t seem to be a configuration supported by modern cljs tools (e.g. shadow), even though cljs repls have to be spawned from clj ones.

respatialized20:09:58

It's not clear to me what the benefits of a single REPL would be. At runtime, these will be distinct processes, so why try to run them out of a single process at dev time? It seems like that's hiding the actual complexity that you'll have to deal with. Are you just trying to keep things consistent across the processes? If you wanted to make sure that your namespaces/functions in .cljc are always up to date, you can use the configuration of each individual REPL and your editor to re-evaluate the file when it's saved. Figwheel supports this: https://figwheel.org/docs/hot_reloading.html

p-himik20:09:03

In fact, using the same process is straight up detrimental as it would mean having the same classpath. Which, of course, doesn't make sense as your backend has different dependencies from your frontend.

p-himik20:09:25

I used to do that by utilizing shadow-cljs functions - basically doing the same what its CLI does but by calling its functions directly. And after about a year I switched back to running separate processes. Having a single process is just not worth it.

vemv20:09:58

@U01EB0V3H39 isn't this what piggieback does? https://github.com/nrepl/piggieback#usage You start a normal jvm clojure repl and can start a cljs repl within it You can :cljs/quit that inner repl and presumably go back in if desired

p-himik21:09:40

As far as I understand, piggieback is just about having a REPL, and OP wants to fully combine backend and frontend operation within a single process ("running a project out of the same REPL"). Just having a CLJS REPL within your CLJ REPL is a rather common thing.

Max21:09:24

Hm, this is good info. I thought a shared REPL might be useful when you wanted to generate frontend code (e.g. via a macro) from backend code: think changing a routing table and having an autogenerated client update without restarting the REPL. In some cases it would be nice to keep the backend code used to generate the frontend code in the backend’s namespaces, rather than in a shared library somewhere, and I thought sharing a REPL might be a way to do that. In other language ecosystems, client code generation is done via a special build target that produces a client library. In Clojure though, we tend to prefer continuous repls over punctuated builds.

vemv21:09:29

In general it seems a decent idea to share the process, this way the clojurescript compiler can be a dev-time Component (or Integrant etc) component and have a very fine-grained code reloading experience. It's probably a not much explored area. (I have in the past e.g. https://github.com/thheller/shadow-cljs/issues/629) As for classpath isolation, I don't think it matters a lot - e.g. there isn't a huge use case for having different clojure.spec versions for backend and frontend. If you need such differences for clojure libs probably you should be wondering what is blocking you from upgrading to $latest. (favoring $latest is an 'official' tools.deps policy) Probably Guava and other yucky Java deps would be affected from the cljs dep tree to the clj dep tree. That's not unsurmountable though, and in fact it's pretty common for .cljc projects to have to deal with classpath management that is affected by such nuances.

p-himik21:09:52

> think changing a routing table and having an autogenerated client update without restarting the REPL Why would you need to restart a REPL here? I've never had that problem with frontend code reloading (when CLJS code changes) and CLJ code evaluation (when CLJ code changes). > I thought sharing a REPL might be a way to do that. At this point, I think I'm confused about what you want. What do you mean exactly by "sharing a REPL"? Right now, you either have a clj => prompt or a cljs => prompt - the current execution context of the REPL is fixed (or whatever the lingo is). Do you want for it to essentially be clj+cljs =>?

p-himik21:09:20

@U45T93RA6 > As for classpath isolation, I don't think it matters a lot Believe me, it does. I have had to deal with enough classpath issues for me to finally switch back from a single process to two separate ones. Just a small example - your backend and frontend code works perfectly, you decide to add a small CLJS dependency. Suddenly, a transitive dependency version changes, becomes incompatible with something in your backend, and everything fails. Now you gonna have fun time figuring out all the :exclude vectors and whether you can use that dependency at all. And when classpaths are separate, changing frontend dependencies simply cannot affect your backend in any way. More than that - a common classpath during development means that you will need to have the same classpath in production, potentially making your uberjar or Docker image or whatever much larger than it needs to be.

lilactown21:09:16

you can have one project that has CLJS and CLJ code, and share a source dir. but I would suggest having separate deps trees and running two separate processes, one for building your CLJS code and another for running your backend. this way you don't need to handle dependency conflicts between your build tooling and your backend application, and ensures you're not deploying a ton of stuff in your backend JAR that might end up being a security problem

lilactown21:09:25

ah similar to what @U2FRKM4TW just said 🙂

👍 2
vemv21:09:22

> Believe me, it does. I have had to deal with enough classpath issues for me to finally switch back from a single process to two separate ones. I've done as much classpath management as any other seasoned Clojurist and I disagree. > More than that - a common classpath during development means that you will need to have the same classpath in production This is what Lein profiles / tools.deps aliases are meant for. A decent setup won't bloat or alter production dependencies, if anything it will make them more explicit (also I'll let anyone know in advance that I'm not interested in arguing with the internet - @U01EB0V3H39 asked about something, I provided a nuanced answer cheers 🍻)

p-himik21:09:11

How can profiles/aliases affect anything here? In development, you need e.g. shadow-cljs with all its dependencies to watch a built, during the whole development process. In production, you need shadow-cljs just once - to build the production JS bundle. After that, you don't need shadow-cljs along with all its dependencies.

vemv21:09:57

You can have a :shadow-cljs profile that is merged into the base profile at dev time, and that is not merged into the base profile for running a JVM server in production. And you can use the :shadow-cljs profile again for the one-off production task of compiling cljs code.

p-himik21:09:14

That will mean that your production backend runs on a classpath that's different from the development backend classpath. Which means that you cannot really reason about how your code works. A bug in production could easily be introduced by a dependency change - and you won't be able to reproduce it in development just because you have a different classpath. It's quite a hassle to debug - been there, done that.

p-himik21:09:37

Frontend and backend are fundamentally different when it comes to production. And conceptually, the stronger the separation between development and production is, the more problems you will have. So it's both simpler and easier to just stick to the approach where frontend and backend are separated at all times.

Max21:09:25

> At this point, I think I’m confused about what you want. > What do you mean exactly by “sharing a REPL”? > Right now, you either have a `clj =>` prompt or a `cljs =>` prompt - the current execution context of the REPL is fixed (or whatever the lingo is). > Do you want for it to essentially be `clj+cljs =>`? @U2FRKM4TW By “sharing a repl” I mean having separate clj=> and cljs=> prompts, but sharing state between them. The fantasy vision for how this would work would be something like this: • I edit the route table in the backend • I evaluate the route table def • In my cljs repl, functions generated from that route table are updated and available

👍 2
vemv21:09:17

> That will mean that your production backend runs on a classpath that's different from the development backend classpath. Which means that you cannot really reason about how your code works. Nice FUD over there, you addressed nothing about what I said and vaguely appealed to experience/authority

p-himik21:09:41

@U01EB0V3H39 With the current state of affairs, I would expect for it to work already. If I change e.g. a macro, the CLJS code is recompiled automatically, making the changes available in the CLJS REPL. If it's not the case for you, perhaps you could create a minimal reproducible example where it doesn't work? Or perhaps I still misunderstand what you mean.

p-himik21:09:38

@U45T93RA6 Sorry, but what was there to address? Yes, you can use profiles/aliases. But my message about it resulting in different classpaths still stands. I'm not trying to appeal to authority and induce FUD. In its basic concept, having different code means a possibility of having different results - why would you want to have that in your project, where production could suddenly start working differently from the dev setup?

vemv21:09:16

if you don't use the mentioned :shadow-cljs profile you don't get any such difference from production A great way to repro prod problems locally is without the :dev profile and with a very minimalistic :test profile. Analogously, one wouldn't add the :shadow-cljs profile when running bugrepros, and also in CI in all cases. There's nothing to remember (i.e. it's not fragile), profiles are absent by default.

p-himik21:09:22

I simply fail to see how such an approach with alias juggling brings more benefit than its cost. To each their own, I suppose.

vemv21:09:51

Indeed to each their own. For one thing one saves processes (extra JVMs) and might get more fine-grained integrations when it comes to code reloading. stop ping the cljs compiler (and other asset compilers) as a Component makes sense to me. Personally I've never been a fan of "compile on save". It also seems nice to run cljs tests from a clj Reloaded workflow. You could run them after every (reset), for 'validation' of your backend changes. To be fair, as mentioned I don't think people are exploring these areas much so it's an open-ended problem that might need more hacking than people can be comfortable with.

p-himik21:09:43

> might get more fine-grained integrations when it comes to code reloading > Personally I've never been a fan of "compile on save". Ah, this seems to be the crucial difference here. I am a fan of that approach when it comes to web UI development. :) But not the backend side.

👀 2
Max21:09:18

@U2FRKM4TW > If I change e.g. a macro, the CLJS code is recompiled automatically, making the changes available in the CLJS REPL. Without a shared classpath, the namespace(s) defining the routes would have to be in a shared library between the front and back end. That’s not the worst thing in the world, but I think it would be preferable to keep the route definitions near the handlers, especially since most existing Clojure routing solutions have the route definitions reference handler functions directly. I suppose another way to frame the problem is “what’s a reasonable way to share code between the front and back end without putting it in a shared library?” I think this is also a bit of an edge case since it’s not the code itself that’s shared, but something generated from the code.

p-himik22:09:27

The classpath is not shared, but parts of it are. E.g. src - I do not split .clj and .cljs files into separate classpaths.

Max22:09:44

@U2FRKM4TW would you mind laying out an example source tree? I’m having trouble visualizing what you’re suggesting

p-himik22:09:30

project
- src
  - some_app
    - core.clj
    - core.cljs
- deps.edn
- shadow-cljs.edn
where deps.edn has its :paths set to "src", among maybe other things. It also has all backend dependencies listed directly in :deps and all frontend dependencies listed in a separate alias. So both backend and frontend have access to all the code within src. The neat thing is that the backend doesn't care about CLJS files - so if you lack the dependencies for those (when you don't specify the frontend alias), it's all fine.

Max22:09:25

How do you tend to structure the src tree itself?

p-himik22:09:08

According to the way I structure namespaces. There's 1-to-1 correspondence between the two, with adjustment to the fact that each namespace can be in just CLJ, in just CLJS, in both files, or in a single CLJC file. And the way I structure namespaces... Well, can't really explain it succinctly. According to isolated pieces of functionality, whatever makes sense for a particular purpose. I'm not a follower of any explicit convention. And while I used to do what e.g. re-frame suggests with splitting all code in subs.cljs, views.cljs, and other files, I stopped doing that quite some time ago.

Max23:09:28

Hm, the idea of pairing feature areas with the backend code that supports them is interesting. Instead of one giant frontend app and one giant backend app, that would feel more like a bunch of mini apps composed together. I’ll have to give that some more thought.

vemv23:09:58

Yes it's basically modularity. I've successfully done it while using a reframe-like framework. See also Polylith for inspiration

Max02:09:17

Interesting ideas! Specifically with the router example, you can imagine you’d end up including the .clj file(s) containing your route definitions in your frontend build as a compile-time dependency so you could macro out a .cljs client library. Since the route definitions directly reference handlers, how would you avoid pulling in your entire backend app into the frontend build? Or is that not a concern? I suspect this an instance of a more general problem where you might need to be careful to make sure clj files required in clj[sc] files don’t have any undesirable dependencies.

p-himik06:09:19

It's not a concern because building frontend is a one-off task, and the resulting bundle has no traces of those libraries.

Max12:09:02

Would it affect fronted build performance?

p-himik12:09:59

I can't possibly do that.

p-himik12:09:58

Even if you copy a bazillion of CLJS files from somewhere else and add a bazillion of dependencies that they require - it won't matter at all if that code is not used because of the DCE, Dead Code Elimination.

p-himik12:09:15

And in case of CLJ, it won't even get to DCE since CLJS compiler won't just use that code at all.

Max12:09:24

That’s the thing: the cljs compiler does read clj files when macros in them are required in cljs files. If you have a macro in a clj file that requires other files in your backend then the compiler is going to have to evaluate all of them to expand the macro.

p-himik12:09:12

Ah, sure - that's gonna affect the compilation time. But no classpath splitting/separation/combination can alleviate that. What you need to be executed for the code to be compiled, will be executed.

dominicm20:09:55

https://github.com/juxt/edge/ is set up like this, and has all the configs you need for cider to figure out the cljs & clj stuff simultaneously.

Drew Verlee23:09:10

Do you think it might be possible a machine could build the url path parameters and query parameters from the react component tree?

p-himik00:09:48

If you can encode that tree as a string, then yes, of course. But to what end? Note that servers in general have a limit on the length of the URL, IIRC it's usually within a few kB or tens of kB.

Drew Verlee00:09:12

I'm not saying encode the tree, I'm saying encode the query and arguments to it. Just like we do now, only make it automatic based off the fns/components.

p-himik00:09:14

Alright. Then I have no clue how that would work or, most importantly, what problem would it solve. Suppose I have this:

(def msg "hello")

(defn span [msg]
  [:span msg])

(defn app []
  [span msg])
where app is the top-level component that gets rendered. So app doesn't have arguments, but span does. But also the value is hard-coded. What would the URL look like? What would changing it achieve? How would it be useful?

Drew Verlee00:09:45

Thanks for chatting. I'm just wrestling with trying to formulate this idea. the short answer to your question would be that the url would be /app because your passing no arguments. I'm not suggesting anything be changed in terms of user experience, i'm suggesting I think it might be possible to making app routing easier to program. Backing up. What is the use of path parameters or query parameters? It's a query that is a combination of browser data (react components) being feed app data (usually form a server), the query when given to a browser gives the result in the form of the view. The issue is that that it isn't expressive. It's at tree walk with a filter on the item found. e.g I select dark mode on my app, now if i share the url to show someone how cool it looks in dark mode where do you put that? app/cats/1?dark=true Maybe would could argue it shouldnt' capture dark mode. But what my page allows for a split sreen mode with cats and dogs and i filter one to show large dogs and another to show small cats um ok my path parameters are... /cat/dog nope how about... dog?size=large|cat?size=small I think it obvious this is going to get gnarly quick. What's more, it's repeating logic i have in the app already. I told my app that i have a top level view that takes essential configuration to filter cats and dogs (defn app {cat {size small} dog {size large}}) Why i'm i now repeating the logic in the url? Is there a better way? What happens when the query gets more complex. e.g small cats buy only ones whose names start with the letter s. I feel like the answer is to move away the informal path parameters and query paramters to something more well thought out like datalog or sql. But their are other considerations.

Drew Verlee01:09:28

This video on fulcro https://www.youtube.com/watch?v=oQpmKWBm9HE and has some intesting ideas

p-himik01:09:20

Your app has a set of all possible states and its current state. As you note with the dark mode, you could want to limit what is exposed via the URL. So when you convert the current state into the URL, you need to: • Leave only the things that are relevant for the URL • Encode those things When converting a URL into a state, you need to: • Decode the URL • Validate the result • Fill in the defaults All in there is pretty much obvious except for the encoding part. Personally, I have settled on this approach: • Always represent the URL-relevant state as a map • Use keys of that map as the keys in the URL query • Encode the values as JSON strings (I like using spec-tools for this) and use those strings as values in the URL query

sova-soars-the-sora02:09:36

All that's great until I want server-side rendering and then I need to provide HTML for each /path/on/the/app

sova-soars-the-sora02:09:03

But I see no reason it could not be automated... Jst syncing up client/server side might not be super clean / obvious.

sova-soars-the-sora02:09:49

Define a set of views in a .cljc have the clojurescript app read those and generate components, have the serverside read those and make compojure endpoints that render the [same] components [as html to up-strap like in rum/hydrate]

lilactown02:09:45

@U0DJ4T5U1 it sounds like you have two problems: 1. what's the appropriate way to route and encode state for my app? 2. how do I handle the bidirectional relationship between state in the URL and state in my app?

lilactown03:09:14

/app/cats/1 is a terrible route if you have a split screen view of both cats and dogs. /app/pets is probably better. any other info would be query params, but it's the split screen view which is the actual "route" in this case AFAICT. as for managing state in both the URL and in some app state, that's a tricky problem that I prefer to defer to a library for, like https://reactrouter.com/. when you want to start providing things like transitions between routes and prefetching data on hover/click, it becomes quite complex to roll your own.