Fork me on GitHub
#clojure
<
2021-08-23
>
simongray07:08:32

I need to extend some Java classes. I’ve been pretty happy with Clojure’s interop so far, but this seems to be an area where interop is a lot more complicated, e.g. a sudden proliferation of namespaces with (:gen-class ...) containing lots of interop-specific key-value pairs. Does it even make sense to attempt interop in this case, or am I better off just writing Java code and importing that into Clojure somehow? What are your experiences?

p-himik08:08:20

Sometimes you can get rather far with just proxy. But if that's not the case, you might want to take a look at these libraries that let you generate JVM bytecode using Clojure, seemingly with no hassle: • https://github.com/athos/JiSEhttps://github.com/jgpc42/insn Clojure itself has something similar in the clojure.asm namespace, but it's more cumbersome to use: • http://polekilroysoft.github.io/ClojureJavadoc/clojure1.0/clojure/asm/package-summary.htmlhttps://gist.github.com/athos/1033052/b332f3786c40f1c5b85abf1ba56012024b977448 And if you decide to go with writing plain Java and using it within a Clojure project, I can give you a few more links.

simongray08:08:14

Thanks. JiSE looks a lot more manageable, while insn reminds me of :gen-class. Do you have any experience using either?

p-himik08:08:15

Nope. :) I just hoard links. I used Virgil in a similar situation - but I didn't know about the above back then. Nowadays I wouldn't recommend it.

simongray08:08:17

Ok. Thanks anyway. Guess I’ll try JiSE and see where that takes me!

👍 2
schmee10:08:56

I’ve had success with writing Java and mixing it in with Virgil in the past

simongray10:08:43

@U3L6TFEJF I’m using tools deps, not Leiningen or Boot, so I guess Virgil isn’t as plug and play.

simongray10:08:18

Hmmmm… JiSE says it does support inheritance, but I can’t figure out how to extend a class. None of the example or the tests seem to do that. I really wish there was a Java-in-deps.edn solution that was good to go.

simongray10:08:33

Still not sure which one to pick. I guess americano seems to be the most friendly option.

p-himik10:08:09

Huh, I didn't know about clojure.tools.build.api including javac, thanks!

👍 2
simongray07:08:55

Just wanted to report back that Americano is incredibly simple to set up. It does require running clj -X:compile-java to recompile (there is no automation of that step AFAIK). In the meantime, I’ve discovered on the mailing list of the Java framework whose class I was extending that I didn’t actually need to extend anything, I could simply mutate an instance of a generic class to get nearly the same result, so I reverted my Java code commit and went with that solution instead. Still, I’m really happy that I tried out Americano and I will definitely use that next time I need to write some Java-in-Clojure.

p-himik08:08:17

> there is no automation of that step AFAIK You should be able to do that by making Americano a part of your app and just calling the compilation code - that's exactly how I used to use Virgil. But not sure if it's a good thing to do.

🙏 3
Steve Pegoraro08:08:48

Hi all, I’m working out the various components to use in a backend web stack today and was wondering what the idiomatic database libraries are right now (excluding Datomic - considering it but would like to see the other options). I’m planning to use Pedestal at this stage but can’t seem to find a database library that works asynchronously. I’m coming from a mostly Rust and Python background the last few years so maybe I’m missing something.

p-himik08:08:38

I would start with selecting a DB itself first, and only then a library. If you're fine with PostgreSQL or any other JDBC-compatible DB, take a look at https://github.com/seancorfield/next-jdbc There's no async support out of the box, as far as I can tell, but you can easily wrap what you need in threads. If you use core.async, there's a handy thread macro.

Steve Pegoraro08:08:19

Thanks for the reply! Postgres is indeed what I’m leaning towards as I have experience with it already. Will using core.async/thread spin up a new thread with all the overhead that entails or will it utilise its own thread pool? I’ve had a quick glance at the source and can see it uses Executor/newCachedThreadPool but I’m not experienced enough with core.async to follow whether that starts one or re-uses one.

p-himik08:08:24

It will use that thread pool, yes.

Steve Pegoraro08:08:04

It’s own, already-running one?

p-himik08:08:24

Thread pools don't run, they manage threads. It already exists. It will launch new threads as needed, and it will cache them. Threads that haven't been reused for long will be pruned from the cache to free resources.

p-himik08:08:51

You also want to use this to avoid opening new connections every time: https://github.com/tomekw/hikari-cp

Steve Pegoraro08:08:30

Ok, got it. Thanks for the help!

👍 2
slipset10:08:27

You seem like you’ve already settled on going async. Why would that be? IMO (not having programmed much async) the sync model is much easier on the brain and the JVM works rather nicely with threads. Put in another way. Unless you know that your app will have a lot of traffic with potentially long running requests, I’d go with a synchronous approach.

2
p-himik11:08:41

Async != core.async. You can have the first without the second. Using threads could also be considered async programming. But up to OP to decide what it means in their context.

Steve Pegoraro11:08:49

I mean async in the sense of non-blocking io with work distributed among threads (reactor). I also prefer threads for almost every task but this is one case where the memory saving from going async is worth the burden. Requests will all have several operations that need to be performed before responding, this affects latency of course but for the application it’s fine.

Steve Pegoraro11:08:20

For context I’m coming at this from a Rust and Python background mostly. I have also done epoll work in C many years ago.

rutledgepaulv12:08:20

Non-blocking database drivers on the jvm have not achieved wide popularity in the clojure ecosystem. If that's something you believe you need I'd probably look at r2dbc or vert.x. For your awareness there is also a jdk project that has been underway for a while that seeks to virtualize threads largely as-is rather than requiring you to use a different api when you want the benefits of non-blocking code. You can read more about it here: http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html

Steve Pegoraro12:08:11

I’m eagerly awaiting the landing of Project Loom!

🙂 2
Oz15:08:22

What's a good way to make the following refactoring? :

|-maindir
|-|-subdir
|-|-|-lib.cljs (ns maindir.subdir.lib)
|-|-|-anotherlib.cljs
|-|-|-yetanother.cljs
|-|-|-alsothis.cljs
*abra kadabra*
      \  /
       \/
|-maindir
|-|-lib.cljs (ns maindir.lib)
|-|-anotherlib.cljs
|-|-yetanother.cljs
|-|-alsothis.cljs

ghadi15:08:08

mv I'm old fashioned about this kind of stuff

p-himik15:08:18

And then sed followed by careful perusing of git diff?

Oz15:08:25

For the sedding and diffing I will use emacs. But yeah, mv is reliable, "one by one" is an acceptable answer 🙂

Oz15:08:35

Actually the only replacement needed is removing ".subdir." from everywhere, not that bad.

Oz15:08:08

Obviously not the only one

pavlosmelissinos16:08:13

I'm probably too late but since you're using emacs, dired should be a better choice than vanilla mv

👍 2
noisesmith17:08:56

this is one of the few cases in clojure where you can do a compiler guided refactor - for every ns form that does not match the path to the file, the compiler will complain, you can repeatedly fix the declarations and reload your namespaces until you get no such complaints

👍 4
Oz18:08:24

Thats what I eneded up doing, I don't know how to use dired yet beyond the most basic survival, but ivy occur edit mode was amazing help in locating and editing names across multiple files at once

noisesmith18:08:53

what I said above was incomplete, of course. you get two compiler errors, one when the ns decl doesn't match the path to the file, the other when the required ns no longer exists due to the refactor

adi08:08:29

Late to the party, but I've previously done this in stages. 1. git mv (not just mv), and commit to do the most disruptive part of the refactor (file renaming) 2. global sed-replace to fix ns declarations and references everywhere (src, test, comments, docs, scripts etc.) sed -i -e <pattern1> -e <pattern2> ... a. After that, it helps to grep -R <token> to identify various unexpected patterns I might have to further sed/replace 3. git add -p to review+stage each rename piece by piece, and of course git commit 4. run tests as well as build jar and run app and exercise it to see if it all appears to work 5. git squash to make the whole refactor a single atomic commit

👍 2
edipofederle22:08:00

Hi, https://github.com/yogthos/migratus is still the most common way to manage DB migrations ? (like Rails migrations or similar)? thanks

didibus22:08:30

I'd say its definitely one of the most popular.

seancorfield22:08:32

https://github.com/weavejester/ragtime is the other common one. If I was starting over, I'd probably use Migratus instead of a custom in-house solution (ours dates back a decade and has slowly evolved).

didibus22:08:19

We use semi-structured data-stores, and manage document versions in application code with Spec. So we write our code so it works will all prior versions of the document spec, and knows how to update it to the latest. Which may or may not be more complicated than a migration strategy :man-shrugging: , but at least it lets us have zero downtime, since we never need to do a big bang migration on the data-store

didibus22:08:43

In our case it does mean that data is forever though, and so overtime, you continue to need to deal with the historical versions of the data you ever had.

seancorfield22:08:15

We generally try to make sure that DB migrations can always be applied while the system is live so when we are planning a DB change, we'll make sure the code will work with either today's DB or tomorrow's DB -- which means we might need to roll out some changes to code first -- then roll out the DB migration then maybe roll out code changes to remove the multi-version handling. But, like @U0K064KQV’s scenario, that's all somewhat meta and tangential to the actual DB migration machinery itself.

Ed09:08:22

I've had a fair amount of success with https://flywaydb.org/ in the past.

edipofederle09:08:21

great, thx guys, I will take a look in these libraries as well.,

steffan13:08:02

In my previous job (non-Clojure) we used Flyway to migrate our AWS Redshift database. I made a CI/CD pipeline based on Flyway, so we could validate schema changes on our 'test' scratch DB instance (schema only) and 'staging' DB instance (schema + test data) before rolling out to the 'production' instance. Flyway was pretty solid and straightforward with all this chicanery going on.

didibus18:08:04

I'm curious, what does it do exactly that's not just running a SQL script? A quick glance and it seems to offer very little over migratus or ragtime but appears to have some commercial aspects to it?

steffan23:08:01

There's a chart comparing free and commercial versions of Flyway on the download page : https://flywaydb.org/download The free version was sufficient for our needs at my old job.

Ed11:08:52

One of the advantages it has over, say ragtime, is that it will lock to ensure that the migrations are only being run by one process at a time. So if you, for example, ran the migrations in a k8s pod at application startup and you ran many replicas, only one of those processes would actually run the migrations. With ragtime that's your problem. I've never used migratus, so I can't comment on that.

didibus20:08:35

Hum... ya okay, I'm probably a Clojurist at heart 😛 so I feel like I'd rather have the choice of locking mechanism if I need one by combining it with another lib, but I can see the appeal of having something that guides you into it