Fork me on GitHub
#clr
<
2023-01-07
>
elena.poot19:01:27

I've been translating some Clojure code to C#, which is as tedious and frustrating as you can imagine. I was wondering if there is an example anywhere of a ClojureCLR project that is called by C# (it's an add-in to another system) that also calls C# APIs (e.g. some of mine and those from the host system), but also includes other Clojure libraries? I'm just wondering how deep this rabbit hole is, and whether using ClojureCLR would be a more effective way to do this, but starting from zero, I can't really envision how to fit the pieces together. The C# APIs I need to call from Clojure are a mix of locally developed NuGet packages, and host APIs for C# objects passed to my code from the containing system. I know my Clojure dependencies would need to be "pure Clojure", i.e. no JVM dependencies, and when I really dig into this, that could end up being a deal breaker. Also, how to manage dependencies?

Frank Henard11:01:33

I thought I saw an example on the ClojureCLR README of calling ClojureCLR from C#. I haven't worked in ClojureCLR much, but it seems like translating Clojure to ClojureCLR would be easier. I hope that works out for you!

elena.poot17:01:21

Oh yeah, being able to use my existing Clojure code would be far easier than rewriting it in C#. It is just a question of whether I can get all the pieces working together in less time than it would take to finish the rewrite. The calls into my code are simple enough. Interop looks workable, and I would build Clojure wrappers so I only have to do each one once. I am more worried about being able to successfully build it all together, get the assemblies linked up, consume nuget packages, etc. I don't know what I don't know. A working example would go a long way towards filling those gaps. I would LOVE to be able to do all the real work in Clojure, and that code is already written and tested, but I need to change out the underlying library and embed it as an addin and that is all C# code and packages.

Frank Henard17:01:48

Unfortunately, I can't answer your question due to my lack of ClojureCLR experience. You're probably aware of this, but my impression is that the tooling is not near what it is in Clojure JVM. I hope someone else can help, and I hope that you're able to accomplish what you need to and maybe also further the improvement of the ClojureCLR tooling.

yes 2
elena.poot18:01:18

Thanks! And yeah, I'm aware of the tooling issues. If we had great tooling already, I probably wouldn't have those concerns. I don't have to know how Maven works because Leiningen handles that. You learn over time how the infrastructure works anyway, but if there is no infrastructure you have to figure all that out before you can start. I honestly expect tooling to get better faster now that the .cljr extension is supported, because it makes building tools SO much easier, especially for editor support. And I read here the other day that someone is working on porting tools.deps. So I'm kind of thinking that if I can get this working, and thus commit to ClojureCLR, I'm right at catching the wave on the development story improving significantly. 🙂

👍 4
2
bobcalco21:01:29

@U4LCNMMGS That would be me and I hope to have good news within a week or so. 🙂

awesome 10
❤️ 4
Emma Griffin14:01:08

@U4LCNMMGS Hi Elena, as far as tooling goes, you have a couple options, none of which I recommend, but all of which in theory should be able to handle your situation, though some manual intervention may also be required

Emma Griffin14:01:37

The most straightforward, but also often the most frustrating choice is https://github.com/kumarshantanu/lein-clr . It's a leiningen plugin for ClojureCLR that allows you to pull in NuGet dependencies for use in your Clojure project. It doesn't play nice with native dependencies though, so if there is some native compiled code for Linux, for example (e.g. a .so file), then lein-clr will just outright ignore it when pulling in dependencies. lein-clr also has a "grab every downloaded .dll " approach to loading in assemblies when starting a repl or running main, so if you have .dll files that are architecture specific, you'll also run into issues.

Emma Griffin14:01:48

Another option is using typical C# dependency management tools to pull in NuGet dependencies. This of course requires some manual work and likely means you have a dummy C# file floating around, but if you need to use something with native dependencies than this is probably your best option.

Emma Griffin14:01:36

Finally, you could manually download NuGet dependencies locally and just load them yourself via assembly-load-from inside your ClojureCLR code. This of course means that your dependencies need to be uploaded with your program/library, or you need to provide users with some script to download dependencies, in the event that this is OSS.

Emma Griffin14:01:17

I'm in the middle of wrangling dependencies myself so I'm here to help if I can, though I may not be much help outside of high level advice

elena.poot15:01:19

Thanks! This is internal and it's all nuget based. My cljr code has to be a nuget package, and my dependencies already are, including those I produce. I just figured out how to get clojure.compile to produce dll. It sounds like maybe lein-clr will work for me if I can get it to use my local package source. I was under the impression that I'd need to do assembly-load-from for all my direct dependencies in any case. Since they're all c# packages, I assume they will handle their own dependencies as usual. But I guess I'm not sure how to do the assembly-load-from part during development. In production, the dependencies will all be in the same folder as my cljr code, but in development, I'm not sure where lien-clr will put them.

elena.poot15:01:37

of course, lein-clr doesn't support .cljr files either. I was really hoping to not have to have separate clojure projects for clr and jvm

Emma Griffin15:01:49

I published a ClojureCLR project on NuGet, but it needs to be updated. I remember just needing to setup .sln and .csproj files manually. That project didn't have any NuGet dependencies of its own though. I'm currently working on a project that does have NuGet dependencies though, I'll let you know how that goes as I finish it, but you may finish your project before I do.

Emma Griffin15:01:28

One word of advice with lein-clr, the newest version may not work for your project if you're pulling in Clojure/ClojureScript dependencies that don't have a ClojureCLR implementation. In this case you should downgrade the lein-clr version to the second most recent (don't remember the number off the top of my head). The issue is if you're pulling in some leiningen dependencies for ClojureCLR and some leiningen dependencies for Clojure or ClojureScript. There's not really a good way to handle that at the moment, that I'm aware of.

Emma Griffin15:01:10

@U4LCNMMGS you can always use .cljc and wrap the whole file in #?(:cljr (do ...)) or implement everything in .cljc files, which does get messy unfortunately. We definitely need support for .cljr files

elena.poot15:01:25

Oh, I've never used the reader conditionals, but I was under the impression you had to wrap each top-level form. Oh I guess that's what the (do ...) is for. 🙂

elena.poot15:01:12

As to whether I'd have any pure clojure dependencies... Off hand I'm not sure, but probably? Thanks for the heads up on that one, I'll keep it in mind.

Emma Griffin15:01:46

Yeah, you should be fine if you have pure Clojure dependencies and NuGet dependencies though. My personal preference while .cljr support doesn't exist yet is to implement everything in .cljc files using reader conditionals. But this isn't best practice (though you can't really do what is best practice until .cljr support exists).

elena.poot15:01:26

Well, .cljr support does exist now, but not in lein-clr

Emma Griffin15:01:51

If you do go the route of putting ClojureCLR implementations in their own .cljc files wrapped with #?(:cljr (do ...)) I also recommend putting clr somewhere in the file name so it's obvious it's only for ClojureCLR

elena.poot15:01:16

yep, the most recent alpha. Scroll back a bit, it was announced here a few days ago

Emma Griffin15:01:19

I thought @U2M7EC8KU and @U45FQSBF1 were still working on it

elena.poot15:01:47

Nope, I tested just a bit ago and it works from Clojure.Compiler.exe

Emma Griffin15:01:47

With ClojureCLR 1.12

Emma Griffin15:01:17

That's awesome! I must have been tired when I read that announcement

Emma Griffin15:01:09

You could probably tweak lein-clr yourself to look first for .cljr. I actually think I saw where in the code base to do that when I was looking through it months ago

elena.poot15:01:11

I saw that and thought of all the hoops I wouldn't have to jump through. Except lien-clr doesn't support it. sigh

Emma Griffin15:01:18

I think just a couple lines need to change

elena.poot15:01:47

I will definitely take a look then. Will certainly make my life simpler if I don't have to copy whole projects and then synchronize changes

Emma Griffin15:01:27

@U4LCNMMGS I would see if lein-clr works with the dependencies you need to pull in, if it does, I'm more than happy to try to add in .cljr support in the plugin via a repo fork or something else

elena.poot15:01:34

I'm trying to figure out how to do snapshot versions right now. VS handles them but I'm not sure exactly how, you can't specify wildcards in the nuget command

Emma Griffin15:01:39

Can't guarantee I'll be fast to help, but it would be worthwhile unless deps.edn support or something else for ClojureCLR is right around the corner. @U2M7EC8KU sounds like he's onto something though 👀

elena.poot15:01:15

Yeah, but even if I have to write a .bat file, I need to prove first that I can build this and it will work, so I have to use what's there now. Very much looking forward to deps.edn support though

Emma Griffin15:01:17

Oh wait, I do think I've done this before, because I think lein-clr didn't work with .cljc files either

Emma Griffin15:01:39

I think you may just need to change this function, iirc. More specifically, you need to change clj-x? to check for .cljr, then .cljc, then .clj

Emma Griffin16:01:29

Not sure about publishing leiningen plugins, but I imagine it's not too difficult to do

elena.poot16:01:21

Well, they're dependencies, so it ought to be able to find a local copy in my local maven repo. If that's the only place it needs to change, that'd be awesome. But since I don't have any cljr code in clj files, for my usage, I think I'd ignore those and just do cljr and cljc. And then I don't have to worry about precedence, since there shouldn't be both.

yes 2
Emma Griffin16:01:12

Yeah, I think lein-clr has been abandoned so I'm tempted to push for it to be moved to the clj-commons repo, but if we have deps.edn support then I'd much rather just use that then try to bring a bunch of new features to lein-clr

elena.poot16:01:56

Yep. I still have to learn deps.edn, but that was on my agenda anyway. 🙂

clojure-spin 2
Emma Griffin16:01:09

I prefer it to leiningen personally

Emma Griffin16:01:19

But mostly for aesthetic reasons rather than practical ones

elena.poot16:01:47

Mostly I figure that's the current official tooling, and I'm trying to modernize myself. I learned what I needed to learn about clojure years ago and have not kept up with the changes. But now I'm switching IDEs and looking around and seeing all the cool new stuff and trying to catch up 🙂

Emma Griffin16:01:30

I'll say that at my job everyone seems to use leiningen so I think it will take some time before deps.edn is the true default for Clojure

Emma Griffin16:01:48

(though I have coworkers who prefer deps.edn even if we don't use it)

elena.poot16:01:50

I'm the only Clojure dev in my company, but that may change (🤞), so I'm also trying to be mindful that I'm setting the standards. I use the lein-parent plugin for dependency management, and I've run into other tools not handling that well. deps.edn kind of handles it, if you don't mind hijacking the personal aliases for it, but it's something I see as required and it's weird it's not better supported.

elena.poot16:01:15

Sorry, that's way off on a tangent lol

Emma Griffin16:01:02

Yeah I had an internship where I snuck ClojureScript into an existing full stack JavaScript web app, so I know how it is to be the only Clojure dev (can be fun but scary). I don't mind going off on a tangent as long as you don't but I'm obviously a rambler

elena.poot16:01:20

I was shocked I got permission to use clojure. But I maintain a significant piece of backend infrastructure, and I could never have built it all myself if I was stuck with C#.

Emma Griffin16:01:24

Part of my motivation toward learning ClojureCLR is spreading Clojure everywhere for all platforms and ecosystems

Emma Griffin16:01:43

I didn't get permission I just started going, but there was only one other dev and he had stopped working on the project

elena.poot16:01:51

I have the luxury of being off in my own little world. Nobody else knows my part of the enterprise and I get to build it as I please to a large extent. 🙂 But it'd be nice to have help.

nice 2
Emma Griffin16:01:25

What IDE are you switching to?

elena.poot16:01:14

VSC/Calva (from Intellij/Cursive). Our shop has switched to Javascript as our primary language, so it brings my tools closer to the rest of the team.

elena.poot16:01:46

I'm going to have to integrate with all that Javascript stuff, but I'd rather not write any Javascript. 🙂

Emma Griffin16:01:09

Going to try to use ClojureScript instead?

elena.poot16:01:37

Probably in some cases. I have a dashboard that I built in cljs, but I don't claim to really know the environment, or the larger js ecosystem, hardly at all. So I may end up with clj, cljs, and cljr, as appropriate. Which would be kind of nice in some ways and probably pretty annoying in others, but at least I'd get to do all my work in Clojure. I feel like I'm being punished when I have to go back to C#

elena.poot16:01:26

And my only significant experience with Javascript was traumatic. 🙂

Emma Griffin16:01:05

Yeah I'd much rather program in JavaScript than C#. I actually think JS and Clojure are fairly similar languages, but Clojure is a lot better still. I find JavaScript a fun language to write code in but not a fun language to read code in (because people don't seem to follow functional guidelines consistently)

elena.poot16:01:18

I had to build the back end side of a Cordova app in javascript. Doing async code with nothing but promises was painful. (I did figure out how to implement mutexes with promises though lol)

Emma Griffin16:01:59

Yeah async programming is always a pain, in my experience, even in Clojure. Just feels harder to debug, at least for me

elena.poot16:01:33

oh yeah, the testing was very much the hardest part of that

yes 2
elena.poot16:01:05

It does look like lein-clr will do what I need for dependencies. Just need to hack the file search and I should be good for building. I'll have to check at runtime whether to load assemblies from . or ..\lib, but that's not too bad. Now to make it work... Thanks for your help! I think this is gonna work. 🙂

elena.poot16:01:13

oh, . or ..\lib\foo\lib\bar\blah.dll. sigh

Emma Griffin17:01:39

yeah they can get fairly involved but I think you should be able to start with a couple directories and only look in all of their children directories

elena.poot20:01:38

yeah, I'll figure it out. Have to pick the right one for some of them (e.g. AWS sdk has many different dlls for different platforms). that one spot in lein-clr does seem to be the only change needed by the way. Took me a while to figure out I had to tell it which namespace(s) to compile rather than just running lein clr compile

elena.poot20:01:47

Oh, and I did have to go back a version, as you advised.

Emma Griffin20:01:23

My advice if you're going the lein-clr route, regarding cleaning up platform/architecture specific .dll files, is to write a script in Clojure (or Babashka or your scripting language of choice if you don't want to deal with the startup times of vanilla Clojure) to remove the necessary dll files. I actually have a crude script like this if you need a reference, but it definitely needs to be cleaned up. You can exploit the predictable path scheme of runtimes/[osx|unix|linux|win]-[arm|x86|x64] to do this, at least in my own experience. And by writing the solution in Clojure you can keep your library or app cross-platform, if that's a priority for you (since a bash script basically locks you out of Windows in most cases).

Emma Griffin20:01:54

With how lein-clr works I found it easiest to simply delete any .dll files that weren't needed based on detecting the end user's os and architecture

elena.poot20:01:09

I'm on windows, and it only creates one dll for each namespace. Not sure what would need to be deleted, unless it behaves differently on other platforms?

Emma Griffin20:01:41

Well if lein-clr downloads any dependencies that themselves have platform-specific dependencies, then you'll likely run into an issue. Not all NuGet dependencies have that issue though, so you may have got lucky and don't have to worry about it.

Emma Griffin20:01:55

If every .dll that gets downloaded is needed by your app/library, then you should be fine. You could also load some .dlls that aren't needed, so long as they don't collide with a .dll that is needed (again, the only case I can think of where this happens is with platform-specific dependencies)

elena.poot20:01:55

you've mentioned native dependencies a lot, and I haven't seen the issue yet that causes your concerns. I don't doubt I will. I wouldn't mind having your script to tuck away against future need.

Emma Griffin20:01:20

Sure, my script isn't general purpose but I imagine you can copy/paste and modify as needed

elena.poot20:01:45

Oh, ok I think you're talking about what I mentioned a bit ago, like the AWS SDKs that have at least half a dozen dlls

Emma Griffin20:01:07

I'm assuming you don't want all of those loaded?

elena.poot20:01:22

ok, I get it now 🙂 LOL heavens no

elena.poot20:01:44

I'm not used to having to wire all this up myself. It's a journey. 🙂

Emma Griffin20:01:02

yeah, I wasn't either when I had to

elena.poot20:01:38

Thanks! (Reminds me of the mega makefile days lol.) (I miss that sometimes when nuget is being ornery.)

Emma Griffin20:01:04

Yeah I've only ever had basic Makefiles but I've read a beefy one from time to time

Emma Griffin20:01:24

but really the tooling should be handling this, not our adhoc scripts

elena.poot20:01:43

I've written them, but it's been a LONG time. But for all the tedium, you do have complete control

elena.poot20:01:55

yep. I'm sure we'll get there.

yes 2
elena.poot20:01:21

Looks like the build artifacts are completely different, but this'll be a great starting place.

elena.poot21:01:00

Although, again in my specific case, I might not have to deal with it, since I'm building a plugin for a C# system, and it's used to picking out the DLLs it needs and ignoring the others.

elena.poot21:01:22

What I do have to figure out is how to package my thing. I don't know if I can cram all my dlls into a single nuget package. If I can't, this'll be nuts and I might want to skip nuget entirely and add direct dependencies in the C# project. Horrible general solution, but far better than a dozen or 3 packages to manage.

elena.poot21:01:46

nuget is another thing I have to learn a lot about in a hurry

Emma Griffin21:01:30

Are you writing an app or library?

elena.poot21:01:18

A library, specifically plugin to an app,

Emma Griffin21:01:58

Hmm, yeah that could be tricky. I think you can publish with the dependencies you need self-contained and then upload that to NuGet

elena.poot21:01:38

I have all the functionality coded in clojure in a different context. I need to insert it into the addin and use the host app's API (actually a wrapper we built around it to hide version differences)

elena.poot21:01:34

It's looking that way. I'm reading the docs on the nuspec file. You can tell it what files to include, it's just usually you don't have to because it reads that out of the .csproj file

elena.poot21:01:21

I'm learning new capabilities of tools I've been using for years, because now I'm using them a different way. 🙂

elena.poot21:01:20

And all at once! LOL

nice 2