This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-12-05
Channels
- # adventofcode (111)
- # announcements (20)
- # babashka (19)
- # beginners (47)
- # calva (7)
- # clojure (56)
- # clojure-dev (27)
- # clojurescript (2)
- # events (1)
- # holy-lambda (1)
- # juxt (2)
- # meander (18)
- # minecraft (4)
- # missionary (107)
- # nextjournal (21)
- # off-topic (30)
- # reagent (7)
- # reitit (19)
- # releases (1)
- # tools-build (8)
- # tools-deps (7)
- # vim (22)
- # xtdb (4)
I'd like to use a project that exposes several namespaces, but aliasing them all to the same name. Is this possible?
You'd need to export the vars to a single namespace.
(1) There is some prior art with doing this at runtime, most notably potemkin/import-vars - but beware this does not always play well with tooling and editors that wants to do static analysis (I've also heard of issues with AOT, but not sure if that's still true).
https://github.com/clj-commons/potemkin#import-vars
(2) You could also do this manually, eg. polylith projects rely heavily on creating interface namespaces that just call into the original implementation, eg: https://github.com/furkan3ayraktar/clojure-polylith-realworld-example-app/blob/master/components/user/src/clojure/realworld/user/interface.clj
(3) You could also have scripts and/or editor tooling support that would automatically generate all of the api wrappers for you (so something you would run as a separate step, but it would just look like (2) for all other tooling). I don't have one good link to post here, but there are at least 3 people who I recall have worked on features along this line and perhaps they may be able to offer some more concrete advice: @borkdude @UE21H2HHD @UKFSJSM38
I've struggled with this before, so I'm also interested if someone can provide suggestions for (3).
@UA7E6DU04 If you're on the JVM, then potemkin can be used - clj-kondo has support for this. But static generation of code is by far the most reliable method for all code including CLJS. See a discussion from yesterday in #clj-kondo. @UDRJMEFSN has started doing this for his dtype.next project, also rewrite-clj is doing this.
So: macros are nice and can be used to generate stuff at compile time, but actually having the code as a "what you see is what you get" thing works the best in all situations.
I feel that we could need a library for this in e.g. #clj-easy that we can all share
One thing, I've suggested before, but have not had time to explore myself - is if we could get eg. a LSP code action that would "import" the external var/docstring/etc into the existing buffer. This doesn't resolve the issue of automatically keeping the API up-to-date; but usually I don't have this issue. I usually find myself wanting to wrap existing APIs with perhaps some custom logic.
Also consider just not doing this and using more than one namespace :)
Which is like, totally fine and requires no additional work :)
The only situation where I “proxy” to other vars is when those other namespaces are implementation detail
until literally every one of your new users complains that it requires too many namespaces and equivalent tools in other ecosystems don't do that and their #1 complaint is discoverability - then you might consider having a system to have fewer namespaces and some system of importing vars or codegen. It is funny how this has come up over and over again over the years.... Or if you have a datastructures that contain definitions that can be transformed automatically into functions of various types - you know - data as code - and then you want to expose this api to users. There can be many reasons to use macros to define vars in a namespace and this is the root of the issue. In any case, codegen based on https://github.com/cnuernber/dtype-next/blob/master/src/tech/v3/datatype/export_symbols.clj#L72 is a reliable way to make it all work across editors.
Is there any project to redo the shell + basic utilities in Clojure? Here I am referring to more than just a bash replacement. "ls" would return a vec of strings. ps aux would return a vec of vec of strings, or a vec of maps. The idea here being that the core utilities, instead of doing string in, string out; does clojure data in, clojure data out. I have never used powershell, but heard it does something like this. Is there any Clojure project to replace the shell + coreutils so everything is Clojure value instead of string oriented ?
we could even give it a really catchy acronym like "Really Easy Programming Logistics"
1Like, I think you could write fns in babashka to replace core utilities (e.g. (grep #"foo" '.) or (wc args))
@potetm: it is not obvious to me how to reliably parse tabular output (say ps aux) in babashka
you probably wouldn't want to parse ps . Instead, you would probably want to get the data more directly
I remember there was a python library that did this sort of thing
Right, this is what I meant by replacing not just bash, but bash + core utils; to have them output not string, but Clojure data structures.
Yeah, I’m saying re-write ps in babashka. (I guess it could wrap a shell call to ps, but that’s a little beside the point).
@qqq Does https://github.com/dundalek/closh satisfy at least part of your requirements?
@borkdude: so I think part of the problem is that ps aux would truncate fields right? so the full data isn't even there, if you had a command that was > 40 chars long, the 'column' that psaux displays would likely only display the first 40 chars of it, so there is no way to recover the rest of the data
Most implementations of 'ps' that I have used have options that avoid truncation, and even sometimes use consistent units for things like resident/virtual memory in order to make them easier for a program to parse (e.g. use a many-digit number in the same units for every process, instead of using M or G as suffixes for mega and giga)
Of course different Linux distributions, and/or BSD OS instead of Linux, and macOS, have variations in their implementation of 'ps' that ideally you would want to hide, but I suspect it might be less work overall to try to parse text output of 'ps' with carefully selected command line arguments than it would be to reimplement a big chunk of 'ps' from scratch on each platform.
@qqq You can use java.lang.ProcessHandle to get information about all processes on your system. I'd just use that (and build a small idiomatic Clojure wrapper around the parts I would need most).
@qqq I’m not suggesting that what you’re asking for exists (though Sean’s link might be the answer). I’m suggesting the pieces are out there for someone to do some fairly light work to get something like what you’re saying up and running.
@seancorfield: Closh looks like an interesting mix of bash + Clojure, but not quite what I want. The point I failed to clarify earlier, is that I think any 'solution' would have to replace not only bash, but bash + core utils. The concrete example of this is: suppose you want to look for something in the output of 'ps aux' A traditional ps aux might give you, say, 80 columns, with 40 of them dedicated to the program + args. Even assuming this could be reliably parsed, there is the problem that the full program + args might be longer than 40 chars, and this truncated (think some long java line specify class paths all over the place). So basically here, by the time 'ps aux' has converted the data to a string format, we've already lost the data we want. Therefore, we probably need something 'ps'-ish, but done so via JVM calls, and returning Java/Clojure data. If we agree with the above argument (and it's a bit handwavey), we need a solution that tries to replace not only bash, but bash + core utils.
It's a cool idea but in bb I'm mostly concerned with JVM compatibility and keeping things cross platform. There aren't any unix/bash specific things in bb. There is babashka.fs for file system utils, there is babashka.process for shelling out / launching processes.
babashka.fs has some things which are like core utils, it has fs/which for example, which looks up an executable on the path, which works in Windows as well
One thing that does give me hope is that because the JVM does so much, core-utils-like on top of JVM is probably a lot less work than core-utils in C.
There could be a little shim over ProcessHandle which acts like ps, I haven't thought about that, since I haven't really needed it yet
I don't know if this matches other's experience, but I think 95%+ of my time in a shell consists of: setting env variable, running a command, or running some type of read only "generate | filter" to get data. Clojure REPL is already great at the first two. We already also have map reduce filter for the third; so the main problem it seems to be: right now, when clojure interacts with core utils, it's getting strings back; and if only we had something like clj.query-utils which returned clojure data, we could probably do everything efficiently in a Clojure REPL.
https://github.com/kellyjonbrazil/jc @qqq this may be of interest both as prior art in wrapping common coreutils to get structured output and as an intermediate solution to your problem that you could shell out to if you want nicely parseable data structures from bash tools. https://www.nushell.sh/ https://github.com/PowerShell/PowerShell both powershell and nushell are based around piping around structured output instead of strings and are both cross-platform. EDN support for one or both could be a powerful way to get a lot of coreutils-like things for a clojure environment from the work that's been put into those shell projects.
Doesn't powershell by default pipe around actual CLR objects, not just byte streams?
@UFTRLDZEW: https://www.nushell.sh/ looks really cool. Thanks!
Another possible solution to this would be if Linux commands, in addition to optimizing for human visual reading, also had an option to dump raw json / edn of the data; which Clojure (and other languages) could then easily ingest.
there's https://github.com/babashka/fs for the filesystem side of this
I think you'd be better off emulating coreutils with built-ins like this is
@corasaurus-hex: for the coreutils case, I agree with you that emulating in Clojure is better. However, for the general case, for things like, say, Docker, one really needs the program itself to support json output. On the positive side, I think I finally get why aws cli tools output json -- they're optimized not so much for human consumption, but for other programs to wrap.
docker has an API you can use https://github.com/into-docker/clj-docker-client
and so does AWS. shelling out is ok but the CLI tools often have strange behaviors that are harder to handle than a proper API.
I say this having wrapped many, many, many tools by shelling out to them at my previous job, it really was a lot less fun
https://github.com/kellyjonbrazil/jc @qqq this may be of interest both as prior art in wrapping common coreutils to get structured output and as an intermediate solution to your problem that you could shell out to if you want nicely parseable data structures from bash tools. https://www.nushell.sh/ https://github.com/PowerShell/PowerShell both powershell and nushell are based around piping around structured output instead of strings and are both cross-platform. EDN support for one or both could be a powerful way to get a lot of coreutils-like things for a clojure environment from the work that's been put into those shell projects.