Fork me on GitHub
#clojure
<
2018-03-30
>
zentrope01:03:00

What do people use to create a runnable uberjar with tools.deps?

zentrope01:03:52

Hm. juxt/pack-alpha doesn’t take a main-class option, it seems.

dominicm09:03:03

I'm expecting pack to land that today

👍 4
dominicm09:03:34

At least for capsule. Doing a big refactor.

zentrope15:03:32

I’ll check it out.

dominicm17:03:03

Just pushed it up 🙂 Let me know.

zentrope17:03:41

Ok. Thanks!

dominicm18:03:35

Not using :gen-class is a nice feature imo

zentrope18:03:27

Are all the params required?

dominicm18:03:21

Nope, I should add that to the --help text.

zentrope18:03:56

I got an NPE from forgetting the -O flag (not the jar name, just the flag).

zentrope18:03:13

I can get away with a jar name and a main class, so it works, far as I’m concerned.

zentrope18:03:58

I can now just clojure -Auberjar.

dominicm18:03:11

application-id and version are quite important, although things generally work without them.

zentrope18:03:58

Interesting. :main-opts doesn’t have to be a vector of strings. Can remove the quotes.

zentrope18:03:03

What are those used for?

zentrope18:03:35

Capsule is a jar of jars? Didn’t know that was possible! ;)

dominicm18:03:38

Caching, speeds up startup time

dominicm18:03:15

The cache is of those jars.

zentrope18:03:34

Is that a JDK9 thing? (the app-id speeding up startup)

dominicm18:03:44

No, it's a Capsule thing 🙂

zentrope18:03:28

Ah, I see a ~/.capsule area.

zentrope18:03:53

I can hand this jar off to someone who only has a JVM and it’ll still work, though, right?

dominicm18:03:57

The whole idea is that jar of jars is transparent to the user, but using jar of jars avoids legal issues around the LICENSE file & avoids performing conflict resolutions.

zentrope18:03:26

Yes, I think it’s great.

dominicm18:03:43

I have a blog post coming out soon explaining why I took the approach I did.

zentrope18:03:50

What are the implications if I never change the version number, but sometimes update the dependency versions?

dominicm18:03:59

I haven't confirmed this, but I fear it may use the old dependency versions. The documentation isn't particularly clear on whether this is entirely true though.

dominicm19:03:04

I shall experiment 🙂

zentrope19:03:32

I changed one of my deps to an earlier version, and ~/.capsule didn’t change.

zentrope19:03:59

BUT, the actual uberjar DOES have the new deps.

dominicm19:03:41

I don't think they will be extracted to ~/.capsule, and therefore will not be used though.

dominicm19:03:21

~/src/github.com/juxt/edge/app master*
❯ java -jar uberjar.jar
20:08:09.468 [main] INFO  edge.httpd - Started http server on port 3080
#object[java.net.URL 0x4c296f0a "jar:file:/home/dominic/.capsule/apps/edgedge_1.0/AA4BA0A6F7E0F7749A09E72B4937AF04B2B758C2A0AE09F57C5DB4955977D555-selmer-1.10.8.jar!/selmer/filter_parser.clj"]
^C
~/src/github.com/juxt/edge/app master* 34s
❯ nvim
~/src/github.com/juxt/edge/app master* 16s
❯ clj -A:user/pack -m mach.pack.alpha.capsule -O uberjar.jar --application-id edgedge --application-version 1.0 -m edge.main
Downloading: selmer/selmer/1.11.7/selmer-1.11.7.pom from 
Downloading: selmer/selmer/1.11.7/selmer-1.11.7.jar from 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See  for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
~/src/github.com/juxt/edge/app master* 15s
❯ java -jar uberjar.jar
20:09:38.522 [main] INFO  edge.httpd - Started http server on port 3080
#object[java.net.URL 0x7d702767 "jar:file:/home/dominic/.capsule/apps/edgedge_1.0/C46F0C8C485D01E2A831E5C332485C0265FBFE0C66D8565F4204C30C90C59D27-selmer-1.11.7.jar!/selmer/filter_parser.clj"]
I'm not sure how this works, but it definitely updated the version.

dominicm19:03:25

I need to read more of the Capsule source code, but java.

zentrope19:03:17

Why is the uberjar loading classes from ~/.capsule?

zentrope19:03:26

When you run the uberjar, it unpacks to that directory.

zentrope19:03:35

(I thought it was a build-time thing, somehow.)

dominicm19:03:45

No, it's a runtime thing.

dominicm19:03:16

The JCL loader doesn't require the unpack step, and works a little differently. But the JCL-style approaches have to do funky stuff for native libraries afaict.

dominicm19:03:45

I need to explore Capsule's limits a little further, and ensure I grasp it.

zentrope19:03:58

Okay. So, I retract my previous observation. The ~/.capsule stuff gets the new dependency once you actually run the jar.

dominicm19:03:31

The trampolining system means you can bake in JVM arguments for example, which is very cool.

dominicm19:03:49

It's a neat idea to say that production always runs with 6G of memory at build time.

zentrope19:03:25

Do production folks like the idea of a ~/.capsule on servers? (Assuming someone’s still left not using docker.)

dominicm19:03:51

Yeah, that's a possible risk in some environments. For that case you need to switch to JCL. I have found something (also called JCL) which is not GPL licensed, so is safe for enterprise use.

zentrope19:03:33

One use case I have is to produce a Jar which some support folks run on a customer’s box for troubleshooting. In that case, having it unpack stuff as a matter of course is probably a bad idea.

zentrope19:03:48

But there’s always lein’s uberjar, full AOT, etc, etc.

dominicm19:03:42

it's worth noting that if it doesn't have permission, it will unpack into a temporary directory.

dominicm19:03:24

it might even be possible to force it to do that :thinking_face:

zentrope19:03:38

It’s not that it won’t work, it’s that people will object. Either the people using it, or the customer.

zentrope19:03:01

It’s just one possible use case for something totally self-contained.

zentrope19:03:22

I don’t know that juxt/pack ought to care about that.

dominicm19:03:44

JCL is definitely the answer for this case anyway 🙂

dominicm19:03:06

Which is a shame, because then you can't have default JVM arguments, which is probably handy on a customer's box.

dominicm19:03:17

I'm trying to balance 2 competing priorities to work on next: 1. Figuring out arguments 2. Integrating Xeus JCL and removing jdsoft JCL. I'd prefer to do 1 before 2.

zentrope19:03:24

GPLv3. Oy. Working for “enterprises” is no fun.

dominicm19:03:40

I work for enterprises too.

dominicm19:03:57

It's why I'm aware that lein uberjars violate the APLv2 and BSD licenses, no fun there either I'm afraid.

zentrope19:03:21

Oh, they do? No one at the legal dept I’ve worked with has noticed.

zentrope19:03:30

They just care about third-party libs and the licenses.

dominicm19:03:11

Yeah, I don't think anyone has really checked, but they're exposed. If someone decided to sue, they could absolutely do so.

dominicm19:03:23

Or rather, whatever breaching their license lets them do.

dominicm19:03:51

I'm mostly happy with tools.cli, but I feel like more could be done, so I want to fiddle with it some more to figure out arguments for basic things like AWS Lambda.

dominicm19:03:16

Then I can do Xeus JCL. Xeus JCL is licensed under the APLv2, so all is good there.

dominicm19:03:19

I am wondering if you could nest Xeus inside a Capsule to have some "best of both" situation here.

dominicm19:03:25

that would be pretty cool.

zentrope19:03:19

How do you get an uberjar to use a specific class loader?

zentrope19:03:33

(I mean, super high level.)

zentrope19:03:38

I guess I can check out the code.

dominicm19:03:52

You have to utilise the entrypoint, in the case of JCL, I actually have some java code which acts as the "Main", which then delegates.

zentrope19:03:27

I’ve used the “stub.clj” technque for avoiding giant AOT jars.

zentrope19:03:53

Ah. I even saw that code before, but didn’t know what it was doing. Interesting.

dominicm19:03:27

I'll end up writing a bootstrap for Xeus too 🙂

dominicm19:03:58

btw, contributions welcome here. If you urgently want to add Xeus, I don't mind it having positional arguments & such. You can probably copy the existing jcl, and modify it slightly.

zentrope19:03:45

I’ve no urgency at all. I’m mainly interested in how much of lein/boot can be replaced with clj for most of what I do.

zentrope19:03:12

BUT, when it comes time to make totally self-contained uberjars, I’ll know where to look.

dominicm19:03:45

Right, I'm off to bed. If you have any more questions, feel free to ask 🙂

zentrope19:03:16

Ok. Thanks for that main-class thing. Helps a lot!

dominicm19:03:42

No problem at all 🙂

dominicm19:03:51

I was asked for it yesterday too.

zentrope01:03:26

I tried depstar, but it doesn’t add a Main-class: either. Hm.

ghadi01:03:51

@zentrope we went minimalist with depstar. we do java -cp thejar.jar clojure.main -m our.main.namespace to run stuff

ghadi01:03:17

PR certainly welcome

zentrope01:03:49

Right. I understand how these things work. But I like an uberjar you can just “java -jar theapp.jar” for ease of use for the folks I want it to be easy for. ;)

zentrope01:03:35

Perhaps adding a META-INF/MANIFEST.MF as some resource (as in the webassets example) would do it. ;)

ghadi01:03:52

you can add the manifest as a post-processing step

ghadi01:03:48

the code should be easy to follow, let me know if you have any questions

zentrope01:03:02

It’s a time thing.

ghadi01:03:34

i'm rolling my eyes a bit at that simple_smile call jar --update --file yourjar --manifest YOURS.MF

ghadi01:03:49

which will add your own custom manifest with whatever you desire

zentrope01:03:03

No, I mean I don’t quite have time at the moment to offer a PR that allows you to set the main-class in manifest.mf.

ghadi01:03:41

ah, ok. a post-processing step should do the trick

ghadi01:03:57

there's an even easier way. jar --update --file yourjar --main-class your.main.class (Keep in mind depstar doesn't AOT)

zentrope01:03:56

And those opts don’t seem to work on my version of jar. Interesting.

ghadi01:03:10

interesting. java version?

ghadi01:03:31

ah, I'm on 9

zentrope01:03:01

The options are there, just single letters.

ghadi01:03:39

might need = after file and main class

ghadi01:03:15

hope that helps!

zentrope01:03:30

Heh. This is crazy. ;) (I mean, me completely breaking jar.)

zentrope02:03:09

jar -ufe the.jar the.class

ghadi01:03:36

jar --update --file=your.jar --main-class=your.main.class (Keep in mind depstar doesn't AOT automatically)

zentrope02:03:59

How do you AOT compile? Huh.

zentrope02:03:53

Oh. (compile my.app.namespace).

zentrope02:03:50

So when you start a clj repl, none of the code on the src path is loaded, even when you (ns path.to.main)?

ghadi02:03:52

that's right -- but that behavior is clojure itself, not clj.

ghadi02:03:29

ns is just a declaration, require or load actually do the loading

zentrope02:03:54

I guess I never quite worked all that out, given the other repls I’ve used.

qqq03:03:25

I'm looking at https://github.com/clojure/tools.reader . If I want the "smallest . simplest" library capable of fully parsing clj/cljs/cljc, is tools.reader the lib to go, or is there something 'more minimal' ?[I already have tools.reader working, but am wondering about alternatives]

arrdem03:03:14

@qqq parsing Clojure is nontrivial and tools.reader is the basis for most of the projects which do so.

arrdem03:03:43

technically it's quite easy but you won't gain anything from doing it yourself, many of the token patterns are finicky and t.r is quite good.

qqq03:03:23

@arrdem: thanks for the insight, must be one of these situations where parsing 90% of clojure is easy, but parsing that last 10% requires lots of extra edge cases.

bronsa06:03:37

@qqq that's exactly the case, same for analysing clojure and tools.analyzer.jvm, it seems like it should be not too hard to do as the number of special forms is very small but it turns out that getting all the nuances right can be quite crucial and quite hard to do - - core. async is a good example of this where they used a custom analyser at first and then switched over to t.a

bronsa06:03:43

for this kind of stuff having a single library responsible for handling all of this hidden complexity is crucial to ensure robustness

👍 4
qqq06:03:32

@bronsa: I appreciate all the commits you made to the tools.reader project!

qqq07:03:17

Is there a tutorial on how to hijack the widgets from Eclipse or NetBeans for building your own IDE? Googling "using Eclipse as a Library" ==> "how to load Libraries in Eclipse" "how to use Eclipse from clojure" ==> "Counterclockwise is a Eclipse plugin for Clojure dev" etc ... This is almost impossible to google.

robert-stuttaford07:03:12

if there is, i doubt #clojure will know 👼 perhaps ask on the counterclockwise google group, or ask known maintainers directly?

4
qqq07:03:05

I figured, if it's possible to use Eclipse/Netbean widgets from Clojure as GUI elements, someone here would know someone who got it to work.

bronsa07:03:04

i dont even know that ccw is still maintained

bronsa07:03:28

oh sorry that wasn't your question, misread

Michael Fiano08:03:06

If I have the parameter vector [[a b] x y] in a defmethod, how can I also bind the whole [a b] vector to another variable, like I can using :as in a let binding?

bronsa08:03:22

[a b :as c]

sveri08:03:24

@qqq eclipse uses equinox as their own osgi implementation. So most libraries are just osgi bundles that you can use. At our company we build a lot of software on top of eclipse and if I had the choice I would not be doing that. Development is clunky and for UI tests you have to use SWTBot which is a PITA, slow and error prone.

sveri08:03:03

Although I have to admit I cannot compare it to something similar, never used electron or developed an intellij plugin. So, might be that the other choices are as "bad" as eclipse.

sveri08:03:22

So what I would do, is download eclipse and start a new plugin project. I am not sure if it makes sense to pick single plugins and use them for your own project. At least I have never seen that before. IBM for instance also takes a barebone eclipse and builds on top of that.

geraldodev10:03:55

What is the name of that site that indexes clojure code ?

qqq11:03:39

@sveri: I just spent the past hour reading about SWT and RCP. What I got was a follows: SWT: very easy to use from Clojure, already got it working Eclipse RCP: Provides more GUI elements ... but it wants to be in charge; I can't figured out how to call it from Clojure; I would have to write Clojure as a 'RCP plugin', and to create this RCP plugin, I have to use their GUI wizard tool, which generates a bunch of XML. Is this correct? Basically: SWT = easy to use from Clojure. RCP = more powerful UI elements, unfortunately, near impossible to use from Clojure

sveri12:03:54

@qqq I read a lot of good things about https://github.com/daveray/seesaw if you want to do a java ui with clojure. Last time I tried there were also two experimental wrappers around JavaFX if you want to stick to the newest. Just wondering why did you want to pick parts of eclipse? Is there a specific reason?

schmee12:03:35

I'm trying to create some custom exception classes for adding some tests to the Clojure compiler

schmee12:03:18

so I'm in the namespace clojure.test-clojure.try-catch, and in this file I've added the line

(gen-class :name try_catch.A1 :impl-ns clojure.test-clojure.try-catch :extends java.lang.Exception)

schmee12:03:30

but when I try to compile this I get ClassNotFound errors

schmee12:03:31

any ideas?

schmee12:03:34

I got the tests working in my own scrap project, but I can't figure out what :name and :impl-ns should be set to in the particular case

bronsa12:03:08

gen-class only works during AOT

bronsa12:03:11

it's a no-op while JIT

schmee12:03:53

yup, I think what I need to do is to put the gen-classes in a separate ns and add that to the compile-test in pom.xml

schmee12:03:58

lemme see if that works

Alex Miller (Clojure team)12:03:50

It’s probably easier to just write them in Java

Alex Miller (Clojure team)12:03:08

But what you’re saying should work

schmee12:03:42

yeah, I thought about writing them in Java, but if I did that I wouldn't learn anything 🙂

bronsa13:03:37

fwiw the only reason why gen-class doesn't do genclassing while JITing is that I forgot to enable it when we did the classloader changes in 1.7, while we did make that change for geninterface

bronsa13:03:00

I'll make a patch + ticket about, it it's a 1 line change

schmee13:03:43

just cause I'm curious, could you link the ticket with the classloader changes you are referring to?

bronsa13:03:57

CLJ-979

👍 4
schmee13:03:10

I have part of macro which looks like this:

`(let [ex# (try
              ~form
              (catch ~class e# e#)
              (catch Exception e#
                (let [cause# (.getCause e#)]
                  (if (= ~class (class cause#)) cause# (throw e#)))))]
    "do some things")
I want to change it so that the last catch clause it not added when the class is Exception
(let [ex
      `(try
         ~form
         (catch ~class e# e#))
      default
      `(catch Exception e#
        (let [cause# (.getCause e#)]
          (if (= ~class (class cause#)) cause# (throw e#))))]
   `(let [ex# ~(if (= class Exception) ex (conj ex default))]
      "do some things"))
but that seems a bit verbose. Is there a better way to "ignore" a form when syntax quoting?

bronsa13:03:48

[email protected](when-not condition [..])

schmee13:03:24

ahh, so nil is dropped when using [email protected]?

bronsa13:03:30

the trick works because [email protected] "disappears"

schmee13:03:39

ahh, nice, thanks 🙂

bronsa13:03:14

it's equivalent to concatting nil

bronsa13:03:45

if you're curious about how syntax-quote works, just stick a quote in front of a syntax-quote expression

cesarmarinhorj14:03:16

Hi all!!! Again in clojure challenge. :-)

cesarmarinhorj14:03:37

I gives up from first time, some months ago, but I am back.... :-)

cesarmarinhorj14:03:01

Ops... Gived up...

cesarmarinhorj14:03:36

Now, after learning more from other languages, fear is more under control...

cesarmarinhorj14:03:52

Back to lisp world again and incredible and amazing clojure lang

cesarmarinhorj14:03:42

In loop nightmares now, but getting slow.....lisp is amazing...

cesarmarinhorj14:03:22

Too many projects in clojure nowadays! Nice to see things like this...

🚀 12
bronsa14:03:16

welcome back! fyi there's a #beginners channel if you're just starting out (or re-starting :) )

lwhorton16:03:15

how would one alias a namespaced keyword to just defer to another namespace? something like:

;; a-ns
(require [my-ns])
{::my-ns/foo 1}

;; my-ns
(require [other])
(def foo ::other/foo)

ghadi16:03:38

double :: always needs a namespace alias

lwhorton16:03:41

this isn’t right because the (def ...) shouldn’t really be defining a var, because it’s just a keyword that exists in foo, not a var. is there some way to just say “all references to my namespaced keyword should really just be this other namespaced keyword”

ghadi16:03:58

after (require '[foo.bar :as bar]) then you can say ::bar/whatever

lwhorton16:03:22

without an alias I can just do :other.ns/bar though right?

✔️ 4
ghadi16:03:34

but you can't just say ::my-ns/foo because my-ns isn't an alias, it's a whole namespace

ghadi16:03:44

you can however, say :my-ns/foo (single colon)

lwhorton16:03:39

so assuming I got the first part right, where I alias my-ns and do ::my-ns/foo… can I define every keyword that might be ::my-ns/bar ::my-ns/baz ::my-ns/buz to instead (inside my-ns) point to ::other-ns/bar, baz, buz?

lwhorton16:03:12

i’m trying to hide the implementation details of some third-party library, essentially, so that consumers only use my code, and therefore only have to require one namespace, and use all ns-keywords via my interface and not third-party lib’s ns

ghadi16:03:21

yes you can alias specs

lwhorton16:03:52

okay cool, so this is valid: (in my-ns) (def bar ::other-ns/bar)

christos18:03:56

Hi, I am trying to add vertices on a TinkerGraph instance: ( def test-graph (TinkerGraph/open)) (.addVertex test-graph T/id "name") But i get the following error : No matching method found: addVertex for class org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph . In java the implementation would be: test-graph.addVertex(T.id, "name"), and it seems that the addVertex instance method accepts some kind of key value pair object. Can someone help ?

hiredman18:03:33

.addVertex takes java varargs, at the jvm level this is implement as passing an array, in clojure you have to construct the array yourself

😂 4
christos18:03:13

my god you are right

christos18:03:18

thank you so much

christos18:03:42

was driving me crazy

borkdude19:03:59

What’s the best channel to ask about manifold?

borkdude19:03:17

Anyway, I’ve got this in a normal let:

(let [ ... ...
xml-deferred (grobid/pdf->xml-async grobid-cfg pdf)
authors (d/chain xml-deferred grobid/authors)
] ...)
 
In grobid/pdf->xml-async it’s possible that an exception happens. I would assume that chaining manifold deferred skips the next step in the chain, but in this case it doesn’t.

greglook19:03:47

d/chain should skip the next step(s) if the deferred results in an error - what makes you think grobid/authors is being called? You can use d/catch to handle deferred exceptions.

greglook19:03:26

if the pdf->xml-async function returns the exception as a value instead of throwing it you’d have a problem; in that case you should wrap the exception in (d/error-deferred ex)

borkdude19:03:10

@U8XJ15DTK pdf-xml-async returns a deferred with xml in it if everything goes well. Then I chain on that deferred.

greglook19:03:22

Right - you said that you expect grobid/authors to be skipped for exceptions, but it’s not. What makes you think it’s running?

borkdude19:03:19

I now put a debug statement in there and I see that being printed. I suspect that aleph client (which I use in pdf->xml-async) doesn’t throw when it gets a bad response

borkdude19:03:23

Thanks, I can solve it now

borkdude20:03:57

When I wrap the body of pdf->xml-async in a catch, the catch works properly. When I don’t do that, grobid/authors gets executed…. That’s a bit odd

borkdude20:03:04

@U8XJ15DTK Hmm:

xml-deferred
        (d/catch
            (grobid/pdf->xml-async grobid-cfg pdf)
            Exception
          (fn [ex] ;; ex :: clojure.lang.ExceptionInfo
            ;; (throw ex) results in unterminating state
            (throw (Exception. “no”)) ;; this gives a correct response
            
            ))
        authors (d/chain xml-deferred grobid/authors)

borkdude20:03:05

if I don’t write the catch, grobid/authors is executed

greglook20:03:49

what do you get when you print out the exception in the catch?

borkdude20:03:54

note that grobid/pdf->xml is itself also a d/chain

borkdude20:03:29

(hope you can open that link)

greglook20:03:34

I’m not sure how rethrowing the exception could result in a hung deferred value

borkdude20:03:38

This is the body of pdf->xml-async:

(defn pdf->xml-async
  “Sends PDF file or byte array to Grobid and returns TEI XML format. Async.”
  [{:keys [host port]} pdf]
  (d/chain
   (client/post
    (str “http://” host “:” port
         “/api/processFulltextDocument”)
    {:multipart [{:name “input” :content pdf}]})
   :body
   ;; assuming UTF-8, GROBID doesn’t send character encoding
   bs/to-string
   ;; Poor man’s dehyphenization.
   #(str/replace % “- ” “”)))

borkdude20:03:03

where client is [aleph.http :as client]

borkdude20:03:42

so the response is a 400

borkdude20:03:56

don’t know what this is: :aleph/complete << false >>

greglook20:03:17

the << foo >> syntax is how manifold deferred values print

borkdude20:03:33

ok, so an error-deferred is returned which should be good?

greglook20:03:02

it certainly looks like pdf->xml-async is doing the right thing

greglook20:03:28

I’d expect you to be able to do something like:

(->
  (pdf->xml-async grobid-cfg pdf)
  (d/chain grobid-authors)
  (d/catch Exception
    (fn [ex]
      ; handle errors...
      ,,,)))

greglook20:03:14

Sorry I can’t be of more help - everything you’ve sent me looks like it ought to work

borkdude20:03:07

No problem, thanks for thinking along

borkdude21:03:38

Somehow the body of the aleph response is causing trouble. When I select it out and re-throw the ExInfo it works…

borkdude21:03:30

E.g. this works:

(d/catch clojure.lang.ExceptionInfo
          (fn [ex]
            (let [d (ex-data ex)]
              (throw (ex-info (.getMessage ex)
                              (update d :body slurp))))))

emccue19:03:16

@xjongar Actually, I want to emphasize that a key function for sorting isn't a clojure exclusive concept. (https://developers.google.com/edu/python/sorting) Basically, its your way of sorting a non-comparable item by computing some comparable property of it. If we try and sort [[1 2 3], [8 9], [10]], then it should be clear that there is no obvious and unambiguous way to sort these lists. What we can do is provide something like count as a key function, in which case we would end up sorting the lists by length. If we provide sum then we would sort the lists by the sum of the items within them. Its an unfortunate case of terminology clash with clojures keyword. if we have a record (def r {:a 1 :b 2 : c 3}) then to extract a property from that record we can use a keyword as a function (:c r) ;; => 3, which as a coincedence we can use as a key function in our sorting. For example: sorting [{:a 3} {:a 1} {:a 2}] with the key function :a would sort everything by the value that record has in its :a field. TL;DR Key Functions are not Keyword Functions but a Keyword Function can be a Key Function and thats the Key to understanding those Functions

noisesmith19:03:08

@emccue overall you are right, but a small pedantic point with that example, clojure does already know how to compare vectors user=> (sort [[1 2] [1] [1 3]]) -> ([1] [1 2] [1 3])

emccue19:03:18

how about sets?

noisesmith19:03:02

yeah, sets are not sortable (but you'd need a more complex sort-by function than just a key) - I wasn't discounting your overall point at all, just pointing something out about that detail

jcr22:03:55

Can tools.deps import a single function?

jcr22:03:01

Is it a planned feature? I would love to have an easy way import those map-vals and map-keys from some gist or something.

jcr22:03:59

It would also (hopefully) help to solve a lot of "omg, why is THAT function not in core?!"-type complains.

hiredman22:03:02

how do you imagine that working?

dominicm22:03:47

:thinking_face: you could load the code into a sandbox, then use a serialization technique to grab just the var+dependencies you need. This essentially fetches dependencies, and then performs tree shaking. Then reverse it in the host.

dnolen22:03:42

both transit-clj & transit-cljs now support writing metadata, give it a spin if you find that interesting

hiredman22:03:53

like, if all you want is to load clojure code from a file over http, that is super easy

dominicm22:03:40

Yeah, I mean that's basically it. With automatic pruning like Rich described though I guess?

hiredman22:03:51

at my last job if we had to monkey patch in production that is usually how I did it, save the code as a private gist, then essentially broadcast a command to all the servers to load the clojure from the file

hiredman22:03:34

(from the url)

tbaldridge22:03:55

You'd have a fair amount of problems with versioning though if it's not done correctly, or have problems with state being off

seancorfield22:03:15

@just_crashed A Gist can be multi-file so it could have a deps.edn as well as the actual source files, yes?

tbaldridge22:03:23

Like imagine if spec's internal format changed between versions, but two different fns required different versions of spec

seancorfield22:03:01

I saw someone using clj to run code from a Gist as a dependency I think...

hiredman22:03:06

I'll believe in clojure tree shaking when I see it

dominicm22:03:21

@hiredman portkey does it.

hiredman22:03:44

I immediately doubt the effectiveness of it

dominicm22:03:30

It seems to work, but I've not thrown anything complicated at it. I'm not sure how it works.

jcr22:03:02

@hiredman haven't given it much thought yet to be honest; the idea is not new though. I bet I have seen something like this in some language, but I can't quite remember which one it was. You could do something like (translated to clojure) (require "http://..." :refer [foo]) - is it possible in golang?.. Eh, anyway. The idea is to allow single functions to be distributed (as opposed to full-blown packages with deps and whatnot). I imagine it would work by creating a mapping from say a gist to a fake namespace, then referring that namespace in the actual code. Wrt transitive deps - not sure, personally I'd be ok if transitive deps wouldn't be allowed - just fetch the code and put it in a ns. In any case, it's just an idea, love to see the discussion

hiredman22:03:44

that is hardly "importing" a single function, that is just load code from a url

hiredman22:03:52

super easy to do

jcr22:03:47

@seancorfield thanks for the link; well, yes, maybe it's actually enough. One can combine it with a simple script that adds deps.edn to the gist and generates a unique namespace for that based on the url or something.

tbaldridge22:03:31

(require '[gist.AF232343DFA])

schmee22:03:18

speaking of dependencies, has anyone tried to use jlink to build Clojure with a tailored Java runtime?

hiredman22:03:48

(with-open [i (.getContent (java.net.URL. "")) r (clojure.lang.LineNumberingPushbackReader. ( i))] (loop [] (let [form (read r false ::foo)] (if (= form ::foo) nil (do (eval form) (recur))))))

hiredman22:03:16

that loads the namespace from that gist

hiredman22:03:09

alternatively you can do things with classloaders, which I think you could make require work as is if you did it at the classloader level

sundarj22:03:08

or slurp + load-string?

hiredman22:03:51

yeah, but slurp reading the string in to memory all at once is gross

sundarj22:03:26

fair enough

dominicm22:03:09

I like the class loader idea, I want to try that.

hiredman22:03:27

I've got some gists for you

dominicm22:03:55

It's a shame clojure doesn't seem to be friendly to swapping the classloader.

dominicm22:03:05

Not from within clojure anyway.

seancorfield22:03:25

@just_crashed here's a Gist with map-vals and map-keys showing how to use it 🙂

hiredman22:03:34

so that is how I would do clojure shebang scripts, loading pomegranate via a classloader, then using pomegranate to load whatever deps

hiredman22:03:13

the shebang line like that doesn't work on linux(I was using osx at the time), but with some changes you can get it to work

jcr22:03:43

@hiredman yeah, I see what you mean; it's a nice trick, but I was thinking about something you can actually include as a dependency (caching, cljs, advanced optimisations, etc). So I guess multifile gist with deps.edn is better in this regard. Also, now thinking of that, it'd kinda sucks without default (project-wide) :require clauses. Say you want to import 5 functions you think have to be in core. That'd mean including 5 [gist.a8fb63ec64704ecb967f :refer [foo]] inside :require's of every ns. I guess it'd be nice to squash all gist-imports into a single namespace upon arrival on your machine. Well, or you can just monkey patch clojure.core... yeeaahh...

jcr22:03:45

@seancorfield haha, nice. Self-contained gists should totally be a thing.

jcr22:03:07

Or rather "self-including"?.. anyway, just thinking aloud

seancorfield22:03:42

@just_crashed What do you need beyond that? I'm not sure what you think is missing?

seancorfield22:03:07

Gist-as-library.

seancorfield22:03:48

The only "downside" is you have to use a single-segment ns because Gists are "flat", so seancorfield-map-utils would be a better name there...

seancorfield22:03:24

(and you have to know the actual SHA as well as the URL)

hiredman22:03:06

there are also two different levels of "import"

hiredman22:03:24

there is make "available to the project", which generally means an entry in your project.clj, build.boot, deps.edn, or whatever

hiredman22:03:44

and "make available in this namespace" which is usually done via require

hiredman22:03:56

and you have to do the first to do the latter

jcr22:03:40

@seancorfield gist-as-a-library is definitely cool (I think you probably should use the url as a namespace though). But still, imagine you had an index of all such gists, and you could just write something like "ok, give me this snapshot of the whole repo as a dependency, plus add this gist and that one, put it all under a single namespace and make it available to my project".

jcr22:03:01

So if someone writes a little handy function, he could just go to http://fn-repo.clojars.org, paste it, and then you can immediately :require it in your project if you have connection. Or you could have "stable" community repo of handy functions and an "unstable" one where nothing is moderated. Or if two guys have written functions with the same name, you can easily tell clojure which version to use by its hash. Stuff like that.

jcr22:03:29

@hiredman oh, yeah, I think I have seen that post!

jcr22:03:59

Probably that's where the idea is coming from. Did Rich talk about it in the spec-ulation?

hiredman22:03:06

this email was brought to my attention many years ago, and at the time the group of people I was working with actually found a random joe armstrong on skype and we added him to our standup to try and ask about it, but he never answered our call. and since then when listing the things you need to accomplish some goal, adding letrec at the end has been kind of an in joke.

jcr22:03:56

(I wonder if it could be the real joe armstrong; I would give my life away to say "Hello, Joe" to him on the phone, lol)

jcr22:03:20

Clojurescript doesn't support in-ns, right?

qqq23:03:47

Besides AWt, String, SWT (Eclipse); is there a list of Java UI Libraries available ?

jcr23:03:51

javafx comes to mind; I think that's all

jcr23:03:05

Well, if you don't count java->js translators (gwt, vaadin etc)

Alex Miller (Clojure team)23:03:37

Rich and I have talked about single function import type use cases. As you discovered, there are a lot of questions that seemed larger than the value provided, so we don’t have any immediate plans. We have kicked around a bunch of ideas regarding ns auto loading and some of that may eventually shake out into a feature in Clojure itself or maybe clj.

dominicm23:03:27

By ns auto loading, what do you mean? Wasn't a patch rejected for this already?

sveri23:03:44

A colleague of mine started talking about nanoservices when microservices were all the new hype. I guess thats what he meant with it, functions as a service. I liked the idea back then. But then, seeing the desaster that npm and leftpad and is-thirteen is, I dont like it that much anymore. Imagine having a leiningen project file that includes 100 lines of https://gist.github.com/seancorfield/6e8dd10799e9cc7527da5510c739e52f It says nothing, so first you would have to find a better way to name things than hashes. And even if there is, something like de.sveri.clj.helpers.core.add-in-map you still dont know what it does by looking at it. And you would have to open a repl or a file where the function is called in your IDE to access the docs. I think cleaning up such a dependency file would be horrible.

sveri08:03:12

Thanks for the link, interesting read. I like the approach about having exactly one entry point. That would make more sense than one function. What I dislike is the notion of the being serverless, while clearly there are servers in play. But that is another topic.

tbaldridge23:03:05

@sveri for even more fun, support that sort of deps for lein :plugins. That's the stuff nightmares are made of