This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-03-30
Channels
- # aws (4)
- # beginners (143)
- # boot (37)
- # cider (31)
- # cljs-dev (53)
- # clojure (303)
- # clojure-conj (5)
- # clojure-dev (106)
- # clojure-dusseldorf (2)
- # clojure-greece (3)
- # clojure-italy (23)
- # clojure-spec (83)
- # clojure-uk (7)
- # clojurescript (328)
- # core-async (25)
- # cursive (2)
- # datascript (2)
- # datomic (3)
- # emacs (10)
- # hoplon (1)
- # jobs (2)
- # lein-figwheel (1)
- # leiningen (13)
- # luminus (6)
- # off-topic (38)
- # onyx (2)
- # parinfer (13)
- # pedestal (2)
- # portkey (5)
- # re-frame (11)
- # reagent (2)
- # shadow-cljs (61)
- # specter (6)
- # unrepl (60)
- # vim (4)
application-id and version are quite important, although things generally work without them.
Interesting. :main-opts doesn’t have to be a vector of strings. Can remove the quotes.
I can hand this jar off to someone who only has a JVM and it’ll still work, though, right?
The whole idea is that jar of jars is transparent to the user, but using jar of jars avoids legal issues around the LICENSE file & avoids performing conflict resolutions.
What are the implications if I never change the version number, but sometimes update the dependency versions?
I haven't confirmed this, but I fear it may use the old dependency versions. The documentation isn't particularly clear on whether this is entirely true though.
I don't think they will be extracted to ~/.capsule
, and therefore will not be used though.
~/src/github.com/juxt/edge/app master*
❯ java -jar uberjar.jar
20:08:09.468 [main] INFO edge.httpd - Started http server on port 3080
#object[java.net.URL 0x4c296f0a "jar:file:/home/dominic/.capsule/apps/edgedge_1.0/AA4BA0A6F7E0F7749A09E72B4937AF04B2B758C2A0AE09F57C5DB4955977D555-selmer-1.10.8.jar!/selmer/filter_parser.clj"]
^C
~/src/github.com/juxt/edge/app master* 34s
❯ nvim
~/src/github.com/juxt/edge/app master* 16s
❯ clj -A:user/pack -m mach.pack.alpha.capsule -O uberjar.jar --application-id edgedge --application-version 1.0 -m edge.main
Downloading: selmer/selmer/1.11.7/selmer-1.11.7.pom from
Downloading: selmer/selmer/1.11.7/selmer-1.11.7.jar from
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
~/src/github.com/juxt/edge/app master* 15s
❯ java -jar uberjar.jar
20:09:38.522 [main] INFO edge.httpd - Started http server on port 3080
#object[java.net.URL 0x7d702767 "jar:file:/home/dominic/.capsule/apps/edgedge_1.0/C46F0C8C485D01E2A831E5C332485C0265FBFE0C66D8565F4204C30C90C59D27-selmer-1.11.7.jar!/selmer/filter_parser.clj"]
I'm not sure how this works, but it definitely updated the version.The JCL loader doesn't require the unpack step, and works a little differently. But the JCL-style approaches have to do funky stuff for native libraries afaict.
Okay. So, I retract my previous observation. The ~/.capsule stuff gets the new dependency once you actually run the jar.
The trampolining system means you can bake in JVM arguments for example, which is very cool.
It's a neat idea to say that production always runs with 6G of memory at build time.
Do production folks like the idea of a ~/.capsule on servers? (Assuming someone’s still left not using docker.)
Yeah, that's a possible risk in some environments. For that case you need to switch to JCL. I have found something (also called JCL) which is not GPL licensed, so is safe for enterprise use.
One use case I have is to produce a Jar which some support folks run on a customer’s box for troubleshooting. In that case, having it unpack stuff as a matter of course is probably a bad idea.
it's worth noting that if it doesn't have permission, it will unpack into a temporary directory.
It’s not that it won’t work, it’s that people will object. Either the people using it, or the customer.
Which is a shame, because then you can't have default JVM arguments, which is probably handy on a customer's box.
I'm trying to balance 2 competing priorities to work on next: 1. Figuring out arguments 2. Integrating Xeus JCL and removing jdsoft JCL. I'd prefer to do 1 before 2.
It's why I'm aware that lein uberjars violate the APLv2 and BSD licenses, no fun there either I'm afraid.
Yeah, I don't think anyone has really checked, but they're exposed. If someone decided to sue, they could absolutely do so.
I'm mostly happy with tools.cli, but I feel like more could be done, so I want to fiddle with it some more to figure out arguments for basic things like AWS Lambda.
I am wondering if you could nest Xeus inside a Capsule to have some "best of both" situation here.
You have to utilise the entrypoint, in the case of JCL, I actually have some java code which acts as the "Main", which then delegates.
btw, contributions welcome here. If you urgently want to add Xeus, I don't mind it having positional arguments & such. You can probably copy the existing jcl, and modify it slightly.
I’ve no urgency at all. I’m mainly interested in how much of lein/boot can be replaced with clj for most of what I do.
BUT, when it comes time to make totally self-contained uberjars, I’ll know where to look.
also see https://github.com/clojure/tools.deps.alpha/wiki/Tools for a list of tools, including depstar https://github.com/healthfinch/depstar
send ’em a pr
@zentrope we went minimalist with depstar. we do java -cp thejar.jar clojure.main -m our.main.namespace
to run stuff
Right. I understand how these things work. But I like an uberjar you can just “java -jar theapp.jar” for ease of use for the folks I want it to be easy for. ;)
Perhaps adding a META-INF/MANIFEST.MF as some resource (as in the webassets example) would do it. ;)
it won't -- we explicitly filter certain stuff in META-INF/ https://github.com/healthfinch/depstar/blob/master/src/hf/depstar/uberjar.clj#L72
No, I mean I don’t quite have time at the moment to offer a PR that allows you to set the main-class in manifest.mf.
there's an even easier way. jar --update --file yourjar --main-class your.main.class
(Keep in mind depstar doesn't AOT)
jar --update --file=your.jar --main-class=your.main.class
(Keep in mind depstar doesn't AOT automatically)
So when you start a clj
repl, none of the code on the src path is loaded, even when you (ns path.to.main)
?
I'm looking at https://github.com/clojure/tools.reader . If I want the "smallest . simplest" library capable of fully parsing clj/cljs/cljc, is tools.reader the lib to go, or is there something 'more minimal' ?[I already have tools.reader working, but am wondering about alternatives]
@qqq parsing Clojure is nontrivial and tools.reader is the basis for most of the projects which do so.
technically it's quite easy but you won't gain anything from doing it yourself, many of the token patterns are finicky and t.r is quite good.
@arrdem: thanks for the insight, must be one of these situations where parsing 90% of clojure is easy, but parsing that last 10% requires lots of extra edge cases.
@qqq that's exactly the case, same for analysing clojure and tools.analyzer.jvm, it seems like it should be not too hard to do as the number of special forms is very small but it turns out that getting all the nuances right can be quite crucial and quite hard to do - - core. async is a good example of this where they used a custom analyser at first and then switched over to t.a
for this kind of stuff having a single library responsible for handling all of this hidden complexity is crucial to ensure robustness
Is there a tutorial on how to hijack the widgets from Eclipse or NetBeans for building your own IDE? Googling "using Eclipse as a Library" ==> "how to load Libraries in Eclipse" "how to use Eclipse from clojure" ==> "Counterclockwise is a Eclipse plugin for Clojure dev" etc ... This is almost impossible to google.
if there is, i doubt #clojure will know 👼 perhaps ask on the counterclockwise google group, or ask known maintainers directly?
I figured, if it's possible to use Eclipse/Netbean widgets from Clojure as GUI elements, someone here would know someone who got it to work.
If I have the parameter vector [[a b] x y]
in a defmethod, how can I also bind the whole [a b]
vector to another variable, like I can using :as
in a let binding?
Thanks
@qqq eclipse uses equinox as their own osgi implementation. So most libraries are just osgi bundles that you can use. At our company we build a lot of software on top of eclipse and if I had the choice I would not be doing that. Development is clunky and for UI tests you have to use SWTBot which is a PITA, slow and error prone.
Although I have to admit I cannot compare it to something similar, never used electron or developed an intellij plugin. So, might be that the other choices are as "bad" as eclipse.
So what I would do, is download eclipse and start a new plugin project. I am not sure if it makes sense to pick single plugins and use them for your own project. At least I have never seen that before. IBM for instance also takes a barebone eclipse and builds on top of that.
What is the name of that site that indexes clojure code ?
@sveri: I just spent the past hour reading about SWT and RCP. What I got was a follows: SWT: very easy to use from Clojure, already got it working Eclipse RCP: Provides more GUI elements ... but it wants to be in charge; I can't figured out how to call it from Clojure; I would have to write Clojure as a 'RCP plugin', and to create this RCP plugin, I have to use their GUI wizard tool, which generates a bunch of XML. Is this correct? Basically: SWT = easy to use from Clojure. RCP = more powerful UI elements, unfortunately, near impossible to use from Clojure
@qqq I read a lot of good things about https://github.com/daveray/seesaw if you want to do a java ui with clojure. Last time I tried there were also two experimental wrappers around JavaFX if you want to stick to the newest. Just wondering why did you want to pick parts of eclipse? Is there a specific reason?
I'm trying to create some custom exception classes for adding some tests to the Clojure compiler
so I'm in the namespace clojure.test-clojure.try-catch
, and in this file I've added the line
(gen-class :name try_catch.A1 :impl-ns clojure.test-clojure.try-catch :extends java.lang.Exception)
I got the tests working in my own scrap project, but I can't figure out what :name
and :impl-ns
should be set to in the particular case
yup, I think what I need to do is to put the gen-classes in a separate ns and add that to the compile-test in pom.xml
It’s probably easier to just write them in Java
But what you’re saying should work
yeah, I thought about writing them in Java, but if I did that I wouldn't learn anything 🙂
fwiw the only reason why gen-class
doesn't do genclassing while JITing is that I forgot to enable it when we did the classloader changes in 1.7, while we did make that change for geninterface
just cause I'm curious, could you link the ticket with the classloader changes you are referring to?
I have part of macro which looks like this:
`(let [ex# (try
~form
(catch ~class e# e#)
(catch Exception e#
(let [cause# (.getCause e#)]
(if (= ~class (class cause#)) cause# (throw e#)))))]
"do some things")
I want to change it so that the last catch clause it not added when the class
is Exception
(let [ex
`(try
~form
(catch ~class e# e#))
default
`(catch Exception e#
(let [cause# (.getCause e#)]
(if (= ~class (class cause#)) cause# (throw e#))))]
`(let [ex# ~(if (= class Exception) ex (conj ex default))]
"do some things"))
but that seems a bit verbose. Is there a better way to "ignore" a form when syntax quoting?if you're curious about how syntax-quote works, just stick a quote in front of a syntax-quote expression
Hi all!!! Again in clojure challenge. :-)
I gives up from first time, some months ago, but I am back.... :-)
Ops... Gived up...
Now, after learning more from other languages, fear is more under control...
Back to lisp world again and incredible and amazing clojure lang
In loop nightmares now, but getting slow.....lisp is amazing...
welcome back! fyi there's a #beginners channel if you're just starting out (or re-starting :) )
Thanks!!!
how would one alias a namespaced keyword to just defer to another namespace? something like:
;; a-ns
(require [my-ns])
{::my-ns/foo 1}
;; my-ns
(require [other])
(def foo ::other/foo)
this isn’t right because the (def ...)
shouldn’t really be defining a var, because it’s just a keyword that exists in foo, not a var. is there some way to just say “all references to my namespaced keyword should really just be this other namespaced keyword”
but you can't just say ::my-ns/foo
because my-ns isn't an alias, it's a whole namespace
so assuming I got the first part right, where I alias my-ns
and do ::my-ns/foo
… can I define every keyword that might be ::my-ns/bar ::my-ns/baz ::my-ns/buz
to instead (inside my-ns) point to ::other-ns/bar, baz, buz
?
i’m trying to hide the implementation details of some third-party library, essentially, so that consumers only use my code, and therefore only have to require one namespace, and use all ns-keywords via my interface and not third-party lib’s ns
Hi, I am trying to add vertices on a TinkerGraph instance: ( def test-graph (TinkerGraph/open)) (.addVertex test-graph T/id "name") But i get the following error : No matching method found: addVertex for class org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph . In java the implementation would be: test-graph.addVertex(T.id, "name"), and it seems that the addVertex instance method accepts some kind of key value pair object. Can someone help ?
.addVertex takes java varargs, at the jvm level this is implement as passing an array, in clojure you have to construct the array yourself
Anyway, I’ve got this in a normal let
:
(let [ ... ...
xml-deferred (grobid/pdf->xml-async grobid-cfg pdf)
authors (d/chain xml-deferred grobid/authors)
] ...)
In grobid/pdf->xml-async
it’s possible that an exception happens. I would assume that chaining manifold deferred skips the next step in the chain, but in this case it doesn’t.d/chain
should skip the next step(s) if the deferred results in an error - what makes you think grobid/authors
is being called? You can use d/catch
to handle deferred exceptions.
if the pdf->xml-async
function returns the exception as a value instead of throwing it you’d have a problem; in that case you should wrap the exception in (d/error-deferred ex)
@U8XJ15DTK pdf-xml-async returns a deferred with xml in it if everything goes well. Then I chain on that deferred.
Right - you said that you expect grobid/authors
to be skipped for exceptions, but it’s not. What makes you think it’s running?
I now put a debug statement in there and I see that being printed. I suspect that aleph client (which I use in pdf->xml-async) doesn’t throw when it gets a bad response
When I wrap the body of pdf->xml-async in a catch, the catch works properly. When I don’t do that, grobid/authors gets executed…. That’s a bit odd
@U8XJ15DTK Hmm:
xml-deferred
(d/catch
(grobid/pdf->xml-async grobid-cfg pdf)
Exception
(fn [ex] ;; ex :: clojure.lang.ExceptionInfo
;; (throw ex) results in unterminating state
(throw (Exception. “no”)) ;; this gives a correct response
))
authors (d/chain xml-deferred grobid/authors)
@U8XJ15DTK https://www.dropbox.com/s/oi0ajp2zuvrkdgt/Screenshot%202018-03-30%2022.21.11.png?dl=0
This is the body of pdf->xml-async
:
(defn pdf->xml-async
“Sends PDF file or byte array to Grobid and returns TEI XML format. Async.”
[{:keys [host port]} pdf]
(d/chain
(client/post
(str “http://” host “:” port
“/api/processFulltextDocument”)
{:multipart [{:name “input” :content pdf}]})
:body
;; assuming UTF-8, GROBID doesn’t send character encoding
bs/to-string
;; Poor man’s dehyphenization.
#(str/replace % “- ” “”)))
you’re hitting this block: https://github.com/ztellman/aleph/blob/master/src/aleph/http/client_middleware.clj#L269
I’d expect you to be able to do something like:
(->
(pdf->xml-async grobid-cfg pdf)
(d/chain grobid-authors)
(d/catch Exception
(fn [ex]
; handle errors...
,,,)))
Sorry I can’t be of more help - everything you’ve sent me looks like it ought to work
Somehow the body of the aleph response is causing trouble. When I select it out and re-throw the ExInfo it works…
E.g. this works:
(d/catch clojure.lang.ExceptionInfo
(fn [ex]
(let [d (ex-data ex)]
(throw (ex-info (.getMessage ex)
(update d :body slurp))))))
@xjongar Actually, I want to emphasize that a key function for sorting isn't a clojure exclusive concept. (https://developers.google.com/edu/python/sorting)
Basically, its your way of sorting a non-comparable item by computing some comparable property of it.
If we try and sort [[1 2 3], [8 9], [10]]
, then it should be clear that there is no obvious and unambiguous way to sort these lists. What we can do is provide something like count
as a key function, in which case we would end up sorting the lists by length. If we provide sum
then we would sort the lists by the sum of the items within them.
Its an unfortunate case of terminology clash with clojures keyword. if we have a record (def r {:a 1 :b 2 : c 3})
then to extract a property from that record we can use a keyword as a function (:c r) ;; => 3
, which as a coincedence we can use as a key function in our sorting.
For example:
sorting [{:a 3} {:a 1} {:a 2}]
with the key function :a
would sort everything by the value that record has in its :a
field.
TL;DR Key Functions are not Keyword Functions but a Keyword Function can be a Key Function and thats the Key to understanding those Functions
@emccue overall you are right, but a small pedantic point with that example, clojure does already know how to compare vectors user=> (sort [[1 2] [1] [1 3]])
-> ([1] [1 2] [1 3])
yeah, sets are not sortable (but you'd need a more complex sort-by function than just a key) - I wasn't discounting your overall point at all, just pointing something out about that detail
Is it a planned feature? I would love to have an easy way import those map-vals and map-keys from some gist or something.
It would also (hopefully) help to solve a lot of "omg, why is THAT function not in core?!"-type complains.
:thinking_face: you could load the code into a sandbox, then use a serialization technique to grab just the var+dependencies you need. This essentially fetches dependencies, and then performs tree shaking. Then reverse it in the host.
both transit-clj & transit-cljs now support writing metadata, give it a spin if you find that interesting
like, if all you want is to load clojure code from a file over http, that is super easy
Yeah, I mean that's basically it. With automatic pruning like Rich described though I guess?
at my last job if we had to monkey patch in production that is usually how I did it, save the code as a private gist, then essentially broadcast a command to all the servers to load the clojure from the file
You'd have a fair amount of problems with versioning though if it's not done correctly, or have problems with state being off
@just_crashed A Gist can be multi-file so it could have a deps.edn
as well as the actual source files, yes?
Like imagine if spec's internal format changed between versions, but two different fns required different versions of spec
I saw someone using clj
to run code from a Gist as a dependency I think...
It seems to work, but I've not thrown anything complicated at it. I'm not sure how it works.
@just_crashed Here's an example of pulling in a Gist as a dependency https://gist.github.com/mfikes/e00202b2de7cc2352fedcf92b1fe60dc
@hiredman haven't given it much thought yet to be honest; the idea is not new though. I bet I have seen something like this in some language, but I can't quite remember which one it was. You could do something like (translated to clojure) (require "http://..." :refer [foo]) - is it possible in golang?.. Eh, anyway. The idea is to allow single functions to be distributed (as opposed to full-blown packages with deps and whatnot). I imagine it would work by creating a mapping from say a gist to a fake namespace, then referring that namespace in the actual code. Wrt transitive deps - not sure, personally I'd be ok if transitive deps wouldn't be allowed - just fetch the code and put it in a ns. In any case, it's just an idea, love to see the discussion
https://github.com/portkey-cloud/portkey/blob/master/src/main/clojure/portkey/ouroboros.clj there's some code in here which seems to perform black magic
@seancorfield thanks for the link; well, yes, maybe it's actually enough. One can combine it with a simple script that adds deps.edn to the gist and generates a unique namespace for that based on the url or something.
(require '[gist.AF232343DFA])
speaking of dependencies, has anyone tried to use jlink
to build Clojure with a tailored Java runtime?
(with-open [i (.getContent (java.net.URL. "")) r (clojure.lang.LineNumberingPushbackReader. ( i))] (loop [] (let [form (read r false ::foo)] (if (= form ::foo) nil (do (eval form) (recur))))))
alternatively you can do things with classloaders, which I think you could make require work as is if you did it at the classloader level
@just_crashed here's a Gist with map-vals
and map-keys
showing how to use it 🙂
so that is how I would do clojure shebang scripts, loading pomegranate via a classloader, then using pomegranate to load whatever deps
the shebang line like that doesn't work on linux(I was using osx at the time), but with some changes you can get it to work
@hiredman yeah, I see what you mean; it's a nice trick, but I was thinking about something you can actually include as a dependency (caching, cljs, advanced optimisations, etc). So I guess multifile gist with deps.edn is better in this regard. Also, now thinking of that, it'd kinda sucks without default (project-wide) :require clauses. Say you want to import 5 functions you think have to be in core. That'd mean including 5 [gist.a8fb63ec64704ecb967f :refer [foo]] inside :require's of every ns. I guess it'd be nice to squash all gist-imports into a single namespace upon arrival on your machine. Well, or you can just monkey patch clojure.core... yeeaahh...
@seancorfield haha, nice. Self-contained gists should totally be a thing.
@just_crashed What do you need beyond that? I'm not sure what you think is missing?
Gist-as-library.
https://github.com/kamranzafar/JCL/blob/master/README.textile#manipulating-class-loading-order--adding-custom-classloaders :thinking_face: I wonder if I could stomach the java to write this.
The only "downside" is you have to use a single-segment ns because Gists are "flat", so seancorfield-map-utils
would be a better name there...
(and you have to know the actual SHA as well as the URL)
there is make "available to the project", which generally means an entry in your project.clj, build.boot, deps.edn, or whatever
@seancorfield gist-as-a-library is definitely cool (I think you probably should use the url as a namespace though). But still, imagine you had an index of all such gists, and you could just write something like "ok, give me this snapshot of the whole repo as a dependency, plus add this gist and that one, put it all under a single namespace and make it available to my project".
So if someone writes a little handy function, he could just go to http://fn-repo.clojars.org, paste it, and then you can immediately :require it in your project if you have connection. Or you could have "stable" community repo of handy functions and an "unstable" one where nothing is moderated. Or if two guys have written functions with the same name, you can easily tell clojure which version to use by its hash. Stuff like that.
Probably that's where the idea is coming from. Did Rich talk about it in the spec-ulation?
this email was brought to my attention many years ago, and at the time the group of people I was working with actually found a random joe armstrong on skype and we added him to our standup to try and ask about it, but he never answered our call. and since then when listing the things you need to accomplish some goal, adding letrec at the end has been kind of an in joke.
(I wonder if it could be the real joe armstrong; I would give my life away to say "Hello, Joe" to him on the phone, lol)
Rich and I have talked about single function import type use cases. As you discovered, there are a lot of questions that seemed larger than the value provided, so we don’t have any immediate plans. We have kicked around a bunch of ideas regarding ns auto loading and some of that may eventually shake out into a feature in Clojure itself or maybe clj.
A colleague of mine started talking about nanoservices when microservices were all the new hype. I guess thats what he meant with it, functions as a service. I liked the idea back then. But then, seeing the desaster that npm and leftpad and is-thirteen is, I dont like it that much anymore. Imagine having a leiningen project file that includes 100 lines of https://gist.github.com/seancorfield/6e8dd10799e9cc7527da5510c739e52f It says nothing, so first you would have to find a better way to name things than hashes. And even if there is, something like de.sveri.clj.helpers.core.add-in-map you still dont know what it does by looking at it. And you would have to open a repl or a file where the function is called in your IDE to access the docs. I think cleaning up such a dependency file would be horrible.
Like https://medium.com/bbc-design-engineering/powering-bbc-online-with-nanoservices-727840ba015b
Thanks for the link, interesting read. I like the approach about having exactly one entry point. That would make more sense than one function. What I dislike is the notion of the being serverless, while clearly there are servers in play. But that is another topic.
@sveri for even more fun, support that sort of deps for lein :plugins
. That's the stuff nightmares are made of