This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-07-01
Channels
- # atom-editor (11)
- # babashka (25)
- # beginners (142)
- # boot (9)
- # calva (3)
- # cider (19)
- # clara (15)
- # clj-kondo (6)
- # cljs-dev (20)
- # clojars (11)
- # clojure (164)
- # clojure-dev (9)
- # clojure-europe (6)
- # clojure-italy (17)
- # clojure-nl (3)
- # clojure-spec (19)
- # clojure-sweden (10)
- # clojure-uk (23)
- # clojurescript (34)
- # code-reviews (31)
- # conjure (20)
- # cursive (14)
- # datomic (54)
- # emacs (1)
- # fulcro (51)
- # graalvm (24)
- # graphql (6)
- # helix (3)
- # jobs (3)
- # kaocha (1)
- # malli (2)
- # meander (15)
- # off-topic (81)
- # pathom (2)
- # re-frame (43)
- # reagent (26)
- # reitit (1)
- # releases (1)
- # sci (12)
- # shadow-cljs (29)
- # sql (22)
- # timbre (3)
- # tools-deps (15)
I have a very simple but long regex, with hundreds of or
:
(foo|bar|hello|world|...)
Performance is acceptable, but I wonder, does the JVM automatically optimize the regex, or should I do it myself?
For example, should I extract the common parts, or is this done automatically?
(aaa|aab) => aa(a|b)
I might consider switching to something like aho-corasick@dromar56 Regexes are "compiled" to an intermediate form before matching is actually done. You can follow along in the JDK here https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/regex/Pattern.java#L1742 but it is gnarly stuff. I can't say this particular implementation compiles out common prefixes but when I've worked with regex parsers before (which was, admittedly, before Java appeared), it was fairly common to compile things to state machines and optimizing common prefixes was often part of that.
It's a small thing really but it has often bothered me... Is there a particular reason that split-with
is implemented as [(take-while ...) (drop-while ...)]
?
It is definitely not the best implementation one can imagine.
like - if you have a lazy infinite seq, that might be the best way to make two independent seqs at the end
@emccue This is a semi lazy version that works with infinite seqs and behaves as I expect (question is, am I the only one to expect that?) https://gist.github.com/dvlopt/03317879b5722baa9e915ff7dde28e3a
I don't think this gist is equivalent to clojure.core/split-with:
(do
(def p #(>= 5 %))
(set! *print-length* 10)
(println (clojure.core/split-with p (range 1e3)))
(println (split-with p (range 1e3))))
[(0 1 2 3 4 5) (6 7 8 9 10 11 12 13 14 15 ...)]
[[] (0 1 2 3 4 5 6 7 8 9 ...)]
@UGJE0MM0W Indeed, this version would reproduce the first result by using #(= % 6)
which I find to be semantically more accurate (but that's a personal opinion).
Or simply with #(>= % 6)
which is closer to what you mean.
Say, if you have java code in a deps.edn project, you have to compile it with a makefile or something right? ^^'
Yes, though there are plugins that can do that from “within” a clj
/ clojure
command line. Some examples here: https://github.com/clojure/tools.deps.alpha/wiki/Tools
The best way to think about “vanilla” deps.edn is that it’s not a build tool - it simply knows how to build classpaths and then run some code. This just happens to be a good first step towards various code-related tools (including build tools), and various plugins provide those kinds of capabilities.
Ah thanks.
Those things aren't necessarily separable though. For example if you have a deps.edn project and it contains java files and a custom build script...
With boot and lein, as long as it's published via the default publishing mechanism -> maven, you're guaranteed to be able to use it.
With deps.edn and a git url and hash, tough luck.
Also I don't think anybody wrote that particular kind of tool... ^^'
> With deps.edn and a git url and hash, tough luck. Unless you commit compiled class files and state them in deps.edn as source paths 😄
😂:rolling_on_the_floor_laughing: good one
Just make Java as deps of your clojure code and do:
{:aliases {:uberjar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}
:main-opts ["-m" "hf.depstar.uberjar" "api.jar" "-C" "-m" "api.core"]}}}}
clj -Spom
clj -A:uberjar
You can do it in a few ways, with pom or without.
more on depstart dockpersonally I think compiling different language dependencies, like for example Solidity (Ethereum for cljs) it should be compiled separately from clojure / java. People often put it in lein, but IMHO it is wrong.
but that still wouldn't solve the recursive java dependency issue, or do I missunderstand?
you'd still have to commit prebuild stuff into your repo
@USB60P79B Right. @U0WL6FA77’s suggestion doesn't help you with Java source code at all.
You still have to compile your Java source to class files somehow. Manually or via some Clojure wrapper for running the javac
process.
What do you mean? Don’t you use Java in Clojure to just use :import
? Maybe I misinterpret the question.
Java source needs to be compiled separately.
This sounds like two different use cases: 1. I have Java code as part of my project that I need to compile (for which there may be deps.edn plugins that do it - see earlier link) 2. There is 3rd party code hosted elsewhere (e.g. github) that I want to use as an upstream dependency, and it isn’t deployed as a binary artifact anywhere (for which there are no deps.edn plugins that I’m aware of, but which may be solved via http://jitpack.io)
@U04V70XH6 Yeah that's what I feared, so basically clojure libraries containing java code are completely broken in the git as a dependency world. 😞
@USB60P79B which use case do you have?
3. I have java code as part of my project and I want to release it as a git dependency.
Yes, any Clojure library that contains Java source code cannot be used as a git dependency, unless it also builds the Java source and adds the classes to git (yuk!) or build it as a separate artifact and make it a normal dependency of the Clojure code.
As part of your project or separately?
as part of the project
Note: clojure.tools.deps
is in this boat now -- it contains Java source code and therefore can no longer be used as a git dependency.
Some of the plugins I listed earlier may do this, or if not you may have to fall back to Leiningen (which absolutely supports this).
@U04V70XH6 yeah exactly, oh my god, I've been gone for 2 years and the eco system is completely broken 😭
(well, you can depend on an early SHA before the Java source code was added 🙂 )
this is insane
@U04V70XH6 I’d (strongly!) argue that projects that contain Java source code should be deployed as “binaries” (JARs, containing the compiled Java code) to Clojars or whatever.
It’s not insane at all, it’s just Java. 😉
I don't think you could ever depend on a mixed language project as a plain git dependency @USB60P79B?
@U04V70XH6 I think lein should support it
You can still build and package a mixed language project (and deploy it to Clojars/Maven). Clojure CLI doesn't support Java out of the box but there are tools (listed on the t.d.a wiki) that support building mixed language projects.
I've never tried to use lein
for git-based projects (and that's not core to Leiningen -- that also needs a plugin, right?)
but I think it manages to resolve correctly but I don't really remember because when everybody publishes jars it's a non issue
(I haven't used Leiningen for close to five years, at this point)
the problem I have is actually that I'm depending on a library that hasn't been released as a jar
they use git dependency only
my library in turn has java files
Sure, and if you package and deploy a JAR from a mixed language project there's no issue (and you can do that with deps-based tooling).
yeah I can use dep and package it
Does the upstream library you want to use contain Java source code though?
but this kind of multi class citizens stuff is exactly what makes other languages so annoying
no it doesn't, so I could build an uberjar from mine
If not, you can use a git/SHA dependency for it, but you’ll still have to build (compile) and deploy your library as a binary.
but that stuff is just ugly
"depending on a library that hasn't been released as a jar" -- how do they expect people to use it? What instructions do they provide for it?
So don’t use Java? :man-shrugging:
performance?
@U04V70XH6 use git ref and deps
So the upstream library is absolutely not an issue here, and we should focus on your mixed language project.
the issue here is that deps is not well thought out
I thought Arachne had been abandoned?
@U04V70XH6 you mean it's stabilised with extreme prejudice ;)
To repeat - Arachne is not your issue. The issue is the Java in your project, and how to build it.
yeah no
I've been writing clojure professionally for 7 years without the gap, and mixing java into clojure has always been one of the main points
Arachne contains a deps.edn, and can be consumed downstream (using Clojure CLI) via a git/SHA coordinate.
we're not talkig about groovy here
yeah nah, I'm just gonna port that stuff to rust 😂
If you choose to mix Clojure and Java, that has nothing to do with your use of upstream libraries.
But as I said originally, I think there are plugins that can build a mixed language project via deps / CLI.
there is not
Badgeon does it, I believe?
(or however it is spelled)
Yeah that was the first one to pop up when I googled this.
There’s a blog post on it too.
mind sharing your google filter bubble?
From Badigeon's README "Compile java sources" -- that's the very first bullet 🙂
I don't see any problem with checking in compiled classes. Distribute binary artifacts, versioned by git? sign me up!
@U04V70XH6 hm yeah that might work thanks!
@U47G49KHQ the issues with that is that you break the reference any hash as a version contract
alternative is you run your build script on every commit and commit the artifacts
Given that Cognitect work with mixed Java/Clojure projects and they've designed the whole deps.edn
stuff and are working on tools.build
(whatever that is), I expect this to become a solved problem with standard Cognitect / core Clojure tooling at some point, in the not too distant future.
which is potentially every commit
@U04V70XH6 yeah but it's apparently been like this for 2 years, that's a pretty long time for a partitioned eco system where you have an officially endorsed rouge player that doesn't go nicely with the others
I understand your frustration that tools-deps is not a full-blown build tool that can compile and run everything
I am certainly looking forward to a build tool from cognitect that they are working on
The design goals of the CLI/`deps.edn` have been very clearly stated from the start -- and dealing with mixed language projects was never on the table. Folks are always free to continue to use Leiningen (or Boot).
but as a "tool to download dependencies and assemble classpaths from them" it is the best I've ever used
It is indeed awesome in that respect -- which is why we switched from Boot to the CLI back in 2018.
We have a build
shell script wrapper for the CLI that handles "build" tasks as opposed to classpath/running code.
yeah I don't mind that it does dependency resolution well, my issue is that it endorses git repositories as an alternative to maven, and that gives people the wrong idea that they don't have to publish their stuff anymore
If tools.build
means we can get rid of (or at least simplify) our build
script, I'll be very happy. If it also handles Java source compilation, we may be more inclined to mix Java into our project, or at least start relying on "building" a couple of 3rd party Java projects that we've otherwise had to manually build and deploy ourselves (since they are not published on Maven).
Anyone who has been following the development of the CLI stuff would not have the wrong idea 🙂
You came back after a few years away, dove into the CLI deps.edn
stuff without reading the design goals etc, and then you complain that it doesn't replace Leiningen 🙂
Which, well, no, it was never designed to replace Leiningen.
I don't care that I doesn't replace leiningen
my complaint is not that it does too little or too much
my complaint is that it breaks other build tools
@U47G49KHQ Yup, you can run the Java compiler in process easily enough. Or at least you used to be able to. I can't remember where it is these days after the tools.jar
changes in the JDK.
@USB60P79B That's a ridiculous argument. How can it possibly break "other build tools"? They are independent.
because of git dependencies
which are advocated for
None of that breaks other tools. They all still work the same as they always have.
You're complaining because you're trying to use Arachne and it's never published an artifact consumable by lein
/`boot` right?
I'm complaining because that they did that is because deps encouraged them to
The reason there are no published artifacts is because the project was abandoned long before it was anywhere near complete 🙂
"clojure.deps is my own personal preferred toolchain as well as others who are currently using this project. However, if you or anyone else wished to submit a PR with a working leiningen or boot config and version 1.0, I'd be happy to push to clojars using that config."
deps gave them very much the impression that they don't have to publish it
I give up. You're just not being reasonable. I'm out.
https://github.com/arachne-framework/arachne-core by the way it checks in class files, so using it as git dep should work
@U04V70XH6 how is quoting the guy give that exact reason not being reasonable?
@U47G49KHQ yeah no, that's just ... urgh, I'm gonna port this stuff to rust, at least they have good error messages and a decent package managing system
You're ranting and raging about tooling whose specific design goal did not include mixed language projects. And you're using as a straw man, a project that was abandoned nearly two years ago and never got an actual release.
@U04V70XH6 because it wasn't abandoned at the time, again I don't complain about deps itself, I complain about that it doesn't give warnings in the documentation about git repository deps
@U04V70XH6 I dont't care how deps does it, I think it's great that there is a minimal way to resolve dependencies, my problem is with the consequences this has on incompatible library publication shemes namely git
@U47G49KHQ yeah, but I've never had compilation issues there, I also never had compilation issues with leiningen or boot, which is one of the reasons I love(d) clojure so much, but apparently those times are over...
Stop @-ing me. I don't want to discuss this any further. Thank you.
@U04V70XH6 have a nice day still!😊
@USB60P79B it sounds like what want is deps.edn support added to http://jitpack.io. Here’s a starting point for you: https://github.com/jitpack/jitpack.io/issues/new?template=feature_request.md&title=Please%20add%20support%20for%20Clojure%20projects%20that%20use%20deps.edn
That way you can consume deps.edn projects just the same as if they were deployed to a Maven/Clojars style artifact repository.
@U0MDMDYR3 thanks for the link looks like a really cool project!
adding deps.edn support would indeed solve the grimes I have with it
I think my main issue I have with it is that clj ignores a lot of best practices that the community build over the years
contrary to seans assertions i've read all of the deps documents and watched all of the conj talks about it and related tooling
but I still find pushing git deps into the world without a note that says "get deps are meant as a convenience during development, and as a tech preview of the capabilities to come, please still publish a maven artifact for backwards compatibility" incredibly careless
Yeah I can understand that. But I also know that Clojure has always had a bit of a culture of “Rich’s way or the highway”. 😜 (grabs popcorn and waits for the pitchfork wielding masses to appear…)
no I agree
the thing is, clj is advocated to newcommers, those are not the people that "followed the developments around deps over the years" and those people will run into these issues head first
And to a great extent, that has served the language very well. Perhaps like you I just feel that sometimes it’s not so great for the community side of things.
Yeah - the git/SHA coordinates stuff absolutely assumes a “clean room” environment of tools.deps only, and that’s not the real world (at all - I mean 99% of what I do with Clojure involves Java libraries).
yeah exactly
I totally agree on both counts
spec for example has made errors worse
the spec 2.0 stuff is super cool
but still, the idea that "just spec your macros and all the difficult errors will become much easier"
was simply a false hope, and now my error messages have become much worse 😞
I haven’t really checked out spec yet. It seems to solve a problem I don’t really have on the small scale hobby projects I’m mostly using Clojure for. That said, one change related to error message that has greatly hurt my productivity was pushing stack traces into a temporary file. I find that frustrating in the extreme.
But perhaps that’s because I came from Java, and so didn’t find stack traces problematic to read.
I get that they’re intimidating for beginners, but most languages I know have them in one form or other, so it’s not like the concept is alien - it’s mostly just the format and length that I think surprises newcomers.
yeah, especially because clojure doens't have a good alternative
if you've used rust for a while the difference becomes unbearable when you go back
it's insane how nice and helpfull good error messages can be, even if you know what you're doing