This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # announcements (1)
- # atlanta-clojurians (1)
- # beginners (198)
- # calva (4)
- # cider (16)
- # clara (8)
- # cljs-dev (14)
- # cljsrn (4)
- # clojure (204)
- # clojure-europe (3)
- # clojure-gamedev (2)
- # clojure-italy (8)
- # clojure-nl (17)
- # clojure-poland (3)
- # clojure-russia (20)
- # clojure-spec (32)
- # clojure-uk (45)
- # clojurescript (59)
- # community-development (1)
- # core-async (25)
- # cursive (20)
- # datomic (47)
- # emacs (7)
- # fulcro (8)
- # iot (1)
- # iotivity (2)
- # jobs (1)
- # jobs-discuss (8)
- # juxt (11)
- # luminus (5)
- # nrepl (4)
- # off-topic (136)
- # onyx (24)
- # other-lisps (1)
- # parinfer (74)
- # pedestal (1)
- # planck (3)
- # portkey (67)
- # random (1)
- # re-frame (28)
- # reagent (11)
- # reitit (9)
- # remote-jobs (3)
- # ring-swagger (2)
- # rum (3)
- # shadow-cljs (96)
- # slack-help (3)
- # spacemacs (6)
- # tools-deps (3)
- # unrepl (1)
- # vim (4)
Did something change about Clojure memory consumption? Hello worlds still consume 256 MB? I am asking in the context of making N instances in environment like kubernetes. Small services can consume very fast too much memory.
right, starting a repl here indeed has a initial heap size of 256 of which only 27 is in use. You could probably use the default memory flags of the JVM to change that
I was trying it a few years ago and my conclusion was Clojure wrapper for Java takes around 256 MB. I have no idea how it looks today.
(I checked with VisualVM, the (initial) heapsize is indeed 256, but its not all used, you could probably further restrict that to 64)
You seem to be able to pass
:jvm-opts in your leiningen project file (similarly for
deps.edn), but the exact correct flags for the JVM are beyond me for now. Check the usual JVM resources
JVM will use more memory than it requires by default. If you want to see how low you can go with memory, you have to limit the memory that is available to the jvm. For example by passing this
also, keep in mind that java 8 cannot determine the memory limits of a container reliably (it will in stead take the memory limit of the host).
You can get past that with -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap
all I wanted to say is that JVM will take more memory that strictly needed by default
heh ok thx 😉 Just curious what is people experience about memory consumption for services in clouds
If you wanted to write as many bytes as possible per second to a file on the file system on clojure-jvm, how would you do it?
Let me specify then: Would you recommend any specific programmatic route? I've tried out http://clojure.java.io/output-stream but it's not anywhere close what I get out of dd.
I never did performance research for such case. I don’t feel competent to answer. From general point of view I will try to read stream from memory if possible, if know size before save create X MB empty file and write to it (not really sure how and if it performance better, but I think it should), if you can use
dd use it instead clojure library.
Maybe use bash input stream instead? But it shouldn’t be the case. I guess it is something about
dd copy whole blocks which is something different than stream.
How would I get the var from my function or macro argument?
(defmacro b [f] (var f)) throws
CompilerException java.lang.RuntimeException: Unable to resolve var: f in this context, compiling:(null:1:17) and
defn has the exact same behaviour.
This is interesting. Looks like the symbol needs to resolve to a var at compile time not runtime. Something like this works:
given that x is already a defined var.
(defmacro b [f] `(var ~f)) (b x)
defmacro are calling
var special form during evaluation of a
defmacro definition (unless it is quoted).
This makes sense because vars represent identities that must not change even though the value that is pointed at by them may change at runtime.
var is a special form - it expects a symbol literal that corresponds with a var.
@U37339DBP I am interested in knowing the exact use case here. What are you trying to accomplish with a macro like that?
@UDHE6D287 That macro is an MWE. The end goal is to retrieve specs with
s/get-spec, which requires a var as argument. I was wondering how to get that var.
Anyone know of any recent libraries similar to
re-ops? Interested in managing infrastructure from Clojure directly
I don’t but I have this feeling I shouldn’t use Clojure as tool for DevOps. I was trying in the past, but… I had bad experience. Just because of such solutions are not enough popular to be well supported and integrated.
Unfortunately I have this impression all open source cicd tools are just bad 🙂 https://concourse-ci.org is open source and very interesting solution - unfortunately it is doing a little to slow progress while paid competitors are much faster. https://codefresh.io and https://circleci.com are good paid alternative IMO if you have budget.
@victorbjelkholm429 but doing this in Clojure is a bad idea, because of the reasons which I mentioned above - just my opinion
@kwladyka yes, so I'm not interested in ci/cd really, but just to manage instances from different cloud hosters. I personally already know Terraform for managing infra, but this project is a bit different. In the end, the UI I'm building is backed by a Clojure application, that app should be able to create/destroy cloud instances for the user, so can't really ask them to just learn Terraform instead 🙂
probably will end up with some common interface, then wrappers around existing cloud libs to support that interface in the end. Seems easy enough but wanted to avoid writing it from scratch
eh, actually quite simple. Just need to be able to create/destroy/stat(get metrics) from hosts.
pallet looked really nice for exactly this but seems outdated. If interested, here is the project described: https://github.com/ipfs-shipyard/cube
I remember there was a library for Clojure to create pipelines in Clojure code. But what it does was draw UI. But I don’t remember the name and can’t find it in google.
https://github.com/flosell/lambdacd this one, but it doesn’t look like it is well supported
@victorbjelkholm429 as alternative maybe you can use kubernetes do deploy / destroy etc. ?
@kwladyka thanks, but kubernetes is the wrong abstraction, still need to create the kubernetes cluster somehow, and that's the missing piece I'm out after right now
ok so as another alternative you can run bash commands. Most of the popular providers let manage instances by bash commands
@kwladyka well, bash is a horrible language. I'll probably end up going the route described in https://clojurians.slack.com/archives/C03S1KBA2/p1547043029371400 (with http APIs)
@kwladyka you mean with
clojure.java.shell? In the end the bash commands you talk about end up doing http requests, so can just do the requests directly from clojure instead...
@myguidingstar yeah, I looked at nixos (specifically nixops) before. Didn't find anything that would concretely help me, but I did get some useful ideas from reading and understanding it
nixpkg is basically a immutable package manager, which is super useful (especially for work machines). Do an upgrade and something broke? Just rollback the entire system to the previous version and stuff is working again. nixos is a operating system using nixpkgs and the immutable ideas
@kwladyka nixops is for the instance setup itself, not the application runtime you run on the instance.
to run your images, you need a host + docker daemon somewhere. Nixops would help you operate that host + daemon
@kwladyka nixpkgs can be used for building package ie your own app, due to its immutability even your app is reproducible (except database and file system)
Well in this case I am thinking about it more in the sense to be early adopter 😉 Just curious what they did, but probably you are right.
personally I hope nix (the package manager) takes over the world. Seems to be the only package manager who does things correctly.
that is why it doesn’t matter for most of the people making it hard to became popular 😉
Surely it can be popular within certain populations. As long as we are speaking package managers it makes sense to speak of popularity with users of package managers.
Yeah, I'm using it right now. My site is super basic, but it's been a really nice experience and the amount of code I had to write was tiny
Asking because maybe I should start read some blog or something, because I miss that info
Oh. I saw it in my research of popular libraries for creating an API server. It was mentioned in multiple places so I just read the docs at https://juxt.pro/yada/manual/index.html
I have a question. How can I get arglist from
clojure (defn func1 ([a]) ([a b])) (:arglists (meta (var func1))) ;;=> ([a] [a b]) (def func2 (fn ([a] (inc a)) ([a b] (+ a b)))) (????? func2) ;; => ([a] [a b])
Slack is kinda weird about triple backticks. To actually get Clojure syntax highlighting you need to add a code snippet with the '+' button.
you could reflect on the class backing that anonymous function and look for the
invoke methods, and you'd get the arg numbers
the doc-string for
sort-by doesn't mention that it is stable but I thought i remember hearing this? Am i mistaken?
that was added in clojure at some point a couple releases ago, maybe corresponding needs to be verified and added in cljs
yeah. just put in #cljs-dev. It's a comment from Stu Halloway from 8 years ago that it is stable but the doc string purposefully doesn't promise it
it's stable right now. But this is asking if cljs will promise this for all future versions essentially
it's not ambiguous i suppose. it doesn't promise stability even though it has it. it just seems like this is something that it can promise though is all
It's ambiguous by not stating. I'm suggesting it state exactly what you said - currently stable, but not guaranteed in future versions
If the docstring is updated, it should be to just add "Not guaranteed to be stable" -- saying it's stable now but might change in the future isn't useful, IMO.
If a docstring says "property X is true but might change", folks can't rely on property X remaining true so either they rely on it today (and might have their code broken in subtle/unexpected ways) or they explicitly don't rely on it at all (even tho' it may never actually change).
It's better to either explicitly guarantee property X or explicitly not guarantee it.
agreed. although i'm hoping that the 8 years of stable sort means it will just be acknowledged as a promise. But sean that's a very good comment you should leave on the ticket. Mark it as not guaranteed stable or guaranteed stable to remove confusion?
I don't agree with that as "not guaranteed to be stable" is ambiguous. It is not stating that is or isn't stable, which was Dan's original question. If the documentation were to state that it is "Stable, but not guaranteed" now it's not ambiguous.
all i'm looking for is what i can bank on for the future. if it is reasonable to promise stability I'm good. If there is no commitment to stability I need to find another solution regardless of current behavior
I think it would be helpful if we had an explanation on why it may not be stable in the future
I'm answering this as someone who spent nearly a decade working on ANSI/ISO standards committees. Saying "Stable, but not guaranteed" is not useful. Saying "Not guaranteed to be stable" provides more useful information.
(because it makes it clear that the stability of the current implementation is just an implementation detail and not part of the contract)
Please explain to me how something that is ambiguous is more useful than something that is not ambiguous
The documentation should reflect the current behavior. If that behavior changes then the documentation should change with it. If it is stable now, then it should state that it is stable. If it becomes unstable then the documentation should change with that.
Behavior in software changes. I don't subscribe to the idea that the documentation should try to predict the future.
Stability of a sort is either part of the contract or part of the implementation. You need to know whether it is part of the contract. Either it is intended to be stable or it is not. That is true across Clojure's APIs in general -- if the implementation happens to provide "more" than the docstring, what's in the docstring is all you can rely on.
In this case, the docstring provides no guidance (although it does not guarantee stability today so it's actually just fine as it is).
If a Clojure docstring provides a guarantee, that is considered when changes are made in the future -- because the core team have a principle of not breaking guaranteed behavior.
If the docstring provides no guarantee, behavior can change without needing to update the documentation (since the behavior was never guaranteed).
It's a courtesy for the docstring to be explicit about not guaranteeing a specific behavior. That's kind of Clojure's basic position on a lot of things.
Documentation of goog.array/stableSort: "Sorts the specified array into ascending order in a stable way. If no opt_compareFn is specified, elements are compared using goog.array.defaultCompare, which compares the elements using the built in < and > operators. This will produce the expected behavior for homogeneous arrays of String(s) and Number(s)."
Right, and if the decision is to guarantee a stable sort -- as the Clojure docstring does -- then updating the (ClojureScript) docstring to make that same guarantee about current and future behavior would be the correct thing to do.
You're missing my point that if there is no intended guarantee of that behavior, then the current docstring is accurate: it makes no guarantee about stability.
Clojure's docstrings are generally very careful about what they do and don't guarantee. If they guarantee something, it is a contract that can be relied on and will not break in the future (because Rich says breakage is bad).
If a docstring does not guarantee a specific behavior, then that behavior can change (because it wasn't part of the contract).
(Maybe put it like this, that it uses stable sort is an implementation detail, it could as well use quicksort, which is not stable, but that wouldn’t matter as the stability is not required by the contract between caller (you calling sort) and the callee (cljs’s sort))
I've understood what you're saying this whole time; I just don't agree with it. I don't believe my point is being understood, which is that the Clojure version is guaranteed stable and there is no reason that can't be true of the ClojureScript version as the function it is using is guaranteed to be stable. The goog.array library could disappear off of the face of the Earth and we'd still be fine because we could recreate the implementation as it is open source and relies only on essential JS operations that aren't going to change. The fact the CLJS code hasn't changed in 8 years should be a testament to that.
You're still missing the point: the question isn't how it is implemented but whether the intent of the design is for a stable sort or not on cljs.
If David Nolen (or whoever can make that decision) says "Yeah, we intended it to guarantee stability" then there's no question: the docstring should be updated to reflect that design intention as a guarantee -- and any future implementation changes should reflect that. If he/they say "Nope, we are not guaranteeing that for cljs" then the current docstring is correct, but I think it would be a courtesy to update the docstring to reflect that stability is (explicitly) not guaranteed.
Please stop accusing me of missing the point when I've stated I just don't agree with what you're saying. I just referred to that indirectly in my above message. Obviously we still need to check with the powers that be what their intentions are. I stated that, objectively, there isn't any reason it couldn't be guaranteed given the expected and current functionality. I am not speaking for the powers that be as that is not my place.
That being said it also begs the question as the implementation that has survived 8 years uses an, explicitly, stable sort function so I would think it should be pretty clear what the intention was when that code was written as they could have used any other sort function, but deliberately chose that one.
That may or may not be meaningful. Clojure libraries have a tendency to be completed for their stated objective(s) and haven't needed to change anything since then.
I am aware of that. However in this specific case Spark is already at 2.4 and the libs are in 2.1
huh, googles impl of stableSort is actually quite interesting. Expected an actual stable sort algorithm, but instead they change the comparator to also take index into account.
So it gains
2n extra time complexity for looping the array twice to add/remove indices, and
n memory for the array of value/index pairs that is actually sorted
Just O(n) extra: "Runtime: Same as Array.prototype.sort, plus an additional O(n) overhead of copying the array twice."
O(2n) are the same complexity class:
O(n), it really loops over the array twice
Or; it was easiest. But I am not yet prepared to accept that google engineers take shortcuts too.
yes. it's using the built in sort on arr which probably changes algo depending on size. i've heard of quick sorts bottoming out into insertion sort at small n, etc
(not saying that is what is happening here but could be the reason not to duplicate that kind of stuff)
Hi all! I've built a small tool (based on
rewrite-clj) that allows to search for a certain values in Clojure files (like
Idea is that one can try it from command line in seconds (no need to install): https://github.com/andrewboltachev/fo0001#clojure-value-finder-like-grep
Not sure if alternatives exist. Feedback appreciated 🙂
not that fancy but debux works for me in those cases: https://github.com/philoskim/debux#81-dbg-inside-dbgn-or-vice-versa
It does exist for java streams: https://plugins.jetbrains.com/plugin/9696-java-stream-debugger
Huh, the above comments from "Tariq" are interesting. That makes it seem like either a) it was a human troll rather than a bot, or b) the bot watches for its (impersonated) name and then says the stuff above. Bit more sophisticated than I expected.
It's definitely a human at the wheel, there were direct back-and-forths in the original attack as well
Turn off inline image display in your preferences, too; it's not often needed in this slack, and better safe than sorry
User deactivated. We're working on deleting the messages wherever they got posted. Feel free to post in #slack-help with links to ones we haven't gotten to yet. And, yes, these bots are able to get around the restrictions that Slack lets us put in place to prevent non-admins using @ everyone/channel/here...
Confirmed via an article one of the other members here sent me (thank you @jayzawrotny!). You can hack the permission client-side in JS and then spam with immunity it seems. Slack have known about it for two and a half years and confirmed a year and a half ago that they won't fix it. Sigh.
Also if you remap caps lock to control the caps lock button still makes the text box take focus rather than copying 😞