Did something change about Clojure memory consumption? Hello worlds still consume 256 MB? I am asking in the context of making N instances in environment like kubernetes. Small services can consume very fast too much memory.

does it consume or request 256m of memory

right, starting a repl here indeed has a initial heap size of 256 of which only 27 is in use. You could probably use the default memory flags of the JVM to change that

I was trying it a few years ago and my conclusion was Clojure wrapper for Java takes around 256 MB. I have no idea how it looks today.

(I checked with VisualVM, the (initial) heapsize is indeed 256, but its not all used, you could probably further restrict that to 64)

You seem to be able to pass :jvm-opts in your leiningen project file (similarly for deps.edn), but the exact correct flags for the JVM are beyond me for now. Check the usual JVM resources

So how many MB are “lost” only to run Clojure app?

I feel confuse how it will behave in practice after such cut of memory

If you wanted to write as many bytes as possible per second to a file on the file system on clojure-jvm, how would you do it?

too general question, but mainly I will buy fast hard drive 😉

Let me specify then: Would you recommend any specific programmatic route? I've tried out http://clojure.java.io/output-stream but it's not anywhere close what I get out of dd.

I never did performance research for such case. I don’t feel competent to answer. From general point of view I will try to read stream from memory if possible, if know size before save create X MB empty file and write to it (not really sure how and if it performance better, but I think it should), if you can use dd use it instead clojure library.

Maybe use bash input stream instead? But it shouldn’t be the case. I guess it is something about dd copy whole blocks which is something different than stream.

I'll probably look at java.nio next.

How would I get the var from my function or macro argument? (defmacro b [f] (var f)) throws CompilerException java.lang.RuntimeException: Unable to resolve var: f in this context, compiling:(null:1:17) and defn has the exact same behaviour.

Anyone know of any recent libraries similar to pallet and/or re-ops? Interested in managing infrastructure from Clojure directly

I don’t but I have this feeling I shouldn’t use Clojure as tool for DevOps. I was trying in the past, but… I had bad experience. Just because of such solutions are not enough popular to be well supported and integrated.

Better to learn good DevOps tool

IMO

Unfortunately I have this impression all open source cicd tools are just bad 🙂 https://concourse-ci.org is open source and very interesting solution - unfortunately it is doing a little to slow progress while paid competitors are much faster. https://codefresh.io and https://circleci.com are good paid alternative IMO if you have budget.

@victorbjelkholm429 but doing this in Clojure is a bad idea, because of the reasons which I mentioned above - just my opinion

@kwladyka yes, so I'm not interested in ci/cd really, but just to manage instances from different cloud hosters. I personally already know Terraform for managing infra, but this project is a bit different. In the end, the UI I'm building is backed by a Clojure application, that app should be able to create/destroy cloud instances for the user, so can't really ask them to just learn Terraform instead 🙂

oh and gitlab cicd

probably will end up with some common interface, then wrappers around existing cloud libs to support that interface in the end. Seems easy enough but wanted to avoid writing it from scratch

hmm I don’t know details to try help

It looks like you have some kind of not standard needs

eh, actually quite simple. Just need to be able to create/destroy/stat(get metrics) from hosts. pallet looked really nice for exactly this but seems outdated. If interested, here is the project described: https://github.com/ipfs-shipyard/cube

I remember there was a library for Clojure to create pipelines in Clojure code. But what it does was draw UI. But I don’t remember the name and can’t find it in google.

If it help in any way (from UI perspective)

https://github.com/flosell/lambdacd this one, but it doesn’t look like it is well supported

But idea was nice 🙂

@victorbjelkholm429 as alternative maybe you can use kubernetes do deploy / destroy etc. ?

then you can use the same way for all cloud providers

@kwladyka thanks, but kubernetes is the wrong abstraction, still need to create the kubernetes cluster somehow, and that's the missing piece I'm out after right now

ok so as another alternative you can run bash commands. Most of the popular providers let manage instances by bash commands

@kwladyka well, bash is a horrible language. I'll probably end up going the route described in https://clojurians.slack.com/archives/C03S1KBA2/p1547043029371400 (with http APIs)

You can run bash commands from Clojure. You don’t have to write bash scripts

@kwladyka you mean with clojure.java.shell? In the end the bash commands you talk about end up doing http requests, so can just do the requests directly from clojure instead...

yes, but it could be easier than interop and update, just saying an alternative

@victorbjelkholm429 if you're serious about functional programming and linux, there's Nixos https://nixos.org/

@myguidingstar yeah, I looked at nixos (specifically nixops) before. Didn't find anything that would concretely help me, but I did get some useful ideas from reading and understanding it

@myguidingstar Interesting, what is the real difference in practice

nixpkg is basically a immutable package manager, which is super useful (especially for work machines). Do an upgrade and something broke? Just rollback the entire system to the previous version and stuff is working again. nixos is a operating system using nixpkgs and the immutable ideas

Hmm but considering for everything I use Docker images

@kwladyka nixops is for the instance setup itself, not the application runtime you run on the instance.

to run your images, you need a host + docker daemon somewhere. Nixops would help you operate that host + daemon

I mean with docker images there is no issues about rollback / update

Just thinking if NixOS can has benefits for me

@kwladyka nixpkgs can be used for building package ie your own app, due to its immutability even your app is reproducible (except database and file system)

anyway, it's a lot to learn, so you may consider cost vs benefit

Well in this case I am thinking about it more in the sense to be early adopter 😉 Just curious what they did, but probably you are right.

personally I hope nix (the package manager) takes over the world. Seems to be the only package manager who does things correctly.

the same here 🙂

True, but I am afraid it is good for developers, not for common people 😉

eh, common people don't use package managers anyways 🙂

that is why it doesn’t matter for most of the people making it hard to became popular 😉

Surely it can be popular within certain populations. As long as we are speaking package managers it makes sense to speak of popularity with users of package managers.

@whoneedszzz did you finish research about https://github.com/juxt/yada ? I am wonder if it is worth to switch to use it

I have a question. How can I get arglist from fn

clojure
(defn func1
  ([a])
  ([a b]))

(:arglists (meta (var func1)))
;;=> ([a] [a b])


(def func2 (fn
             ([a] (inc a))
             ([a b] (+ a b))))

(????? func2)
;; => ([a] [a b])

you can't

even using reflection?

you could reflect on the class backing that anonymous function and look for the invoke methods, and you'd get the arg numbers

but blargh

why do you need this?

I want to call that function, using map. (do-someting func1 {:a 1 😛 2})

(do-someting func1 {:a 1 :b 2})

https://gist.github.com/hiredman/71b71e4e07e7666fe1b9def9a476c765#file-lrun-clj-L11-L13

the doc-string for sort-by doesn't mention that it is stable but I thought i remember hearing this? Am i mistaken?

https://clojuredocs.org/clojure.core/sort-by

"Guaranteed to be stable"

@dpsutton

thanks. wanted to make sure i wasn't just going on rumor

oh. i'm in cljs. and that is conspicuously absent from the docstring

that was added in clojure at some point a couple releases ago, maybe corresponding needs to be verified and added in cljs

yeah. just put in #cljs-dev. It's a comment from Stu Halloway from 8 years ago that it is stable but the doc string purposefully doesn't promise it

hey peeps, any advice which library to use with Spark ?

both flambo and sparkling haven't done any commits the last 6+ months..

huh, googles impl of stableSort is actually quite interesting. Expected an actual stable sort algorithm, but instead they change the comparator to also take index into account.

So it gains 2n extra time complexity for looping the array twice to add/remove indices, and n memory for the array of value/index pairs that is actually sorted

that's an interesting find @lennart.buit

I guess that calling into v8s default sort algorithm is so much faster, that implementing a stability guarantee in javascript is more performant than implementing a complete stable sort alg.

Or; it was easiest. But I am not yet prepared to accept that google engineers take shortcuts too.

yes. it's using the built in sort on arr which probably changes algo depending on size. i've heard of quick sorts bottoming out into insertion sort at small n, etc

(not saying that is what is happening here but could be the reason not to duplicate that kind of stuff)

yeah, lists of less than 10 are actually instertion-sorted

Hi all! I've built a small tool (based on rewrite-clj) that allows to search for a certain values in Clojure files (like grep) Idea is that one can try it from command line in seconds (no need to install): https://github.com/andrewboltachev/fo0001#clojure-value-finder-like-grep Not sure if alternatives exist. Feedback appreciated 🙂

https://twitter.com/djleeds/status/1082838230662348800

someone go make that for Clojure! :)

It does exist for java streams: https://plugins.jetbrains.com/plugin/9696-java-stream-debugger

Not for Clojure yet, but it’s on some list I have somewhere.

Anybody around?

Can someone ban this guy (Tariq Nasheed)

lol 3...2...1...

yeah, it’s annoying

Admins know about it

Stop!

dude!!!!! dont do that!

Last time this happened, they were impersonating an existing user, too.

whoa

that has to be the most confusing meme i’ve ever seen

I didn’t type that

how do you block someone?

just like that

UFA3W5N4U is the account that posted

fyi, /collapse can be used to collapse all media in the room

maybe this the catalyst for the clojure community moving on from slack?

and btw: do not download any attachments posted

Perhaps that’s how they’ve been able to impersonate?

funky

Turn off inline image display in your preferences, too; it's not often needed in this slack, and better safe than sorry

Preferences > Messeges & Media

😓

User deactivated. We're working on deleting the messages wherever they got posted. Feel free to post in #slack-help with links to ones we haven't gotten to yet. And, yes, these bots are able to get around the restrictions that Slack lets us put in place to prevent non-admins using @ everyone/channel/here...

yeesh, client-side permissions enforcement?

Confirmed via an article one of the other members here sent me (thank you @jayzawrotny!). You can hack the permission client-side in JS and then spam with immunity it seems. Slack have known about it for two and a half years and confirmed a year and a half ago that they won't fix it. Sigh.

Also if you remap caps lock to control the caps lock button still makes the text box take focus rather than copying 😞

Probably the only thing we can do at this point is to change the auto-inviter to require moderation somehow.