This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-03-23
Channels
- # babashka (15)
- # beginners (102)
- # biff (4)
- # calva (15)
- # cider (51)
- # clojure (5)
- # clojure-dev (3)
- # clojure-europe (3)
- # clojure-france (1)
- # clojure-india (1)
- # clojure-korea (2)
- # clojure-norway (13)
- # clojurescript (20)
- # data-science (1)
- # datalevin (6)
- # datascript (2)
- # emacs (3)
- # events (2)
- # fulcro (4)
- # gratitude (2)
- # introduce-yourself (8)
- # lsp (3)
- # malli (1)
- # meander (1)
- # nbb (9)
- # off-topic (11)
- # releases (1)
- # ring (1)
- # yamlscript (5)
Hi all, I have a question. I believe Clojure concurrency is based on STM model. In concurrency, there are STM, Actors and CSP. (not sure they are more, but often I heard these 3) Like Erlang/Elixir, they are based on Actors model which is message passing. So far can see most chats and some streaming platforms taking advantage of BEAM (actor based). Assume I want to create a live streaming platform with many many concurrent users. Which concurrency model will work best? Will Clojure STM model fit to such? Is there a popular live streaming currently using Clojure STM? Just to understand how Clojure (JVM) fits in live streaming? or is a wrong choice which BEAM will suit better? Thanks.
My feeling is that they are somewhat orthogonal. In real-world Clojure programs, only atom
s are widely used -- ref
s are almost never needed -- so the much-vaunted STM is mostly theoretical, IMO.
For a live-streaming platform, you're going to want to rely on (Java) threads, possibly the new virtual threads (available since JDK 19 with --enable-preview
, and now in JDK 21 without that).
How you would write such a platform will be radically different between a BEAM-based language and a JVM-based language I suspect...
I assume virtual threads is kinda new, which is not really proven compare to what BEAM can offer. So I guess this is something probably BEAM shines better than what Clojure/JVM can offer. Having said that, I assume Netflix is not live streaming, but movie streaming platform with possibly lots of users streaming at the same time. Netflix is based on JVM stack (presume the streaming part), assume JVM can handle such load too?
The JVM can handle pretty amazing loads -- but not everyone is Netflix so not everyone needs whatever Netflix has done to support their platform.
BEAM isn't anywhere near as widely used as the JVM so...
https://robots.net/tech/what-technology-stack-is-netflix-built-on -- lots of things at all levels.
Thanks for sharing. I assume JVM can handle such load, but problem is do I have to throw in either scale a powerful vertical servers or many horizontal servers? Unlike BEAM, less servers can handle the load. I am not sure is a good comparisons where I saw some case studies of Elixir or Erlang (can't recall which), 20 servers down to 2 servers. Can a standard JVM server handles that many request? Assume compare apple to apple (1 JVM server vs 1 BEAM server having the same features) handling concurrent requests.
I saw this in elixir forum (post #3) https://elixirforum.com/t/what-are-the-main-benefits-of-elixir-compared-to-clojure/29959/2 .. what BEAM can offer. Seems BEAM can handle concurrency / parallism really well.
This is all theoretical. Build something and see how it actually scales. What problem are you actually trying to solve? What sort of traffic do you actually expect for it?
With the JVM, you can scale horizontally with "small" servers or vertically -- but which you need depends on exactly what sort of traffic and what sort of computation & data processing you are doing. Trying to compare BEAM and JVM from a purely theoretical p.o.v. is just silly.
Disney/Hulu is mostly powered by AWS services (a lot of which are based on the JVM -- I don't know if any of them are based on BEAM?). Amazon Prime Video is all based on AWS. Netflix already mentioned above is based on a lot of JVM stuff. Not sure about other streaming services. But if you're not building something at that scale, it really doesn't matter anyway...
(some of this analysis is not helped by the fact that AWS includes Apache Beam which is nothing to do with BEAM and is 65% Java đ )
Is this a good place to ask about trying to use the http://practical.li CLI setup?
I'm trying to use the practicalli CLI to create a project, but I'm getting the following error message:
clojure -T:project/create :template practicalli/service :name practicalli/todo-list :component :donut
No function found on command line or in :exec-fn
the :project/create
alias that's in <https://github.com/practicalli/clojure-cli-config/blob/main/deps.edn> has an :exec-fn
, so if that deps.edn is in place, then that error shouldn't occur. Are you also getting a warning about an undeclared alias? Does the alias content in the GH file match up with what you have in one of your deps.edn files (I'm guessing probably user-level).
No undeclared alias warning and I just checked the file and it's an exact match.
I'm not sure what a user-level deps.edn file is, but I cloned the repo to my ~/.config/clojure
folder
You can check clojure -Sdescribe
to see which deps.edn files are contributing, and maybe make sure there aren't any ambiguous definitions (as in, the same alias defined in more than one of those files).
Hold on, I am getting that error with other aliases in that file
I think it may not be pulling that file
But that file is in the list of deps.edn files
And there are no other aliases that are the same
Does it matter that it has a ~
instead of my unqualified path name?
The first two variables
:config-files ["usr/share/clojure/deps.edn" "~/.config/clojure/deps.edn
And :config-user "~/.config/clojure/deps.edn"
also, if an alias is found but misbehaving, and clojure is reporting that it can't find other aliases ("I am getting that error" <- not sure if this means the exec-fn message or the undeclared alias message), I would speculate that maybe there's a tiny syntax error.
I'm not sure if the ~
is problematic or not
I'm getting the undeclared alias error for : project/new and other lines in that file
Hrm... A syntax error? But I made no changes, just fork and clone
after some local testing, it seems like the tilde might be problematic - can't hurt to try expanding it and see if that changes anything
Ok, how would I do that?
Wait, figured it out
Got it!
It must have been the ~
Hey Everyone, Iâm having the weirdest issue. After a long time I fired up my lein to create an app project. But I got a âCould not transfer artifact X from/to central repo errorâ for all my dependencies when I ran lein deps So I thought my libraries could be out of date and reinstalled lein, jdk and clojure based on the instructions in https://clojure.org/guides/install_clojure yet the same error persists. I dont think Iâm behind a proxy, what am I doing wrong?
Is lein itself up-to-date? A while ago (kind of a long time actually) Maven Central switched from http to https.
Delete your ~/.m2/repository anyway because the dependency fetching library may cache a failure there and respect that failure instead of ever trying again
@U0HG4EHMH I dont think I overridden default args. Deleting .m2 didnt help unfortunately
I performed a quick Google search on "Could not transfer artifact X from/to central repo error" and from the kind of answers I saw there, it seems to be a Maven problem.
According to https://help.salesforce.com/s/articleView?id=001118073&type=1 , it could be a proxy problem or a misconfiguration of Maven which causes it to fail to fetch a dependency for you. You may want to delete ~/.m2
(Maven cache) and recreate it.
I tried that didnât help. This is also wired because the lein is fresh and I did not explictly install mvn
@me1221 should be solvable, maybe hititng an old enpoint or something, can you provide more tangible results such as which deps are saying they are unfindable? is it all of them? what version ? lein -v
@v1nc3ntpull1ng I sure hope so, lein -version Leiningen 2.11.2 on Java 21.0.2 OpenJDK 64-Bit Server VM Even the template lein new app application doesnât seem to work. All the libraries seems to have an issue.
When a message is selected, the right arrow key or t
opens the thread for that message. There's also an option in the hover menu on messages to "Reply in thread".
@v1nc3ntpull1ng I think so, this is the entire error message
lein deps
Could not transfer artifact org.clojure:clojure:jar:1.11.1 from/to central ( ): transfer failed for
Could not transfer artifact org.clojure:clojure:jar:1.11.1 from/to clojars ( ): transfer failed for
Could not transfer artifact nrepl:nrepl:jar:1.0.0 from/to central ( ): transfer failed for
Could not transfer artifact nrepl:nrepl:jar:1.0.0 from/to clojars ( ): transfer failed for
Could not transfer artifact org.nrepl:incomplete:jar:0.1.0 from/to central ( ): transfer failed for
Could not transfer artifact org.nrepl:incomplete:jar:0.1.0 from/to clojars ( ): transfer failed for
Failed to read artifact descriptor for org.clojure:clojure:jar:1.11.1
Failed to read artifact descriptor for nrepl:nrepl:jar:1.0.0
Failed to read artifact descriptor for org.nrepl:incomplete:jar:0.1.0
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
See if you can access http://maven.org at all
Thank yâall for switching to a thread. Try not to flood the channel with follow up messages to the same issue
Hm, is it possible that just your shell is behind a proxy? I don't get how the browser would be able to get the .jar but not the shell. which shell are you using @me1221?
Iâm using the default mac terminal, let me try iTerm. the curl seems to hit maven url though
Cool, that's weird, and likely means something weird within that specific program. 85-88% + is where i'm placing my chips, optimistically better results w/ iTerm đ¤
Try using âlein pomâ to generate a pom.xml and then âmvn dependency:treeâ
That gets you out of lein, and you might get a better error from mvn
I clicked on https://repo1.maven.org/maven2/org/clojure/clojure/1.11.1/clojure-1.11.1.jar in the error message in my browser (Google Chrome) and it opened a dialog for saving a downloaded file. So the problem looks like a connection problem. You may want to try the same in your browser. Then, if necessray, try to ping http://repo1.maven.org. When I tried https://repo.clojars.org/org/clojure/clojure/1.11.1/clojure-1.11.1.jar , my browser yielded the response:
NoSuchKeyThe specified key does not
So this could be a transient http://repo.clojars.org problem.You should also look at ~/.lein to see if you have some bad config there
I donât think this is a connection or proxy problem, itâs something in your config
@U064X3EF3 thanks Alex, trying the mvn dependency:tree command yields same result. mvn dependency:tree [INFO] Scanning for projects... Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-clean-plugin:3.2.0: Plugin org.apache.maven.plugins:maven-clean-plugin:3.2.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:3.2.0 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-resources-plugin:3.3.1: Plugin org.apache.maven.plugins:maven-resources-plugin:3.3.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:3.3.1 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-jar-plugin:3.3.0: Plugin org.apache.maven.plugins:maven-jar-plugin:3.3.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-jar-plugin:jar:3.3.0 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-compiler-plugin:3.11.0: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.11.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-compiler-plugin:jar:3.11.0 âŚ
Hmm, not sure what config affects both of those. Can you share the project.clj? Any config in your ~/.m2/settings.xml?
Can you just curl -O
one of those urls directly?
Was the url you tried a .pom?
this is the project.clj BTW (defproject make-music-test â0.1.0-SNAPSHOTâ :description âFIXME: write descriptionâ :url âhttp://example.com/FIXMEâ :license {:name âEPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0" :url âhttps://www.eclipse.org/legal/epl-2.0/â} :dependencies [[org.clojure/clojure â1.11.1"]] :main ^:skip-aot make-music-test.core :target-path âtarget/%sâ :profiles {:uberjar {:aot :all :jvm-opts [â-Dclojure.compiler.direct-linking=trueâ]}})
The curl gets me the jar -> curl -0 https://repo1.maven.org/maven2/org/clojure/clojure/1.11.1/clojure-1.11.1.jar --output file.jar
Both lein and mvn are ultimately using http to download those urls, not much different than curl, so itâs definitely not a proxy or dns problem.
Could potentially be ssl due to an old Java maybe?
What java -version
java -version openjdk version â21.0.2â 2024-01-16 LTS OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)
Well thatâs new and shiny
I donât know, Iâm out of ideas. Maybe rm -rf ~/.m2/repository in case there is something in that dir with bad permissions or something
Hmmm⌠ls -l ~/.m2 total 0 drwxr-xr-x 4 vinoth staff 128 23 Mar 17:11 repository vinoth@192 make-music-test % vinoth@192 make-music-test %
correct, but it seems like no clojure app is running now. despite reinstalling lein, clj and jdk
@me1221 that is certainly befuddling, are you working from a particular tutorial or getting started guide?
@v1nc3ntpull1ng not really, all I did was lein new app <app-name> and https://clojure.org/guides/install_clojure
@U02UHTG2YH5 not quite, I have MacOS 14.3.1. But looks more like a java error though. How do I validate it?
Then you should be safe from that. https://blogs.oracle.com/java/post/java-on-macos-14-4
Did you ever try rmâing .m2 ?
@U064X3EF3 yep, I did. Same issue for all of my projects
did you try clj
?
Folks, please use threads instead of posting a lot of top-level messages -- in particular if you are answering someone's question, do so in a thread (@w1 @v1nc3ntpull1ng).
And, yes, I deliberately posted this as a reminder to everyone here in the #C053AK3F9 channel.