Fork me on GitHub
#beginners
<
2024-03-23
>
Eric Chua05:03:34

Hi all, I have a question. I believe Clojure concurrency is based on STM model. In concurrency, there are STM, Actors and CSP. (not sure they are more, but often I heard these 3) Like Erlang/Elixir, they are based on Actors model which is message passing. So far can see most chats and some streaming platforms taking advantage of BEAM (actor based). Assume I want to create a live streaming platform with many many concurrent users. Which concurrency model will work best? Will Clojure STM model fit to such? Is there a popular live streaming currently using Clojure STM? Just to understand how Clojure (JVM) fits in live streaming? or is a wrong choice which BEAM will suit better? Thanks.

seancorfield05:03:47

My feeling is that they are somewhat orthogonal. In real-world Clojure programs, only atoms are widely used -- refs are almost never needed -- so the much-vaunted STM is mostly theoretical, IMO. For a live-streaming platform, you're going to want to rely on (Java) threads, possibly the new virtual threads (available since JDK 19 with --enable-preview, and now in JDK 21 without that). How you would write such a platform will be radically different between a BEAM-based language and a JVM-based language I suspect...

Eric Chua05:03:00

I assume virtual threads is kinda new, which is not really proven compare to what BEAM can offer. So I guess this is something probably BEAM shines better than what Clojure/JVM can offer. Having said that, I assume Netflix is not live streaming, but movie streaming platform with possibly lots of users streaming at the same time. Netflix is based on JVM stack (presume the streaming part), assume JVM can handle such load too?

seancorfield05:03:12

The JVM can handle pretty amazing loads -- but not everyone is Netflix so not everyone needs whatever Netflix has done to support their platform.

seancorfield05:03:38

BEAM isn't anywhere near as widely used as the JVM so...

Eric Chua05:03:52

Thanks for sharing. I assume JVM can handle such load, but problem is do I have to throw in either scale a powerful vertical servers or many horizontal servers? Unlike BEAM, less servers can handle the load. I am not sure is a good comparisons where I saw some case studies of Elixir or Erlang (can't recall which), 20 servers down to 2 servers. Can a standard JVM server handles that many request? Assume compare apple to apple (1 JVM server vs 1 BEAM server having the same features) handling concurrent requests.

Eric Chua05:03:14

I saw this in elixir forum (post #3) https://elixirforum.com/t/what-are-the-main-benefits-of-elixir-compared-to-clojure/29959/2 .. what BEAM can offer. Seems BEAM can handle concurrency / parallism really well.

seancorfield05:03:25

This is all theoretical. Build something and see how it actually scales. What problem are you actually trying to solve? What sort of traffic do you actually expect for it?

seancorfield06:03:13

With the JVM, you can scale horizontally with "small" servers or vertically -- but which you need depends on exactly what sort of traffic and what sort of computation & data processing you are doing. Trying to compare BEAM and JVM from a purely theoretical p.o.v. is just silly.

seancorfield06:03:00

Disney/Hulu is mostly powered by AWS services (a lot of which are based on the JVM -- I don't know if any of them are based on BEAM?). Amazon Prime Video is all based on AWS. Netflix already mentioned above is based on a lot of JVM stuff. Not sure about other streaming services. But if you're not building something at that scale, it really doesn't matter anyway...

seancorfield06:03:37

(some of this analysis is not helped by the fact that AWS includes Apache Beam which is nothing to do with BEAM and is 65% Java 🙂 )

Eric Chua08:03:13

Thanks for sharing. Interesting to know.

Jonathan Bennett12:03:30

Is this a good place to ask about trying to use the http://practical.li CLI setup? I'm trying to use the practicalli CLI to create a project, but I'm getting the following error message: clojure -T:project/create :template practicalli/service :name practicalli/todo-list :component :donut No function found on command line or in :exec-fn

Bob B13:03:28

the :project/create alias that's in <https://github.com/practicalli/clojure-cli-config/blob/main/deps.edn> has an :exec-fn, so if that deps.edn is in place, then that error shouldn't occur. Are you also getting a warning about an undeclared alias? Does the alias content in the GH file match up with what you have in one of your deps.edn files (I'm guessing probably user-level).

Jonathan Bennett14:03:52

No undeclared alias warning and I just checked the file and it's an exact match.

Jonathan Bennett14:03:58

I'm not sure what a user-level deps.edn file is, but I cloned the repo to my ~/.config/clojure folder

Bob B14:03:10

You can check clojure -Sdescribe to see which deps.edn files are contributing, and maybe make sure there aren't any ambiguous definitions (as in, the same alias defined in more than one of those files).

Jonathan Bennett14:03:29

Hold on, I am getting that error with other aliases in that file

Jonathan Bennett14:03:38

I think it may not be pulling that file

Jonathan Bennett14:03:46

But that file is in the list of deps.edn files

Jonathan Bennett14:03:55

And there are no other aliases that are the same

Jonathan Bennett14:03:09

Does it matter that it has a ~ instead of my unqualified path name?

Bob B14:03:30

what is "it" that has the tilde?

Jonathan Bennett14:03:03

The first two variables

Jonathan Bennett14:03:54

:config-files ["usr/share/clojure/deps.edn" "~/.config/clojure/deps.edn

Jonathan Bennett14:03:32

And :config-user "~/.config/clojure/deps.edn"

Bob B14:03:42

also, if an alias is found but misbehaving, and clojure is reporting that it can't find other aliases ("I am getting that error" <- not sure if this means the exec-fn message or the undeclared alias message), I would speculate that maybe there's a tiny syntax error. I'm not sure if the ~ is problematic or not

Jonathan Bennett14:03:24

I'm getting the undeclared alias error for : project/new and other lines in that file

Jonathan Bennett14:03:41

Hrm... A syntax error? But I made no changes, just fork and clone

Bob B14:03:37

after some local testing, it seems like the tilde might be problematic - can't hurt to try expanding it and see if that changes anything

Jonathan Bennett14:03:06

Ok, how would I do that?

Jonathan Bennett14:03:40

Wait, figured it out

Jonathan Bennett14:03:06

It must have been the ~

Jonathan Bennett14:03:10

Thank you!

👍 1
jf13:03:03

any I ok using graalvm instead of a “regular” JDK to run clojure?

Bob B13:03:26

In general, yes, you can use a GraalVM distro as a JVM

geeky_vin14:03:03

Hey Everyone, I’m having the weirdest issue. After a long time I fired up my lein to create an app project. But I got a “Could not transfer artifact X from/to central repo error” for all my dependencies when I ran lein deps So I thought my libraries could be out of date and reinstalled lein, jdk and clojure based on the instructions in https://clojure.org/guides/install_clojure yet the same error persists. I dont think I’m behind a proxy, what am I doing wrong?

phill14:03:01

Is lein itself up-to-date? A while ago (kind of a long time actually) Maven Central switched from http to https.

phill14:03:06

You had not overridden the Central url in any ~/.lein config file, had you?

phill14:03:13

Delete your ~/.m2/repository anyway because the dependency fetching library may cache a failure there and respect that failure instead of ever trying again

geeky_vin16:03:47

@U0HG4EHMH I dont think I overridden default args. Deleting .m2 didnt help unfortunately

Nundrum20:03:34

Do you have a web proxy set up? Maybe http_proxy or https_proxy in your shell env?

geeky_vin14:03:38

I’m running all this in a late 2019 macbook

Omer ZAK14:03:12

I performed a quick Google search on "Could not transfer artifact X from/to central repo error" and from the kind of answers I saw there, it seems to be a Maven problem. According to https://help.salesforce.com/s/articleView?id=001118073&amp;type=1 , it could be a proxy problem or a misconfiguration of Maven which causes it to fail to fetch a dependency for you. You may want to delete ~/.m2 (Maven cache) and recreate it.

👍 1
geeky_vin14:03:37

I tried that didn’t help. This is also wired because the lein is fresh and I did not explictly install mvn

Vincent15:03:19

@me1221 should be solvable, maybe hititng an old enpoint or something, can you provide more tangible results such as which deps are saying they are unfindable? is it all of them? what version ? lein -v

geeky_vin15:03:04

@v1nc3ntpull1ng I sure hope so, lein -version Leiningen 2.11.2 on Java 21.0.2 OpenJDK 64-Bit Server VM Even the template lein new app application doesn’t seem to work. All the libraries seems to have an issue.

Bob B15:03:03

please use a thread for the convo

1
👍 1
👁️ 1
Omer ZAK16:03:28

What Slack command forces the convo to continue in a thread?

Bob B16:03:16

When a message is selected, the right arrow key or t opens the thread for that message. There's also an option in the hover menu on messages to "Reply in thread".

👍 1
Vincent15:03:41

it's hitting clojars right?

geeky_vin15:03:59

@v1nc3ntpull1ng I think so, this is the entire error message

lein deps                                       
Could not transfer artifact org.clojure:clojure:jar:1.11.1 from/to central (): transfer failed for 
Could not transfer artifact org.clojure:clojure:jar:1.11.1 from/to clojars (): transfer failed for 
Could not transfer artifact nrepl:nrepl:jar:1.0.0 from/to central (): transfer failed for 
Could not transfer artifact nrepl:nrepl:jar:1.0.0 from/to clojars (): transfer failed for 
Could not transfer artifact org.nrepl:incomplete:jar:0.1.0 from/to central (): transfer failed for 
Could not transfer artifact org.nrepl:incomplete:jar:0.1.0 from/to clojars (): transfer failed for 
Failed to read artifact descriptor for org.clojure:clojure:jar:1.11.1
Failed to read artifact descriptor for nrepl:nrepl:jar:1.0.0
Failed to read artifact descriptor for org.nrepl:incomplete:jar:0.1.0
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

Vincent15:03:30

See if you can access http://maven.org at all

geeky_vin15:03:10

via the browser?

geeky_vin15:03:34

yes, I can access it via browser

dpsutton15:03:57

Thank y’all for switching to a thread. Try not to flood the channel with follow up messages to the same issue

👍 1
Vincent16:03:51

Just wanted to let all 18,000 members know I'm alive

😅 1
Vincent16:03:53

Hm, is it possible that just your shell is behind a proxy? I don't get how the browser would be able to get the .jar but not the shell. which shell are you using @me1221?

geeky_vin16:03:25

mac terminal

Vincent16:03:02

which one? iterm?

Vincent16:03:38

might recommend installing iTerm or maybe others have recommendations

geeky_vin16:03:06

I’m using the default mac terminal, let me try iTerm. the curl seems to hit maven url though

Vincent16:03:54

Cool, that's weird, and likely means something weird within that specific program. 85-88% + is where i'm placing my chips, optimistically better results w/ iTerm 🤞

geeky_vin16:03:27

unfortunately, iTerm gave the same error

Alex Miller (Clojure team)16:03:35

Try using ‘lein pom‘ to generate a pom.xml and then ‘mvn dependency:tree‘

Alex Miller (Clojure team)16:03:23

That gets you out of lein, and you might get a better error from mvn

👍 1
Omer ZAK16:03:08

I clicked on https://repo1.maven.org/maven2/org/clojure/clojure/1.11.1/clojure-1.11.1.jar in the error message in my browser (Google Chrome) and it opened a dialog for saving a downloaded file. So the problem looks like a connection problem. You may want to try the same in your browser. Then, if necessray, try to ping http://repo1.maven.org. When I tried https://repo.clojars.org/org/clojure/clojure/1.11.1/clojure-1.11.1.jar , my browser yielded the response:

NoSuchKeyThe specified key does not 
So this could be a transient http://repo.clojars.org problem.

Alex Miller (Clojure team)16:03:55

You should also look at ~/.lein to see if you have some bad config there

Alex Miller (Clojure team)16:03:06

I don’t think this is a connection or proxy problem, it’s something in your config

geeky_vin16:03:21

@U064X3EF3 thanks Alex, trying the mvn dependency:tree command yields same result. mvn dependency:tree [INFO] Scanning for projects... Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-clean-plugin:3.2.0: Plugin org.apache.maven.plugins:maven-clean-plugin:3.2.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:3.2.0 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-resources-plugin:3.3.1: Plugin org.apache.maven.plugins:maven-resources-plugin:3.3.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:3.3.1 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-jar-plugin:3.3.0: Plugin org.apache.maven.plugins:maven-jar-plugin:3.3.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-jar-plugin:jar:3.3.0 Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom [WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-compiler-plugin:3.11.0: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.11.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-compiler-plugin:jar:3.11.0 …

Alex Miller (Clojure team)16:03:40

Hmm, not sure what config affects both of those. Can you share the project.clj? Any config in your ~/.m2/settings.xml?

Alex Miller (Clojure team)16:03:59

Can you just curl -O one of those urls directly?

geeky_vin16:03:37

curl -0 on the url gets me a xml response

geeky_vin16:03:06

I dont have settings.xml in m2 folder

Alex Miller (Clojure team)16:03:30

Was the url you tried a .pom?

geeky_vin16:03:01

this is the project.clj BTW (defproject make-music-test “0.1.0-SNAPSHOT” :description “FIXME: write description” :url “http://example.com/FIXME” :license {:name “EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0" :url “https://www.eclipse.org/legal/epl-2.0/”} :dependencies [[org.clojure/clojure “1.11.1"]] :main ^:skip-aot make-music-test.core :target-path “target/%s” :profiles {:uberjar {:aot :all :jvm-opts [“-Dclojure.compiler.direct-linking=true”]}})

Alex Miller (Clojure team)17:03:46

Both lein and mvn are ultimately using http to download those urls, not much different than curl, so it’s definitely not a proxy or dns problem.

Alex Miller (Clojure team)17:03:12

Could potentially be ssl due to an old Java maybe?

geeky_vin17:03:12

java -version openjdk version “21.0.2” 2024-01-16 LTS OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)

Alex Miller (Clojure team)17:03:34

Well that’s new and shiny

Alex Miller (Clojure team)17:03:09

I don’t know, I’m out of ideas. Maybe rm -rf ~/.m2/repository in case there is something in that dir with bad permissions or something

geeky_vin17:03:07

Hmmm… ls -l ~/.m2 total 0 drwxr-xr-x 4 vinoth staff 128 23 Mar 17:11 repository vinoth@192 make-music-test % vinoth@192 make-music-test %

geeky_vin17:03:18

I see the same user in the permission list

Vincent18:03:53

... so this isn't like some app you were working on it's a blank project?

Vincent18:03:02

yeah throw it into the recycle bin and keep moving man

geeky_vin18:03:00

correct, but it seems like no clojure app is running now. despite reinstalling lein, clj and jdk

Vincent22:03:04

@me1221 that is certainly befuddling, are you working from a particular tutorial or getting started guide?

geeky_vin21:03:44

@v1nc3ntpull1ng not really, all I did was lein new app <app-name> and https://clojure.org/guides/install_clojure

Nundrum20:03:19

Do you have macOS 14.4 with that lovely new bug that causes Java to fail?

geeky_vin22:03:09

@U02UHTG2YH5 not quite, I have MacOS 14.3.1. But looks more like a java error though. How do I validate it?

Alex Miller (Clojure team)23:03:43

Did you ever try rm’ing .m2 ?

geeky_vin19:03:02

@U064X3EF3 yep, I did. Same issue for all of my projects

geeky_vin19:03:18

I’m running a 2019 Intel based Macbook BTW

seancorfield17:03:07

Folks, please use threads instead of posting a lot of top-level messages -- in particular if you are answering someone's question, do so in a thread (@w1 @v1nc3ntpull1ng).

thanks2 1
😳 1
seancorfield17:03:56

And, yes, I deliberately posted this as a reminder to everyone here in the #C053AK3F9 channel.