This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # announcements (1)
- # bangalore-clj (1)
- # beginners (47)
- # boot (16)
- # calva (33)
- # cider (14)
- # clj-kondo (3)
- # clojure (46)
- # clojure-india (1)
- # clojure-italy (6)
- # clojure-nl (4)
- # clojure-uk (5)
- # clojurescript (10)
- # dirac (8)
- # emacs (1)
- # fulcro (1)
- # leiningen (14)
- # lumo (1)
- # off-topic (178)
- # pathom (9)
- # planck (17)
- # quil (2)
- # reagent (5)
- # reitit (6)
- # shadow-cljs (55)
- # tools-deps (3)
https://nypost.com/2019/04/12/software-engineer-close-to-julian-assange-arrested-while-trying-to-leave-ecuador/ -- that's pretty shocking (re: Ola Bini)
https://www.imf.org/en/News/Articles/2019/03/11/ecuador-pr1972-imf-executive-board-approves-eff-for-ecuador March 11, 2019 - IMF Executive Board Approves US$4.2 Billion Extended Fund Facility for Ecuador
In Lisbon for a few days, hotel's a bit gloomy. Anyone know a nice place with wi-fi to work?
Working with multiple micro-services, many teams, multiple contexts for the same business entity, and multiple sources for the same entity. It seems like the tower of babel story, we've build something so big that every teams starts to speak its own language. I see that this independence brings success, but it also seems I'm writing structure transformations all the time. I've also seen this happen at my previous job, and also many coworkers have seen this before at different scales. ¿Have you deal with these "tower of babel integration" problem? ¿Did you enforce a common API across all teams, or did you solve it by automating the transformations, or making a catalog? ¿Or do you just code the transformations?
How I think it would work best is all using Kafka for messages, with the same 'type' of massages. For example strings as keys, and values avro encoded with schema's. And on each topic there is only one producer. Of all JVM you could use specific records if you want. If you also makes sure the schema's are always backwards compatible I think most things are tackled. Depends a bit on what the product is, but towards the front-end I would expose multiple topics with GraphQL.
This is a big topic for me also, another question I had was: do the microservices need to be written in the same language? I would say of course not but many Enterprise companies are worried that it would be a mess recruiting people...dunno
What do you mean by “multiple sources for the same entity”? Different teams are duplicating work so there are multiple models for the exact same data?
@U0C8489U6 I think it’s fine for microservices to use different languages, but for a number of reasons I advocate that companies limit “approved” languages to two or three with very few exceptions.
Could you please describe a specific example of a structure transformation? I think using Message Brokers like RabbitMQ and ZeroMQ can really help reduce duplicate code. And you could even use foreign language interfaces… e.g. how Clojure can call other languages inside of it. I am researching Vert.x because I could write it in Clojure, Python/Jython or even Jruby/Ruby. You can compare and contrast that to Elixir Umbrella Projects that try to combine monolithic with micro services.
Kafka: I guess a big enterprise bus is one way to solve it, or at least for a subsystem. Polyglot: multiple languages is not an issue.(Just use one lang for service) I guess the issue I tried to mention is being a mediator between multiple "bounded-contexts", that's not exactly model duplication, but thats the part where we start to speak different tongues
You might try Vert.x because it builds a message bus into the tool. They built it for providing micro services (reactive programming) and polyglots. It supports most of the JVM languages. But I need to do a proof of concept… to make sure it’s not another callback hell async problem. It has a promises/futures library to fix that, I believe.
I love the idea of it. But I can’t say yet until I put it in production and load test it.
It’s also not the easiest to setup. But sometimes the easier stuff to setup is a headache in the long term.
The only disadvantage for me so far with Vert.x is that it doesn’t auto scale as easily as Elixir. But Vert.x has so many more advantages. JVM, libraries, machine learning, foreign language interfaces, actor model + reactive model + blocking thread worker pool… and more
Elixir gives you rapid dev with a monolith and umbrella style micro services. But it’s not a general purpose tool for machine learning, cpu number crunching. And Vue JS can give me the rapid monolith advantage.
From my experience using Kafka is a lot easier then VertX. The Kafka ecosystem is huge making it easy to connect basically whatever you want. If you have a legacy service you still need, you can create events from the changes to the the database. Also you have access to almost all languages for clients thanks to librkafka. But maybe most importantly Kafka allows parallel async programming as a whole, with execution of 'simple' synchronous code.
Can anyone explain to me… if a computer has 8 cores, it can have 16 threads (if hyper threading, intel). But why do I see code setting up 100's of threads? Are the threads virtualized and pooled? And there is a difference between actual real world parallel threads versus pooled threads?
Yes they are, its the task of your operating system to schedule threads and processes on the CPU
Ahh ok. So it’s a virtualized thread count. They CPU interleaves them (schedules them) on the actual hardware threads?
Well, what your operating system is concerned, a octa-core with HT is actually just 16 cores
ahh, the hyper threading makes it look like 2 cores from the operating system’s perspective?
I am researching Clojure threading primitives in comparison to just the actor model. e.g. clojure agents
threads are "abstraction" for "concurrency", but we you go down the hardware there is a limit to what is actually done in "parallel"
JVM will introduce (cooperative) fibers\\continuations in near future, which is a lightweight replacement for (pre-emptive) threads, it will allow JVM to manage async tasks in more efficient way, mapping hundreds of thousands fibers to a thread pool matched to real cores in size.
@UH16CGZC2 so something similar to what Go or Erlang does? That sounds amazing! What I was able to find was "Project Loom". Very curious when this will be available in a stable JVM release 🙂
@U8ZN5EHGU correct, in a year or two, also probably tail call recursion, since they changing how stack frames being handled
Can't wait! Also curious how this will be integrated into Clojure since Clojure still needs to support older JVM versions..
Java Fibers will be similar to Actors in Erlang where no locking is needed? I am not sure how Golang fits under the Actor model since it’s really CSP (Communicating Sequential Process) and actually relies heavily on locking/mutexes?
Reading this… it seems Java Fibers are the equivalent of actors with async ability?
@UGQJZ4QNR java fibers is green threads scheduled by JVM with ability to pause and continue by saving fiber stack on heap
@UGQJZ4QNR pretty much, regular Thread will become lower level of abstraction rather for continuous calculations
wow. Vert Sync allows for async non blocking calls but with a syncronous style, no callback hell. I am reading dzone website.
it says the Fiber is blocked… but the underlieing thread is not blocked. So you get the best of both worlds. And it doesn’t turn your code into spaghetti callback hell
but generally with green threads, its your app, or host platform (f.e. the JVM) that does the scheduling
I am curious because this weekend is my concurrency/threading study time. I need to write a deadlock on Clojure to learn how to simulate it.
(lol, google is showing me an image of a green ball of yarn when looking for “green threads”)
well, everything can deadlock, its just that Clojure has a mitigation strategy when sharing memory between threads
I am asking because I am trying to figure out if I need lower level threading like Clojure provides. Lower level than actor model.
(its actually funny that you deleted the question, you kinda illustrated the problem that threaded execution can have, if you are not careful the “world” can change under your feet)
also idk what you’re using to learn Clojure, but http://shop.oreilly.com/product/0636920013754.do has a really (insert punhusky here) thorough chapter on concurrency
My conclusions, faulty as they can be, is that Clojure rules the realm of data crunching because of these lower level concurrency patterns?
I installed Spacemacs because I knew VIM. And I really don’t want to go back to Jetbrains since that.
(A very famous deadlock problem: https://en.wikipedia.org/wiki/Dining_philosophers_problem )
I will try to write that dinner problem in Clojure… Right now I am comparing Elixir to Vert.x
I think I will bow out; I’m not a specialist on concurrency / threaded programming enough to compare languages and say which one is better 🙂
I am either going to put Clojure on a message bus… or use Clojure inside Vert.x
yeah, same. I have a theoretical basis because of my CS degree, but I wouldn’t know the ins and outs of the concurrency model used in various languages
but Clojure can give you lower level concurrency, shared memory and also use actor model
You’ve got 3 different models in Clojure, plus anything you can steal from Java, plus lazy evaluation which is kinda generators painted over, and generators are kinda coroutines painted over if you squint 😉
Anyway, I’m off to sleep b/c it’s near midnight here. Thanks for bringing this up, it reminded me I need to explore this area in Clojure more 🙂
the only reason I’d use Elixir is because it can do micro services (umbrella) without the headaches
(if you’re wondering about what language to choose for a project, my advice would be: don’t look at the language, look at the libraries/ecosystem.)
oh yeah, also this is an interesting library in development: https://github.com/reborg/parallel
@UGQJZ4QNR You may find this Reddit thread interesting (and the PDF it links to moreso): https://www.reddit.com/r/programming/comments/3behbh/a_comparison_of_the_jvm_and_the_erlang_beam_vm_pdf/
Ahh, reading it now. The first thought is… actor model alleviates the pausing garbage collection problem because the entire actor’s memory is freed up.
Pausing is much less of a problem with the G1 collector than some of the previous collectors -- and GC is always always improving in the JVM.
Wow, I did’t know that. I haven’t looked at Java in 5 years. This is super good.
I think 1 use case for no pausing GC is high frequency trading algorithms on the NYC exchanges
Yeah, there are some people who think that HFT and Oracle's JVM are not compatible. The makers of the Azul JVM claim that theirs is suitable for HFT.
I am thinking of how an email client will thread a bunch of sent emails when I hear green threads?
whatever you do leave some ram for off-heap memory. JVM store "the objects" in "heap", but it also needs off-heap memory
want to know a very interesting comparison of Clojure versus Elixir data type for lists?
I am leaning towards putting Clojure on top of Vert.x if my proof of concept shows me I can scale it across machines
I want to find out if I can run Jython and Clojure side by side. And not have to use Cython. I want to utilize JVM for my entire stack.
yes, but Graal is an experimental polyglot VM that allows you to run jvm languages and python intertwined. That said, it is limited in some senses. For one, it disallows some forms of reflection
no, generics are really only a concept in the Java compiler. As soon as your program is compiled, they are gone
Cython tree recursion was as fast as Clojure lazy eval. But I don’t know if Cython did some under the hood compile optimization.
You can certainly run multiple languages side by side on the JVM. Sometimes the interop between them is awkward tho', depending on how they compile their constructs for the JVM.
For example, some aspects of Scala can be difficult to call from other languages.
would you advise against calling languages inside other languages? And just use a message bus instead?
It depends on the languages involved and the JVM-level APIs they expose in terms of data structures etc.
Writing the code was the easy part. But supporting Jruby on the server was a headache
It doesn’t work with monoliths because every single library needs to be thread safe in Jruby… I am pretty sure