This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # beginners (25)
- # boot (30)
- # cljs-dev (22)
- # cljsjs (2)
- # cljsrn (1)
- # clojars (4)
- # clojure (223)
- # clojure-boston (1)
- # clojure-dusseldorf (1)
- # clojure-gamedev (8)
- # clojure-italy (5)
- # clojure-russia (122)
- # clojure-sg (3)
- # clojure-spec (26)
- # clojure-uk (42)
- # clojurescript (69)
- # clojuresque (10)
- # core-async (25)
- # cursive (10)
- # datascript (5)
- # datomic (12)
- # emacs (18)
- # garden (1)
- # interop (1)
- # jobs (1)
- # jobs-discuss (10)
- # leiningen (2)
- # liberator (1)
- # lumo (21)
- # nyc (2)
- # off-topic (210)
- # om (11)
- # om-next (3)
- # onyx (1)
- # pedestal (6)
- # re-frame (10)
- # rum (9)
- # specter (38)
- # uncomplicate (1)
- # vim (23)
- # yada (22)
@roberto @noisesmith : erlang vs clojure: fight! 🙂
I like functions and values as substrate, objects for stateful components, and queues between systems.
@qqq I'd just have you read my Quora answer to the question https://www.quora.com/Functional-Programming-Why-choose-Clojure-over-Elixir/answer/Didier-Adot?share=fec7786b&srid=hNqI
- data writes to refs, yes. But we tend to read a lot more than write, and reads can be uncoordinated, and furthermore almost nobody actually uses refs.
Clojure uses the software transactional memory concurrency model. That is, each access to data is made inside a transaction, and must either go through fully, or not at all.
That's true, but the only part that's novel is STM. I could have mentioned CSP, but I feel that would have gotten too deep in details of actors vs csp
Maybe I could reword this a bit though. It could sound like that's the only way to access data.
Actors are async OOP, why take something as painful as mutable objects then make them async?
There's nothing oop about actors, unless you consider anything that deals with state oop
@tbaldridge : I feel like in Clojure, if you have a bunch of different agents/atoms/refs, you need to coordinate their communication. They don't talk to each other until you tell them to.
In an "erlang/actor" model, I get the impression that up front, you tell them who to talk to, ... and then they just talk to each other.
This mess of 'routing messages/updates' between parts -- if manually handled in Clojure and seems defined once and behind the scenes in actor models.
@qqq that's one of the reasons why it's highly recommended to have one atom that holds all the state of your application. I don't understand how the answer to hard engineering problems needs to be "more mutability! More non-deterministic code!"
@tbaldridge : Most complex systems I see are individual components acting on their own. "One centralized atoms" feels alot like a mayor of a city dictating the action of every citizen via Sim 3 or something.
But that's the way "real life" works 🙂
To quote "Are we there yet", you don't send a message to the quarterback of the super bowl to ask him what the state of the football is.
The "real world" can be viewed as a succession of "universe states", with each old state existing forever as an immutable fact, and future states being produced by
(f state) => new-state.
@tbaldridge: in https://www.youtube.com/watch?v=096pIlA3GDo&feature=youtu.be&t=30m19s what does "userspace" mean ? (in the context of actors, dataflow, core.async)
And to be fair, that's exactly how most games are programmed. They may use mutable datastructures, but they are synchronous, it's very hard to enforce game rules on an async concurrent system.
@qqq yeah I didn't define it in the talk, I mean "outside of library or framework development".
Or the code in which you spend 80% of your time and comprises your business logic.
@tbaldridge : aren't you glad that the internet is an actor model and not a single atom 🙂
But it's not the actor model
I feel like at certain levels of compelxity, one can't micro manage everty part, and just code components and let them interact
many machines have multiple IP addresses (mailboxes) and a server can't magically spawn new servers.
the internet is more closely related to CSP than actors.
@tbaldridge : I must be misunderstanding "actor model" then; what do you call something like the internet with lots of connected machines, each with their own state, and can only talk via message passing
Not to mention that actors often assume infinate mailbox sizes and ignore backpressure.
If you were to simulate the internet in clojure, what would you use to represent each indivudla machine?
@tbaldridge in your talk you mention immutable / purely functional actors, do you mean stateful processes are a bad thing ?
So to start with, I'd recommend everyone watch this informal talk given by the guy who helped invent actors: https://channel9.msdn.com/Shows/Going+Deep/Hewitt-Meijer-and-Szyperski-The-Actor-Model-everything-you-wanted-to-know-but-were-afraid-to-ask
So we probably have to start with the problem that what is described in that talk ^^ is rather different from Akka, Erlang, etc.
@leonoel but yes, one of the biggest problems I have with actors is that they (like CSP go loops) are very hard to debug because there's no good way to get at their internal state.
So you can improve the model given by Erlang/Akka by defining actors as a application of a function over old state to produce a new state, function and messages to be sent
(f state msg) => [next-f next-state [[dest msg] [dest2 msg2] ...]]
And then the system that runs the actors could allow you to deref @state just like any other clojure ref
well, given that actors (like go blocks, or agents) process one message at a time, they're free of concurrency issues, so they're not that hard to get right
As that talk mentions ^^ "conceptually one message at a time", the actor system is free to optimize and run many messages through an actor at once if it knows that the actor will only ever read from the state
@tbaldridge: that video is tedious
well if you want to understand hard concepts sometimes you got to do some research 😛
yeah, but I think there's probably 2 pages of dense math equations somewhere instead of these three guys chatting
Perhaps, but I'd rather sit and talk to "these three guys" talk anyday. Remember, one guy helped invent actors, the other helped write parts of Haskell, Linq, and C#.
But I think these very abstract "chats" are a good way of removing conceived notions of what actors are.
But at the core actors are: 1) a process 2) that can send messages to other actors 3) based on a message it receives
@tbaldridge : yeah, this video is not for me; but what I did get is that csp (communicating sequential processes) != actor; and, as you mentioned erlang != actor, go != actor ... right?
go lang is csp, that's for sure. Erlang bends a lot of actor rules, and adds things. For example Erlang allows for mailbox searching, and IO inside of an actor's message processing loop.
Mailbox searching can result in degrading performance over time. And IO inside an actor has the same problems of any IO inside a function, it's impure.
where does core.async go loops fall in this? I feel it's a "strict subset" of go, in the sense that <! and >! can only happen in the "top level of the go loop" and not "nested in functions" since
go is some magical macro
Both go and core.async are implementations of CSP.
The problem isn't nested functions, but that both CSP and actors often recommend a pattern of programming that is impure: namely IO inside functions that perform decision logic. This is why transducers are fantastic, we can test transducers in isolation, in a deterministic manner, then we can use them in a impure context like a channel. This allows us to test our impure code in one place, our transform logic in another.
So for example (chan 1 (map f)). How do we test this? We don't. We know
chan is tested by core.async, and
map is tested by clojure.core. So all we need to do is test
f. Once we compose this channel into a larger system we can also use integration tests to make sure it all works correctly.
Is the crux of your argument: 1. pure functions = easier to test than impure functions 2. <! and >! perform IO and are thus impure 3. any function with <! or >! nested inside of them are impure and thus hard to test 4. thus, we'd like <! and >! to be top level, so the other functions involved are pure (if so, I buy the argument)
yep, and that's about 50% of my talk at Clojure/West 🙂
Don't get me wrong, I love the idea of actors, and continue to play with them from time-to-time. However I have yet to run into a situation where actors made a problem clearer than it would have been using immutable data and "normal" clojure code.
@tbaldridge a channel with a transducer is not unlike an actor
They are nothing alike
Actors have unbounded message boxes, unknown outputs, can "become" new actors, have internal state.
Most of those are completely different from channels + transducers
@tbaldridge : IIRC (can't find reference), Alan Kay once drew a reference between OO and biological cells. Cells (1) have their own state (2) do their own thing (3) can't view/poke the state of other cells and (4) communicate only via message passing. This also describes individual machines on the internet. Using core.async, how would one model cells/individual-machines via go blocks ?
You haven't watched my talk from Clojure/West have you?
It's a little over 35min of talking about different models that handle different tasks better than raw core.async or even actors.
So in my case, what I prefer is the Dataflow model. Nodes with explicit inputs/outputs connected via a configuration graph. True it's not as clean in a distributed model. But even stuff like Onyx can be viewed as a dataflow graph.
"using emacs leaves me with no closure" (facepalm)
@tbaldridge the model you describe as "dataflow and FRP" has working clojure implementations ?
Although I have a non-OSS implementation that is 100% immutable.
Yes they can. Rx is often built by attaching objections into chains, so yes, it looks somewhat like javelin.
In most situations I prefer to build up a dataflow graph as data, but that's an implementation detail.
If you store the entire state of the dataflow graph in immutable data you can do step-by-step debugging, time travel, etc.
well, what annoys me with frameworks like Rx is that you handle "live" objects, and the graph itself is a mutable thing
Which interestingly enough is also a problem with actor systems.
It's currently part of a rules engine, I should just pull it out as a separate library. Maybe I'll look into doing that this Friday.
hello guys, I am looking to extract some PDF document in Clojure. I did some proof of concept with
Apache Tika and
PDFBox. For both solution I can easily extract text but I have no clue how to get back
titles from the original PDF document
any ideas on how to achieve this ?
Hi ... is anyone using gradle and emacs/cider together? ... could do with help jacking-in to an existing project 😉
there's a long build.gradle script that i'm not really sure i want to dig into too far yet ... so I downloaded clojuresque.jar v1.7.0 and put it in the gradle home dir (/usr/share/gradle/lib/plugins)
@tbaldridge you said... > So in my case, what I prefer is the Dataflow model. Nodes with explicit inputs/outputs connected via a configuration graph. True it’s not as clean in a distributed model. But even stuff like Onyx can be viewed as a dataflow graph. prefer as opposed to what? you linked your talk to core.async, but i’m having a hard time comparing core.async to onyx.  I mean, couldn’t someone build a dataflow model using core.aysnc?
@mobileink fair point ... i'll ask there too if I get no joy here ... thanks 😉
@drewverlee the gist of the talk is that core.async can be used to create higher level constructs like dataflow, pedestal interceptors, etc. But just like bare actors don't give you a whole lot, the same is true with core.async.
So I'd prefer to find a higher-level model that allows me to not have to think about concurrency or parallelism on a daily basis. I should be able to write pure functions, and then something else can take care of those details for me.
@mobileink nope ... i'm joining a clojure project that uses gradle as a build tool
@drewverlee Onyx is the same way, you define nodes that do things: read from files, sum results, etc. And then wire them together with a config. What Onyx does with that config is unrelated to the work performed by the nodes themselves.
not as far as i can tell... everyone seems to use cursive ... i'm the only emacs user on the team
probably not ... its 425 lines long and probably contains a bunch of things that shouldn't be shared in a public forum
using gradle for clojure, omg, i spent a bunch of time learning gradlecin order to use clojuresque and ended up running screaming for the exits.
the last place i worked, there were a bunch of java devs moving to it from maven ... the stuff i worked on was all lein 😉
do your wholevteam a big favor, and get a big raise, by rewriting it in boot. or leinengen.
i'm sure i can extract the classpath from gradle, then add nrepl & cider middleware and just run java -cp ....
but if you've got gradle stuff that alreafy works, can't you just run it and cider-connect?
maybe ... i just need something that injects the correct version of nrepl, the cider middleware and maybe the clj-refactor stuff ... emacs knows what those versions are, and i think i can work out how to get the cp out of gradle ... but it's 10pm here, and i should get some sleep ... got an early flight this morning, and could do with the sleep ... thanks for your sympathy @mobileink 😉
@tbaldridge Ok yea, that makes sense. I’m not sure Onyx supports a multithreaded local version. I mean, they run their tests using core.async, so its possible to simulate the model locally, but i don’t know what it would be like to use it in production. There was a talk at clojure remote about a higher level abstraction on top of core.async. https://www.youtube.com/watch?v=-SMHkPAEGIk I just confirmed that isn’t opensource though 🙂
@l0st3d @mobileink Gradle’s actually not that bad, I use it but with a plugin I wrote: https://github.com/cursive-ide/gradle-clojure
de gustibus non disputandem. me, i 'd rather slug drano than so much as look at groovy again.
did not know about gradle-clojure. did that come out after clojuresque? looks good, the more the merrier.
my foggy recollection is that one reason i came to hate gradle was because it was almost impossoble to figure out how to write plugins.
Hm, gradle plugins can be plain groovy / java code. So technically they have the same power as boot.
I agree that clojure is more expressive than java / groovy. But you can do the same stuff with gradle that you do with boot, thats all I said.
Hehe, one year ago we switched from several 1000 lines of ant to gradle and it was mostly straight forward, compared to the ant stuff 🙂
i don't doubt that, best wishes to amybody who wants to. but there is a pain factor.
fwiw i found gradle/groovy almost totally impenetrable. which is not what you want in your build tools.
As with every framework, as long as you stick with the suggested / proven paths its fine. Everything beyond gets messy, complex and hard to maintain.
that's exactly the problem. frameworks are great, until they're not. i gradually discovered that gradle did not in fact solve the problem, it just moved it, begat a different kind if complication. boot is a whole nother thing, it actually solves the problem.
put it this way: show me anybody who has mastered both boot and gradle (or any other fancy bld system) and i will bet my next paycheck that said person will pick boot.
the whole idea that we need specialized "build systems" like gradle or whatever is wrong from the git-go, imho. it's all just peogramming.
I haven’t mastered boot, but when I tried playing with it I didn’t like it and went back to lein for clojure and gradle for java/kotlin projects. 🙂
@roberto: how much of "didn't like it" is attributable to sucky docs? curious, cause i think that's a big problem.
that could well be the biggest turn off for me. It has so much hype but I found it much harder to even grasp the mental model and do anything with it.
It is difficult to justify investing time on it these days with so many things coming our way: build systems, deployment pipelines (kubernetes, docker, vault, etc), front end frameworks, other programming languages
learning clojure and emacs already required some effort, and to pile learning boot on that was just too much for me.
plus, we are a consultancy, so sometimes we have to learn new languages rather fast. Had to learn Kotlin for a project while on the job. So time is very limited.
I decided to not learn any new js framework, no new testing framework (no matter which language, if using Kotlin/java stick to JUnit, if using clojure sticking to clojure-test)
I do tend to go off the deep end learning new concepts, right now I’m fascinated by type systems, especially the concept of expressing a program as a state machine and focusing on making impossible states …. impossible …. (I think that is what the creator of OCaml calls it).
I think Dave Thomas gave a talk recently at GOTO expressing the same concepts, but I think his lack of familiarity with more advanced type systems hurt his presentation
fascinating thing: you might think that as time goes on we would settle on the best stuff and be done with it. instead we see the opposite: massive proliferation of new languages and tools. it's about 3 fulltime jobs just to keep an eye on things, let alone master anything.
I am actually enjoying this new age of innovation in programming languages. It is the proliferation of different tools that do the same thing that I’m tired of. Especially in the JS space and the “devops” space.
There are all these tools in the devops space that I can hardly understand what they do
heh. fortunately i can ignore the js churn. devops? i hear its cool, that's about as far as i go. i've heard of this thing they call "docker", does that count?
hehehe, yeah, I actually like docker. It is the orchestration tools that I find intimidating.
you could probably make a good living just knowing what those things mean and bsing clients. 😉
We started using Kafka before it became hype, and it was a headache. I remember having to read the source code if we wanted to do anything with it.
submit your homepage, we'll vet it for buzzword compliance - and alert you to the latest! That'll be $20K, please.
E.g. of an actual job description that reminds of the
buzzword compliance idea:
We are a multi-award winning information and advisory company that helps leading global companies to improve their strategic decision-making and operational implementation through the best use of data and analytical techniques. our open plan offices enable our people to collaborate with industry leaders and incredible minds
roberto: "industry leaders, and also people with incredible minds"
@mobileink clearly it’s an award with arbitrary first class dispatch using either the implicit global hierarchy of award types or optionally a local first class hierarchy argument.
roberto: "industry leaders, and also people with incredible minds"
@mobileink Actually gradle-clojure is written in Kotlin, and you can now use Kotlin for build files too.
depends on your needs, i guess. if you're developing clojure code, why would a kotlin build tool be a good thing when you already have boot and leinengen?
@mobileink Because I need access to lots of other JVM stuff I don’t want to have to reimplement myself (like compiling Kotlin, running profilers etc). Basically, my project is a JVM one, not pure Clojure.