Fork me on GitHub
#fulcro
<
2019-10-10
>
cjmurphy03:10:09

@tony.kay I made a repo to illustrate the point about lookups not being recognised by default merge behaviour. I'm starting to think I need to put Pathom on the client. Interested in your advice as to whether that could be a solution here. And/or if it would be sensible for default app state merging to not always just remove attributes that are not returned, but rather think of just an id being returned as a case where a lookup ident needs to be placed in app state. https://github.com/chrismurrph/lookups-test

tony.kay04:10:44

Pretty sure you can handle that case with :pre-merge

cjmurphy04:10:58

That sounds like the thing I'd be looking for, sounds like the right place. Thanks I'll look into it...

cjmurphy05:10:47

Great that worked. Using :pre-merge on Product, having the function return (merge current-normalized data-tree), does the trick. I updated the test project.

cjmurphy13:10:15

Another answer is to do the df/load query completely on non-UI defsc-s where the edge defsc are 'stubs'. i.e. stub meaning the lookup component only has the id attribute. The :pre-merge trick only works right at the edges - your lookup must not do any further lookups, or else you will fall afoul of the shape of the query not matching the shape of the response.

Arto Kalishian12:10:18

Why is fulcro not included in the benchmark tests of techempower? https://www.techempower.com/benchmarks/#section=data-r18&amp;hw=ph&amp;test=fortune I wonder if anyone did any performance benchmarks against other known frameworks.

Ludwig12:10:10

I think it is because fulcro is mostly front end framework, you can use any backend.

Arto Kalishian12:10:52

Fulcro is Full Stack as far as I understand and the Pathom is in the back-end.

Ludwig12:10:48

not really, Pathom is just a EQL parser engine, you can use Fulcro without Pathom.

Arto Kalishian13:10:41

And what is your estimation of Fulcro's performance based on your experience using it? Does it compare to Java or Python or Node?

Arto Kalishian13:10:39

I would expect it to be on par with Elixir/Phoenix.. maybe not as fast but at least equivalent..

Arto Kalishian13:10:54

Since both handle concurrency well.

Ludwig13:10:09

well, fulcro, imo, should be compared against react+redux, angular, vue ... etc , not to Java or Python

Arto Kalishian13:10:03

Not sure if I fully understand. So Fulcro does not process http requests on the server?

Ludwig13:10:10

no it don't, you will need a webserver for that job, like immutant, aleph ... etc

wilkerlucio13:10:36

Fulcro only talks EQL, it sends EQL queries and mutations, someone needs to process this, Pathom is the current favorite option, and Pathom can run both on the client or on the server, so both Fulcro and Pathom are agnostic around HTTP part, you may have none (for example, you can write a client app that stores data in IndexedDB, that would use Fulcro and Pathom, but no HTTP at all). makes sense?

Arto Kalishian13:10:09

It will make sense maybe when I get to that part in the documentation.

Arto Kalishian13:10:30

But thanks for explaining. At first I thought Fulcro is a full stack solution as the web-page claims.

kszabo13:10:33

That was true in Fulcro 2, when it had server-side support

kszabo13:10:59

now there is no need for that, since the interface between frontend/backend is just EQL

Arto Kalishian14:10:41

Finally I got to that part in the documentation

Arto Kalishian14:10:59

Since pathom can respond to client requests I believe we can consider it as an alternative to a web server right?

Arto Kalishian14:10:59

This again means that the performance of concurrent processing of requests can be measured just like any other framework.

Arto Kalishian14:10:13

For me at least :)

Arto Kalishian14:10:43

Maybe the only thing that's not possible compared to a web server is that transferring images will not be possible through transit. Does it?

kszabo14:10:18

That’s correct, transit uses JSON or messagepack under the hood. Since those don’t have any native binary types, you need to resort to strings (base64)

kszabo14:10:48

but EQL is used for ‘rich’ data, blobs are best served through other channels like CDNs

👍 4
kszabo14:10:14

same as you wouldn’t put an image blob in a REST API response

👍 4
henrik14:10:45

I’m not sure how valuable the TechEmpower benchmarks are anyway. For most cases, I think the most useful point to be at is “good enough performance, for a reasonable amount of effort, maintaining good enough flexibility for alterations in the future”, which benchmarks can’t tell you.

Arto Kalishian14:10:43

But good enough is too general.. is it as good enough as rails or hapiJS or django? :)

henrik14:10:25

Effort and flexibility are hard to measure, because the tools can only help you so far with that, they depend a lot on you, and are hard to estimate without experience. But pretending like they are not concerns doesn’t make the benchmarks any truer when estimating the value of a solution.

wilkerlucio14:10:41

one thing that may make Fulcro hard to measure here, is the same problem trying to measure GraphQL requests, different from REST where endpoints have a sort of consistent response time, EQL/GraphQL requests will vary a lot depending on the user request, so just have to think what exactly is been measured there, IME most of the time is spent on the user code (requesting other services, requesting databases...) so that needs to be taken in consideration, I don't know TechEmpower but looking quickly it seems to be measure the raw overhead of the libraries, which is important only up to a point, or maybe I just not understanding it correctly

henrik14:10:22

For certain parameters, benchmarks are fine as a decision-making tool in my opinion. For example, if Reitit is faster than Compojure for routing (or vice versa), you can go with whatever is faster, because it’s not going to be vastly different anyway. But for choosing between Django and Fulcro, I’d saying that there are a lot of other parameters worth considering as well.

henrik14:10:33

Same with picking between http-kit/aleph/undertow/what-have-you as a server.

henrik14:10:06

But for example, what about the hard-to-measure-up-front-but-incredibly-interesting speed of development as a function of application complexity as a function of business goals?

👍 16
👏 12
Arto Kalishian15:10:39

The reason I wanted a performance benchmark is much simpler than what most will think. A group of my tech friends told me that one vps may not be enough for my website project because Django consumes a lot of ram if more than 100 users are concurrent. I couldn't reply with which framework I'm intending to use because Fulcro isn't popular at all let alone Clojure in my country. So considering clojure is a language more optimised for concurrency I only have an assumption that fulcro is much lighter in terms of resource consumption and will be able to handle load better. But a benchmark estimate would have been much better to know how much vps resources I may need before things go bad.

tony.kay15:10:15

100 concurrent users doing what? You see, I could either find that laughable or applaudable, depending on the stuff the users are doing. If it is a simple website for filling out forms: laughable. If it’s a data analysis platform it might be quite respectable. Microbenchmarks won’t help you.

tony.kay15:10:47

But you did hit on something that is very much worth considering: popularity. Clojure is not popular. Fulcro is not popular. That can change, but it means you will face problems in staffing, face resistance from doubters, and have to do some cheerleading. Those aspects are absolutely critical points to consider. I’ve seen devs literally pee on co-workers computers and quit over library choice (yeah, put that person on a do-no-hire list)…people problems are very real.

😲 4
Arto Kalishian15:10:28

What kind of coworkers are these

tony.kay15:10:36

but true story

4
Arto Kalishian16:10:38

Well my website will be simple very few images and 2 or 3 tables. It will basically be a form where people can log their experiences in certain fields. Other uses will be able to give reviews on these experiences. And that's all. But i expect the website to have a lot of traffic incrementally when it is indexed on Google. I haven't said the whole idea because actually no such website exists in the field I'm intending to focus on.

Arto Kalishian16:10:00

But the data on this website will need to have high integrity because it will be a future reference for many who will redirect on it as a sort of organized wiki.

Arto Kalishian16:10:21

Let's just assume the closest thing to it is a wiki.

Arto Kalishian16:10:02

Will probably use postgresql but I'll see if pathom is enough first.

tony.kay16:10:03

With Fulcro and a JVM your bottleneck with be the database. Pathom is not a database. PostgreSQL can be the database of record behind Pathom. Pathom is a parsing technology for interfacing the EQL requests from Fulcro into the data store of your choice.

Arto Kalishian16:10:56

And you have better recommendation than postgres?

tony.kay16:10:57

I would expect a JVM on a $15/mo EC2 instance to easily handle quite a bit of load. The costs for RDS would be where you’ll have to ramp things up.

tony.kay16:10:09

and that will be true independent of your back-end, for the most part.

tony.kay16:10:46

“better” is relative. PostgreSQL is a great piece of tech

tony.kay16:10:36

if your write-load is super high, you will commonly need to make some amendments to the infrastructure over time (caching, write queues, possibly add in something like Kafka), but I would not predesign for that kind of load unless I knew it was materializing. Lots of complexity and costs around that.

Arto Kalishian16:10:42

What makes postgres great I think is proven reliability that came out of software maturity.

Arto Kalishian16:10:05

There doesn't seem to be any graph based db that had enough time to prove reliable

Arto Kalishian16:10:00

That said I may consider dgraph, neo4j and crux one day

tony.kay16:10:03

you don’t need a graph db

tony.kay16:10:17

unless you personally need the features of a graph db

tony.kay16:10:00

Fulcro uses a graph on the wire and client because it makes sense there (the UI is a tree)

Arto Kalishian16:10:52

The reason I think about graph dbs are 2: 1. I think schemas are complex 2. Why not get closer to the data structure of the clj lang? Makes it simpler.

Arto Kalishian16:10:06

And above all that if performance is better...why not.

tony.kay16:10:53

(1) you’re mixing concerns. Graph dbs and schemas are not orthogonal.

tony.kay16:10:20

(2) The data structure of your materialized view will often not match the graph in (whatever) database.

tony.kay16:10:42

You will be using Pathom to bridge the gap.

tony.kay16:10:49

and Pathom makes the gap pretty small

tony.kay16:10:22

so, use the database that makes the most sense for your overall business concerns: cost, availability in deployment envs, complexity, atomicity support, etc.

Arto Kalishian16:10:42

I guess I will find out soon when I am near deploy then. Thank you Tony for your advice and input, and everyone else contributed to the thread. I hope I can comprehend the framework as soon as possible.

tony.kay16:10:43

BTW, from earlier in the thread: I still consider Fulcro to be a full-stack solution. Fulcro defines an optimal way for you to interact in a full-stack manner. It considers the complexity of both sides in its design. Most front-end libs say absolutely nothing about how you talk to a server and what might go on there. Fulcro doesn’t shove you down the path of a particular database, but it does push you down a path and pattern of interaction…and that is the most important part of the “full stack” consideration IMO. Going one step further would be overreach: as soon as I dictate a database half of the users “lose”. Fulcro 2 was no different, but it did have more server functions for giving you “entry points” to hooking up your real storage. Pathom is a better implementation of those. I see the story of Fulcro 3 as a better full-stack system than F2, not any kind of abandonment of it.

💯 4
tony.kay16:10:29

All that said, I’d love to see one or more community template projects where a particular database is selected and a bunch of automation scripts are provided.

tony.kay16:10:46

to get ppl going that already know that’s what they want

henrik19:10:03

Depending on the read/write ratio, Datomic Cloud is a nice way to both host the app and get a database to go with it. Datomic is generally better where you expect the data to be read more often than written.

kszabo19:10:08

I think you mean Datomic Ions, no? That has the ability to run functions on the DB nodes locally.

henrik19:10:12

Yeah, Cloud is the service, an Ion is the moniker for the pod that that your app runs inside.

henrik19:10:51

You may have multiple Ions running in a single Datomic Cloud deployment.

👍 4
kszabo20:10:28

Yup, that’s correct, except that there is no ‘pod’ (at least in Kubernetes sense) you are literally on the same process/classpath as Datomic AFAIK

henrik21:10:09

True, it’s just a metaphor. We have a GraphQL API running in this manner, and I’m running an (admittedly PoC) webapp this way as well. Works great.

tony.kay15:10:11

Frankly, I think this line of microbenchmarking to be about the most insane waste of developer productivity on the face of the planet. I’ve only ever seen one web dev env where performance was so horrible that it dwarfed the real logic: RoR (and even then it is about the most popular lib on the planet for web dev). Pretty much everything else, in 35 years of development, has boiled down to crappy algorithms in application space, poor choice of back-end database, poor caching strategies. I’m not saying that a web library cannot have performance issues…but there are no apples-to-apple comparisons that make any sense. I can tell you the two places where Fulcro can have performance problems you care about: how fast does the query run on the UI, and how fast can React update the DOM. Everything else is pretty much under your control, and in fact, depending on your design, the UI query and React story is quite a bit under your control as well.

Arto Kalishian15:10:25

I think that people don't choose their frameworks on their performance most of the time but at the same time since I'm pretty sure clojure/Fulcro can give very competent results matching pedestal or java frameworks that it would be a good kind of marketing for whoever visits these results and it will make them curious to check fulcro.

tony.kay15:10:59

I don’t want those users 😜

tony.kay15:10:26

(sold on marketing based on benchmarks on a thing where benchmarks make little sense)

tony.kay15:10:30

BUT, since you posted a thing that does matter in the thread (memory footprint and hosting requirements), perhaps we should re-frame the discussion

tony.kay15:10:35

You can run whatever back-end you want. It need not even be Clojure. You can use a BaaS. Fulcro has very little bearing on what you have to use there. However, you will get the best benefits if you run something that can do Clojure since Pathom is a nice add on that side. Now you’re talking JVM requirements. There, you’ll run into the fact that since you’re running a very mature but feature rich VM. The JVM is about the most efficient user of CPU there is, depending on the lib (it’s really a compiler, so expect C/C++ speeds from the raw code).

tony.kay15:10:13

memory is cheap in hosting envs, and any VM-based soln is going to like lots of it…but it isn’t your real cost. I’ve seen at least one RoR shop running Fulcro side-by-side where they are still running with a single JVM for the Fulcro stuff, and the RoR stuff is eating most of their profits in hosting.

tony.kay15:10:54

In business terms: The Fulcro-based product is profitable. The RoR one is not.

tony.kay15:10:14

so, yes, that matters, but you won’t figure it out from micro-benchmarks

tony.kay15:10:05

and there are boat loads of RoR projects that are quite profitable.

tony.kay15:10:14

so everything with a grain of salt 😉

tony.kay15:10:29

At the end of the day a well-designed system with any library: The systems runtime costs are usually dwarfed by HR costs.

tony.kay15:10:28

so, back to the thing I’m interested in: how hard is it to keep the project decomplected so you can reach business goals with minimal pain and manpower?

Arto Kalishian20:10:55

I guess I am using a framework in order not to need manpower at all 🙂

Arto Kalishian20:10:09

But the only thing where I will seriously need a dedicated person to handle, is the hardening of the VPS server in terms of security. Because a framework can't do that and because doing that myself while I am no expert, is a waste of my manpower/development time.

Arto Kalishian20:10:58

But of course I am talking only in relative to the size of my project. Maybe someone wants to do an ERP with Fulcro... they might need man power for each department then.