Fork me on GitHub
#off-topic
<
2020-08-29
>
Vincent Cantin17:08:01

I came to the weird realization that GraphQL does not collect a graph of data. It is only collecting a tree. It is not a query language to gather graphs.

phronmophobic17:08:38

I thought trees were technically graphs. even though it doesn't collect a graph, doesn't it search a graph?

p-himik17:08:32

By itself, it doesn't search anything - it's just a query language. How the actual implementation is done is, well, an implementation detail.

Vincent Cantin17:08:50

GraphQL was made for decorating html trees, that’s why it returns a tree.

p-himik17:08:58

Do you have a source that confirms the first part of that sentence? Because that by itself sounds unlikely.

Vincent Cantin17:08:35

well .. graphs can be implemented as trees, but I recently met a situation where I really need to represent a graph of values, and GraphQL and EQL can’t do that.

p-himik17:08:22

Hold on. You need to represent a graph. But you're looking for a query language. I don't see how the two are connected.

phronmophobic17:08:24

you can represent a graph of values, but some of the nodes would need to represent references to other parts of the tree

Vincent Cantin17:08:58

I have no source, just my impression that GraphQL was made for improving data loading at the time where everybody was doing React.

Vincent Cantin17:08:10

@U2FRKM4TW I encountered a situation where I need to express a query in the shape of a graph instead of a tree.

p-himik17:08:10

This is a graph with a single cycle represented as an EDN structure with an ID convention on top of it:

[{:id 1, :sister-id 2}
 {:id 2, :brother-id 1}]
Any query language that I have heard of can return similar data. Or query based on similar data. @U8MJBRSR5 Can you give an actual example?

3
Vincent Cantin17:08:58

Yes. Imagine that we are developing a re-frame app. There is a directed graph of values being computed in the subscriptions. Now, imagine that we want to tell the server about this graph, so that it plays a role to update its values.

p-himik17:08:06

> There is a directed graph of value being computed in the subscriptions I do not understand this sentence. Can you provide some [pseudo]code?

Vincent Cantin17:08:56

I don’t have code for my example.

Vincent Cantin17:08:15

(let [c (+ a b)
      d (* a b)
      e (- a d c)])

p-himik17:08:20

Consider this:

(reg-sub :x
  ...)

(reg-sub :a
  :<- [:x]
  ...)

(reg-sub :b
  :<- [:x]
  ...)

(reg-sub :c
  :<- [:a]
  :<- [:b]
  ...)
Is this what you mean by "a directed graph of value being computed"?

Vincent Cantin17:08:34

Let’s imagine that this is the graph I am talking about.

Vincent Cantin17:08:49

Yes, that’s what I mean.

p-himik17:08:51

OK. So what is the problem?

phronmophobic17:08:02

any graph can be represented by a list of nodes and a list of edges. a list of nodes and edges can be represented by a tree:

{:nodes #{:a :b :c}
 :edges #{ [:a :b] [:b :a] [:a :a] ....} }

Vincent Cantin17:08:09

The problem is: if I want to describe the usage of the subscriptions by the views, I cannot do it with a tree. I cannot create a EQL structure and tell the server “this is what I am currently using”, because some nodes will be duplicated while in reality I reuse those nodes (that’s the difference between a tree and a directed graph: some leaves can be joined into the same nodes).

Vincent Cantin17:08:57

So, as @U7RJTCH6J said, I will have to transform the graph and use a custom format. Not EQL or GraphQL.

p-himik17:08:16

And when it's a bit unwieldy, you can just encode edges within the nodes themselves:

[{:id :x}
 {:id :a, :dependencies [:x]}
 {:id :b, :dependencies [:x]}
 {:id :c, :dependencies [:a, :b]}]

Vincent Cantin17:08:34

I am not complaining, I am just sharing the anecdote .. because the name is “GraphQL”

phronmophobic17:08:24

my point is that you could construct graphql query that represents a graph

p-himik17:08:36

@U8MJBRSR5 Hold on, again. :) You want to "tell a server" about something. Why do you want to use a query language for that? But even with a query language, it's still possible, although it wouldn't make much sense IMO. Every map in my vector above can be represented with EQL.

3
p-himik18:08:58

FWIW, if I had to deal with the scenario above, I would just create a POST request (or a WebSocket message) and feed it with the relevant EDN data, that's it. Regular HTTP API. Not REST, not GraphQL, not EQL, not whatever else - just plain old HTTP API. If I really had to use e.g. EQL for that, I would use mutations: https://github.com/edn-query-language/eql#28-mutations And it would work in exactly the same way as with the regular HTTP API, only the actual data would be wrapped in some fancy EQL just to make the query parser happy. Same works for GraphQL.

Vincent Cantin18:08:24

I am brainstorming on the design of a very different web framework, and exploring ideas. Some of what I say may not make sense - that’s normal.

👍 3
lilactown18:08:12

GraphQL represents a graph query that returns a tree

👍 3
☝️ 3
lilactown18:08:11

so in that regard, the underlying data source can form a graph, and graphql can easily query that graph. but the return value is a static tree, because that’s what we can serialize and send across a wire

lilactown18:08:56

ex.:

query {
  user(id: 1) {
    name
    friends(id: 2) {
      name
      friends(id: 1) {
        name
      }
    }
  }
}
this data we’re querying is recursive (friends of user 1 are also friends of user 2), so we could keep going and infinitely nest this as many layers deep as we want. however, we almost always want to bottom out somewhere. computers are not great at representing infinite graphs, and we can’t serialize recursive data structures easily. so the query language describes the tree we want to get from the graph we’re querying

phronmophobic18:08:20

there's a recipe for serializing recursive data structures, this isn't graphql, but the same idea applies: https://netflix.github.io/falcor/documentation/jsongraph.html#reference

phronmophobic18:08:28

> Using References, it is possible to model a graph in JSON.

lilactown18:08:38

yep, falcor does have a way of serializing it but then you need an interpreter on the frontend to realize the graph. that’s a tradeoff.

lilactown18:08:24

using an actual graph data structure on the frontend also has some tradeoffs, like the inability to console.log it effectively

lilactown18:08:25

IMO GraphQL made the right tradeoff: return a plain JSON tree so that any frontend can construct a query & get the response with plain fetch

phronmophobic18:08:39

sounds good. my point wasn't that it's the best solution for every problem, it's that we already have a recipe for representing graphs that some users have had success with

3
Aron19:08:32

What I don't understand is, why would I use graphql when there is datalog?

Aron19:08:09

or falcor. In 2015 I evaluated both and we decided to use graphql, we have gone trough all what you have discussed above and in the end I still had to do lots of pre and postprocessing on the js side to deal with the results I got from our implementation of GraphQL. Not to mention all the non-standard extensions to it.

Aron19:08:41

Falcor appeared to me back then as something that was started and then abandoned, or at least the public stuff was not very ready for usage at all.

lilactown19:08:43

the problem with datalog is that it’s not amenable to front end caching

lilactown19:08:36

graphql (and to a certain extant, eql) can be statically analyzed to determine what entities your query returns, and use that to track whether to use the cache & subscribe to changes to those entities

lilactown19:08:58

you basically use typename + ID to figure it out, e.g.:

query {
  user(id: 1) {
    ...
  }
}
since GraphQL is typed, you can know that the user query returns a User type, and then any other query that returns information of User type with id equal to 1 will update the same entity in the cache.

lilactown19:08:54

if you have a front end framework that uses a cache like this, you can skip the request entirely if you already have data

Aron19:08:11

is helix considered a frontend framework? 🙂

Aron19:08:16

I know react is not one.

lilactown19:08:37

helix is just a wrapper for React

lilactown19:08:55

I’m referencing something like apollo-client or facebook’s relay

lilactown19:08:29

using a cache like this also allows you to subscribe to queries. so you can use the query above to subscribe to any changes of User w/ id: 1

Aron19:08:02

Right, so my problem is that all this is conditional that 1. I can implement what is necessary 2. it's easier to implement this than add caching to datalog

Aron19:08:02

regarding 1. I can, but it was hard last time and no one said anything since (in fact, since then I met ten times as many companies who wanted to ditch GraphQL than those who wanted to adopt it) 2. I don't think it's that hard to add caching to datalog, you just need collaborating environments on both end, with deterministic hashing for values. Then you can hash your queries and responses to the queries and only send the diff.

lilactown19:08:18

all of this is pretty much impossible to do with general datalog because datalog has certain dynamic properties which prevent you from statically analyzing it. e.g.:

'[:find ?name
  :where
  [?e :config ?name-attr]
  [?e ?name-attr ?name]]

Aron19:08:40

but I do not need to statically analyze a query to cache the results to it

lilactown19:08:08

you do if you want to relate the query to entities within the cache, to know whether you should fetch or use the cache

Aron19:08:25

I am not sure what you mean

Aron19:08:59

I don't 'fetch or use the cache' I always fetch. The cache only sends what's necessary.

lilactown19:08:02

you can do really simple caching that says “the exact same query will yield the same data”, but you can’t do something like: Query 1: I want the user 1s name, age, and phone number … around the same time, another query … Query 2: I want the user 1s name & phone number w/ GraphQL, you can easily return the cached value for the second. w/ datalog, it can be difficult to know that these two queries are about the same thing

Aron19:08:52

of course you can do that, why wouldn't you be able to do that?

Aron19:08:15

I assume the queries come from components of the application?

lilactown19:08:17

because of the reason I said above: datalog is too dynamic to do this for any query

p-himik19:08:30

With GraphQL and the scenario that you describe, how do you guard against situations where between Query 1 and Query 2 the value of e.g. the user 1's phone number was changed?

Aron19:08:59

there is probably a timeout or something

lilactown19:08:14

@U2FRKM4TW you can control this. if you want to fetch fresh data you can always request it. but if you’d rather use stale data, you can do that too

Aron19:08:48

I do not do it this way. I do not keep a dedicated cache on the client side separate from client data. The cache is online and sends only updated values. I always send full queries and last timestamps so that the server can know what has changed since.

lilactown19:08:05

more importantly is that you can know that new data is being loaded. so you can run query 1 first, and then a subscriber to query 1 can get notified when query 2 is fetching fresh data

lilactown19:08:30

or it can ignore that if it chooses, but it will get updated w/ fresh results once query 2 resolves

Aron19:08:35

This is exactly the kind of programming I want to avoid. Having to think about which query is being loaded when.

lilactown19:08:35

it’s all about controlling the UX based on what data has already been loaded. e.g. you could show part of the page while some of the other data is loading, if you’ve already populated the cache with some of it

Aron19:08:14

but I do not want to control the ux based on the data, the data already controls the UX, no special wiring, no interfacing, no additional metada over this is required?

lilactown19:08:31

anyway, my point is that datalog can run e.g. arbitrary functions in the query. it can dynamically choose keys to use based on other parts of the query. it’s too dynamic

Aron19:08:59

and my point? 🙂

lilactown19:08:05

so you can’t use as simple of a solution as reading the query and knowing what entities it relates to

Aron19:08:13

because yours was definitely expressed clearly several times already

lilactown19:08:50

you’re basically saying, “the complexity you’re talking about doesn’t matter because it doesn’t match the way that I want to make my apps”

lilactown19:08:17

which is fine to say for yourself, but I’m talking about a general application solution a la fulcro, relay, & apollo

lilactown19:08:26

I have needs for my applications that using datalog does not meet

lilactown19:08:40

I’m trying to explain the constraints which prevent me from using it

lilactown19:08:53

and you’re saying they don’t matter. so excuse me for not directly addressing your point

Aron19:08:16

let me give you a tip. every time you consider saying or writing "you are basically saying" to stop and think if the person you are talking to does really say that

Aron19:08:50

because that's absolutely not what I was saying, not even close

p-himik19:08:59

What were you saying? Because I've been following the discussion and so far what lilactown has said is completely in line with my own understanding of what's going on.

Aron19:08:05

What he said is not out of line with my own understanding either.

lilactown19:08:46

then I don’t understand what you’re saying. > I do not do it this way. > This is exactly the kind of programming I want to avoid. > but I do not want to control the ux based on the data there is either a misunderstanding on my end or yours. I’m trying to explain use cases which I have encountered in my day-to-day, and your rhetorical method has left me with the feeling that you don’t understand or don’t care about them enough to the point where you think that it’s not worth talking about. if datalog matches your use cases well, then you should continue to use it. I will continue to not use it, because it does not match mine.

Aron19:08:01

you are quoting wrongly

Aron19:08:54

the last line requires the part where I say 'it already does that', that's why I don't 'want' it, since wanting something that is given already by default is impossible.

Aron19:08:28

I do not want to cast doubt on your use cases. Nor do I want to change how you work. If anything, my part in this discussion was first to ask why GraphQL fits your needs, to this you said things that I understood implicitly in a way that I questioned them. So it's one thing to state 'my day-to-day' and another to say that 'datalog is too dynamic'. I cannot question your day-to-day, but I can question that it's not possible to write the same thing you do - but differently - with datalog. I understand completely why doing the same thing the same way wouldn't work with datalog, you really overexplained that. Maybe I missed it in the long discussion, but have you touched on why you need to have a separated cache on the client side? Isn't it the case perhaps - and pardon me for being egotistical enough to guess - here: that you have been given the tools first then you had to work out the solution? If you already use all the things required for GraphQL like Relay or apollo (or similar), if you already have a UX toolkit that falsely assumes that everything requires a cache and a fetch and some loader, then all this can just happen without notice.

lilactown19:08:40

there’s too much nuance to what we’re talking about to boil it down to what you’re saying. different applications (or parts of an application) may need different behavior. e.g. if you click a back button, I may want to show cached data. if you navigate to a page by clicking a link, I may want to show some of the data that’s cached but also fetch new stuff. OR I may want to show a loading spinner until all data has completed. this requires knowledge at the call site of a query of what specific behavior you want and the action that brought it there. it’s complicated. to support all of these different behaviors, I need a way of knowing what entities a query refers to before I fetch the query from the server.

lilactown19:08:14

yes in order to maintain consistency across the UI, I need a front end cache. even just the use case of: Component A renders & fetches some data Component B renders & fetches the same data Component B re-fetches and renders with new data unless I’m okay with Component A and Component B being out of sync, you need a single place to put data that Component A and B can both refer to

Aron19:08:55

they all read the same state, they can't be out of phase unless I am doing some react experiments

lilactown19:08:30

so that state then is your cache. it doesn’t have to be outside of the React tree

Aron19:08:39

It's probably worth noting that I completely agree that this is nuanced and complicated.

Aron19:08:35

But my attitude after this realization is to 1. avoid general solutions 2. avoid solutions that require those who work on the visuals and interactions to think about network requests and data fetching

Aron20:08:02

and no, the react state is not the cache, that would be the register

Aron20:08:29

my cache is the same cache that was always the cache, sometime the website can't handle the requests and stores the calculated responses. this is what we called the cache, not the one from the cpu analogy. This cache is definitely nothing like react state.

p-himik20:08:56

"Cache" is a concept, not a specific place. Arguing semantics is not very productive.

Aron20:08:30

anyway, thanks for the answers. I now understand that you have decided first that you need a client side cache and then used a solution for this problem. I have no arguments in this area - obviously.

Aron20:08:43

Yes, I do realize that arguing semantics is not productive, however we have already been in an unproductive discussion and realizing that we meant slightly different things by the same word, in this case making what we meant by it clear helped to make me understand the other side much better.

👍 3
zackteo22:08:30

Hey guys! Not sure if this is the right channel for this but wanted to get some thoughts. So I happened to see this Clojure question on Stackexchange via Zulip. And thought hey maybe I could give a shot at answering. Okay, wanted to actually comment at first, given that the qn wasn't too clear ... but couldn't cause I haven't accumulated enough rep >< So granted, I ended up giving a short half answer/half question in my original response which okay fine I got 1 downvote for. But because of the comment not to ask questions, I rephrased my answer and gave quite a bit of due thought to trying to tackle it? But, I ended up finding that 2 people deleted my answer? 😕 Maybe it is because an answer can't be provided before clarification? I don't know? Just me trying to rationalise if I am justified to feel a bit angry or annoyed. Any thoughts? 😮 (Also should I undelete my answer? Also moved this from #clojure ) https://stackoverflow.com/questions/63642728/how-to-convert-from-a-string-into-an-argumentfunction-using-clojure?answertab=active#tab-top

❤️ 3
Giovani Altelino02:08:03

I mean, you gave some tips that could have helped him to debug, and also you "... is a new contributor. Be nice ... ". So yeah, feel free to be annoyed. 😆

zackteo05:08:26

Thanks 😄 hahaha. At this point after getting advice from @seancorfield, I'm good! Tho I am curious if anyone has had similar experiences with StackOverflow. Hopeful not a wrongful perception, but one of my WTH moments was when I realised that those that deleted my post don't seem to even do Clojure Development (I know StackOverflow is for tech in general but still thought What The Heck)

seancorfield05:08:51

That's very common. My friends in the ColdFusion (CFML) community are always having their posts deleted by SO moderators who know nothing about CFML...

seancorfield05:08:34

This is part of why we have http://ask.clojure.org now. To avoid the weird partisanship of SO.

zackteo05:08:18

Mmmm, I figured! Quite sad tho that it then seems to others that the Clojure community might be "dead". Especially cause of our removal from the StackOverflow survey this year (speaking of which did anyone actually find out why? 😮 )

seancorfield05:08:23

The SO survey is not a useful measure of anything, given the politics of SO...

👍 3
seancorfield05:08:39

Tiobe is also not a useful measure, because it arbitrarily decides what is a language and what isn't. The same goes for all these "language popularity" sites.

👍 3
zackteo05:08:39

Yeap, I saw a discussion on that on reddit. I do agree it might not be useful to those who understand. But to those who aren't in the loop, people who would have potentially gotten into Clojure that doesn't help that part i guess? That's how I feel at least

seancorfield05:08:05

But you can't go out and convince all those people, you just can't reach them. This is just one of those annoying things about misinformation. There's nothing you can do.

zackteo05:08:54

Mmmm, yeap that's the unfortunate reality :x

seancorfield05:08:52

Gotta be "zen" about it... otherwise it will drive you crazy 🙂

Jcaw13:08:29

I bet you they removed clojure because it was consistently topping the "most lucrative languages" measure

😁 3
zackteo05:08:26
replied to a thread:Hey guys! Not sure if this is the right channel for this but wanted to get some thoughts. So I happened to see this Clojure question on Stackexchange via Zulip. And thought hey maybe I could give a shot at answering. Okay, wanted to actually comment at first, given that the qn wasn't too clear ... but couldn't cause I haven't accumulated enough rep >< So granted, I ended up giving a short half answer/half question in my original response which okay fine I got 1 downvote for. But because of the comment not to ask questions, I rephrased my answer and gave quite a bit of due thought to trying to tackle it? But, I ended up finding that 2 people deleted my answer? :confused: Maybe it is because an answer can't be provided before clarification? I don't know? Just me trying to rationalise if I am justified to feel a bit angry or annoyed. Any thoughts? :open_mouth: (Also should I undelete my answer? Also moved this from #clojure ) https://stackoverflow.com/questions/63642728/how-to-convert-from-a-string-into-an-argumentfunction-using-clojure?answertab=active#tab-top

Thanks 😄 hahaha. At this point after getting advice from @seancorfield, I'm good! Tho I am curious if anyone has had similar experiences with StackOverflow. Hopeful not a wrongful perception, but one of my WTH moments was when I realised that those that deleted my post don't seem to even do Clojure Development (I know StackOverflow is for tech in general but still thought What The Heck)