The primary issue with pagination in RAD has more to do with the existence and integration with indexes on the server (if that’s what you’re referring to). Otherwise pagination is already a simple matter on the client. Server-side pagination state machines for reports already exist in the repo, and I have various other ones internally on my own projects. Again, it’s the server side that is the issue, and that is largely “can I resume where I left off without killing the server in a multi-user scenario?”

Do you mean the integration of Pathom's indexes?

No, in order to do server side pagination you have to make it possible to say, in a database-specific manner: “OFFSET 100 LIMIT 10” Fulcro and Pathom cannot help you here

it is NOT database agnostic unless you pull all results to the client, pull all results to the server and cache them (for god knows how long?) or have a sorted index on that paginated sort order that the server can efficiently use to offset/limit

in SQL this is making sure you’re using an index with OFFSET/LIMIT, and in Datomic this is index-pull

totally depends on db tech, and may require you to build an index system to support your needs if you’re using something that doesn’t have it

That is why Fulcro RAD has no default solution for this. There isn’t a db agnostic one.

Would the support not go into the DB-specific adapter in any case?

It technically could. Feel free to write that 😄

Consider: You want 5 cols in a report. Which ones are sortable? For every sortable column, is there a secondary sort order? For EACH of those combinations, you have to create an index, and you have to communicate between the client and server as to which of them you decided to support. Do you auto-generate a cross product of every possible column combination as indexes in advance?

So at a minimum you need a way to declare that you want server-side pagination support with enough details that the adapter can “do the right thing” and not something totally insane in your infra

But, it is pretty trivial to manually create what you need as you create the report itself. So, that is what I do.

Yes, I see how the complexity can get out of hand and making some user-friendly thing in RAD to capture it all is difficult, especially across different DBs. > But, it is pretty trivial to manually create what you need as you create the report itself. So, that is what I do. You just manually create your indexes and use them for your OFFSET sorting?

Yes. I decide which columns are sortable, then create the proper indexes and resolvers to support that

Tjhe state machine is reusable, so it’s largely a logical task with some infra/deployment details

auto-generating a resolver would be nice, and I’ve started to take a stab at that in rad datomic adapter, but it isn’t well-tested/refined yet (see index-scan)

Do you even need a state machine for it? Couldn't the client just include some offset parameters in the request?

It’s just a modified state machine for the report that does exactly that

Okay, sounds straight forward enough. I started a second RAD project, just using SQL for the DB. Decided to skip using a DB adapter for it for now and it's working out okay. I'll take a look at your SM code in the repo to get an idea of how you put it together. Thanks very much @U0CKQ19AQ.

shoot…it looks like I have not published the paginated report state machine.

prob because it isn’t quite library quality yet

https://gist.github.com/awkay/023c974212ec3c928695b6f299565b00

The idea is that the return of the result will be {:results […] :next-offset n} where n == -1 means there are no more

If you can quickly get the total number of results on the server, then expanding that a bit to include :total m is useful so you can show a pagination control that allows you to jump to a particular page instead of just being able to go “next”

what you end up wanting on top of all of this is filtering, which means it gets more expensive. In Datomic index-pull works fine, but if you want to filter on top of that it is a post-step and it makes finding the total potentially expensive. The next-offset is also about filtering. If you asked for next page of size 10, and you get that, the filtering might have caused the index scan to walk 150 things, so your next offset actually grows by 150 instead of just 10. So, in that state machine the assumption is we know we have a lot of results, and we can get a page of them with relative ease, but we need the client to track where we last left off scanning for efficiency (beyond just a page number…we need the real index offset)

in SQL this is kind of hidden for you, but unless you’re really careful you can really kill your database with a combo of pagination and filtering done on the server.

Thanks for the added detail. I'm not quite ready to start using it quite yet but I wanted to familiarize myself with it a little before beginning. Will review the gist alongside your comments.