If I remember correctly, there was some talk about backup/restore functionality for datomic cloud being in the works, but can’t find any news about it. Is that still being worked on?

am I misunderstanding the :limit option to index-pull?

(count
 (d/index-pull db {:index :avet
                   :selector [:db/id]
                   :start [:story/group group-id]
                   :limit 5})))
10

is this feature implemented only for Datomic Cloud? It’s described in the on-prem index-pull documentation.

You can Call (take 5 (d/index-pull ...

Yes, I know, but I was planning to use :limit in conjunction with :offset to do pagination without realizing the full collection of results. (`:offset` does not appear to have any effect either for me.)

@enn Do you have a link to the docs you're reading?

@joe.lane https://docs.datomic.com/on-prem/query/index-pull.html

Thanks

And you're using on-prem, correct?

Yes

peer api or client api?

This is on a peer

Hi @ennhttps://docs.datomic.com/on-prem/clojure/index.html#datomic.api/index-pull does not include :limit. This is implemented in the client-api which is accessible in the latest client-pro release https://docs.datomic.com/on-prem/overview/project-setup.html#client-setup

The reason for this is at the top level of client in https://docs.datomic.com/client-api/datomic.client.api.html#top:

Functions that support offset and limit take the following
additional optional keys:

  :offset    Number of results to omit from the beginning
             of the returned data.
  :limit     Maximum total number of results to return.
             Specify -1 for no limit. Defaults to -1 for q
             and to 1000 for all other APIs.

I can see how this is confusing in our docs, given the example shows the usage of :limit without the added context above. I will update the docs to reflect that.

I need to also discuss with the team if peer api will ever support index-pull with limit, but as Joe said, you can still take 5 etc.

Is the pull realized by advancing the outer seq, or only by reading each entry? E.g. if we go (drop 100 result-of-index-pull), does that do the work of 100 pulls or 0?

(I’m trying to discern if drop is an exact workalike to :offset or potentially much more expensive in the peer api)

My understanding is it does the work of 100 pulls. But I need to validate that understanding and am running that by Stu.

@enn Ideally when implementing a pagination api though, you wouldn't use offset like that. Rather, you would grab the last element of the prior page and use that in the value position of :start, or in your case, :group-id.

@favila ^^

“Cursor based pagination” is the concept Joe Lane is referring to :)!

Sure, ideally. But obviously it’s important enough that the client api has :limit and :offset 🙂

@jaret thanks for the clarification, I appreciate it. If you hear anything back on whether this will be supported in the future I’d love to hear.