xtdb 2023-07-18 | Slack Archive

Soumil Shekdar06:07:53

Query Construction I am interfacing with the XTDB Http Client using GO, and constructing queries based on the general structure below:

func listDocs(args *ListDocArgs) string {
  whereClauses := []string{}
  if args.Argument1 != "" {
    whereClauses = append(whereClauses, "[where filter]")
  ... for all arguments
  query := "{query" + where clauses + "limit }"
  return query
}

So was wondering what the best practices are in terms of query construction, and if code injection could cause harm here (and how we might prevent that)

refset11:07:39

Hey @U05EY098T09 I am not very familiar with Go so can't really suggest what best practices may exist, however I suspect this level of string concatenation could be problematic from an injection perspective. Have you evaluated https://github.com/go-edn/edn or anything similar?

Soumil Shekdar06:07:03

Pagination I want to implement a query with pagination. The query I have currently is:

# with no page token
{:query {
  :find [(pull ?e [*]) ?id]
  :where [[?e :xt/id id]]
  :order-by [[?id :asc]]
  :limit 100
}}

# with previous_id as the page token
{:query {
  :find [(pull ?e [*]) ?id]
  :where [[?e :xt/id id]
    [(> id "previous_id")]]
  :order-by [[?id :asc]]
  :limit 100
}}

Now while consuming results we are using the results of pull ?e [*], but these could not be used for ordering since they were not in find , so had a couple of questions: 1. Can we access fields on pull ?e[*] for ordering since it already contains the id (which is being used to order)? 2. What is the best practice for pagination?

hifumi12307:07:27

the official documentation has some clues for pagination https://docs.xtdb.com/language-reference/datalog-queries/#ordering-and-pagination

hifumi12307:07:38

> Basic :offset and :limit options are supported however typical pagination use-cases will need a more comprehensive approach because :offset will naively scroll through the initial result set each time.

✅ 2

hifumi12307:07:12

So if you have a large record set, you may want to implement pagination using a :where clause with some predicate like >

➕ 2

refset11:07:15

This old discussion is probably also of interest (regarding the limitations of ordering in general) https://github.com/xtdb/xtdb/discussions/1514

2023-07-18

Channels