Fork me on GitHub
#honeysql
<
2024-01-07
>
seancorfield07:01:21

☝️:skin-tone-2: This still doesn't compose as nicely as the vector/keyword-based version since you need to know what symbols to replace and those might get conditionally added by functions that build parts of the DSL tree, but at least you control what symbols you'd want to replace (and their values). The formatf version being positional relies on knowing the complete list of parameters in a fixed order, which may not actually be possible for conditionally constructed subexpressions in the DSL.

slipset10:01:00

This probs doesn’t answer your question, but I figured you might be interested to see how we use honey at Ardoq:

(defn by-workspace-filter-rule [workspace-id]
  (pq/and (by-workspace-id workspace-id)
          [hpg/at> [:-> :filters "workspaceFilter" "rules"]
           (pq/lift [{:value workspace-id}])]))
Where pq/and is defined as:
(defn and [& clauses]
  (let [clauses* (filter seq clauses)]
    (if (seq clauses*)
      (into [:and] clauses*)
      none)))
Then our queries, when querying for stuff that’s “self contained” in a table, look like
(repo/query! some-entity-repo/config ctx (some-entity-repo/by-workspace-filter-rule workspace-id)
Which takes the clauses above and creates a proper select query and executes it. So for us, the ability to compose larger queries from smaller fragments is quite important.

seancorfield18:01:31

TBH, this is pretty straightforward stuff compared to a lot of the conditional query building we have to do at work where we have cond-> and a big pipeline of conditions and DSL-builder function calls, that each may add select, join, where, having, etc clauses 🙂

seancorfield18:01:36

It's interesting the split between folks who "mostly" use the helpers vs those who "mostly" use the DSL and Clojure core functions to build stuff. All the examples seem to reinforce my thinking that folks just don't use the "quoted symbolic" DSL much -- they either use the helpers or they use the vector/keyword DSL (or a combination of both). I'd be interested in hearing from @potetm on this because when this topic last came up ~6 months ago, he mentioning quoting and unquoting and I'd like to see examples of that (it's the unquoting part that I'm struggling with when using the "quoted symbolic" DSL -- which I like a lot).

potetm03:01:56

Just saw this! Sorry for the delay. So most of our cases around this look something like:

{:where [['and
          '[= tx.type "SalesOrd"]
          '[in tx.status ["B" "D" "E"]]
          '[= tx.custbody_3pl_exported "T"]
          ['in 'so.location (map loc-id
                                 [:
                                  :])]
          ['in 'tx.custbody_sm_order_source
           (map source-id
                #{:
                  :})]]]}
It's usually in the where clause, it's usually a mix of quoting at the operator level vs symbol level, and often we need full-fledged evaluation, not just simple symbol resolution. We also occasionally pass clause fragments into the fn. From what I can tell formatf appears to be most appropriate for my use case. I would prefer a {:params [arg1 arg2]} addition to the option map over the var args option, both because varargs are often a pain and because I would rather have all format options available. (Alternatively {:params {:arg1 a, :arg2 b}}, but I would probably use the positional variant most frequently, and it seems fairly straightforward to support both.) My personal problem is the constant SHIFT-; motion when typing the query. My normal type speed is fast, but repetitively reaching for that sequence is not what I'm optimized for, and it interrupts my thoughts. Adding net more keystrokes at the end of the query is actually less frustrating because those motions are more regular.

seancorfield04:01:31

Okay, that wasn't the "unquoting" I was imagining, so thanks for clarifying and providing feedback. I'll respond in more detail tomorrow when I'm at a computer instead of on my phone!

seancorfield20:01:37

@potetm Gathering my thoughts: > My personal problem is the constant SHIFT-; motion when typing the query. Yes, good point, and that's why I'd like to make the quoted-symbol DSL easier to use. > I would prefer a {:params [arg1 arg2]} addition to the option map over the var args option Yes, I'm not very happy with the var args option either -- but formatf also requires you use ?1, ?2, etc and "fakes" the map of params with (zipmap (range) param-vals) so it's as if you had {:params {:1 arg1 :2 arg2}} and the point is to save you the keystrokes (and, of course, they have to be keywords right now which raises the typing issue again. Perhaps (sql/formatf 'dsl [arg1 arg2] {opts}?) might be good compromise here instead? No var args but you'd have to wrap a single arg as a sequence. At least that would allow for (sql/formatf 'dsl {opts}) since the second arg could be either sequential (arguments) or not (options hash map). I wonder what your thoughts on the new formatv function? That lets you specify local bindings that should be substituted as values into a quoted-symbol DSL. I guess it depends on how often you would "unquote" just a local symbol vs an entire expression...

potetm15:01:11

Yeah (sql/formatf 'dsl [arg1 arg2] {opts}?) seems like a good option here. I'm actually struggling to find any instances in the codebase where formatv would be useful. We use honeysql almost exclusively for building queries. If we have a simple sql statement, we usually just write the sql. This often means that query fragments get passed across scopes, so any scope-based solution is gonna be difficult to use. Actually, now that I say that out loud, having named parameters would be necessary in some contexts. (Or I would just fall back to manual quoting in those instances. This is actually the most likely outcome if I'm honest.)

seancorfield00:01:18

That's good feedback, thanks!

👍 1
slipset10:01:24

The thing that I and I believe my colleagues trip over the most is how many vectors you need to sprinkle around to make things work. And, we tend to use the helpers to write out the full queries when that’s needed, ie (-> (hh/select :foo) (hh/from :bar) (hh/where …)) and I kind’a like that. Only problem is that some times we need to drop down to the data dsl when things become a bit hairy, but that’s not a big deal.

seancorfield18:01:10

If there are situations where you need to "drop down to the data dsl" that you think could be simplified with extra helpers or smarter helpers, LMK.

slipset18:01:09

There is one super edge-case thingy where you have several CTE’s and at least one of them is recursive. I think that’s hard to express with the helpers.

slipset18:01:18

It might also have been me…

seancorfield18:01:20

I don't have much (any!) experience with CTEs so it's entirely possible that area has some rough edges. I'm still on Percona 5.7 at work and nearly all of my database experience has been MySQL (with a bunch of -- OMG! -- Oracle before that while at Macromedia, before we switched to MySQL).

slipset18:01:51

Oracle - the good old days 🙂 One thing I remember with fondness of oracle was that they had a dashboard where you could see all kinds of interesting stats. I miss that.

seancorfield18:01:36
replied to a thread:This probs doesn’t answer your question, but I figured you might be interested to see how we use honey at Ardoq: (defn by-workspace-filter-rule [workspace-id] (pq/and (by-workspace-id workspace-id) [hpg/at&gt; [:-&gt; :filters "workspaceFilter" "rules"] (pq/lift [{:value workspace-id}])])) Where `pq/and` is defined as: (defn and [&amp; clauses] (let [clauses* (filter seq clauses)] (if (seq clauses*) (into [:and] clauses*) none))) Then our queries, when querying for stuff that’s “self contained” in a table, look like (repo/query! some-entity-repo/config ctx (some-entity-repo/by-workspace-filter-rule workspace-id) Which takes the clauses above and creates a proper select query and executes it. So for us, the ability to compose larger queries from smaller fragments is quite important.

It's interesting the split between folks who "mostly" use the helpers vs those who "mostly" use the DSL and Clojure core functions to build stuff. All the examples seem to reinforce my thinking that folks just don't use the "quoted symbolic" DSL much -- they either use the helpers or they use the vector/keyword DSL (or a combination of both). I'd be interested in hearing from @potetm on this because when this topic last came up ~6 months ago, he mentioning quoting and unquoting and I'd like to see examples of that (it's the unquoting part that I'm struggling with when using the "quoted symbolic" DSL -- which I like a lot).