Fork me on GitHub
#announcements
<
2023-03-04
>
seancorfield00:03:12

HoneySQL -- Turn Clojure data structures into SQL -- com.github.seancorfield/honeysql {:mvn/version "2.4.1002"} -- https://github.com/seancorfield/honeysql -- this is the @p-himik release! Almost every issue here was raised by him and several were fixed by pull requests from him -- thank you! gratitude • Address https://github.com/seancorfield/honeysql/issues/474 by adding dot-selection special syntax. • Improve docstrings for PostgreSQL operators via PR https://github.com/seancorfield/honeysql/pull/473 https://github.com/holyjak. • Address https://github.com/seancorfield/honeysql/issues/471 by supporting interspersed SQL keywords in function calls. • Fix https://github.com/seancorfield/honeysql/issues/467 by allowing single keywords (symbols) as a short hand for a single-element sequence in more constructs via PR https://github.com/seancorfield/honeysql/pull/470 @p-himik. • Address https://github.com/seancorfield/honeysql/issues/466 by treating [:and] as TRUE and [:or] as FALSE. • Fix https://github.com/seancorfield/honeysql/issues/465 to allow multiple columns in :order-by special syntax via PR https://github.com/seancorfield/honeysql/pull/468 @p-himik. • Fix https://github.com/seancorfield/honeysql/issues/464 by adding an optional type argument to :array via PR https://github.com/seancorfield/honeysql/pull/469 @p-himik. • Address https://github.com/seancorfield/honeysql/issues/463 by explaining :quoted nil via PR https://github.com/seancorfield/honeysql/pull/475 https://github.com/nharsch. • Address https://github.com/seancorfield/honeysql/issues/462 by adding a note in the documentation for set operations, clarifying precedence issues. Follow-up in #C66EM8D5H

🎉 30
👏 6
serioga07:03:57

The parser combinator library parsesso has been released. https://github.com/strojure/parsesso

🎉 20
borkdude09:03:35

Congrats! I love parser combinators and having a maintained one in the CLJ ecosystem is great. I'm interested in using this in bb for scripts. I noticed you're dropping down to .first interop in clj/cljs. Does this really matter much for performance? I notice micro-benchmarks in the repo, but I'd be interested in macro-benchmarks, as in, parse a 1mb file 1000 times and then see if it matters. At least in bb it won't work today, but with a small tweak it will:

borkdude09:03:43

Are you open to a small PR for this?

borkdude10:03:36

https://github.com/strojure/parsesso/pull/2 (also added a CI config to run lein test + bb test:bb)

serioga12:03:45

> Does this really matter much for performance? Well, checking for if (x instanceof ISeq) on every token has some effect...

borkdude12:03:20

instanceof is the cheapest operation there is

serioga12:03:39

Don't we have some reader conditional for everything what is not :clj/:cljs ?

borkdude12:03:43

but of course it could matter in microbenchmarks

borkdude12:03:49

yes, check the PR :)

serioga12:03:20

> instanceof is the cheapest operation there is and totally useless in the context...

👍 2
serioga12:03:17

> yes, check the PR you used :bb, I've meant “everything else”

borkdude12:03:48

true, but in bb's case, :bb has to go before :clj, else it will pick the :clj branch, which most of the time is what you want

serioga12:03:33

Will not :default work for bb?

borkdude12:03:46

no, because it picks the :clj branch as soon as it encounters it

borkdude12:03:11

that's how reader conditionals work: the language picks the first one that is applicable

borkdude12:03:50

you're right that in a JVM it can matter significantly:

user=> (time (dotimes [i 100000000] (.seq [123])))
"Elapsed time: 9.2935 msecs"
nil
user=> (time (dotimes [i 100000000] (seq [123])))
"Elapsed time: 489.359167 msecs"
especially in a hot loop of course which parsers are prone to

borkdude12:03:45

but it seems the instance check itself isn't the performance problem:

user=> (time (dotimes [i 100000000] (instance? clojure.lang.ASeq [123])))
"Elapsed time: 6.648 msecs"

borkdude12:03:24

anyway, I understand your trade-offs better now

serioga13:03:01

> Does this really matter much for performance? https://github.com/strojure/parsesso/blob/default/test/perf/bench.clj#L127 this benchmark becomes 2x slower with first/next.

👍 2
serioga13:03:54

ISeq and ASeq differ significantly 🙂

(instance? clojure.lang.ASeq [123])
;; Execution time mean : 4.182330 ns

(instance? clojure.lang.ISeq [123])
;; Execution time mean : 36.959700 ns

serioga13:03:22

> instanceof is the cheapest operation there is it depends on tested type more classes inherited, more expensive test (when missed)

borkdude13:03:46

I haven't looked at the implementation but how important is it that the input is a seq? it could e.g. also be a vector so you don't have to go through the seq abstraction at all? Just thinking out loud. You're right about the inheritance chain, yeah

serioga13:03:57

Anyway a difference is not tiny so let's keep it.

👍 2
serioga13:03:11

> (also added a CI config to run lein test + bb test:bb) this made your PR not so small 🙂

borkdude13:03:38

would you rather not have tests run on CI and let regressions creep in? ;)

serioga13:03:25

I mean it would be better to have separate commits...

borkdude13:03:51

ok, sure. would you like to have the CI config at all, or should I just remove it?

serioga13:03:24

I hope you would not mind it I take your changes manually...

borkdude13:03:36

not at all

2
borkdude13:03:03

feel free to take whatever you want and leave out whatever you want, it's your project :)

serioga13:03:39

the downside is that you will not be listed in collaborators 🙂

borkdude13:03:30

you can do something about that. Add a blank line followed by:

Co-authored-by: Michiel Borkent <[email protected]>
in the commit message

borkdude13:03:02

I think the :cljs thing will work in bb as well

borkdude13:03:18

and is also faster, since interop is quite slow in bb

serioga13:03:22

should I add :cljs thing for :bb?

borkdude13:03:34

I think that would be better yes

serioga13:03:35

for me the rule looks like “use :bb before any :clj with cross-platform implementation”

borkdude13:03:07

and you can leave :bb out if :clj works already well for bb

borkdude13:03:36

for comparison:

(require '[clojure.string :as string])

(def c1 \f)
(def c2 \g)
(time (dotimes [i 1000000]
        (.equals ^Object (Character/toLowerCase ^char c1)
                 (Character/toLowerCase ^char c2))))

(time (dotimes [i 1000000]
        (= (string/lower-case c1)
           (string/lower-case c2))))
$ bb /tmp/perf.clj
"Elapsed time: 2007.004583 msecs"
"Elapsed time: 112.205875 msecs"
$ clj -M /tmp/perf.clj
"Elapsed time: 19.156291 msecs"
"Elapsed time: 49.091 msecs"

borkdude13:03:50

so not using interop is much cheaper in bb :)

2
serioga13:03:27

nbb uses same conditionals as bb?

serioga13:03:51

what is best for bb here?

(defn- c
  "Cross-platform char."
  [s]
  #?(:cljs s, :clj (first s)))

borkdude13:03:58

nbb uses :org.babashka/nbb and it has the same rule with respect to the :cljs branch

borkdude13:03:28

bb has JVM semantics but I noticed that str/lower-case also works on chars

borkdude13:03:50

user=> (clojure.string/lower-case \C)
"c"

serioga14:03:17

it would be cool if you check before release

borkdude14:03:52

ok, I'll check locally I think adding a test runner for bb would be good as a follow up? this is better than checking manually imo

borkdude14:03:30

$ bb test:bb

Running tests in #{"test"}

Testing strojure.parsesso.char-test

Testing strojure.parsesso.expr-test

Testing strojure.parsesso.parser-test

Ran 52 tests containing 320 assertions.
0 failures, 0 errors.
This is what I had in my PR

serioga14:03:53

> adding a test runner for bb would be good as a follow up? ah, what is it for 🙂

borkdude14:03:08

I think I didn't understand your last reply. Are you asking what a test runner is for?

borkdude14:03:01

Adding that would enable you to run the tests in test with babashka, as a verification that nothing is broken

borkdude14:03:40

And the .github/workflows that was also part of my previous PR then runs both lein test and bb test:bb in CI on every commit, so you automatically know that everything still works, without having to run things locally

borkdude14:03:53

It allows you to use your library as a deps.edn dependency (local or via git). This is both useful for clojure users as babashka users, as they both use the deps.edn dependency manager, which is the official clojure tooling

borkdude14:03:23

In the bb.edn test runner file I am using your dependency as a local deps.edn dependency

borkdude14:03:28

so I'm able to run tests on it

serioga14:03:31

so do I understand correct? 1. Build and run tests • ci.yml + bb.edn 2. Use library as deps.edn • deps.edn

borkdude14:03:46

yes, but 1 needs 2 in this case

2
serioga14:03:51

or all 3 files required?

borkdude14:03:39

yes. also 2 allows you to use your library as a git dependency so people can test it even before you release it to clojars

serioga14:03:58

how do you maintain this dependency changes?

:extra-deps {com.cognitect/test-runner
                        {:git/url ""
                         :sha "a522ab2851a2aa5bf9c22a942b45287a3a019310"}

borkdude14:03:29

the above means that I'm using https://github.com/cognitect-labs/test-runner as a library at a specific commit. https://github.com/cognitect-labs/test-runner/commit/a522ab2851a2aa5bf9c22a942b45287a3a019310 Does this answer your question? If not, please clarify

serioga14:03:31

It does not How do you monitor if version of test-runner should be changed

borkdude14:03:01

You have tools for this. How do you do this for project.clj? I will give you a link to a similar tool for deps.edn

borkdude14:03:25

In general, I don't expect this library to be necessary to update. It's very stable

borkdude14:03:35

This tool is a popular one for deps.edn: https://github.com/liquidz/antq

borkdude14:03:51

Also #C03KCV7TM6F can do it

borkdude14:03:03

In a similar way people can use your library with:

{:deps {com.github.strojure/parsesso {:git/sha "..."}}}
It's pretty cool :)

borkdude14:03:34

When you use com.github. or io.github. you don't have to specify the :git/url, then it can be inferred from the library name

borkdude14:03:10

You can also specify a :git/tag in which case you can use a short :git/sha (7 characters)

borkdude14:03:08

:sha and :git/sha are synonyms

serioga14:03:17

now bb tests failed on github...

borkdude14:03:35

ok, I'll take a look :)

borkdude14:03:58

hmm, locally it works:

$ bb test:bb

Running tests in #{"test"}

Testing strojure.parsesso.char-test

Testing strojure.parsesso.expr-test

Testing strojure.parsesso.parser-test

Ran 52 tests containing 320 assertions.
0 failures, 0 errors.

borkdude14:03:48

I will make a PR just to debug this

borkdude14:03:54

don't merge it

serioga14:03:00

I see the error

borkdude14:03:35

ah do you? ok

borkdude14:03:56

ah I see yes

borkdude14:03:16

clj-kondo should add better support for linting multiple languages, currently it only does :clj and :cljs

borkdude14:03:54

I got another error: Character can't be cast to CharSequence

borkdude14:03:03

I think in (re-find #".." character) probably

borkdude14:03:49

something like this

serioga15:03:11

green now 🥳

🎉 2
serioga15:03:27

Thank you too!

serioga15:03:20

Mentioned @U04V15CAJ in README 🙂

🙇 2
Jakub Holý (HolyJak)23:03:36

@U0HJNJWJH I’d appreciate a paragraph in the readme explaining what parser combinators are & are good for 😅 🙏

2
serioga10:03:06

@U04V15CAJ Maybe you have somewhere a configuration for cljs tests on node as well?

Jakub Holý (HolyJak)11:03:11

An eluded (we all know parser parse strings) simplified, more pragmatic explanation? Perhaps worth an example, and most importantly a list of cases you use it for? How does it differ eg from instaparse, which also parses text into data?

borkdude11:03:12

@U0522TWDA A parser combinator library is a library with functions that can be composed into a parser

borkdude11:03:40

instaparse takes a grammar specifiction, but in a parser combinator library you build the specification from functions, rather than a DSL

serioga11:03:22

@U0522TWDA kern has the best docs I saw in various Clojure implementations https://github.com/blancas/kern/wiki

Jakub Holý (HolyJak)11:03:11

Borkdudes 2 comments is exactly what I needed

serioga09:03:54

OK, added the answer in FAQ

borkdude10:03:16

looks great!

2
borkdude10:03:00

you can unduplicate this config by replacing this line: https://github.com/strojure/parsesso/blob/bd18de34e04aba22faec6dd0709c7f90c64e8c4f/.clj-kondo/config.edn#L9 with

:config-paths ["../resources/clj-kondo.exports/com.github.strojure/parsesso"]
on the top level

2
serioga11:03:54

> looks great! only looks or also works? I could not test in Cursive. The confusing part in documentation is if in path should be ns strojure.parsesso.parser or piece of project name com.github.strojure

borkdude12:03:58

project name

borkdude12:03:15

in cursive you can test via #C02UN1B0998

borkdude12:03:04

but you can also test it from the command line

borkdude12:03:09

In a project with your dependency

mkdir .clj-kondo
clj --lint $(clojure -Spath) --dependencies --copy-configs
clj-kondo --lint src
You can test your commit with:
{:deps {com.github.strojure/parsesso {:git/sha "..."}}

serioga12:03:56

not sure if clj-extras uses --dependencies --copy-configs flags...

serioga12:03:28

ah, additional command before use not very convinient 🙂

borkdude12:03:28

clojure-lsp does this automatically

Jakub Holý (HolyJak)23:03:17

@U0HJNJWJH When should I pick parser combinators over EBNF? Do they offer the same and it is only question of which one I prefer to learn or is there some distinct advantage over a DSL such as EBNF? Perhaps it is easier to describe more complex grammars b/c I can make my own helper functions, or something?

serioga05:03:56

Looks like here is good explanation https://softwareengineering.stackexchange.com/questions/338665/when-to-use-a-parser-combinator-when-to-use-a-parser-generator. I learned parser combinator for parsing mustache which grammar is context dependent.

👀 2
Jakub Holý (HolyJak)06:03:00

I gathered this as the answer: > in general, parser combinators such as parsesso are for creating top-down (i.e. LL) parsers, with the ability to reuse common code (this lib). Parser Generators typically generate a finite state automaton for a bottom-up (LR) parser. Though nowadays there are also combinators for LR grammars and generators for LL ones (e.g. ANTLR). “Which one you should use, depends on how hard your grammar is, and how fast the parser needs to be.” Especially if the grammar has lot of non-trivial ambiguities then it might be easier with the more flexible combinators approach.

2
serioga08:03:54

ring-control — More controllable composition of Ring middlewares. https://github.com/strojure/ring-control

🎉 18
borkdude21:03:36

https://github.com/babashka/json: a JSON abstraction library It lets you choose your preferred JSON implementation (currently data.json and cheshire, more to come) by providing them on the classpath, while not coupling your libraries or scripts to them.

🙌 16
🎉 23