Fork me on GitHub

I’ve released version 0.3.3 of, a framework for structurally scraping whole sites in Clojure. Get it from Clojars: {skyscraper/skyscraper {:mvn/version "0.3.3"}} Changes in this release: • Feature: To facilitate debugging, processors can now set the :skyscraper/description key on contexts. These descriptions will be logged when downloading, instead of the URL, and won’t be propagated to child contexts. • Fix: Skyscraper now properly closes the cache when using scrape! and one of the processors throws an exception. • Fix: Skyscraper no longer complains when the server returns a quoted charset in the Content-type header. • Fix: :skyscraper.traverse/priority is no longer propagated to child contexts. • Infra: Skyscraper’s dependencies are now managed with cli-tools, with Kaocha being used for testing.

🎉 6

Turned out I rushed with the announcement. Due to a misconfiguration (a failed attempt at migrating away from Leiningen to deps.edn), the pom.xml from 0.3.3 declared no dependencies. I've now pushed 0.3.4 which has a correct dependency listing and is otherwise the same as 0.3.3. Apologies and happy scraping!


com.seancorfield/honeysql {:mvn/version "2.3.928"} -- turn Clojure data structures into SQL: • Address by clarifying that INTERVAL as special syntax may be MySQL-specific and PostgreSQL uses difference syntax (because INTERVAL is a data type there). • Address by supporting DEFAULT values and DEFAULT rows in VALUES. • Address by auto-quoting unusual entity names when :quoted (and :dialect) are not specified, making HoneySQL more secure by default. • Fix by adding :replace-into for :mysql dialect. • Address by adding honey.sql.protocols and InlineValue with a sqlize function. • Address by flagging a lack of WHERE clause for DELETE, DELETE FROM, and UPDATE when :checking :basic (or :checking :strict). • Fix by adding support for WITH / (`NOT`) MATERIALIZED -- via PR Follow-up in #honeysql

❤️ 10
👍 4
🎉 7

And I have another announcement to make! Announcing, a small experimental library that helps write more comprehensive tests. From the README: > In the life of a developer who writes tests for their software, there are always two (or more) possibilities. In an A/B test, either your user is assigned to group A, or to group B. Either that API call succeeds, or it fails. The list goes on. > Oftentimes, we are tempted to only test the happy path. But it would be good to test all the combinations – a Cartesian product of the “dimensions”, if you will. Or at least a reasonably-sized subset of them, if the possibility space is large. > This library helps you to do just that. I’d love to hear any feedback on this one. Do people think it’s a good idea? The library is so experimental that I’m not yet building any artifacts nor publishing them on Clojars, but if there’s interest I’ll start doing so.

🎉 1

This is interesting. I think one of my biggest problems when trying to test across combinations is how to specify what the expected outcome should be, especially if there's a failure mode that throws an exception. How are you typically mapping from "possibilities" to outcomes in your combination tests?


Great question! It will broadly depend on the particular combination of possibilities (the v in the example), but how exactly depends on the test. For example, in, I’m checking for the :succeeding dimension to determine expected behaviour. Admittedly a more advanced example would be in order, though.


This seems similar in spirit to property-based testing (eg test.check). I'm interested in your thoughts on the overlaps and differences with this approach?


I was thinking this too! Think this is a “sub-concept” of property-based testing: more coarse-grained and arguably easier to grasp. I found writing a test.check generator can be tricky. But I guess cartestian could be theoretically rewritten in terms of test.check. 🙂


A new version of #babashka! Lots of updates, bug fixes and error location improvements. 0.9.162 (2022-09-04) • Fix postgres feature • add .ssl.SSLException and classes ( • Fix satisfies? with marker protocol (no methods) • Update rewrite-clj • Update deps.clj • Update babashka.cli • Update org.clj-commons/clj-yamlbabashka.fs: fix expand-home on Windows • babashka.fs: expose :win-exts • nREPL: preserve stacktrace on exception • Fix improve error location • Fix add-watch segfault • Fix resolve record name ending with dot. (Also, there's a new project brimming in #babashka-bbin which will be announced later, after more people test it.)

🎉 17
babashka 15