Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-07-02

Channels

#sql
<
2022-07-02
>
DenisMc19:07:31

Hi, I’m using next.jdbc and honey.sql to good effect in my app, everything going well. For much of my code, I use a single db execute! function that takes in vector of honeysql maps and returns the results. These statements are all run in a single jdbc transaction, allowing me to easily mock out the db for various tests and return arbitrary data from the ‘database’ in order to examine how the system works in various scenarios. I have built a mocking function that allows me to match a honeysql map, return whatever I want when a matched map is encountered, and examine the contents of that honeysql map to confirm that it is being populated as expected - happy days. I am testing the db directly as well of course in other tests but this model does away with the db population set up work, which can get tricky as the db gets more complex. So far, my db transactional requirements have been straightforward, and having a single transaction in this function has been sufficient for my needs. However, I am now encountering more scenarios where I need to handle more complex database-related logic, and I need to wrap more code with jdbc/with-transaction. My db mocking feature breaks down when I need to do this, - because jdbc/with-transaction is a macro and I can’t figure out a way to ‘mock’ this call (or do something that has the same effect) so I can concentrate on the correctness of my honeysql maps in my unit tests and not worry about a db. Any help on this would be appreciated, and happy to provide further information if helpful.

DenisMc21:07:37

I’ve solved this. I can create an in-memory datasource (using derby for example) using code from here https://github.com/seancorfield/next-jdbc/blob/03092d023f334b56b9c41bdc13be7dd343480319/test/next/jdbc/test_fixtures.clj#L121. I pass this datasource to the with-transaction macro - which keeps the macro happy - while I can keep mocking away with abandon elsewhere.

emccue01:07:32

you could make a version of "with-transaction" that takes a function as an argument in lieu of a macro

emccue01:07:22

(defprotocol Transactable
  (with-transaction [datasource callback]))
and then make the implementation for a regular datasource be 
(extend-type Datasource
  Transactable
  (with-transaction [datasource callback]
    (jdbc/with-transaction [transaction datasource]
      (callback transcation))))
so now you need to be a bit more verbose normally 
(transactable/with-transaction 
  db
  (fn [tx]
    (jdbc/execute! tx ...)))
but you have a protocol behind which you can mock how a transaction is made

emccue01:07:56

actually that is kinda so obvious that...

emccue01:07:17

you can just provide a thing which implements this protocol and go along your merry way

seancorfield04:07:45

Just use a local database. Seriously. If you use PostgreSQL there's an embedded version you can use that runs entirely in the JVM. At work we just use a throwaway Percona instance in a Docker container.

seancorfield04:07:23

The problem with mocking is that you're no longer testing anything useful.

James Amberger12:05:12

Just use a local database. Seriously. If you use PostgreSQL there’s an embedded version you can use that runs entirely in the JVM.I’ll be grateful for any more advice in this vein. My ultimate database is a hoary old Oracle installation at $large_institution where typically people connect to the dev/tst/etc databases in development. I need to work on this stuff while remote. As per your advice above I created a sqlite db and started retrieving DDLs for the relevant tables from the Oracle database, removing unnecessary or Oracle-specific directives etc and translating column types etc to sqlite3 equivalents, and creating the tables and inserting records. Somewhat tedious, but different RDBMS are different or they wouldn’t be different RDBMS, right? Was this smart? How could it have been smarter?

igrishaev13:05:10

Well, this is partially true. Using a local db is amazing for development and testing, but I doubt about a embedded version of Postgres. Native Postgres works everywhere, and there is always an option to run it in Docker. Regarding your case with Oracle, I'm not aware about if Oracle ships docker images for their database, but quick googling gives some evidence.

igrishaev13:05:50

Perhaps you can find an appropriate docker image of Oracle DB made for development purposes, so why not using it?

seancorfield16:05:22

> I doubt about a embedded version of Postgres Based on the dependencies you provide, it seems to be a Java wrapper around JNI to run PostgresSQL via its native libraries, but I'd have to dig into the docs for any incompatibilities: https://github.com/zonkyio/embedded-postgres

James Amberger12:05:12
replied to a thread:Hi, I’m using next.jdbc and honey.sql to good effect in my app, everything going well. For much of my code, I use a single db execute! function that takes in vector of honeysql maps and returns the results. These statements are all run in a single jdbc transaction, allowing me to easily mock out the db for various tests and return arbitrary data from the ‘database’ in order to examine how the system works in various scenarios. I have built a mocking function that allows me to match a honeysql map, return whatever I want when a matched map is encountered, and examine the contents of that honeysql map to confirm that it is being populated as expected - happy days. I am testing the db directly as well of course in other tests but this model does away with the db population set up work, which can get tricky as the db gets more complex. So far, my db transactional requirements have been straightforward, and having a single transaction in this function has been sufficient for my needs. However, I am now encountering more scenarios where I need to handle more complex database-related logic, and I need to wrap more code with `jdbc/with-transaction`. My db mocking feature breaks down when I need to do this, - because `jdbc/with-transaction` is a macro and I can’t figure out a way to ‘mock’ this call (or do something that has the same effect) so I can concentrate on the correctness of my honeysql maps in my unit tests and not worry about a db. Any help on this would be appreciated, and happy to provide further information if helpful.

Just use a local database. Seriously. If you use PostgreSQL there’s an embedded version you can use that runs entirely in the JVM.I’ll be grateful for any more advice in this vein. My ultimate database is a hoary old Oracle installation at $large_institution where typically people connect to the dev/tst/etc databases in development. I need to work on this stuff while remote. As per your advice above I created a sqlite db and started retrieving DDLs for the relevant tables from the Oracle database, removing unnecessary or Oracle-specific directives etc and translating column types etc to sqlite3 equivalents, and creating the tables and inserting records. Somewhat tedious, but different RDBMS are different or they wouldn’t be different RDBMS, right? Was this smart? How could it have been smarter?