clojure 2023-06-25 | Slack Archive

h0bbit06:06:28

Hello All! I’m using next.jdbc on a project where I write to a SQLite database and my colleague on a Python project reads from it. My requirement is to seamlessly read and write JSON to the DB for certain complex fields. I read the https://cljdoc.org/d/com.github.seancorfield/next.jdbc/1.3.874/doc/getting-started/tips-tricks of next.jdbc and cobbled together something that works so smoothly! : First, I change Clojure vectors and maps to JSON automatically when writing to the DB using the SettableParameter protocol:

(extend-protocol prepare/SettableParameter
  clojure.lang.IPersistentMap
  (set-parameter [m ^PreparedStatement s i]
    (.setObject s i (to-json m)))

  clojure.lang.IPersistentVector
  (set-parameter [v ^PreparedStatement s i]
    (.setObject s i (to-json v))))

Then, when reading from the database, I define a builder-fn which converts JSON to Clojure if the column type is JSON:

(def builder-adapter-from-json
  (rs/builder-adapter
   rs/as-maps
   (fn [builder ^ResultSet rs ^Integer i]
     (let [rsm ^ResultSetMetaData (:rsmeta builder)]
       (rs/read-column-by-index
        (if (#{"JSON"} (.getColumnTypeName rsm i))
          (from-json (.getObject rs i))
          (.getObject rs i))
        rsm
        i)))))

I’m very happy with the end result and how simple it has made reading / writing data into SQLite, so I thought I’d share here on the channel. If anyone has a way that’s even simpler, please let me know!

emccue14:06:17

I would use jsonb and not json tbh

emccue14:06:02

there is a bit of a downside to setting those globally at the protocol level in that all next.jdbc code has to be okay with vectors/maps = json. Depending on your project and how you think about that sort of thing, just working with the normal PGObject that comes out isn't that bad.

h0bbit10:06:41

I’m using SQLite and not PG, so PGObject and jsonb isn’t available to me. Other than that, yes +1 to hidden gotchas of doing this at a protocol level. But this is a fairly small and contained code-base so I think that this was an acceptable optimisation

Dallas Surewood15:06:59

Are you all running integration tests from a dev REPL or as a separate process? I am using Integrant, and it seems difficult to have my dev integrant system running and then setup a second test system. Is it normal to just not have any processes running and run a test command instead of doing it from the REPL?

enn16:06:05

I run tests from the REPL most of the time. Our tests set up a second test system (with some memoization for performance). We use Component, I don't know much about Integrant.

seancorfield16:06:13

Same here: we use Component but run a lot of tests via the REPL, including tests that spin up multiple processes to test cluster behavior etc.

Dallas Surewood16:06:58

Integrant is similar, although the more I'm digging into it, the more it might be hard to run two systems at once without leaving the REPL. It allows you to read a different configuration and start that system. So I can just say load-test-config and it will set all of my stuff to mocked systems or test databases. But then if I start it, my other one closes. There might be a way to set it up where I can run two systems at once, but it may require significant refactoring from how Kit bootstraps the project, which is a bit confusing. I'm not sure what they were going for.

p-himik16:06:32

> But then if I start it, my other one closes. You might be using integrant.repl then. It's meant to be a keeper of the system singleton. If you stop using that namespace and keep track of the active systems yourself, you can have multiple ones side by side with no problems.

Dallas Surewood16:06:10

I am. So essentially instead of using go or reset from that namespace, I should be making my own functions that call init-key?

Dallas Surewood16:06:22

Or I mean, ig/init

p-himik16:06:38

Yep, since you will be the one responsible for managing the system.

p-himik16:06:08

You can still use that namespace for the "main" system that will be concerned with all the reloading. And have some side system that you manage outside of that namespace.

Dallas Surewood16:06:37

Alright, that sounds good. I'm noticing the kit bootstrap has an interesting block of code when starting and stopping the system in production.

(defn stop-app
  []
  ((or (:stop defaults) (fn [])))
  (some-> (deref system) (ig/halt!))
  (shutdown-agents))

(defn start-app
  [& [params]]
  ((or (:start params) (:start defaults) (fn [])))
  (->> (config/system-config (or (:opts params) (:opts defaults) {}))
       (ig/prep)
       (ig/init)
       (reset! system))
  (.addShutdownHook (Runtime/getRuntime) (Thread. stop-app)))

This is for a jetty server. It seems like the shutdown part is done manually even though I was using halt just fine before. But the program crashes when it hits shutdown-agents. I'm not sure why this part is necessary.

Dallas Surewood16:06:24

Actually nvm. It seems like shutdown-agents just crashes my program entirely without anything running

igrishaev17:06:08

> Are you all running integration tests from a dev REPL or as a separate process? Both. Well-developed tests can be run either in REPL or with lein test. Usually, you have a fixture that runs a system, then runs a test, then tears down the system. You can extend starting and stopping the system such that they know the current state of the system and don't start/stop it if it has been run before manually.

Dallas Surewood17:06:11

Alright, thanks for the help. I wrote a function called run-integration-tests that automatically starts a test system if one doesn't exist and uses it for side-effectful functions. Then it shuts down when the tests are done.

Dallas Surewood19:06:54

Seems like cljdoc is down. Are there any libraries that let you run tests from the repl? It seems like cognitect test-runner is meant to be used from the command line.

Dallas Surewood19:06:02

(clojure.test/run-tests makes you list every namespace you're testing which seems odd? Is there not a way to just recursively run tests?

seancorfield20:06:21

https://clojuredocs.org/clojure.test/run-all-tests

Dallas Surewood20:06:38

Doesn't that run every dependency you have?

Dallas Surewood20:06:50

Oh I see the regexp

seancorfield20:06:19

Your dependencies will have source but not tests (in general - there might be some rare exceptions).

Dallas Surewood20:06:32

I ended up going with koacha just because the output was better

seancorfield20:06:48

And there are quite a few test-running functions in that namespace to suit all needs 😄

BuddhiLW20:06:09

I'm trying out Java's jdsp library for signal procressing. And, I can't nail the interop. The code for the in com.github.psambit9791.jdsp.io.Wav, reads: (we have the readWav method (?) and I don't know how to access it)

public class Wav {
    private WavFile wf;
    public Hashtable<String, Long> props;
    private double[][] data;

    public void readTemplate() throws WavFileException, IOException {
        File f = null;
        try {
            f = new File(this.getClass().getClassLoader().getResource("sample.wav").getFile());
        }
        catch (NullPointerException e) {
            System.out.println("File Not Found.");
        }
         = WavFile.openWavFile(f);
        this.computeProperties();
    }

    public void readWav(String filename) throws WavFileException, IOException {
        String ext = filename.substring(filename.lastIndexOf(".") + 1, filename.length());
        if (!ext.equals((Object)"wav")) {
            System.out.println("File is not of WAV format");
        } else {
            File f = null;
            try {
                f = new File(filename);
            }
            catch (NullPointerException e) {
                System.out.println("File Not Found.");
            }
             = WavFile.openWavFile((File)f);
            this.computeProperties();
        }
    }

I'm trying out (I tried other things, didn't work):

(let [wav (Wav.)]
  (.readWav wav))

But, throws me:

No matching field found: readWav for class com.github.psambit9791.jdsp.io.Wav

seancorfield20:06:53

Looks like the readWav method expected a string argument?

BuddhiLW20:06:40

Yes, the path for the wav file

BuddhiLW20:06:21

But, I didn't even get to call the readWav yet, I suposse

phronmophobic20:06:01

try something like:

(let [wav (Wav.)]
  (.readWav wav "my.wav"))

✅ 2

BuddhiLW20:06:08

Yeah, worked fine

🎉 2

2023-06-25

Channels