Fork me on GitHub
#sql
<
2020-02-10
>
teodorlu11:02:47

Hello! I'm using jdbc.next to work with a PostgreSQL table with a column containing JSON. When I select from the table, I get data like #object[org.postgresql.util.PGobject 0xb1fe4db ... . Any recommendations for working with JSON using jdbc.next? Do people just "convert to maps" when running the select query?

teodorlu11:02:20

I'm aware that I can use str with a normal JSON parser, but I'm suspecting that ad-hoc json/string conversion all around the codebase might not be desirable:

(with-open [conn (jdbc/get-connection db-datasource)]
  (-> (jdbc.sql/query conn ["SELECT * from my_table LIMIT 1"])
      first
      :my_table/data
      str
      json/read-str))

Andrew11:02:59

I haven’t tried this, but perhaps it’ll work / be useful. I’d extend ReadableColumn to PGobject as described here https://github.com/seancorfield/next-jdbc/blob/master/doc/result-set-builders.md#readablecolumn When the type of the object as reported by .`getType` is "jsonb" or "json" call json/read-str on the result of .getValue, otherwise return the result fo .getValue

👍 4
teodorlu13:02:06

Found an article that seems to cover insertion as well, but it's for clojure.jdbc and not jdbc.next: https://web.archive.org/web/20161024231548/http://hiim.tv/clojure/2014/05/15/clojure-postgres-json/

Andrew13:02:02

This says how to do the reverse

teodorlu13:02:36

That's right, the article documents how to do it both ways. Both articles rely on switching on the object type before converting to PGObjects. Thanks.

jwoods13:02:57

Does anyone have an example of importing data from csv into postgres or any other db?

teodorlu14:02:16

At least psql can output to CSV, but I haven't found a way to input. From man psql:

-F separator
       --field-separator=separator
           Use separator as the field separator for unaligned output. This is equivalent to \pset
           fieldsep or \f.

dharrigan15:02:16

\copy my-funky-table from '/home/foo/this-is-my.csv' with delimiter as ',' csv header

jwoods16:02:03

Instead of passing a file can I do it with csv data in memory?

dharrigan15:02:29

you can remove the header if you csv doesn't include a header.

fabrao22:02:12

hello all, about jdbc.next what is the way to use jdbc driver that is not in list of available

noisesmith22:02:02

you provide jdbc drivers via the classpath, which should be getting it via project.clj or deps.edn depending on how you manage your project

fabrao22:02:42

I got

Unhandled clojure.lang.ExceptionInfo
   Unknown dbtype:

noisesmith22:02:06

OK - that's either an issue with the classpath where the db driver wasn't loaded, or you misconfigured the db spec itself - I can't help with the latter

noisesmith22:02:14

can you show more of the message / context?

fabrao23:02:34

Unhandled clojure.lang.ExceptionInfo
   Unknown dbtype:
   {:class "dbmaker.jdbc.ws.client.Driver",
    :dbname "DEFAULT",
    :user "SYSADMIN",
    :password "1234",
    :host "127.0.0.1",
    :port 1054}
            connection.clj:  201  next.jdbc.connection/spec->url+etc
            connection.clj:  141  next.jdbc.connection/spec->url+etc
            connection.clj:  267  next.jdbc.connection/eval959/fn
             protocols.clj:   14  next.jdbc.protocols/eval723/fn/G
            connection.clj:  283  next.jdbc.connection/eval979/fn
             protocols.clj:   24  next.jdbc.protocols/eval753/fn/G
                  jdbc.clj:  134  next.jdbc/get-connection
                  jdbc.clj:  117  next.jdbc/get-connection
                      REPL:   12  app.core/eval13801
                      REPL:   12  app.core/eval13801
             Compiler.java: 7177  clojure.lang.Compiler/eval
             Compiler.java: 7132  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
                  main.clj:  437  clojure.main/repl/read-eval-print/fn
                  main.clj:  437  clojure.main/repl/read-eval-print
                  main.clj:  458  clojure.main/repl/fn
                  main.clj:  458  clojure.main/repl
                  main.clj:  368  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj:  660  clojure.core/apply
                regrow.clj:   18  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   79  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   55  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  142  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  171  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  170  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  748  java.lang.Thread/run

fabrao23:02:16

for

(jdbc/get-connection
 {:class "dbmaker.jdbc.ws.client.Driver"
 :dbname "DEFAULT"
 :user "SYSADMIN"
 :password "1234"
 :host "127.0.0.1"
 :port 1054})

seancorfield23:02:35

You must provide :dbtype.

seancorfield23:02:55

Per the docs:

If you want to use a database that is not in this list, you can specify a new :dbtype along with the class name of the JDBC driver in :classname. You will also need to specify :port. For example:

{:dbtype "acme" :classname "com.acme.JdbcDriver" ...}

seancorfield23:02:33

Add :dbtype "dbmaker" to your hash map and it should work.

seancorfield23:02:31

next.jdbc caches certain aspects of driver handling based on the db type so you have to provide a unique value for each different type of database (driver) you want to use.

fabrao23:02:56

sorry about that. I only read the get-datasource help

fabrao23:02:54

any other advice? :dbtype "dbmaker" didn´t work

seancorfield23:02:27

Oh :class is wrong. Should be :classname

seancorfield23:02:38

get-datasource says

In the first format, these keys are required:

:dbtype -- a string indicating the type of the database

seancorfield23:02:47

and

:classname -- if you need to override the default for the :dbtype (or you want to use a database that next.jdbc does not know about!)

seancorfield23:02:14

Let me know if I can make that clearer.

seancorfield23:02:49

I guess I could link from the get-datasource docstring to the #dbtypes section of the next.jdbc.connection namespace...

fabrao23:02:52

oh, you right, I´m dumb*ss , that was the problem, :classname

fabrao23:02:49

the docs of driver says that we have to include other parameter to it

fabrao23:02:00

Class.forName("dbmaker.jdbc.ws.client.Driver");
Connection conn =
DriverManager.getConnection("jdbc:dbmaker:
T", "SYSADM", abc");

fabrao23:02:25

what is the way to include :type3 ?

fabrao23:02:13

or the best way is to provide the url directly?

seancorfield23:02:20

:dbtype "dbmaker:type3" should work.

seancorfield23:02:58

That's what the DB type is: the driver-specific segment of the JDBC URL.

fabrao23:02:40

Best regards Sean, thank you, now is working

4
fabrao23:02:06

have you heard about this database?

seancorfield23:02:13

Nope, never heard of it.

seancorfield23:02:29

BTW, going back to the docstring for that dbtypes Var:

The value of :dbtype should be the string that the driver is associated with in the JDBC URL, i.e., the value that comes between the jdbc: prefix and the ://<host>... part. In the above example, the JDBC URL that would be generated would be jdbc:acme://<host>:<port>/<dbname>.

seancorfield23:02:16

(I wanted to double-check I was describing the relationship between :dbtype and the JDBC URL somewhere -- and I would have added that if it wasn't already present!)

fabrao23:02:22

my customer is using COBOL in JVM, 🙂

fabrao23:02:44

and this database

fabrao23:02:17

the old new technology

fabrao23:02:57

the Cobol code is transpiled to Java and is compiled

dpsutton23:02:09

You are taking to the absolutely perfect person then :)