Fork me on GitHub
#sql
<
2018-01-16
>
seancorfield00:01:35

Can you share the code that is causing that exception @josmith2016?

Will00:01:34

Here are the lines of code that are causing the issue: (def simpleQuery ["SELECT * FROM table"])

Will00:01:51

(j/query db2 [simpleQuery]
           {:as-arrays? true})

seancorfield00:01:05

Ah, you've nested your vectors.

Will00:01:20

What do you mean?

seancorfield00:01:45

What you have is equivalent to (j/query db2 [ ["SELECT * FROM table"] ] {:as-arrays? true})

seancorfield00:01:27

What you want is (j/query db2 ["SELECT * FROM table"] {:as-arrays? true})

Will00:01:38

You're right

seancorfield00:01:17

Or, since you have no parameters to your SQL statement, you can omit the brackets altogether: (j/query db2 "SELECT * FROM table" {:as-arrays? true})

seancorfield00:01:04

(but it's probably better while you're getting used to the library to use a vector, even when there are no parameters, just for consistency)

Will00:01:28

Lol thanks for the help, sorry it was such a beginner mistake.

seancorfield00:01:49

No problem! As you said, you're just getting started with Clojure so this sort of thing should be expected -- and we all try to be as helpful and welcoming as we can here!

Will00:01:09

Would I be able to pm you for these kinds of question? or is it ok to post here?

seancorfield00:01:39

Probably best to post here -- on the rare off-chance that I'm not actually around 🙂

Will00:01:56

I have another one, after querying the database I get Exception in thread "main" java.sql.SQLDataException: DATE/TIME value not in standard format.

Will00:01:08

querying an ingres database

seancorfield00:01:09

(I'm on Pacific US time and I'm nearly always online -- but I do sleep occasionally and sometimes I even go on vacation!)

Will00:01:00

I really appreciate all the help. It's been taking me days to figure out simple stuff like this because I didn't realize this group existed until today

seancorfield00:01:01

OK, so that exception is coming from the JDBC driver itself, not java.jdbc, and I'm not familiar with Ingres (it's not one of the databases I test java.jdbc against).

seancorfield00:01:14

@josmith2016 I haven't been able to find any information about that exception online...

Will00:01:17

I haven't either

seancorfield00:01:20

Are you supplying a date/time value as a string in your query?

Will00:01:18

All it's doing is running that query

Will00:01:45

There may be null dates / timestamps in the table, would that be an issue?

seancorfield00:01:57

Hmm, not sure what to suggest -- that exception is coming from the layer below clojure.java.jdbc I suspect (feel free to share the stacktrace as a snippet so I can double-check that).

seancorfield01:01:16

I would expect null fields to come back into Clojure as nil.

Will01:01:41

Exception in thread "main" java.sql.SQLDataException: DATE/TIME value not in standard format.
        at com.ingres.gcf.util.SqlExType.getSqlEx(SqlExType.java:99)
        at com.ingres.gcf.util.SqlExFactory.get(SqlExFactory.java:68)
        at com.ingres.gcf.util.IngresDate.getTimestamp(IngresDate.java:1323)
        at com.ingres.gcf.util.IngresDate.getObject(IngresDate.java:1364)
        at com.ingres.gcf.jdbc.JdbcRslt.getObject(JdbcRslt.java:4246)
        at clojure.java.jdbc$dft_read_columns$fn__405.invoke(jdbc.clj:462)
        at clojure.core$mapv$fn__8088.invoke(core.clj:6832)
        at clojure.lang.LongRange.reduce(LongRange.java:233)
        at clojure.core$reduce.invokeStatic(core.clj:6747)
        at clojure.core$mapv.invokeStatic(core.clj:6823)
        at clojure.core$mapv.invoke(core.clj:6823)
        at clojure.java.jdbc$dft_read_columns.invokeStatic(jdbc.clj:462)
        at clojure.java.jdbc$dft_read_columns.invoke(jdbc.clj:458)
        at clojure.java.jdbc$result_set_seq$row_values__415.invoke(jdbc.clj:508)
        at clojure.java.jdbc$result_set_seq$thisfn__417.invoke(jdbc.clj:518)
        at clojure.java.jdbc$result_set_seq$thisfn__417$fn__418.invoke(jdbc.clj:518)
        at clojure.lang.LazySeq.sval(LazySeq.java:40)
        at clojure.lang.LazySeq.seq(LazySeq.java:49)
        at clojure.lang.RT.seq(RT.java:528)
        at clojure.core$seq__5124.invokeStatic(core.clj:137)
        at clojure.core$map$fn__5587.invoke(core.clj:2738)
        at clojure.lang.LazySeq.sval(LazySeq.java:40)
        at clojure.lang.LazySeq.seq(LazySeq.java:49)
        at clojure.lang.Cons.next(Cons.java:39)
        at clojure.lang.RT.next(RT.java:706)
        at clojure.core$next__5108.invokeStatic(core.clj:64)
        at clojure.core$dorun.invokeStatic(core.clj:3134)
        at clojure.core$doall.invokeStatic(core.clj:3140)
        at clojure.core$doall.invoke(core.clj:3140)
        at clojure.java.jdbc$query$fn__543.invoke(jdbc.clj:1098)
        at clojure.java.jdbc$execute_query_with_params.invokeStatic(jdbc.clj:994)
        at clojure.java.jdbc$execute_query_with_params.invoke(jdbc.clj:987)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1016)
        at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:996)
        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1090)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
        at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1063)
        at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
        at app.core$_support.invokeStatic(core.clj:49)
        at app.core$_support.invoke(core.clj:40)
        at app.core.support(Unknown Source)
        at app.Application.main(Application.java:17)

seancorfield01:01:28

Yeah, so that's coming from the .getObject() call on the ResultSet object, before clojure.java.jdbc gets the value.

seancorfield01:01:00

So I'm going to guess that the Ingres JDBC driver forces you to call a different method for certain SQL data types...

seancorfield01:01:53

That code is generic in clojure.java.jdbc and works for every database I've encountered so far 😐

Will01:01:21

I probably have some configuration wrong or something, I'll play around tonight and see what I come up with

seancorfield01:01:57

Are there other JDBC drivers you could try for Ingres? Maybe some are more "compliant" than others...?

Will01:01:18

I'll chekc

seancorfield01:01:35

I only see one on Maven Central: [com.ingres.jdbc/iijdbc "10.2-4.1.10"]

Will01:01:00

this is the one i'm using com.ingres.jdbc.IngresDriver

seancorfield01:01:08

It just looked like there were several other options when I searched in general for Ingres JDBC driver.

seancorfield01:01:37

Do you know which version of the JDBC driver you're using? (from your Gradle build file)

Will01:01:52

I'm using an older, I'll try updating

seancorfield01:01:08

compile 'com.ingres.jdbc:iijdbc:10.2-4.1.10'

seancorfield01:01:55

According to Maven, that's the way to get the latest version in Gradle... (I don't know, I haven't used Gradle for a decade!)

Will01:01:31

Lol that is the correct syntax

Will01:01:08

That didn't fix it

seancorfield01:01:10

I last touched Groovy back in... 2008 I think...

Will01:01:21

That's along time

Will01:01:18

What do you use now?

seancorfield01:01:46

Clojure -- since 2011.

seancorfield01:01:15

I first learned Clojure in 2010. Went to production on 1.3 Alpha 7 or 8 in 2011. We're on 1.9 in production now.

Will01:01:45

Nice, this is the first time I've used clojure

seancorfield01:01:56

My arc was roughly C / assembly / COBOL -> C++ -> Java -> Groovy -> Scala -> Clojure.

Will01:01:02

I've been a java programmer for like 7 years

seancorfield01:01:06

I loved Java when I first switched from C++ (around '97)... but got fed up with it by the time Java 5 appeared I think...

Will01:01:45

I haven't been around for that long but after only seeing java syntax or similar the clojure syntax feels so alien lol it's taking me a while to understand it

Will01:01:25

For all my backend development I've only used java, besides one time I used c# for a college class

seancorfield01:01:46

Back at university, I did Lisp, Prolog, APL, Pilot... so I was exposed to a lot of non-C-family stuff.

Will01:01:19

What does this mean? Exception in thread "main" java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.String

Will01:01:18

I guess I did use MIPs for some classes, but very few and it was a long time ago

seancorfield01:01:19

Well, it means you passed a lazy sequence to something that wanted a string or tried to convert it to string... stack trace and code?

seancorfield01:01:08

You'll quickly learn to interpret that exception in the context of your code -- and this one java.lang.ClassCastException: <sometype> cannot be cast to clojure.lang.IFn which means you have something in the "call" position of an expression that is not a function, e.g., ("foo" 1 2 3)

Will01:01:41

Here is the exception:

Exception in thread "main" java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.String
        at app.core.support(Unknown Source)
        at app.Application.main(Application.java:17)

Will01:01:06

All I did was add a where clause to that simple query I posted above\

seancorfield01:01:21

Show me the code...

seancorfield01:01:41

(since that stack trace is... less than helpful 🙂 )

Will01:01:14

(def simpleQuery "SELECT * FROM table WHERE col = 2")
  (j/query db2 [simpleQuery])

seancorfield01:01:30

Is app.core.support your Clojure namespace?

Will01:01:48

Ya from what I've seen the stacktraces are not as helpful as java, unless I don't quite understand them

Will01:01:01

My namespace is app.core

seancorfield01:01:10

How exactly are you calling your Clojure from Java?

Will01:01:11

The method being called is support

Will01:01:36

I've just been calling it in the static main function (which could possibly be an issue) to play around with core.support("FIRST");

seancorfield01:01:14

I need to see more code than that...

Will01:01:52

public static void main(String[] args) {
        core.support("FIRST");
        SpringApplication.run(Application.class, args);
    }

seancorfield01:01:57

I don't understand the context of that code -- what is core? how is suppport defined? what does your ns form look like.

Will01:01:49

(ns app.core
  (:require [clojure.java.jdbc :as j]
            [clojure.string :as str])
  (:gen-class
        :name app.core
        :methods [#^{:static true} [support [String] String]]))

Will01:01:32

(defn -support [x]
  (def simpleQuery "SELECT * FROM table WHERE col = 2")
  (j/query db2 [simpleQuery])
)

seancorfield01:01:48

Ah, so you've declared -support to return a String, but you're returning a sequence -- the result of j/query is a sequence of hash maps.

Will01:01:28

How do I specify that?

Will01:01:56

Would it just be map?

seancorfield01:01:19

Trying to learn Clojure by calling it from Java is going to be a pretty painful way to do things... you're trying to learn gen-class (which folks generally don't use) and combining Java types and Clojure types and so on...

seancorfield01:01:28

j/query should return a fully realized sequence so that part itself won't be lazy but I suspect the default attempt to turn it into a String creates a lazy sequence somewhere.

Will01:01:01

Ok, is there a way thats different from gen-class?

seancorfield01:01:11

Off the top of my head, I'm not sure what Java interfaces would be implemented by that sequence -- I'd have to try it in a REPL.

Will01:01:18

Or should I learn it separately?

seancorfield01:01:28

Learn it separately would be my advice.

seancorfield01:01:02

There's a huge amount of "incidental complexity" in the Java/Clojure interop that will get in the way of learning and debugging Clojure (e.g., that terrible stack trace)

seancorfield01:01:20

Ah, yes, you do get a lazy sequence back from j/query but it is fully realized under the hood (as it consumed the ResultSet).

Will01:01:39

Maybe I'll split the project into 2, I've been trying to get it working this way because at my work we already have java services, I was hoping I could get them to work together in the same project

seancorfield01:01:08

user> (query {:dbtype "mysql" :dbname "worldsingles" :user "myuser" :password "secret"} "select * from status")
=> 
({:name "approved", :id 1}
 {:name "new", :id 2}
 {:name "rejected", :id 3})
user> (type *1)
=>
clojure.lang.LazySeq
user> (ancestors *1)
=>
#{java.lang.Object
  clojure.lang.Obj
  clojure.lang.ISeq
  clojure.lang.IPending
  java.util.Collection
  clojure.lang.IPersistentCollection
  clojure.lang.IHashEq
  clojure.lang.IObj
  clojure.lang.IMeta
  clojure.lang.Sequential
  clojure.lang.Seqable
  java.io.Serializable
  java.util.List
  java.lang.Iterable}
So you could declare it to return java.util.Collection or java.lang.Iterable and then manipulate it that way.

seancorfield01:01:25

(working from the REPL is so much easier!)

Will01:01:44

How do I add dependencies to the REPL?

Will01:01:00

like the clojure.java.jdbc for example

seancorfield01:01:25

There are several possibilities: use the command line clj script with a deps.edn file; use Boot (a build tool); use Leiningen (another build tool). I recommend Boot -- http://boot-clj.com

seancorfield01:01:53

Because then you can just do boot -d org.clojure/java.jdbc repl and get a REPL with the latest version of clojure.java.jdbc running.

Will01:01:02

Thats awesome

Will01:01:07

I'll try that

seancorfield01:01:13

(you can use -d to specify anything that gradle would pull in for you)

seancorfield01:01:40

Because you'll need to pull in the Ingres JDBC driver as well...

seancorfield01:01:24

boot -d com.ingres.jdbc/iijdbc -d org.clojure/java.jdbc repl should do that.

seancorfield01:01:30

@josmith2016 if you hit any Boot-specific questions, there's a #boot channel but I'm happy to answer them here in the context of getting you up and running with SQL stuff 🙂

Will01:01:44

Thank you 🙂

Will02:01:29

What IDE do you use to develop clojure @seancorfield?

seancorfield02:01:51

Atom with ProtoREPL. I used to use Emacs (for years!).

danielcompton02:01:56

@josmith2016 did you find the Ingres driver you needed? It looks like https://www.actian.com/data-management/ingres-sql-rdbms/ is the current product

Will15:01:48

Yes I did, thank you @danielcompton

Will15:01:54

As I query a database with the clojure.java.jdbc library, it is returning a clojure.lang.LazySeq. How do I change this to a list or map I can use in java?

seancorfield16:01:04

@josmith2016 As I noted above, a LazySeq implements java.util.Collection and java.lang.Iterable so you can use it like either of those Java types.

Will16:01:20

I get this stacktrace when trying to cast it to a collection: Exception in thread "main" java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to clojure.lang.IFn

seancorfield17:01:44

You're trying to use the result of the query as a function I expect. Again, show the code and the top of the stacktrace (down to where it comes into your code).

seancorfield17:01:06

(That was one of the two exceptions I called out specifically above as being common for folks new to Clojure)

Will17:01:44

Here is the clojure method

(defn -support [x]
  (def simpleQuery "SELECT * FROM table")
  (j/query db [simpleQuery])
)

seancorfield17:01:04

How do you have it declared in gen-class now?

Will17:01:29

(:gen-class
        :name app.core
        :methods [#^{:static true} [support [String] java.util.Collection]])

seancorfield17:01:50

OK, and what does your Java look like now?

Will17:01:59

Collection list = core.support("FIRST");

seancorfield17:01:33

And the top of the stacktrace?

seancorfield17:01:28

BTW, do not use def inside defn -- def is for top-level definitions only.

Will17:01:43

What should I use instead?

seancorfield17:01:56

If you want a local "variable", use let

seancorfield17:01:09

(let [simple-query "SELECT * FROM table"]
  (j/query db [simple-query]))
And we use "kebab-case" in Clojure, not camelCase.

seancorfield17:01:08

(getting a coffee -- back in five)

Will17:01:02

I guess that worked, I got the error when trying to cast it to an array list in my java code. System.out.println((ArrayList<Object>) list);

Will17:01:33

I got it working by declaring a new ArrayList like this: System.out.println(new ArrayList<Object>(list));

Will17:01:17

Would the simple-query variable only exist until the end of that end parenthesis? or would it be available throughout the rest of the method?

seancorfield17:01:34

Bindings introduced in let exist until its closing )

seancorfield17:01:01

def always defines a Var and a top-level binding (even if it nested inside a function).

Will17:01:35

Ok that makes sense, I tried using let but couldn’t get it working because I thought it lasted the whole method like in java

Will17:01:18

Thanks so much for all your help, You saved me a ton of time. Multiple days worth of work probably

Will17:01:20

One more question

Will17:01:48

I’ve seen something where you can declare keywords

Will17:01:53

for example

Will17:01:18

If I’m declaring a database connection map like this

(def db-connect {:dbtype "ingres"
           :classname "com.ingres.jdbc.IngresDriver"
           :dbname ":dbname"
           :host "host"
           :port "I17"
           :user "user"
           :password "pass"})
Can i declare what the :dbname is equal to later? after already declaring the db connection?

jgh17:01:26

you can use a function for it, as long as it emits a string

Will17:01:39

Can you show me an example?

jgh17:01:55

e.g. for my prod code i do

{
    :port 4000
    :host "0.0.0.0"
    :uri ""
    :db  {:dbtype "postgresql"
          :dbname (slurp "/run/secrets/postgres-db")
          :host "postgres"
          :user (slurp "/run/secrets/postgres-user")
          :password (slurp "/run/secrets/postgres-passwd")}
}

jgh17:01:47

(this is in a docker swarm)

seancorfield17:01:16

The other thing you can do is use assoc to produce a new hash map with :dbname updated:

(def db {:dbtype ... :dbname ...})
(j/query (assoc db :dbname "newdb") ["select * from whatever"]))
Note that db is not changed -- assoc produces a new hash map with :dbname set to "newdb"

seancorfield17:01:32

@josmith2016 We do something similar to @jgh’s suggestion at work: we pull all of the settings that can change from an environment/configuration library (that uses EDN files on disk mostly), and we build all our db-specs at application startup since we use c3p0 for connection pooling.

seancorfield17:01:24

Examples of reusing connections (directly in clojure.java.jdbc or via connection pooling) in the docs http://clojure-doc.org/articles/ecosystem/java_jdbc/reusing_connections.html