Fork me on GitHub
#xtdb
<
2020-03-31
>
Akshay C. Gollapalli04:03:08

I can't seem to turn off hostname verification when configuring kafka using :crux.kafka/kafka-properties-map {"ssl.endpoint.identification-algorithm" "", <other ssl options>} When I try to connect to my kafka broker in azure I get: "Execution error (CertificateException) at sun.security.util.HostnameChecker/matchIP (HostnameChecker.java:137). No subject alternative names present"

refset09:03:17

Hmm, that's interesting. Does a properties file work?

refset09:03:05

This is how a properties file gets read:

(require [ :as io])
(import [java.util Properties])
(import [ Reader])

(defn load-properties [^Reader in]
  (->> (doto (Properties.)
         (.load in))
       (into {})))

(defn read-kafka-properties-file [f]
  (when f
    (with-open [in (io/reader (io/file f))]
      (load-properties in))))

refset09:03:42

Do you need to set this property on the Kafka side also? And if so, have you done that?

refset09:03:32

Yeah it looks like a broker property also: https://docs.confluent.io/current/installation/configuration/broker-configs.html kafka-properties-map is only for the Kafka client settings

Akshay C. Gollapalli04:04:51

Oh, I didn't change it on the broker side, so maybe that's what it was.

Akshay C. Gollapalli04:04:35

I ended up just using Confluent Cloud to get the job done. But i'll probably revisit this in a couple months, after evaluating cost-benefit of the managed solution.

👍 4
Akshay C. Gollapalli07:04:43

Oddly enough, I can write to db fairly easily, but I can't seem to sync. I have maybe 3 documents in my test db, and one node, and it won't sync. Await-tx hangs as well. It worked fine with the locally installed kafka instance though.

refset07:04:04

Oh, you may have hit a config bug we've been looking at recently, to do with the polling frequency and SSL handshakes. Something changed on the Confluent end that means we had to increase the poll time in the Kafka client, but I don't think we've released since, I'll investigate this morning and report back

👍 4
refset08:04:27

Just spoke to the team - we're cutting a new release for you today which will include the fix: https://github.com/juxt/crux/issues/726 🙂

👍 4
refset09:04:15

@US65YEL1M done - please try again with 20.04-1.8.1-alpha

👍 4
Akshay C. Gollapalli18:04:27

Thanks! it works!

🎉 12