This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-10-05
Channels
- # aws (1)
- # beginners (57)
- # boot (3)
- # cider (6)
- # clara (49)
- # cljs-dev (47)
- # cljsjs (23)
- # clojure (144)
- # clojure-dev (2)
- # clojure-finland (1)
- # clojure-germany (1)
- # clojure-sg (1)
- # clojure-spec (25)
- # clojure-uk (245)
- # clojurescript (39)
- # core-async (3)
- # cursive (6)
- # datomic (117)
- # emacs (3)
- # fulcro (6)
- # hoplon (10)
- # jobs (7)
- # juxt (5)
- # leiningen (11)
- # om (27)
- # pedestal (4)
- # perun (2)
- # re-frame (22)
- # reagent (35)
- # ring-swagger (11)
- # shadow-cljs (333)
- # spacemacs (10)
- # specter (10)
- # sql (20)
- # vim (8)
while a single thread in a single process is responsible for writing transactions, the transactor can still take advantage of multiple cores for other purposes, right?
right: http://docs.datomic.com/monitoring.html#add-more-cores sorry for noise
I’m trying to get Datomic working in k8s with Cassandra as the storage service, my peers cannot find the transactor after connecting to storage. I’ll describe the setup:
Here’s the transactor config:
protocol=cass
host=0.0.0.0
alt-host=datomic
port=4334
license-key=<redacted>
cassandra-table=datomic.datomic
cassandra-host=cassandra
cassandra-port=9042
memory-index-threshold=32m
memory-index-max=512m
object-cache-max=1g
The transactor is running in its own pod and there’s a service called datomic
with port 4334 that other pods can connect to
the peer reports that it’s trying to connect to the transactor at localhost and alt-host nil
It’s my understanding that the transactor stores its connection info in storage and the peer picks it up there. But it appears not to get the configured information. Is that not correct? Why does the peer not try datomic:4334
?
a different transactor or config is running; peer is connecting to wrong storage; peer cannot resolve "datomic" to an address?
There is without a doubt no other transactor or no other Cassandra because in both cases I’m launching fresh pods in fresh clusters. As for the peer not resolving it, it’s not even trying it:
clojure.lang.ExceptionInfo: Error communicating with HOST localhost on PORT 4334 {:alt-host nil, :peer-version 2, :password "ckGYIzP97L6l+ERbbcnZuzCiSB/v3S1HfzUZdyIFLdE=", :username "f8HVTTNhf8bmknxqrO2TINx2xmPH5KAJLwQ5q/Cs0J8=", :port 4334, :host "localhost", :version "0.9.5561.59", :timestamp 1507230053031, :encrypt-channel true}
fyi, there’s a service called cassandra and the peer connects to it (clear from logs(
I think not. I think 4335 is dev transactor storage access, 4336 is the h2 GUI console when using dev storage
I'm mostly suspicious that the wrong config file is pulled or another transactor is running against that storage
But here’s what’s strange. I’ve run this in k8s before just fine with this setup when it was the dev transactor without having to do anything that complex.
dev transactor's "storage" is the peer, so your connection string routes to that peer already
There’s only one transactor pod. I don’t know where it’s even possible for there to be another.
maybe you could kill all transactors you know about, try to connect a peer, see if you get a different error
Same error, but that doesn’t make any sense. This is a completely fresh Minikube with nothing on it by Cassandra. Is it possible that a transactor that is no longer running but was once running connected to Cassandra and left it’s connection info there and it’s just not getting updated?
ps axf | grep datomic
doesn't show anything? you mentioned port forwarding. maybe you tried a transactor outside the cluster earlier and forgot about it?
can also confirm in the logs that the transactor did actually startup and connect to cassandra. maybe it never did and the settings in there are from an earlier test, like you suggested
Launching with Java options -server -Xms4g -Xmx4g -XX:+UseG1GC -XX:MaxGCPauseMillis=50
Starting datomic: ...
System started datomic:
Right, and I’m giving it config/transactor.properties and that’s the only file at that location
also datomic.transactor - {:event :transactor/start, :args {:cassandra-port 9042, :cassandra-table "datomic.datomic", :log-dir "log", :alt-host "datomic", :protocol :cass, :rest-alias "cass", :memory-index-max "512m", :cassandra-host "cassandra", :port 4334, :memory-index-threshold "32m", :data-dir "data", :object-cache-max "1g", :host "0.0.0.0", :version "0.9.5561.59", :encrypt-channel true}, :pid 1, :tid 12}
this is what’s in the Cass table: {:key "[\\"0.0.0.0\\" \\"datomic\\" 4334 \\"XnH1k0PQkm/Hz/Y4ISZpE7fpHcBMP7ui8pz8wwNcPXk=\\" \\"afmWR0zouI8Bee4gld/5zM48H4pecIzmHeNjeCODSfI=\\" 1507233334931 \\"0.9.5561.59\\" true 2]"}
you should see events like :peer/get-connection
:coord/lookup-transactor-endpoint
, and :peer/hornet-connect
and :peer/hornet-connect-failed
I'm having some trouble trying to start a dev transactor inside of docker-compose. Well, it's not trouble exactly...
I'm able to start the transactor and connect to it OK and it seems to work, but every time I connect to it I get bunches of these tracebacks in the peer:
ERROR 16:54:30.264 o.a.activemq.artemis.core.client: AMQ214016: Failed to create netty connection
java.nio.channels.UnresolvedAddressException: null
at sun.nio.ch.Net.checkAddress(Net.java:123)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:208)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:203)
at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1226)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:549)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:534)
at io.netty.handler.ssl.SslHandler.connect(SslHandler.java:438)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:549)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:534)
at io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:50)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:549)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:534)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:516)
at io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:970)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:215)
at io.netty.bootstrap.Bootstrap$2.run(Bootstrap.java:166)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:402)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at java.lang.Thread.run(Thread.java:745)
My transactor props looks like this:
protocol=dev
host=0.0.0.0
alt-host=dev-datomic
port=4334
...and the docker-compose bit looks like this:
services:
dev-datomic:
image: ""
ports:
- "4334-4336:4334-4336"
volumes:
- "datomic-data:/opt/datomic/data"
From the peer I'm connecting to datomic: