Clojurians Log v2

@turbopape 👍

Hi guys, I have a problem

When using mllib, I run into incmpatibilities with kryo serialization

Seems it’s a known bug, and I need to be going with spark 2.X

problem is, When I reaquire spark 2.X, it tells me that it can’t find logger class...

any idea ?

here is the relevant trace:

com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Can not set final scala.collection.mutable.ListBuffer field org.apache.spark.mllib.fpm.FPTree$Summary.nodes to scala.collection.mutable.ArrayBuffer Serialization trace: nodes (org.apache.spark.mllib.fpm.FPTree$Summary) org$apache$spark$mllib$fpm$FPTree$$summaries (org.apache.spark.mllib.fpm.FPTree) at com.esotericsoftware.kryo.serializers.FieldSerializer$Obje

seems it does not like fptree

… (conf/set "spark.kryo.classesToRegister" "scala.collection.mutable.ArrayBuffer,scala.collection.mutable.ListBuffer")

But still waiting for it to finish ...

Yes.

This definitely worked !