onyx 2016-12-18 | Slack Archive

A question, I’m getting this error in my onyx.log when using the amazon-s3 plugin.

java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.ClientConfiguration
    clojure.lang.ExceptionInfo: Caught exception inside task lifecycle. Rebooting the task. -> Exception type: java.lang.NoClassDefFoundError. Exception message: Could not initialize class com.amazonaws.ClientConfiguration
       job-id: #uuid "0f3cdf70-462f-4a22-b109-79287f0e2808"
     metadata: {:job-id #uuid "0f3cdf70-462f-4a22-b109-79287f0e2808", :job-hash "ee57c6e7e986279e9d56ce5f2c42d51bf887608c28218e30297c36e074f3"}
      peer-id: #uuid “a655141a-dbf9-415f-b61b-07d5008c1742”

Is the aws client executed when the task is assigned to the peer or do I need to add (s3util/set-region (s3util/new-client) “eu-west-1”) in my code to fire off a client session first? There’s no documentation making it clear what I’m supposed to do. If I do add the set-region then I get the following exception:

$ java -cp target/peer.jar testapp.core submit-job "kafka-job" -c resources/config.edn -p :default
Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findDefaultDeserializer(BasicDeserializerFactory.java:1529)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.findStdDeserializer(BeanDeserializerFactory.java:167)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:132)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:403)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:352)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:461)
	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3838)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3732)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2679)
	at com.amazonaws.internal.config.InternalConfig.loadfrom(InternalConfig.java:236)
	at com.amazonaws.internal.config.InternalConfig.load(InternalConfig.java:249)
	at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:304)
	at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:141)
	at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:136)
	at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:97)
	at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:63)
	at com.amazonaws.ClientConfigurationFactory.getDefaultConfig(ClientConfigurationFactory.java:46)
	at com.amazonaws.ClientConfigurationFactory.getConfig(ClientConfigurationFactory.java:36)
	at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:469)
	at onyx.plugin.s3_utils$new_client.invokeStatic(s3_utils.clj:17)
	at onyx.plugin.s3_utils$new_client.invoke(s3_utils.clj:17)
	at testapp.jobs.basic$basic_job.invokeStatic(basic.clj:26)
	at testapp.jobs.basic$basic_job.invoke(basic.clj:16)
	at testapp.jobs.basic$fn__21445.invokeStatic(basic.clj:67)
	at testapp.jobs.basic$fn__21445.invoke(basic.clj:32)
	at clojure.lang.MultiFn.invoke(MultiFn.java:233)
	at testapp.core$_main.invokeStatic(core.clj:86)
	at testapp.core$_main.doInvoke(core.clj:68)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at testapp.core.main(Unknown Source)

jasonbell14:12:20

Does anyone have any ideas?

jasonbell18:12:32

Wondering if there’s a dependency issue with the AWS sdk?

yonatanel18:12:05

https://github.com/FasterXML/jackson-databind/issues/1101

yonatanel18:12:39

Somewhere someone is causing an older jackson to be resolved

yonatanel18:12:05

@jasonbell Can you do a lein deps :tree on the peer project and see what you get?

jasonbell18:12:44

I just did a lein clean and rebuilt from scratch