Morning!

morning

Morning

morning

morning all!

måning

I'm having an interop issue. Something that is a oneliner in Kotlin, doesn't work in Clojure, getting an IllegalAccessError in Clojure 😞

what’s the one liner?

A protobuf object that I call toByteArray on, i.e., message.toByteArray(). Works perfectly in Kotlin. The same in clojure (.toByteArray message) throws me an IllegalAccessException (JVMs in both cases is OpenJDK 12) and I'm on Clojure 1.10.1), here's what clojure is telling me...

Execution error (IllegalAccessError) at com.foo.Foo$Bar/getSerializedSize (Bar.java:7881).
class com.foo.Foo$Bar tried to access private field com.google.protobuf.AbstractMessage.memoizedSize (com.foo.Foo$Bar and com.google.protobuf.AbstractMessage are in unnamed module of loader 'app')

It's been stumping me all Sunday 😞

I'm even considering just writing a tiny kotlin/java library/jar that does that for me, but I thought that clojure/java interop would come to my rescue 🙂

tried to access private…..

Yes, but it works perfectly in Kotlin, zero illegal access issues

so what does the Kotlin/Java code look like?

and I would hazard a guess that if I did the same in plain old java, it would be the same, after all, .toByteArray is a valid, legal method on the protobuf object, not hidden.

i.e.,

morning

@dharrigan at a guess, clojure is attempting to reflectively access the field, and kotlin is linking - there are some JVM bugs around reflective access not implementing precisely the same access semantics as linking. i have encountered similar before, and the only solutions were either [a] a compiled proxy or [b] dynamically removing the access restriction if your security manager permits it

@mccraigmccraig that's very useful information! thank you 🙂

I have done this as well (set! *warn-on-reflection* true) and not seeing any warnings

does that help?

dunno @dharrigan, it's been a while

i was just looking for the issue in the 3rd party java library where i encountered the problem, but i can't even remember which lib it was in ...

There are days @dharrigan than the ease of writing some Java to do all the nasty stuff you’re trying to interop and shimming in you Clojure code just wins.

I’ve written stuff around the google apis in Clojure and they’re a pain.

All depends on the assumption that others will be able to maintain it all going forward.

> I’ve written stuff around the google apis in Clojure and they’re a pain.

fixed that for you

true

(also, hi! 411 I'm a Sunday roast >< )

(also... I know Google has a huge reputation, but has there been anything Googley that is not a pain / works well released recently?)

(...I guess a lot of people are happy with go, so maybe that?)

Let me think: BigQuery Java API, an utter pain….. GoogleAds Java API, an utter pain….. nope can’t think of anything else.

<-- not happy with golang

the biggest upside with golang imo is that there is only one way of doing things generally, and golang doesnt give you very much rope to hang yourself with

it’s hard (er) to write poor code in golang than say, in any proper OOP language

@jasonbell Yes, I think so to. Clojure interop gets you there most of the time, but sometimes, a bit of java/kotlin to ease things isn't bad

I'm doing protobuf stuff and man, the sooner we move to a more reasonable dataformat the better

what is the problem with Protobuf @dharrigan (sorry if I missed this earlier on) and what you prefer instead?

(besides your little interop problem you are having)

@lady3janepl GKE is very nice (mostly)

Certainly a lot better than doing it yourself and better than AWS/Azure's version

Internally, we're discussing using other formats such as messagepack, cap'n proto (by the original other of protobu (https://capnproto.org/)), CBOR (since we deal with IoT devices and it's RFC spec'ed), Flatbuffers etc..

We find protobuf a bit clunky for our needs and other formats can encode more efficiently thesedays

We're also still consdiering doing everything (internally) as JSON, since it's ubiquitous.

and most importantly, easy to debug

🙂

surely you mean simple? 😉

and that 🙂

😉

Morning 🙂 New Clojure In article with the team at AppsFlyer is finally live. I hope you enjoy reading. Please share some retweet love if you can ❤️ https://twitter.com/yogidevbear/status/1145647419989925888

for those with a passing interest, @alexmiller helped me solve my problem with the interop with protobuf

> I had a rogue import of a protobuf library all the way from rebel readline which included the cljs libraries which included the closure compiler which included a version of protobuf that I am not using!

this solved it

:rebel {:extra-deps {com.bhauman/rebel-readline {:mvn/version "RELEASE" :exclusions [org.clojure/google-closure-library
                                                                                       rewrite-cljs/rewrite-cljs]}}

this tool helped a lot (Alex's recommendation)

so, there you go. classpath issue. 🙂

.

aws is always a dns issue

jvm is always a classpath issue

c++ is always LD_LIBRARY_PATH

I was wondering if people could help me with some naming. 😄

I'm outputting things to charts, csv, and excel files atm. For each thing I need to do some manipulation to get the data into the right shape

for excel and csv, the shapes are the same. It is a vector of vectors where the first vector is the header

for the charts it varies a lot more, there are different ways of pivoting the data to create grouped or stacked bar charts. I'm moving on to some multi line charts and heatmaps soon that will have even yet more shapes.

I've been struggling quite a bit with good names (and looking at some d3 examples where things like x0 and x1 are used for domain and subdomains on charts, I can see that even Mike Bostock is too)

anyone got any good ways of thinking about these output, but not domain specific manipulations?

sometimes I can fake it by doing anonymous functions, but some of these I want to re-use or are too big to go in as an anon-fn

if you are at all interested in the concrete detail you can see it in a public repo here: https://github.com/MastodonC/witan.send.ingest