Fork me on GitHub
#java
<
2023-08-11
>
RAJKUMAR01:08:48

Hi I'm facing some issues with respect to java interop

RAJKUMAR01:08:01

I've namespace myorg.helpers.fs-input-stream -Its definition is

RAJKUMAR01:08:09

(ns myorg.helpers.fs-input-stream)
(gen-class
 :name "myorg.helpers.FsInputStream"
 :extends java.io.ByteArrayInputStream
 :implements [org.apache.hadoop.fs.Seekable org.apache.hadoop.fs.PositionedReadable]
 :main false
 :exposes {buf  {:get getBuf}
           pos  {:get getPos}
           mark {:get getMark}}
 :init-impl-ns false)

RAJKUMAR01:08:18

I've another namespace myorg.plugin.orc-output which uses the class generated by above namespace. Its definition is -

RAJKUMAR01:08:58

(ns myorg.plugin.orc-output
  (:import
   [myorg.helpers.fs-input-stream FsInputStream]
   [ URL]
   [ File]
   [java.nio.file Path]
   org.apache.hadoop.conf.Configuration
   [org.apache.orc OrcFile]
   [org.apache.hadoop.fs FileSystem FileSystem$Statistics FSDataInputStream FSDataOutputStream]))

(set! *warn-on-reflection* true)

(defn to-path
  [x]
  {:post (instance? Path %)}
  (cond
    (instance? URL x) (Path. (.toURI ^URL x))
    (instance? File x) (Path. (.getPath ^File x))
    (string? x)                (Path. ^String x)
    (instance? Path x)         x))

(def dummy-path (to-path "dummy"))

(def orc-header-length (count OrcFile/MAGIC))

(defn input-stream-filesystem [buffered-bytes]
  (proxy [FileSystem] []
    (open [_]
      (FSDataInputStream. (FsInputStream. buffered-bytes)))))

(defn create-orc-reader [buffered-bytes]
  (OrcFile/createReader dummy-path (doto (OrcFile/readerOptions (Configuration.))
                                     (.maxLength (count buffered-bytes))
                                     (.filesystem (input-stream-filesystem buffered-bytes)))))

RAJKUMAR01:08:08

Then I've test for namespace myorg.plugin.orc-output called myorg.plugin.orc-output-test. It has very basic test

RAJKUMAR01:08:16

(ns myorg.plugin.orc-output-test
  (:require [clojure.test :refer :all])
  (:require [myorg.plugin.orc-output :refer :all]))

(deftest orc-create
  (is (= 1 1)))

RAJKUMAR01:08:23

When I run the test I get error

RAJKUMAR01:08:45

Syntax error (ClassNotFoundException) compiling at (myorg/plugin/orc_output.clj:1:1). myorg.helpers.fs-input-stream.FsInputStream Full report at: /var/folders/b_/_t677thx43n6pd8l8x0c1qjr0000gp/T/clojure-13616919898936783952.edn

RAJKUMAR01:08:17

How can I fix this?

hiredman01:08:17

You are not aot compiling the namespace with the gen-class in it

hiredman01:08:03

You also are not requiring the namespace that defines the class from the namespace that uses it

hiredman01:08:12

You also are not using the correct name for the class, you need to replace all - with _

hiredman01:08:05

The docstring for gen-class helpfully tells you it does nothing when "not compiling", but unhelpfully clojure is always compiled to bytecode, so in this case "compiling" means aot compiling

RAJKUMAR21:08:40

but I'm stuck more on this

RAJKUMAR21:08:24

how can I aot compile this in deps.edn?

RAJKUMAR21:08:07

do I have to add (require 'myorg.plugins.helper.FsInputStream) ?

hiredman21:08:38

in general aot compilation is a separate step your build tool is usually responsible for

hiredman21:08:00

clj and deps.edn are not really a complete build tool, but they work together with tools.build (https://clojure.org/guides/tools_build) #C02B5GHQWP4

hiredman21:08:14

aot (a head of time) compilation in clojure means generating jvm bytecode classfiles and writing them to disk, instead of just keeping it in memory (what the clojure compiler normally does with bytecode)

hiredman21:08:18

if you are new to clojure, gen-class is pretty terrible to work with, in general if you can it is much better to use reify/deftype/defrecord then proxy, then if you must gen-class (a lot of people would prefer to write some stub java than use gen-class)

RAJKUMAR21:08:14

Thanks hiredman