java

RAJKUMAR 2023-08-11T01:27:48.417589Z

Hi I'm facing some issues with respect to java interop

RAJKUMAR 2023-08-11T01:28:01.352099Z

I've namespace myorg.helpers.fs-input-stream -Its definition is

RAJKUMAR 2023-08-11T01:28:09.787859Z

(ns myorg.helpers.fs-input-stream)
(gen-class
 :name "myorg.helpers.FsInputStream"
 :extends java.io.ByteArrayInputStream
 :implements [org.apache.hadoop.fs.Seekable org.apache.hadoop.fs.PositionedReadable]
 :main false
 :exposes {buf  {:get getBuf}
           pos  {:get getPos}
           mark {:get getMark}}
 :init-impl-ns false)

RAJKUMAR 2023-08-11T01:28:18.535829Z

I've another namespace myorg.plugin.orc-output which uses the class generated by above namespace. Its definition is -

RAJKUMAR 2023-08-11T21:06:26.899039Z

Thanks @hiredman

RAJKUMAR 2023-08-11T21:06:40.076769Z

but I'm stuck more on this

RAJKUMAR 2023-08-11T21:07:24.143269Z

how can I aot compile this in deps.edn?

RAJKUMAR 2023-08-11T21:08:07.492839Z

do I have to add (require 'myorg.plugins.helper.FsInputStream) ?

2023-08-11T21:08:11.897289Z

no

2023-08-11T21:08:38.272749Z

in general aot compilation is a separate step your build tool is usually responsible for

2023-08-11T21:10:00.998579Z

clj and deps.edn are not really a complete build tool, but they work together with tools.build (https://clojure.org/guides/tools_build) #tools-build

2023-08-11T21:11:14.128439Z

aot (a head of time) compilation in clojure means generating jvm bytecode classfiles and writing them to disk, instead of just keeping it in memory (what the clojure compiler normally does with bytecode)

2023-08-11T21:15:18.760539Z

if you are new to clojure, gen-class is pretty terrible to work with, in general if you can it is much better to use reify/deftype/defrecord then proxy, then if you must gen-class (a lot of people would prefer to write some stub java than use gen-class)

RAJKUMAR 2023-08-11T21:30:08.086949Z

ohh okay

RAJKUMAR 2023-08-11T21:30:14.233679Z

Thanks hiredman

RAJKUMAR 2023-08-11T01:29:58.489969Z

(ns myorg.plugin.orc-output
  (:import
   [myorg.helpers.fs-input-stream FsInputStream]
   [ URL]
   [ File]
   [java.nio.file Path]
   org.apache.hadoop.conf.Configuration
   [org.apache.orc OrcFile]
   [org.apache.hadoop.fs FileSystem FileSystem$Statistics FSDataInputStream FSDataOutputStream]))

(set! *warn-on-reflection* true)

(defn to-path
  [x]
  {:post (instance? Path %)}
  (cond
    (instance? URL x) (Path. (.toURI ^URL x))
    (instance? File x) (Path. (.getPath ^File x))
    (string? x)                (Path. ^String x)
    (instance? Path x)         x))

(def dummy-path (to-path "dummy"))

(def orc-header-length (count OrcFile/MAGIC))

(defn input-stream-filesystem [buffered-bytes]
  (proxy [FileSystem] []
    (open [_]
      (FSDataInputStream. (FsInputStream. buffered-bytes)))))

(defn create-orc-reader [buffered-bytes]
  (OrcFile/createReader dummy-path (doto (OrcFile/readerOptions (Configuration.))
                                     (.maxLength (count buffered-bytes))
                                     (.filesystem (input-stream-filesystem buffered-bytes)))))

RAJKUMAR 2023-08-11T01:30:08.200759Z

Then I've test for namespace myorg.plugin.orc-output called myorg.plugin.orc-output-test. It has very basic test

RAJKUMAR 2023-08-11T01:30:16.484199Z

(ns myorg.plugin.orc-output-test
  (:require [clojure.test :refer :all])
  (:require [myorg.plugin.orc-output :refer :all]))

(deftest orc-create
  (is (= 1 1)))

RAJKUMAR 2023-08-11T01:30:23.570239Z

When I run the test I get error

RAJKUMAR 2023-08-11T01:30:45.894999Z

Syntax error (ClassNotFoundException) compiling at (myorg/plugin/orc_output.clj:1:1). myorg.helpers.fs-input-stream.FsInputStream Full report at: /var/folders/b_/_t677thx43n6pd8l8x0c1qjr0000gp/T/clojure-13616919898936783952.edn

RAJKUMAR 2023-08-11T01:30:59.975919Z

The project is in https://github.com/rajcspsg/gen-class-error-demo as well.

RAJKUMAR 2023-08-11T01:31:17.125369Z

How can I fix this?

2023-08-11T01:36:17.736779Z

You are not aot compiling the namespace with the gen-class in it

2023-08-11T01:38:03.430939Z

You also are not requiring the namespace that defines the class from the namespace that uses it

2023-08-11T01:39:12.146699Z

You also are not using the correct name for the class, you need to replace all - with _

2023-08-11T01:43:05.440399Z

The docstring for gen-class helpfully tells you it does nothing when "not compiling", but unhelpfully clojure is always compiled to bytecode, so in this case "compiling" means aot compiling