This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-08-30
Channels
- # announcements (1)
- # babashka (28)
- # beginners (68)
- # cider (12)
- # clj-kondo (5)
- # clojure (37)
- # clojure-spec (4)
- # core-async (5)
- # cursive (3)
- # data-science (1)
- # datomic (70)
- # fulcro (18)
- # graalvm (1)
- # joker (1)
- # luminus (2)
- # malli (11)
- # off-topic (21)
- # shadow-cljs (3)
- # spacemacs (11)
- # test-check (3)
- # vrac (8)
Howdy y'all! I have a little clojure web application and i'm trying to embed jasper reports. When i run my application with lein run or lein repl it works! When i create an uberjar and run it it throws a null pointer exception on some reports
java.lang.NullPointerException
at java.base/java.io.File.<init>(File.java:278)
at net.sf.jasperreports.engine.JasperFillManager.fill(JasperFillManager.java:317)
at net.sf.jasperreports.engine.JasperFillManager.fillReport(JasperFillManager.java:850)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:167)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:332)
at vgs.api.report$get_pdf$fn__17267.invoke(report.clj:80)
and a jruntime exception on others
at net.sf.jasperreports.engine.util.JRQueryExecuterUtils getExecuterFactory "JRQueryExecuterUtils.java" 103
at net.sf.jasperreports.engine.fill.JRFillDataset createQueryDatasource "JRFillDataset.java" 1251
at net.sf.jasperreports.engine.fill.JRFillDataset initDatasource "JRFillDataset.java" 726
at net.sf.jasperreports.engine.fill.BaseReportFiller setParameters "BaseReportFiller.java" 457
at net.sf.jasperreports.engine.fill.JRBaseFiller fill "JRBaseFiller.java" 584
at net.sf.jasperreports.engine.fill.BaseReportFiller fill "BaseReportFiller.java" 414
at net.sf.jasperreports.engine.fill.JRFiller fill "JRFiller.java" 120
at net.sf.jasperreports.engine.fill.JRFiller fill "JRFiller.java" 103
at net.sf.jasperreports.engine.JasperFillManager fill "JasperFillManager.java" 530
at net.sf.jasperreports.engine.JasperFillManager fillReport "JasperFillManager.java" 954
at jdk.internal.reflect.NativeMethodAccessorImpl invoke0 "NativeMethodAccessorImpl.java" -2
at jdk.internal.reflect.NativeMethodAccessorImpl invoke "NativeMethodAccessorImpl.java" 62
at jdk.internal.reflect.DelegatingMethodAccessorImpl invoke "DelegatingMethodAccessorImpl.java" 43
at java.lang.reflect.Method invoke "Method.java" 566
at clojure.lang.Reflector invokeMatchingMethod "Reflector.java" 167
at clojure.lang.Reflector invokeStaticMethod "Reflector.java" 332
at vgs.api.report$get_pdf$fn__17267 invoke "report.clj" 80
but not a single one works from the jar'ed app.
Any suggestions on troubleshooting the problem.they're both failing on get-pdf
in report.clj (on line 80). where are the arguments for that call coming from?
my guess is that it's trying to find a file, but the environment changed in way that makes it not find the file it's looking for. what's the working directly when you run from the repl? is it the same working directory when you try to run the jar?
how do get the pwd in clojure? trouble translating the answer here https://stackoverflow.com/questions/320542/how-to-get-the-path-of-a-running-jar-file
return new File(MyClass.class.getProtectionDomain().getCodeSource().getLocation()
.toURI()).getPath();
I was referring to whatever directory you're running the jar from
usually, the file path is relative to the current working directory (ie. pwd
), which may be different than the location of the jar fiile
Line 80
(try (JasperFillManager/fillReport
(get @jasper-reports (keyword reportname))
(map->java params)
(jdbc/get-connection db))
(catch JRRuntimeException jrrx (vgerror :get-pdf|fill (seq (.getStackTrace jrrx))) (JasperPrint.))
(catch JRException jrex (vgerror :get-pdf|fill (seq (.getStackTrace jrex))) (JasperPrint.)))
the first parameter is a JasperPrint which is loaded in memorywhat about params
?
has report parameters which i convert to a java borrowed the function
(defn map->java
"Convert Clojure hash-map to Java HashMap object"
[data-map]
(let [hashmap-reducer (fn [m [k v]] (doto m (.put (name k) v)))]
(reduce hashmap-reducer (java.util.HashMap.) data-map)))
right, but what is the value of params
before it's passed to fillReport
?
ok, then I would check what the value of (get @jasper-reports (keyword reportname))
it's either one of the 3 arguments to fillReport
that is different or the environment is somehow different
and are you sure that (get @jasper-reports (keyword reportname))
is non-nill when run in a jar?
positive. I added some printlns to count the reports loaded in memory but let me try that call on its own and see.
So I wrote this program to parse some multiple GB json log files.
(ns loghell.main
(:require [cheshire.core :as cs])
(:require [ :as io])
(:require [clojure.pprint :as pp]))
(def log13h "ut5-2020-08-28_13H.log")
(defn parsed [line]
(cs/parse-string line true)) ; for keywords instead of strings
(defn pred [i]
"AND of predicates to filter the logs"
(and
(not= true (get-in i [:jsException :isJoi])) ;ignore joi validations
(= (:level i) 50))) ;get only errors
(defn needed [i]
"what properties/keys are needed from the log"
(select-keys i [:msg :name :time]))
(defn loginv [pred? needed log]
"investigate logs by doing an AND of predicates"
(with-open [r (io/reader log)]
(let [s (line-seq r)]
(-> needed
(pmap
(filter #(pred? %)
(pmap parsed s)))
doall))))
(time (loginv pred needed log13h))
I’m a bit unsure about doall - it puts the whole seq in memory; is there a way to avoid it?
I think dorun
is what you’re looking for instead of doall
— it does not retain the whole seq in memory.
I tried dorun, but I guess I don’t understand how it works. I think I need to process everything in one pass because it doesn’t retain the head of the seq
You should use dorun only in the very end, to “force” the lazy sequence. @ULE3UT8Q5
I tried to just replace doall with dorun but instead of the lazyseq with maps I get a nil
so it makes me think I’m using it wrong somehow 🙂
Actually, I read your code once again, and yeah, my mistake — you need the resulting sequence
Since you’re filtering, I don’t think the whole source file is retained memory, don’t worry about that
is it a map vs filter difference? anyways I will try to do a parse, filter and select keys in one function and see how it goes
I mean to do all these actions per line
Actually, can you try not keeping the seq in the local variable s
, but running it directly in-place?
I believe right now the sequence s cannot be garbage-collected because you keep the head, so actually the whole file is in-memory
Hmm running it directly in place doesn't seem to change anything
I tried visualvm on the process but the heap doesn't grow at all when I run the function
@ULE3UT8Q5 I think in this case, Clojure’s compiler “sees” that you don’t use the local variable after some point, and nulls it behind the scene (which allows the GC to clean it up): https://clojure.org/reference/compilation#_locals_clearing So I was wrong again, and Clojure compiler is smarter than I thought :)
also is it the final seq after the pmap and filter, or does it load the whole multi-GB file in memory?
the time I got is 5 seconds for a 1 gb file, but I want to parse and analyse even longer ones
I’m also concerned that I’m doing pmap, filter and pmap again, should I use transducers so I can do it in one pass?
If I am not mistaken, nesting pmaps is superfluous. Also, I believe pmaps are no magic bullets, and they will help mostly only if individual item processing is rather heavy.
for what it is worth these pmaps give a 5s processing time compared to the 8s maps 🙂
That’s certainly more than I expected :)
this file has lines that are pretty big, with nested json 🙂
Hello, I know there is a better way to do this: [`(.getYear timestamp) (.getMonthValue timestamp) (.getDayOfMonth timestamp)]`
If they were clojure fns and not java methods you could use juxt easily; also depends on what you want to do with the result - maybe you can leverage date formatter instead
You'd have to do something like this since java methods aren't first class Clojure fns:
((juxt #(.getYear %) #(.getMonth %) #(.getDay %)) (java.util.Date.))
;;=> [120 7 0]
Interesting
Thanks a bunch
No problem; as I said, there might be a better approach depending on how you are going to use this representation
Just wanted to get a date (year month and day)
I want to remove the repetition. Do you mean bean the first func call and then thread it?
You could bean, then juxt
Although I think what you were doing is probably the most efficient answer
Thanks @alexmiller the closest I came to was this: (select-keys (bean date) [:year :month :day])
That versus ((juxt #(.getYear %) #(.getMonth %) #(.getDay %)) date)
I am not sure how to combine those two as you suggested - but these seem to work. I am just curious how to combine them. I am assuming that date
is an instance of (java.util.Date.)
The repetition is triggering my OCD
Or rather DCD
Ooooh, this is scary, I come up with a bean-ish question and this channel is discussing beans! Anyway, here I am messing with datomic seriously for the first time and using the low-level API I am retrieving #datoms such as #datom[83562883711053 10 :orange-ish 13194139533330 true]
and I want to take it apart. bean
complains about being unable to process keywords or sth unhelpful. I put up a hail mary and (:a <datom>)
works. Yay, But (:t d)
does not. So what are the other keys (I wondered before I found the java doc). :tx
I had been able to guess, but the active value? OK, I sorted it out thx to the doc, but why did not bean
work, and what else could I have used. Thx!
@hiskennyness Out of curiosity, did you try calling fails with Don't know how to create ISeq from: datomic.core.db.Datumkeys
on that #datom
object?
(my assumption would be that it is probably a record, so I would expect wrong... I guess it's a Java type or a keys
to work...)deftype
You can use org.clojure/java.data
to get the datoms as hash maps:
$ clj -Sdeps '{:deps {com.datomic/dev-local {:mvn/version "0.9.195"} org.clojure/java.data {:mvn/version "RELEASE"}}}'
...
user=> (def y (last (d/datoms db {:index :eavt})))
;; in my case that is:
#datom[83562883711058 10 :yellow 13194139533320 true]
user=> (require '[clojure.java.data :as j])
nil
user=> (j/from-java-shallow y {:exceptions :omit})
{:v :yellow, :e 83562883711058, :tx 13194139533320, :t 8, :p 19, :a 10, :assertion true}
user=>
^ Just in case that helps @hiskennynessCool, thx, I will give it a try! Yeah, I had tried keys
when I saw to my surprise things like :a
work. Surprised again when it did not. 🙂 And that error you show is reminiscent of what bean
failed with.
keys
and bean
both expect certain things about maps that Datomic datoms don't provide. The keyword lookup protocol is ILookup, and that one is supported. I'd refer to the Datomic docs for what expectations they provide (or ask in #datomic)
Looks like a fairly limited set of keys are supported:
user=> (:v y)
:yellow
user=> (:e y)
83562883711058
user=> (:tx y)
13194139533320
user=> (:t y)
Execution error (IllegalArgumentException) at datomic.core.db.Datum/valAt (db.clj:374).
No matching clause: :t
user=> (:p y)
Execution error (IllegalArgumentException) at datomic.core.db.Datum/valAt (db.clj:374).
No matching clause: :p
user=> (:a y)
10
user=> (:assertion y)
Execution error (IllegalArgumentException) at datomic.core.db.Datum/valAt (db.clj:374).
No matching clause: :assertion
user=>
Which makes me curious what T
and P
are in a Datom?The rest are all obvious.
Things not behaving like hash maps is partly what I added from-java-shallow
(and from-java-deep
) to java.data
with the options to control what happens if a "getter" doesn't actually behave like a getter. You can also omit "properties" that line up with get*
methods that "do bad things" (are mutating / do I/O / etc).
Thx all! It felt like a beginner Java interop question, but I see after playing with from-java-shallow
that (as you say, @alexmiller) datoms march to the beat of a different drummer. I'll go see about these "expectations" ... 🙂