Hello! Can anyone recommend a library for accessing the google api, especially the google drive part?

@blmstrm Google’s Java API doesn’t do the trick? https://developers.google.com/api-client-library/java/

There seem to be a few Clojure wrappers as well. https://github.com/SparkFund/google-apps-clj https://github.com/mikeflynn/gmail-clj https://github.com/ianbarber/clj-gapi

@twashing Thanks! I’ve checked out google-apps-clj already. I’ll look over the other options.

Cool beans 👍:skin-tone-5:

I'm not looking for anything complica5ted. It doesn't even have to be as performant as bitcask / leveldb / rocksdb. Is there a simple persistent kv store for clojure in pure clojure/java ?

Basically, I want something better than spit + flush on every change.

There should be some type of append only log that appends all my changes, and there should be a GC / compactation phase.

@ymilky As a follow on. Using just the raw java KafaConsumer, I also get an emptry result. https://pastebin.com/QWY8KjJ1 This is the output from my dockerized kafka output. https://pastebin.com/MGv65iV4 Any idea what I’m missing? I’m sure sure it’s something simple. But I’ve been looking at this too long.

…

@ymilky Nevermind. I figured this out for the raw Java version. Turns out that calling .poll once doesn’t guarantee that data will be returned. http://grokbase.com/t/kafka/users/155m9916sv/consumer-poll-returns-no-records-unless-called-more-than-once-why#20150520qpwf3n0xdt20k0kg3fx83h1dmg

So I just keep calling it until the expected records come out.

(defn bind-all-but-first [f & args]
          #(apply f % args))

(defmacro tae [e]               '(thrown? java.lang.AssertionError ~e))

        (macroexpand-1
         '(tae (assert false)))
        

        (ct/deftest
          (ct/is                        (tae (assert false))))

why does this not work?

I'm trying to write tae as a shorthand for (thrown? java.lang.AssertionError)

@qqq Wrong kind of quote. Use the backtikc

@rauh: thanks, turns out there's a second error, I need a name symbol hname after the deftest 🙂

Hi All, http://http-kit.org seems to no longer work unfortunately 😞

anyone who knows how to contact one of the maintainers?

(list? (1 2 3 a))` => false / is ther esomething wrong with my clojure?

@thomas: possibly file issue on https://github.com/http-kit/http-kit

@qqq Looks like you're using a back-tick instead of a quote in front of the list. (list? `(1 2 3 a)) ;;=> false (list? '(1 2 3 a)) ;;=> true

I think back-tick expands to a (seq (concat ...))

@petterik: indeed, the backtick is on purpose

using type, I seem to be getting a Cons from the backtick

@qqq list? is almost always wrong, you probably want seq?

is the search on https://clojuredocs.org/quickref broken at the moment?

@ztellman Hi Zach, I’m having a look at your automat library but noticed the repo has been flagged as archived and read-only. Does that mean something in particular like: “don’t use it”, or “i’m not gonna work on it anymore” ?

Ah, it is in Chrome when using https. https://github.com/zk/clojuredocs/issues/183

When using ->, (doto clojure.pprint/pprint) is a nice debugging trick. Does anyone know a simple way to do that when using ->>?

The best I've come up with is (#(clojure.pprint/pprint %))

This is not a joke. Completely serious: 1. I have a Clojure function, which when executed, returns a String. 2. This String contains raw java source code. Which we can dump out via (spit "foo.java" foo-str) 3. Now, after I make a system call to javac, we get foo.class 4. I need somewah to load foo.class + call the function in foo.class === I need help with step 4.

(The XY problem is writing fast numeric code in clojure, but using a clojure DSL instead of actually writing java.)

that will already work if foo.class is in your classpath

user=> A/foo
CompilerException java.lang.RuntimeException: No such namespace: A, compiling:(NO_SOURCE_PATH:0:0)
user=> A/foo
"asd"

This is black magic to me. I don't even know what I don't know.

1. When you recompile A.java, does Java auto reload the new A.class ?

2. Is it safe / unsafe to add cwd to the java classpath?

3. Can I modify the classpath at run time?

4. What is the correct way to setu pall this up?

5. Should I create a directory ~/dynamically-generated-java-classes/ , add that to class path, and stuff all runtime generatee *.class files there ?

1 doesn’t make sense

java doesn’t do any auto reloading

the jvm has a classpath, when you initialize a class, the classloader will scan that classpath

if the classpath includes filesystem directories other than jar files, it will look for the directory state at run time, not at the jvm load time

1a. I see. My lack of JVM basics is showing. Let me rephrase -- how can I force reload after I recompile A.java ?

I would reconsider your approach of generating java as a string & loading it in memory

My basic use case is a follows: 1. I'm in eamcs. I modify some clojure AST. 2. I run some function, it does "clojure dsl ast -> java source string" 3. I want to compile this source string, get a class, have the JVM reload this class, and call a function from this class.

Okay, I have no idea wtf I'm doing. What do you mean by "loading it in memory" and what do you recommend? I'm not attached to writing out a *.java file at all.

>(The XY problem is writing fast numeric code in clojure, but using a clojure DSL instead of actually writing java.)

if you’re set on using clojure instead of going straight to java

I would have your DSL either supported by some java classes, or emit JVM bytecode for whatever clojure can’t compile

Hi!

I have a DSL in clojure taht spits out .c / .cu files for C/Cuda. I want to extend it to also spit out *.java files.

You are recommending I generate JVM bytecode directly?

yes

How would I properly use the Socket REPL? I tried to pass -Dclojure.repl=localhost:5555 as JVM arguments (in Cursive), but that does not seem to start the REPL.

I'm okay with that, as the only data I deal with are float [] and int; and I only use basic arith expressions. What library do you recommend for "clojure generating jvm bytecode" @bronsa

https://github.com/jgpc42/insn

x86 doesn't scare me; how bad can jvm bytecode be?

jvm bytecode is easy

and that library allows writing it symbolically which is quite good

@bronsa: looking at it now; while you're here -- at some point, I'm going to run into a brick wall due to my lack of JVM bytecode knowledge; what reference would you recommend for studying JVM bytecode ?

i used the jvm spec

why not use primitive math or some static java functions and combine them in clojure with proper type hints?

^

that’s highly likely to be the best solution

@bronsa: I have no idea what "JVM spec" is -- are you referring to https://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf ?

yes

@bronsa: cool; thanks! anything else I should know?

primitive math converts your (+ 1 2 3) into java static function calls via macros, belive me it's fast enough

@tsulej: 1. I've tried primitive math before -- gone as far as implementing fast histograms in it, -- ended up not liking it.

@tsulej: I also already have a "DSL" which outputs .c and .cuda files -- I wnat to extend it to output jvm bytecode too.

@qqq A hybrid approach may not be bad, don't emit Clojure math, emit calls to clojure.lang.Number/*

That's going to be pretty much as fast as Java.

(clojure.lang.Number/add 1 (clojure.lang.Number/add 2 3))

ad.2. ok, clear, but imho it could be path of pain

I know that (+ 1 2 3) should be almost exactly the same, but my experience trying to performance tune such code leads me think otherwise

@tbaldridge not for floats

ah right

ad.1. curious why?

@tsulej: I ended up developing these complicated macros that wraps around clojure's loop to simulate for loops

looks like you went too far 😉 I do all calculations with clojure structures just backed by primitive math instead of clojure numbers. And for me it's as fast as java. I make some generative/math art stuff.

@tbaldridge FWIW it should be exactly identical btw

user=> ((clojure.lang.Compiler/isInline #'+ 3) 1 2 3)
(. clojure.lang.Numbers (add (. clojure.lang.Numbers (add 1 2)) 3))

uhm, except letf vs right fold

but I don’t believe that makes a difference

with all due respect, as both of you know far more abut clojure / jvm than I do, I find these (+ 1 2 3) examples silly

do something like implement taking the convolution of an image, and I'll be impressed

the code is the same.

like (n*n) float image by (k*k) float kernel => (n-k+1) * (n-k+1) float image

no, it's not clear to me at all how you do the looping part

Show me the Clojure code you generated, and we can probably help better.

It's much more likely that you forgot a typehint somewhere than that Clojure's compiler is producing ugly code.

@tbaldridge: "show me the clojure code" <-- is this referring to me?

@qqq for convolution I would go for gpu or use https://docs.oracle.com/javase/7/docs/api/java/awt/image/ConvolveOp.html

Now I have gone the route of using Java code for this, but that was mostly do to me being unhappy with the type hints I had to track.

@tsulej : right, for convolution, I would use cuda ... but it "convolution" was the simplest examples I could come up with // actual image op is something else, but has similar looping structure

@tbaldridge: are you assuming that these ops can be easily expressed via map/reduce ?

I'm not sure if we are making the same assumptions.

not at all

let me throw out this challenge: for something like: https://rosettacode.org/wiki/Canny_edge_detector I think java is far better than clojure; here, "java" includes "clojure code tht generates string representing java code"

loop and recur work pretty well, but yes, if you need something like goto and break/continue, then go the bytecode route

the JVM is a stack interpreter so going from an expression based DSL to java bytecode is rather trivial.

@qqq I'll take a look at this. Nice, not trivial problem. Thx for pointing at this. As a side note, in my opinion generating java code is worst possible solution. I would just write java code parts and call from Clojure.

the other idea is to call OpenCV lib (via JavaCV)

for such problems

I'm really kind of stuck doing "my dsl -> c, cuda, jvm" output since I need the same algorithm to run in both CLJS and CLJ -- CLJ for training side, CLJS for execution directly on tablet; so basically, besides BLAS, everything elese needs to be "hand written"

@tsulej, @bronsa , @tbaldridge: thanks for the insightful discussion; I'm going to go play around with insn and enjoy the 800+ pages of the jvm spec 🙂

FYI a bit of familiarity with JVM bytecode will also help you figure out performance stuff like this on your own. Next time you have a Q just write 2 functions and look at the decompiled bytecode to figure out which one is fastest ;)

probably counting instructions would do; as the faster one = one calling jvm instructs directly; slower one = the one that have to jump around + make dispatches since it doesn'T assume anything about types of args

not always

The most important JVM optimization is method inlining.

Things like static method calls, or calls to a single type of object through a interface may be almost completely optimized away.

@tbaldridge: lol, I'm "not even in the ballpark of being wrong"

@bronsa: insn is amazing, I just got the basic example working, to sanity check, in

(def class-data
  {:name 'my.pkg.Adder
   :fields [{:flags #{:public :static}, :name "VALUE", :type :long, :value 42}]
   :methods [{:flags #{:public}, :name "add", :desc [:long :long]
              :emit [[:getstatic :this "VALUE" :long]
                     [:lload 1]
                     [:ladd]
                     [:lreturn]]}]})

I don’t know what jvm asm intrs means

:emit is a list of symbolic JVM bytecodes

right, by jvm asm instrs I meant to say "jvm's equiv of assembly instructions", which I believe is what "JVM bytecodes" are

sorta, yes

Is there a section of https://docs.oracle.com/javase/specs/jvms/se7/html/index.html which documents "function calling convention? in C, this would be things like how the arguments are passed via the stack frame" -- I think the three things . need are: 1. basic instructions: all documented https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5 2. understand how to get arguments / return result 3. understand how to read/write to parts of float []

lol, am I reading https://stackoverflow.com/questions/41693637/whats-the-calling-convention-for-the-java-code-in-linux-platform correctly? it seems to state that JVM spec does NOT define how args are passed for func calls

they’re talking at the ASM level

all you care about is the bytecode level

Getting the following error when trying to send email with [com.draines/postal "2.0.2"]

javax.mail.AuthenticationFailedException: 534-5.7.14 <https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=<key> Please log in via your web browser and
534-5.7.14 then try again.
534-5.7.14  Learn more at
534 5.7.14   i39sm10355859qte.19 - gsmtp

(defn send-email
  "function to send an email"
  [to subject body-text attachment-file-path]
  (send-message {:host ""
                 :user <username>
                 :pass <password>
                 :ssl true
                 :port 465
                 }
                {:from <username>
                 :to to
                 :subject subject
                 :body [{:type "text/html"
                         :content body-text}
                        {:type :attachment
                         :content (java.io.File. attachment-file-path)}]}))

not sure what I'm doing wrong

Hi all - is there a recommended oauth2 impl in clojure ?

Hey, all, I'm trying to get a :gen-class section in a ns form working. My function returns an array of Integers. I'm trying to add this to the methods section: ^:static [foo [Integer] "[Ljava.lang.Integer;"] to no avail. I was trying to follow the example from the bottom of https://groups.google.com/forum/#!topic/clojure/YRcvWPdUY9Y to get it working. I keep getting this problem clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:. Any ideas?

@markbastian You'll have to share some details of your code and the actual error for us to be able to help...

Hi all, could you please recommend a good library for PGP encryption? I need to send encrypted data to an external server via SFTP given a public key. Would also very much appreciate your help if I can get some examples for pgp data encryption.

if I were doing something like that I'd go for the most stable well known and well used java lib available and write the interop myself

encryption is not a thing I want to have to rely on an intermediary for updates to

Here's a complete example:

(ns sandbox.itoa
  (:gen-class
    :name sandbox.ITOA
    :methods [^:static [intToArrayOfInts [Integer] "[Ljava.lang.Integer;"]]))

(defn -intToArrayOfInts [id]
  (into-array [id]))

The basic issue is "How do I define a type hint for an array of items in a gen-class method?"

Hmm, based on my bit of web searching, that syntax looks like it is supposed to work (and perhaps used to work)...

...yup, confirmed that works on Clojure 1.8.0.

Maybe @alexmiller can shed some light?

I had an example of that I wrote up for someone else here, but it’s fallen off the end of the message buffer

Yeah, I just saw both of these guys: * https://stackoverflow.com/questions/2398478/declare-array-as-return-type-in-gen-class-method-declaration-in-clojure * https://stackoverflow.com/questions/3318671/clojures-gen-class-and-double-arrays?noredirect=1&amp;lq=1 Perhaps there's a new and improved way. For now, I'll revert to 1.8.

I expect there's some sort of Class/forName dance that you can do to satisfy the new-in-1.9 ns spec...

I presume the issue here is the "[Ljava.lang.Integer;" ?

yes

spec is written to take a symbol there

Can I do this in a ns-declaration (symbol "[Ljava.lang.Integer;")?

no

try ints

Basically I want to put this into my return type in gen-class:

(type (into-array [1 2 3]))
=> [Ljava.lang.Long;

longs then

altho i'm not sure if genclass supports that syntax

I tried that (I think), let me try again.

give it a try

well the spec won’t fail with it

but it’s not obvious without digging whether that does the intended thing

I think this is probably a bug in the spec and ticket would be welcome

https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L184 is the line in question

simple-symbol? should be ::class-ident

I'd be happy to file it. Thanks for the help!

https://dev.clojure.org/jira/browse/CLJ for spec bugs

https://dev.clojure.org/jira/browse/CLJ-2314

done

greetings! how can I "introspect" list of all required namespaces in some/current ns?

and, is (:require [foo.bar :as baz]) – not the same as just (:require [foo.bar]) from the "required ns'es code is loaded" pov?

having weird issue with onyx, where peers ns requires only job ns, w/o explicitly requiring namespaces job ns uses, and I get "no class def found" exception. So either I don't fully understand how :require'ing works in chains of namespaces requiring each other, or I might have not eval/test/etc. the right way.