Fork me on GitHub
#clojure
<
2017-06-10
>
qqq00:06:23

what exactly does seasel/piggieback add? looking at github, they seem to do sxomething to nrepl, but I can't figure out what

noisesmith00:06:59

isn't piggieback the thing that lets a cljs repl take over your clj nrepl session?

noisesmith00:06:16

or was that weasel? maybe weasel uses piggieback?

noisesmith00:06:26

OK yeah - just refreshed my memory, piggieback is the part that latches onto an nrepl with your cljs, weasel is the part that makes the cljs eval in the browser

qqq00:06:12

so if I am just doingpure clj nrepl, I can ditch both piggieback & weasel from my dependencies ?

qqq01:06:35

@noisesmith : alright, removing the two works; thanks for the clarification

qqq01:06:41

Quoting: http://rundis.github.io/blog/2015/buddy_auth_part1.html

openssl genrsa -aes128 -out auth_privkey.pem 2048
You should probably use something stronger than -aes128. You’ll need to fiddle with your JVM, but might be worth it unless it’s important for you that your government agencies have access to decrypting your token signatures.
In this case, what algorithm should be used besides aes128 ?

eoliphant03:06:35

Hi, I have a macro question. I’m trying to have a macro generate some defrecords based on some values passed from an array I have something like the following

(defmacro testmac3
  [ss]
  `(do [email protected](map makerecsimple ss)))
Where makerecsimple returns (defrecord <RecordName> [a]) I can’t figure out how to get the syntax quoting working properly. In the code above ss is a list of strings. When I run it as is, I get a CompilerException java.lang.IllegalArgumentException: Don’t know how to create ISeq from: clojure.lang.Symbol, which appears to be due to the list not being evaluated. If I remove the @, and just unquote the list with ss, then the macro returns a list of (defrecord) statements but doesn’t evaluate them. any suggestions?

noisesmith04:06:17

so I assume makerecsimple is another macro?

noisesmith04:06:34

no, it can't be, or you'd get an error

qqq04:06:35

needs to be a function

qqq04:06:40

since it's passed to map

noisesmith04:06:34

works here

+user=> (defn makerecsimple [s] `(defrecord ~s [~'a]))
#'user/makerecsimple
+user=> (makerecsimple 'foo)
(clojure.core/defrecord foo [a])
+user=> (defmacro testmac3 [ss] `(do [email protected](map makerecsimple ss)))
#'user/testmac3
+user=> (macroexpand-1 '(testmac3 [foo bar baz]))
(do (clojure.core/defrecord foo [a]) (clojure.core/defrecord bar [a]) (clojure.core/defrecord baz [a]))
+user=> (testmac3 [foo bar baz])
user.baz
+user=> (foo. 1)
#user.foo{:a 1}
+user=> (bar. 1)
#user.bar{:a 1}
+user=> (baz. 1)
#user.baz{:a 1}

eoliphant12:06:19

noisesmith: ah crap I forgot to mention that yes it works fine with a ‘literal’ list or vector. The problem arises when i have something like

(def mylist ["A" "B" "C"])
(testmac mylist)
HEre’s the full code almost exactly identical to yours
(defn makerecsimple
  [tag]
  (let [name (symbol tag)]
    `(defrecord ~name [~'a])))

(defmacro testmac3
  [ss]
  `(do [email protected](map makerecsimple ss)))
So yes (testmac3 ["A" "B" "C"]) works perfectly but (testmac3 mylist) creates the IllegalArgumentException Don’t know how to create ISeq from: clojure.lang.Symbol error presumably as mylist is what’s being passed to map as opposed to the actual list it represents

noisesmith14:06:37

yes, this is an issue with macros, fundamentally

noisesmith14:06:51

they don't work on runtime data, they work with literal input

noisesmith04:06:26

@eoliphant I just did literally what you described, and the code works

didibus04:06:17

I'm facing a strange issue, it could be with my setup, but I'm trying to call clojure from Java using the Clojure.java.api. I get a var unbound exception, but the thing is, my code works if I run it from a unit test. But the way I have it setup it prod is that it runs from Spring, with MethodInvokingFactoryBean. Anyone knows if there would be something with having Spring call in to require a var that maybe its too early it the startup and something isn't working right?

noisesmith04:06:52

@didibus are you calling require before accessing the var?

noisesmith04:06:42

with unit testing I could easily imagine a scenario where some other code caused the ns to be required

didibus04:06:00

Ya I am. ITs quite strange. I have a java static that requires the namespace, and than gets the var and calls it.

didibus04:06:22

If I call it from a test, the method works. But from Spring, it does not

noisesmith04:06:53

what var comes back as unbound - does it come directly from the namespace you require?

didibus04:06:14

Its a method

noisesmith04:06:31

wait, a method?

didibus04:06:50

I mean function

didibus04:06:03

getting confused with the back and forth with Java lol

noisesmith04:06:28

so you resolve require, call it to require ns, resolve the function, call it

noisesmith04:06:36

sometimes the function comes back as unbound?

noisesmith04:06:00

is there possibly a code path that resolves the function before the require is done?

didibus04:06:25

I have a Java static method which gets require var from clojure.core, and then uses it to require a Clojure namespace. I then get a var from that namespace and when I invoke it, I get the unbound exception

noisesmith04:06:37

because require shouldn't return until all applicable vars are bound

didibus04:06:21

Hum, well, its not sometime, its always. When my app start, it does with spring, and spring calls my method which does the clojure interop.

noisesmith04:06:31

unless there's something bad like a defn inside a defn and the outer one didn't get called

didibus04:06:40

But in a unit test, a java unit test, f I call that same java method, it works

noisesmith04:06:12

what if you put a log at the top level of the file, at the bottom, to verify whether the message is logged before the unbound exception?

didibus04:06:35

The namespace you mean?

didibus04:06:40

the clojure namespace?

didibus04:06:47

Ya, I'll try that

noisesmith04:06:33

if the var just didn't exist, you'd get NPE - to get an unresolved var it would have to be declared but not have a value yet...

noisesmith04:06:45

which is super weird

didibus04:06:14

Ya, this normally happens if you forget to require the namespace, but I definitely am. I'm wondering if its something to do with Spring and the way its invoking that Java static method

didibus04:06:22

Which could interfere

noisesmith04:06:29

no no not requiring the namespace can't do that

noisesmith04:06:34

an unbound exception is weird

noisesmith04:06:44

you just get an NPE if the resolve fails

noisesmith04:06:59

resolving and getting unbound is another thing entirely, there's very few ways to get that state

didibus04:06:09

"java.lang.IllegalStateException: Attempting to call unbound fn"

noisesmith04:06:30

right, you need declare to get that

noisesmith04:06:42

or a defn that is inside another defn that didn't get called

noisesmith04:06:05

or a defn just got called before it finished running...

noisesmith04:06:35

or someone created a var using the clojure.lang.Var constructor but never gave it a value

noisesmith04:06:48

but you should be using resolve, so that wouldn't happen

noisesmith04:06:59

(if you aren't using resolve, fix that)

didibus04:06:06

Wait, resolve?

noisesmith04:06:06

oh never mind, with interop you call Clojure.var instead

noisesmith04:06:20

I misremembered

didibus04:06:18

Thats my clojure

didibus04:06:51

So if I have Spring call initClojure when m app starts, it gives the unbound exception. If I call initClojure from a test, it works.

noisesmith04:06:00

OK - reading the source for clojure.java.api.Clojure calling it's var method can give you an unbound var

noisesmith04:06:05

if the var doesn't exist yet

noisesmith04:06:19

wait, why are you using a one segment namespace?

noisesmith04:06:23

that is wrong and bad

noisesmith04:06:39

I don't know that this is causing your problem, but I also don't know that it isn't...

noisesmith04:06:54

single segment namespaces cause weird bugs

didibus04:06:07

I'm not, but its like work code, so I just replaced all identifiable things by test, sorry

didibus04:06:52

If it wasn't work, I would be doing 100% Clojure 😛

noisesmith04:06:35

another thing is you could try finding all-ns and seeing if your ns is in the result

noisesmith04:06:47

and calling ns-publics on your ns symbol to see if the function is actually in there

noisesmith04:06:11

these would be debugging tactics, since I really don't know what would be breaking that code

didibus04:06:32

I'll try your suggestions and keep digging, thanks for the help

seancorfield05:06:36

@didibus Given that you're calling this via Spring as a static function at startup, I wonder if this is because Clojure's runtime isn't being properly initialized at startup?

seancorfield05:06:42

When you can it via a test, Clojure's runtime will have been initialized prior to that...

didibus05:06:43

I call Class.forName("clojure.java.api.Clojure");

didibus05:06:10

A stackoverflow post from Alex Miller mentioned that's the new way to make sure the runtime is initialized

didibus05:06:54

I know that my build does AOT compilation, I wonder if this could be part of it. Is clojure.java.api compatible with AOT compilation?

seancorfield05:06:06

Yup, that rings a bell. I remember there being a special way to bootstrap Clojure.

noisesmith05:06:40

@didibus are you sure you need to aot compile? in my experience avoiding it tends to reduce a lot of complexity

seancorfield05:06:56

We certainly had some problems getting AOT-compiled Clojure loaded. We had to isolate the things that specifically needed AOT compilation and leave the rest as source.

seancorfield05:06:15

AOT compilation = bad in general.

didibus05:06:18

Hum, I'm not sure I need too. We've got custom build tools, and its just the way things integrate with it. I never looked into seeing if I could change that

seancorfield05:06:02

If you're AOT-compiling, you need to package that as JAR files specifically and ensure those are on your classpath directly (and you'll probably need to something specific to "load" each compiled class). Way more complex than just having the source on your classpath and loading it via the Clojure API...

didibus05:06:38

Prior to this, we'd always relied on gen-class for interop

seancorfield05:06:17

I'd say treat AOT-compilation as regular compiled (Java) code and interact with it via native interop. The Clojure API feels like it's designed to interact with compile-on-demand source code...

didibus05:06:19

Right, I could go back to that. I was trying out the Clojure API, as I felt it was simpler, but gen-class has worked for me perfectly in the past

didibus05:06:00

How would I not do AOT compilation? So, we use clojure to compile. We have an ant script that does it using Clojure. What would I do instead, I'd just copy my src to the build folder? Wouldn't I still need to package them inside a jar, I'm confused.

seancorfield05:06:55

Most Clojure code is packaged as source JARs.

seancorfield05:06:16

Why would you AOT anything?

didibus05:06:26

I guess gen-class needs AOT. But my question was more, if I dont compile clojure, I just need to package its src in a jar and put that on my classpath?

noisesmith05:06:15

usually I use lein uberjar for this

didibus05:06:48

I'll have to use ANT unfortunatly, to integrate with my company build infrastructure

didibus05:06:08

But ok, I find its almost too simple lol

didibus05:06:34

not even having to compile, the build step seems like it barely does anything

noisesmith05:06:51

yeah, it's just a packaging step

didibus05:06:27

One question, can you mix AOT compiled jars with non aot compiled? Say I depend on 2 libs, both are in jars on my classpath, one contains compiled clojure .class files, the other clojure source files. Is that ok?

seancorfield05:06:29

We don't AOT compile anything.

seancorfield06:06:08

We previously used Ant to call Leiningen to deal with Clojure. We've replaced all of that with Boot these days.

qqq06:06:37

When running a war, it is obvious where the resourdces directory comes from. When running directly in a repl, how dc I figure out (or better yet, set) where the resources directory is?

didibus06:06:53

I'm assuming at a minimum the main clojure namespace must be compiled, so that it can bootstrap the gen-class which has main on it?

didibus07:06:14

Thanks, I'll read it.

didibus07:06:53

BTW, problem solved. Nothing to do with Clojure, just deployment issues, somehow I kept getting an old clojure source that actually was missing the var inside of it

triss11:06:46

Is there a simple way to test if a float, Ratio or Integer is equal to 1?

triss11:06:06

(= 1 1.0) -> false

triss11:06:24

(= 1N 1.0) -> false

triss11:06:50

oh I see. We have ==

hoff12:06:59

is there a simple way to make a spec that only accepts one particular value? E.g. '(spec/def ::42 (spec/of 42))' only accepts the value 42 and has a generator just repeating 42 over and over

eoliphant12:06:25

Ok I got it to work in both cases by doing the following, but it just feels weird lol

(defmacro testmac3
  [ss]
  `(do [email protected](map makerecsimple (eval ss))))

pradyumna13:06:09

Any advise regarding selecting web server with NIO. At present I have used ring and compojure deployed as uberwar to Jetty running on Azure web app. However my other colleague swears by nodejs for its NIO and future scalability. Is NIO is really that crucial or you can configure Jetty to be as performance.

noisesmith14:06:38

node doesn't perform as well as java does... it absolutely needs to use nonblocking IO because it doesn't do threads

noisesmith14:06:07

but if you find you need a non blocking IO based ring server, you can switch to aleph

hoff14:06:53

I thought spec/unform was supposed to be the inverse of spec/conform. Then why does

(spec/def ::i (spec/coll-of (spec/or :i int?)))
(spec/unform ::i (spec/conform ::i [1]))
yield [[:i 1]]?

pradyumna14:06:02

I was reading on aleph and yada which looks promising. I have not found a way yet how to deploy to Azure we app. At present only Jetty or Tomcat.

noisesmith14:06:14

deployment with an uberjar consists of making the uberjar then making sure you have java installed on the server

noisesmith14:06:34

you don't actually need one of those java container server things

noisesmith14:06:31

(I actually use the apache commons daemon, the unix port is called jsvc but there's also a windows version - but it's not neccessary)

pradyumna14:06:03

That's interesting. Usually I was worried about keeping it running. But as I rethink, Azure worker service is already there to manage the java process. I should give it a try.

noisesmith14:06:05

right, this is what jsvc does

noisesmith14:06:25

it hooks into whatever native features exist to keep a process running / restart on reboot etc.

pradyumna14:06:32

Anyway, I am really curious is it really important? So far I have not faced much issue. But again I lack much experience about these.

noisesmith15:06:06

with higher loads async io handlers can help, but they are neither neccessary nor sufficient

noisesmith15:06:15

async is a really big deal when you only have on thread though

noisesmith15:06:21

but we don't have that problem

potetm15:06:08

Curious about @seancorfield's experience here. > AOT compilation = bad in general.

potetm15:06:41

I've used it for one smallish standalone app and it worked exactly as expected.

noisesmith15:06:06

it has very few benefits, and introduces sources of bugs that don't otherwise exist

noisesmith15:06:52

even with jsvc, I used a shim approach to minimize AOT to a single namespace (since jsvc needs to be able to find a class on startup)

potetm16:06:24

Speed of of startup was the reason I used it. I'm genuinely not sure what the speedup is though. Wasn't exactly rigorous about the decision to use it

tokenvolt16:06:20

What is the approach to background processing in Clojure? Is there something like Sidekiq from Ruby world?

noisesmith16:06:02

typically you can just use a ScheduledThreadPoolExecutor

noisesmith16:06:22

or one of the various wrappers like quartzite

noisesmith16:06:12

I use a stpe - it's an easy thing to use via interop

tokenvolt16:06:41

I've looked at quartzite and if I understood correctly it doesn't provide persistent queue

noisesmith16:06:55

you want persistent over restarts of the vm?

noisesmith16:06:06

if so, use a db for the queue

tokenvolt16:06:37

yes, I need persistence between restarts

mpenet16:06:54

durable-queue is great for this kind of things

tokenvolt16:06:55

so I need to implement it on my own?

noisesmith16:06:02

I have a table that represents the scheduled tasks, including last run time etc.

noisesmith16:06:16

there might be something decent, I haven't really looked, sorry

mpenet16:06:24

Well depends on the use case

noisesmith16:06:20

yeah - for me disk backed doesn't cut it because we have N servers and any one of the machines might disappear at any moment

noisesmith16:06:26

so I need a db

noisesmith16:06:38

@tokenvolt oh, my mistake - quartzite which I mentioned above does support a queue that persists across restarts of the vm

noisesmith16:06:45

so I would just go with that

tokenvolt16:06:59

hm, good news, I'll check it

potetm16:06:11

@mpenet What is this durable queue of which you speak?

potetm16:06:58

You don't happen to know of somebody building an SQS-compatible dev service on that?^ We use elasticmq and it currently drops everything across restarts, which is a bit of a hassle for us.

noisesmith17:06:05

kafka is a durable queue - it has a configurable backlog you can rewind and read through

potetm17:06:28

Yeah, really just want something that re-implements SQS semantics.

potetm17:06:38

Don't really care what's backing it I guess 🙂

mpenet17:06:56

Yes that lib from factual/ztellman. Super userful/reliable, highly recommended if that fits your use case

qqq17:06:36

(deftask build-jar []
  (comp 
   (aot :all true) 
   (uber)
   (jar :main 'z.app
        :file "uberjar.jar")
   (target :dir #{"output"})))
when I do "java uberjar.jar" does it try to run z.app.clj/-main ? if not, what does it try to run ?

seancorfield17:06:08

You mean java -jar output/uberjar.jar? Yes, I believe it will run z.app/-main. We generally omit :main and specify -m when we run the JAR file. That runs the main Clojure entry point and tells it to then run the -main in the namespace specified via -m — which lets us have multiple -main functions (in different namespaces) in a single uber-jar file (convenient for bundling a whole series of cron tasks into one JAR, for example).

noisesmith17:06:43

you can also do that with java -cp uber.jar clojure.main -m ..., overriding the jar's main

qqq17:06:20

The XY problem is that (1) I am deploying to elastic beanstalk and (2) it's not calling my main function, and (3) I haven't figured out how to get the logs in a useful manner.

qqq17:06:41

java -jar output/uberjar.jar Error: Could not find or load main class z.app

qqq17:06:52

do I need an AOT flag ; no wait, I have (aot :all true) already

qqq18:06:57

figured out the problem; apparently I need a (:gen-class) on the main 'entry point for clj' file

mobileink18:06:22

if you aot everything why would you need gen-class? asking 'cause i don't know.

qqq18:06:57

I don't know eiehter.

qqq18:06:09

but adding a (;gen-class) made it go from not working to working.

noisesmith18:06:27

even with aot, you don't create a .main method on your class, so yeah, you need gen-class

noisesmith18:06:39

or to point the server at clojure, and tell clojure which ns to run

noisesmith18:06:57

(the above java -cp snippet)

noisesmith18:06:25

I guess there are other ways to make something with a main method? reify or proxy or whatever

mobileink18:06:46

i thought (defn -main ... ) would take care of that. no?

qqq18:06:08

I really wish this was slightly better documented, I still don't understand -main :gen-class or :aot fully

noisesmith18:06:15

defn creates a class that implements IFn with an invoke method

noisesmith18:06:22

it doesn't create a method of that name

noisesmith18:06:59

aot just runs the clojure compiler and writes class files to disk, it doesn't create classes from namespaces or methods from functions

mobileink18:06:37

so, nothing special about defn -main? (i'm going by stuff like http://ben.vandgrift.com/2013/03/13/clojure-hello-world.html.)

noisesmith18:06:04

not without gen-class no

noisesmith18:06:21

gen-class is the thingy that makes classes out of namespaces and methods out of defns

mobileink18:06:00

oops, never mind. read further in the article and saw gen-class. what's the icon for embarrassment?

mobileink18:06:37

(i must say i'm with @qqq, this needs mo' betta docs.)

noisesmith18:06:55

well, doc for defn leads to fn which links to this https://clojure.org/reference/special_forms#fn which is pretty clear about what it creates

mobileink18:06:29

sure, we can also read the source, haha. 😉

noisesmith18:06:08

so what's your alternative?

mobileink18:06:39

noisesmith: someday, maybe, i hope, i'll find the time to put my money where my mouth is and write some good docs regarding gen-class, aot, etc. until then i can only kvetch and hope somebody else will beat me to it. :)

mobileink18:06:15

noisesmith: i'm fine with reading the source, but a good high level explanation would have saved me mountains of time and effort, that's all.

mobileink19:06:29

yeah. the prob (as i see it) is that is already takes a huge amount of knowledge to even begin to grok that page. i think we need sth better, esp. wrt gen-class, which routinely gets beaten up for no goid reason i can see.

mobileink19:06:33

look at it this way: neither @qqq nor i just fell off the tomato truck. yet we both have had to put in an inordinate amount of work to understand aot, gen-class, etc. that's a problem, imo.

noisesmith19:06:05

it's beaten up for good reasons - relying on aot compilation (which gen-class requires) leads to all sorts of avoidable problems

mobileink19:06:50

oy vey. gen-class is no different than any other construct - use it correctly and it solves your problems; use it incorrectly, not so much. there is absolutely nothing wrong with gen-class per se, akaics.

mobileink19:06:56

that's a documentation problem.

mobileink19:06:46

maybe it would be more accurate to say: aot must be used corrctly.

mobileink19:06:19

i personally have never had a problem with aot/gen-class, maybe that's why i don't understsnd the complaints.

noisesmith19:06:35

among other problems it leads to errors caused by cached byte code on disk, code that is broken and you won't know it's bad even after a namespace reload; you only see the breakage after a full clean of class files from disk and process restart; those cache based errors will cause nonsense stack traces in irrelevant places that aren't related to the root problem. Aot compiling code leads to bizarre Foo cannot be cast to Foo errors if you use protocols, records, deftype.

noisesmith19:06:39

that's off the top of my head

noisesmith19:06:11

There's a few ways to address this. Learn all the counter intuitive ways it breaks and what the misleading messages actually mean; treating clojure as a non-reloadable language and do a clean of disk cache and restart of the repl after each change; or simply not using aot. The last option is a lot simpler.

qqq19:06:41

@noisesmith : I don't think the argument is "make gen-class easy to use so everyone will use it", but rather: if you want to deploy on GAE or ElasticBeanStalk or Tomcat, you WILL have to gen-class at some point -- and if it's better documented, alot of pain could be saved there. 🙂

mobileink19:06:04

are we talking :aot all? in my use cases i only needed to aot the file that contains my gen-class. i've not run into those kinds of problems.

mobileink19:06:33

gen-class needs aot, why would one need to aot anything else?

mobileink19:06:36

fwiw i never put implementation code in a gen-class ns. everything gets delegated to the :impl-ns. so e.g. "cached byte code on disk" problems never occur.

mobileink20:06:18

to me what gen-class is all about is delegation.

noisesmith20:06:49

aot is recursive

noisesmith20:06:03

aot in a specific class always aot compiles every class it requires

mobileink20:06:11

@qqq more generally: if your runtime involves a container which will be asked to load something, and it is designed to look for that something on disk, then u need gen-class. but you do not need to implement your stuff in gen-class.

mobileink20:06:03

@noisesmith no. that is easily overcome. i don't have the code at hand, alas, but it is easy to aot a single file without aoting its deps.

noisesmith20:06:37

by using thedeps indirectly via runtime require and runtime resolve yes

eoliphant18:06:28

Hi, I have a question about (deftype). I’m sending some stuff off to Kafka and using the franzy lib. For testing, I was initially using the simple string serializers which worked fine, but I’m having trouble using the ones that come with the library, there’s for instance a KeywordSerializer that’s clojure-aware The issue is that in a snippet like the following

(let [pc {:bootstrap.servers (-> env :kafka :bootstrap.servers)
            :retries           1
            :batch.size        16384
            :         1
            :buffer.memory     33554432}
        key-serializer (serializers/edn-serializer)
        value-serializer (serializers/string-serializer)
string-serializer works fine, edn-serializer does not, it fails with a class not found for the EdnSerializer class which is defined by franzy. In looking at the source and playing with this, the string-serializer and others that map directy to those provided by kafka work fine. The EdnSerializer is defined by the library
(deftype EdnSerializer [opts]
  Serializer
  (configure [_ _ _])
  (serialize [_ _ data]
    ;;TODO: process + inject more options? better defaults via configure or opts?
    ;;no reason to close bos, but we do so to keep clean
    (with-open [bos (ByteArrayOutputStream. 1024)]
      (with-open [w (if opts ( bos opts) ( bos))]
        (binding [*print-length* false
                  *out* w]
          (pr data)))
      ;;death to efficiency, but easiest way without writing something low-level to encode a stream directly into Kafka
      (.toByteArray bos)))
  (close [_]))

(defn edn-serializer
  (^EdnSerializer [] (edn-serializer nil))
  (^EdnSerializer [opts]
   (EdnSerializer. opts)))
What I think is happening is that the (deftype EdnSerializer …) isn’t being evaluated when I require the namespace that contains it, but I’m not sure what I’m supposed to do to address this. edn-serializer is accessible in my namespace but the EdnSerializer type blows up when it’s executed

weavejester19:06:19

What’s the error?

eoliphant19:06:29

ah nvm. It worked fine in the repl, so kept digging. looks like I need to have it :aot in the project config

eoliphant19:06:40

though based on other stuff I’ve seen here, wondering if that’s going to cause other issues

noisesmith19:06:29

if you need aot you need aot, it makes things more complicated and introduces new kinds of errors but it can be managed...

eoliphant19:06:09

So is this something that can be avoided via doing something differntly in the library, etc? Like I said, after I posted, tried it in the repl, worked fine. Adding :aot :all to my dev config to see if that works

eoliphant19:06:08

ugh, yeah so according to HLS’ note here, I need to add :aot :all but I’m not sure where… would it go in my lein :profiles. Just tried it in my :dev config and now other stuff is blowing up NoClassDefFoundError etc, exceptions when I do a lein run

urbank19:06:55

In clojure (or functional programming in general), how do you solve problems where in an imperative language you'd set a flag? If in the process of doing something to a collection, you discover that something will have to be done after? With an imperative language I'd set a flag and put an if statement after the loop.

eoliphant20:06:46

Do you have an example? @urbank typically the functional approach is say creating a new collection based on a functional transformation of the original. So something like newcoll = checksomething(oldcoll)

eoliphant20:06:39

where newcoll is what you’d want to do something with based on your ‘flag’ in the imperative world

urbank22:06:19

eoliphant: Hmm... sorry, but I'm not quite clear on what checksomething does/returns. For some reason I'm having difficulty interpreting this sentence: > where newcoll is what you’d want to do something with based on your ‘flag’ in the imperative world As for an example: I'm running reduce over some state to do a transformation, but if this transformation results in let's say nilfor one of the key value pairs, I have to do some other action after the reduce. I suppose one way would be to reduce over a tuple, where one element is the state I'm changing, and the other holds the equivalent of a flag. But that's perhaps too literal a mapping from the imperative version. Or maybe not.

eoliphant22:06:33

sorry if it wasn’t clear. checksomething could be a filter, etc. hmm, well a reduction is going from many to one, so there’d be no nil for a specific entry per-se. your ‘reducer’ would just need to handle these cases based on your requrements. You might want to say ‘prefilter’ the collection to look for these ‘bad’ cases, then apply the reduce to the ‘good’ collection, etc In the simplest case, I guess you just ignore the bad ones. If you actually need to take action, then you can use a filtered ‘bad’ collection and act on that.

noisesmith22:06:20

Having a tuple, or even a hash full of state values in an accumulator in reduce is petty normal

eoliphant22:06:38

true enough the ‘one’ can be some aggregate

noisesmith22:06:27

in fact, it's so common that transduce even expects your function to support a arity 1, which gets your actual final state out of your accumulator

urbank10:06:27

A prefilter works a lot of the time, but I think I have a case where a tuple/hashmap in the reduce would be a better solution

qqq20:06:36

when I run "java -jar foobar.jar" where does the resource "/" correspond to? is it the root of the di4rectory when I unzio foobar.jar ?

noisesmith20:06:35

it's the classpath

noisesmith20:06:19

which follows the structure of the jar, yes, but also can be manipulated by arguments to the java command or by environment settings etc.

qqq21:06:53

(wrap-resource ... "public") ;; works in repl, works in jar (wrap-resource ... "/public/") ;; works in repl, fails in jar

noisesmith21:06:51

oh - I think I misremembered about leading slashes - I don't know what they would indicate actually

eoliphant22:06:15

leading slash roots it to the ‘top’ of the classpath. “public” alone is relative to the location where it’s being referenced, generally speaking. Now having said that functions/methods may do some other interpretation internally

noisesmith22:06:21

@eoliphant

+user=> (io/resource "clojure/core.clj")
#object[java.net.URL 0x115667d "jar:file:/home/justin/bin/bench!/clojure/core.clj"]
+user=> (io/resource "/clojure/core.clj")
nil

noisesmith22:06:46

with a classpath, what is "the location it is being referenced" ?

eoliphant22:06:52

i always use the leading slash to avoid any ambiguity. I beieve that the first would get resolved to <current package/namespace>/clojure/core.clj

noisesmith22:06:14

namespaces aren't parts of the classpath

noisesmith22:06:09

can you show me an example of http://clojure.java.io/resource resolving something that starts with / ?

eoliphant22:06:17

fair enough, I’m newer to clojure lol. But from a ‘pure’ JVM/Java perspective, thats the usual behavior

eoliphant22:06:41

and I thought io/resource was just a thin wrapper around

eoliphant22:06:54

the relatve Java/JVM api

noisesmith22:06:06

it is - I'm asking an honest question

noisesmith22:06:21

because clearly a couple things I thought I knew about this are wrong

eoliphant22:06:11

ok actualy ugh.. I’m looking at some code I just wrote.. I created a clojure wrapper that would parse Avro .idl files

eoliphant22:06:44

(defn parse-protocol-idl
  "Parse an avro protocol inputstream"
  [protocol-res]
  (with-open [stream (io/input-stream (io/resource protocol-res))]
    (let [idl (Idl. stream)
          protocol (.CompilationUnit idl)]
But I called it like this:
(avroprot/parse-protocol-idl "BOOT-INF/classes/authorization.avdl")
and it works fine. Ah.. hmm, if it calls getSystemResource then it would start at the classloader root, and that should work..

noisesmith22:06:18

that's not a leading slash though

eoliphant22:06:59

yeah but by attempting to use the Context or System classloader then it works

eoliphant22:06:12

just looking at the source

eoliphant22:06:25

(defn ^URL resource
  "Returns the URL for a named resource. Use the context class loader
   if no loader is specified."
  {:added "1.2"}
  ([n] (resource n (.getContextClassLoader (Thread/currentThread))))
  ([n ^ClassLoader loader] (.getResource loader n)))

eoliphant22:06:52

ok that makes more sense now, by using the contextclassloader, that ‘roots’ it though im not sure how to explain qqq’s issue based on what we see here

noisesmith22:06:57

so is there any context where using a leading / on the resource would find what I'm looking for?

eoliphant22:06:54

i need to play with it, but based on the way io/resource is defined, not sure why leading slash wouldn’t work

noisesmith22:06:54

that's funny because I can't seem to get a string with a leading slash to find anything

eoliphant22:06:00

leading slash means it’s absolute, but the contextclassloader unless there’s some custom behavior or something should also ‘start’ at the root so an unquaified path should start there as well

eoliphant22:06:08

yeah i’m going to modifiy the code i have

eoliphant22:06:11

to see if it works

noisesmith22:06:13

I need to learn a lot more about classloaders and such...

eoliphant22:06:21

they are a PITA lol

eoliphant22:06:38

especially once you start doing servlet containers, etc

noisesmith22:06:09

modularity's so often a mess isn't it

eoliphant22:06:34

OSGI etc was a clunky attempt to clean it up, now the modules, etc in Java 9 are supposed to help. But it’s apparently a huge point of contention among the spec leads

noisesmith22:06:02

yeah, java 9 looks like it'll be a huge problem for clojure

noisesmith22:06:22

and even more so the tooling, leiningen in particular

eoliphant22:06:11

yeah i think in the long run it will be helpful, but the impact to existing tooling etc may be a pain

alexmiller22:06:39

@noisesmith I don't see Java 9 as a huge problem at all

alexmiller22:06:23

There are some minor things to smooth over but they don't seem like a big deal