Fork me on GitHub
Garrett Hopper01:02:00

How do I configure USE_CONTEXT_CLASSLOADER to be false? I've got a Java file loading my Clojure code, however it fails to find clojure/core__init.class. Doing some debugging, it looks like if that variable was false, it would use a classloader that is able to find the class.




it has a function var() that returns you a var

Garrett Hopper02:02:30

How would I use that exactly? Clojure.var("clojure.core", "use-context-classloader")? It doesn't look like that has the bindRoot function that RT.USE_CONTEXT_CLASSLOADER has. (The latter doesn't work, because it tried to load clojure/core__init.class before it's changed.)


forget about context classloader


IFn RR = Clojure.var("clojure.core", "requiring-resolve");


IFn code = RR.invoke("your.ns/entrypoint"))




that is the official way to invoke clojure from java

Garrett Hopper01:02:26

Do I just set the variable, or is there a setting/function somewhere?

paul a01:02:35

i have two methods that share the same options map argument, including the same defaults - i would like to extract the defaults, like so, but this doesn't work at all:

(defn some-fn-0 [{:keys [foo bar] :or {foo :you bar :me}}]
  (prn "foo" foo "bar" bar))
(defn some-fn-1 [another-arg {:keys [foo bar] :or {foo :you bar :me}}]
  (prn "foo" foo "bar" bar))

;; i want to refactor the above code like so:

(def default-options {foo :you bar :me})
(defn some-fn-0 [{:keys [foo bar] :or default-options}]
  (prn "foo" foo "bar" bar))
(defn some-fn-1 [another-arg {:keys [foo bar] :or default-options}]
  (prn "foo" foo "bar" bar))
i tried :or ~default-options as a shot in the dark, but that didn't work either. i'd appreciate any advice.

Alex Miller (Clojure team)03:02:32

the destructuring options map is a literal syntax

Alex Miller (Clojure team)03:02:47

another way to approach this is to not do it in destructuring

Alex Miller (Clojure team)03:02:09

but instead merge the incoming map into a default map

Alex Miller (Clojure team)03:02:12

(def default-options {:foo :you :bar :me})
(defn some-fn-0 [m]
  (let [{:keys [foo bar]} (merge default-options m)]
    (prn "foo" foo "bar" bar)))


Curious, is there such a thing as uberjaring without compiling .clj files at all? (neither mine or transitive) I'm mostly intrested in uberjar as means of bundling dependencies + resources, but otherwise I don't particularly want AOT. I'd favor a slower deployment over running into AOT oddities


I think I've run into AOT issues in production only once (at most), but anyway they are an occasional pain point in #clojure so I think I'd be happy to kill the mere possibility of these issues


With a previous client I regularly built non-aot'd uberjars, mostly because they were being deployed as storm topologies and pomtemkin (a dependency) was causing some odd behaviour post-aot. One option is to include a java shim in your source that requires your namespace and calls the main fn. Set that java shim as the main class in the manifest, e.g. :manifest {"Main-Class" ""}


Main looks a bit like


import clojure.lang.IFn;

public class Main {
    public static void main(String[] args) {
        IFn require = Clojure.var("clojure.core", "require");

        Clojure.var("your.system", "start!").invoke();

Alex Miller (Clojure team)13:02:09

Yes, you can make a non aot uberjar


With which tool? I gave a quick shot to :aot [] in Lein, no luck

Alex Miller (Clojure team)13:02:12

I’m just talking conceptually, they are independent things

Alex Miller (Clojure team)13:02:29

In clj land, I think depstar will do that

👀 4

> Note that depstar does no AOT compilation by default Indeed! Thanks for the pointer


Same for pack :)

✌️ 4
paul a13:02:12

yeah, it did occur to me to merge the default-options map inside of the function body. thanks alex!


hello everyone


Hi alex, how u doing?


People, I'm having an issue here regarding aws lambda converting a map to json here's my method signature: [^{:static true} [getValue [java.util.Map] clojure.lang.IPersistentMap]] When i'm handling an exception, i return a map that look's like this: {::a/category ::a/incorrect ::a/message error-msg :exception-data response} This :exception-data key is a map response from clj-http, and it's type is: clojure.lang.PersistentHashMap Here is the error message from aws lambda:

An error occurred during JSON serialization of response: java.lang.RuntimeException
java.lang.RuntimeException: An error occurred during JSON serialization of response
Caused by: .UncheckedIOException: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.UnsupportedOperationException) (through reference chain: clojure.lang.PersistentArrayMap[":exception-data"]->clojure.lang.PersistentHashMap[":http-client"]->org.apache.http.impl.client.InternalHttpClient["params"])
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.UnsupportedOperationException) (through reference chain: clojure.lang.PersistentArrayMap[":exception-data"]->clojure.lang.PersistentHashMap[":http-client"]->org.apache.http.impl.client.InternalHttpClient["params"])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(
	at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(
Caused by: java.lang.UnsupportedOperationException
	at org.apache.http.impl.client.InternalHttpClient.getParams(
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(
	at java.lang.reflect.Method.invoke(
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(
	... 11 more


and btw @borkdude i'm using clj-kondo and it helps me a lot! Thanks for this 🙂

❤️ 12

wait! i think it might be this that come's inside :exception-data :http-client #object[org.apache.http.impl.client.InternalHttpClient 0x55917c8d "org.apache.http.impl.client.InternalHttpClient@55917c8d"]


i think this jackson lib that aws uses is unable to parse this into JSON


yes! exactly this


just dissoc'ed from the map and it worked


Has anyone done fake HTTP basic auth for clj-http-fake


@grounded_sage basic auth is just a http header, how would you like to fake it/


I just want to return a 403 status when the headers aren’t right.


Though I can’t seem to see how to use this lib properly.


Even when I provide it incorrect form-params it also matches against anything.


Looks like it ends up in body but it’s unclear how to catch that without doing it in the return function


I’m just doing it in the return function instead


There's #:: {:_/key "value"} syntax. But I can't find anything about the bogus _ namespace in the documentation. Is it undocumented for some reason? If not, could someone please give me a link?


Oh, never mind - I'm just blind.


keywords not being garbage collected. I'm running some tests on visualvm but it seems it's being garbage collected fine. the code I'm running:


keywords are now stored in a weak table as weak refs so they can be gc'd but still cached


symbols can't be cached since two of the "same" symbol could have different metadata stored


cool. I was afraid of using the usual json parsers that keywordize stuff from wire because of that thread 😅


if you're accepting uncontrolled input, don't keywordify keys


otherwise you open yourself up to an easy DOS attack


how's that?


Unbounded keys being stored in that hashtable noisesmith linked


but if they're gc'd, what's the issue?


at least from the script I used to test it seemed fine. I was generating random 10-char strings and keywordizing it


I think the problem is if you have 10,000 char strings


if it was genuinely untrusted input, you could exhaust memory pretty quickly by just generating a lot of really long random strings and sending them over to be keywordised ... that table is going to grow quickly, unless it's doing something clever


hmm, gotcha. I'll test this out later


It's not just the memory. It's a hash table. Meaning, if you know the hash function and you control the input, you can make it much more slower than it usually is.


that's a separate issue though


you can grow it past the size of usability -- depends on Xmx size


A separate issue, but still a DOS, no?


also, keywordizing json blindly still isn't always the best idea - json allows keys that are nonsensical for keywords, and I'd only keywordize if you need to eg. cross reference with map literals


yeah. keywordizing json only plays a role here to work with clojure.spec/spec-tools, since we don't have an easy way of defining a keys spec for string keys. I'd rather have the keys as strings btw, and was considering using plumatic schema for API contracts, but content negotiation (like reitit does) is good too and it gets a bit more messy if I go the string-as-key route


How do I pass in a function to a macro someone else has defined? Then have it evaluated?


Do I have to wrap it in another macro?


macros are transforming a source form to a new source form


if they pass the name of a function in as a symbol, you can return the same symbol in your output, that's about all you can do


I am using clj-http-fake and wanted to conditionally return statuses based on parameters passed. But I can’t seem to do figure out how I can pass a function into the with-fake-routes macro


hmm, something like (defn foo [f] (with-fake-routes {some-route f} ....)) should actually work - maybe you're doing something weirder?


the macro gets f, and returns f, and the function makes sure f is the right value in the resulting code


It’s an asynchronous clj-http request


So perhaps I need to do a .get on it since it returns a future?


For now I just have a deftest for each case but it could all be modelled as conditionals inside of with-fake-routes return function


yeah. keywordizing json only plays a role here to work with clojure.spec/spec-tools, since we don't have an easy way of defining a keys spec for string keys. I'd rather have the keys as strings btw, and was considering using plumatic schema for API contracts, but content negotiation (like reitit does) is good too and it gets a bit more messy if I go the string-as-key route


How can i use (gen-class) to override the toString method of java ?


This doesnt seem to work:

(ns example
   :name         test.override
   :methods      [ ^{Override {}} [toString [] String]]
   :state        state
   :init         init
   :constructors {[String] []}))

(defn -init
  [[] (atom name_)])

(defn -toString [this]
  (deref (.state this)))


I get a Caused by: java.lang.ClassFormatError: Duplicate method name "toString" with signature "()Ljava.lang.String;" in class file test/override


@caio @ghadi btw this is a helper function I use to look at clojure internals - it can usually take you straight to the gh page for an object from clojure.core

(defn ghdoc
  "looks up some java in the clojure repo, based on the `javadoc` function"
  (let [target (if (class? class-or-object)
                 (class class-or-object))
        target-str (string/replace (pr-str target) #"\." "/")
        url (str "" target-str ".java")]
    (browse/browse-url url)))
where browse is and string is clojure.string

💯 8
Chris Lester20:02:01

I'm playing with the idea of reading feature flags in and defining them as state with mount ... and am running into: "Don't know how to create ISeq from: clojure.lang.Symbol class java.lang.IllegalArgumentException" when calling (defflags [{:flag :test :value "val"}]). I'm not familiar enough with macro's yet to see where I've screwed this up. Any help appreciated 🙂


@activeghost this problem could come up if you called deffflags with the name of a vector instead of a vector literal


you could use (doseq [{flag# :flag value# :value} ~flagsmap] ....)` except you need to generate the defstate calls


might be easier with another macro that takes a keyword and generates a defstate call

Chris Lester20:02:13

Thanks @noisesmith will try your suggestion and composing macros and see what I can get. It is a vector name, unquoting it in the for expression gives me an attempt to call unbound fn (which at least is different :)).

Chris Lester21:02:36

Just to follow up .. this macro works for defs. I don't actually need to define these as states since they aren't stateful .. just compile time data , and since attempting to do so with `(defstate (symbol (name flag)) :start value) doesn't work just moving over to defs ... defstate isn't seeing the :start keyword (have unquoted it, quoted it, etc. ...still doesn't register).


the original you showed didn't provide :start at all


a keyword is self-evaluating

user=> (= :a `:a ':a (keyword "a"))



user=> (symbol :a)
(symbol (name x)) usually isn't needed since symbol knows what to do with keywords already


Only since 1.10 right? Or do I misremember that?


that's true. IMHO there isn't much reason to write new code for older versions, but that is a concern

Chris Lester21:02:13

Yes, had left that out in the original (important, but realized that later). Thx, simplifies it.


I'm trying to enable Jetty's gzip compression by copying Large responses don't contain Content-Encoding: gzip in the response headers. I'm using ring-jetty-adapter 1.8.0. Any idea if something has changed?


is the content type in the list and is the content greater than the min size you set?


Ah, had to enable debug logs. This appears to be it:

2020-02-06 02:12:01 qtp1212997867-539 [org.eclipse.jetty.server.handler.gzip.GzipHandler] DEBUG - GzipHandler@4cdc18be{STARTED,min=100,inflate=-1} excluded by method Request(POST //localhost:8880/api/v1/eql)@3e5cfa48


Is there a reason to exclude certain HTTP methods from gzip? Off the top of my head, it seems ok to enable across all methods.


it's predicated on HTTP content-type, not HTTP method


Jetty's default excluded POST.


After adding (.setIncludedMethods (into-array ["GET" "POST"])) in, it worked as expected.


Same. Seems like I should just enable it for all methods unless I'm missing something.


from the vieiwpoint of the REST spec, GET and maybe POST make sense because they return arbitrary data, the others either have minimal result or aren't normally used for documents


Jetty also lets you set (.setMinGzipSize) to whatever you want. You cannot set it below the gzip breakeven point. For those methods that return small responses, gzip would be skipped.


I’m calling a function inside a deftest that is in another namespace and does a swap on an atom. I am using that to keep some state.. is there any way to redfine that and have it scoped lexically or is it trapped holding onto the one on the other namespace?


you should be able to with-redefs the atom in the namespace


It doesn’t appear to be working


(deftest confirm-entity
  (with-redefs [failed-requests (atom 0)]
    (let [response (fn [id] (with-global-fake-routes {{:address (:confirm-changed-entity url-list)}
                                                      (fn [request]
                                                        (let [form-params (slurp (:body request))]
                                                            (= "Identifier=8" form-params) {:status 200}
                                                            (not (= "Identifier=8" form-params)) {:status 500})))}
                              (confirm-entity! id)))]
      (is (= :success (response 8)))
      (is (and (= :failed (response 5)) (= 1 @failed-requests))))))


(defn confirm-entity!
  (http/post (:confirm-changed-entity url-list)
             {:async? true
              ;:oncancel #(println "Confirming entity " id " was cancelled")
              :accept :json
              :basic-auth auth
              :form-params {"Identifier" id}}
             (fn [_response] :success)
             (fn [exception]
               (swap! failed-requests inc)
               (println "failed requests " @failed-requests)
               (println "exception message is: " (.getMessage exception))


It works as expected with (defn confirm-entity! …)is in the same namespace.


probably need to prefix with the namespace the atom is in


(require '[project.core :as some-ns])

(with-redefs [some-ns/my-atom (atom nil)] ...)


I’ve tried that as well 😞


or you can use the fully qualified namespace


with-derefs will replace the atom with a different atom, it won't change the value of the exiting atom


best thing to do is not def mutable things, but pass them in as arguments where needed


I was trying to keep my code clean. But all these exceptions have made me have to do some annoying stuff and now with testing I have to do even more 😛


the async? true in your http request will also play havoc with dynamicly scoped things like with-redefs


e.g. your http request is put a threadpool to be executed when the with-redef is in effect, and then execution leaves the scope of the with-redef, then the http request is actually executed


async has made my code a mess lol. Though I don’t know how else to handle this scenario


(not sure what http client library you are using or what :async? true actually does)


you have to ensure that with-redefs remains in effect until after the http request has been made


it allows you to catch the return of the request in a callback. Returns a future or something. I don’t quite understand it that much.


so you need to stop, and wait inside the scope of the with-redef, until that future/callback/what is resolved


This testing library I think uses a macro to test the http request. So it doesn’t escape??


sounds terrible


I get that code needs to be testable. But the modifications being required I think are a bit unjustified at this point.


you can do stuff and make this work, but you have to understand what you are doing, and it is easier to understand things if you are directly passing arguments and call functions then it is to understand macros and how dynamic scope interacts with multiple threads


it isn't really about making it testable, it is about making it understandable as a decomposition of parts instead of having to understand the entire thing at once.


making testing easier is a side benefit of that


you seem to think what you have now is easier to understand, but I would point out if you understood it you would be able to write a test for it, which suggests maybe re-evaluating what you have


I’ve got one thread that gets a list of entities to fetch and feeds another thread that acts as a queue (using core.async channels). They throttle making requests to a server that can’t handle excessive load. Then I have async on http-clj so I don’t block on that waiting for the response.


I don't think clj-http-fake even knows how to handle async requests


I’m writing tests now because I’m having to build up a series of exceptions to handle a volatile API.


I've had coworkers write tools to do things like mocking out http requests for that kind of thing, but I have never found it particular useful, if I want to test out making http requests or handling http requests I start up an http server and make requests of it


oh, I guess clj-http-fake does support async? it has a test for it at least


Yea I’m thinking this is the way to go.


Maybe I just reset! the atom in the core namespace during the test..


I mean that works but seems a little wrong lol


opinion: when you have (is thing "some message") generally should the message describe what you expected or should it describe what you assume went wrong i the assertion failed?


eg. the docstring has Example: (is (= 4 (+ 2 2)) \"Two plus two should be 4\") - would you translate that hypthetically to "Oops 2 + 2 wasn't 4" or "2 + 2 = 4" (if you had to pick one)


I was surprised not to see a recommendation about this in the Clojure style guide


I translate "Two plus two should be 4" in a failing test to "The test writer expected two plus two to be 4, and checked for that, but the condition was false"


I brought up the translation in order to attempt to "drive a wedge" into "describe the problem" vs. "describe the expectation" because I think that example walks the line between the two


I have written tests such that when they fail, they print out more details about actual vs. expected results, especially when those actual and expected values are collections, as an aid to starting to debug what went wrong.


it could be the right answer is c) it doesn't matter, either way is fine


I mean, the problem is "I got results I didn't expect". Trying to guess what that might have gone wrong could be done, but how good that guess is really depends upon how much you know about the implementation, and what might have gone wrong.


yeah, the concrete circumstance is a PR where a bunch of is clauses try to describe the failures, and I'm deciding whether it's worth pushing back on haha


That guess could be spot on, or if the implementation has changed significantly since when the test was written with those guesses, and the tests were not also updated to match, the guesses could be way off.


Such tests could be interpreted as over-specifying or over-constraining the solutions, I suppose.


If a development team is willing to write them, but not willing to update them if/when the underlying implementation changes, then they have created tests that will confuse new team members, for sure.


I have written functions that check conditions on the internal structure of trees like Clojure's PersistentVector data structure (the Java internal implementation details, not just at the API level) when trying to look for bugs in the core.rrb-vector library. If I or someone else changes the internal implementation details, but not those internal-structure-checking functions, the results could be anywhere from slightly to majorly confusing and wrong.


right - "white-box" testing of that sort is a conditional tech debt that kicks in if the implementation is refactored


I knew that, eyes open before hand, when I did it, because if I didn't write those functions, I would have missed many opportunities for finding actual bugs. It also helped me learn some invariants of those data structures that I didn't know going in. If I had illusions that those checker functions would somehow be eternally matching a changing implementation, well, they would be just that -- illusions.


yeah, I've done the same - tested things that are really implementation details because it made it easier to write a correct implementation


This may sound like a leap in the conversation, but somehow it reminds me of this slide title in one of Rich Hickey's talks "Programming is an economic activity" (this talk -- ). I didn't realize that fully 30 years ago, but I have gradually realized it over time. Lots of arguments over development practices are between people who realize this versus those who do not, or between people who have different guesstimates of which practices will have different ROI

❤️ 8

Where ROI isn't always measured in money, but can more generally be regarded as happiness/satisfaction/effectiveness versus time spent to achieve them.


this kind of thing really tips me off into the deep end contemplating "the industry", my profession, and my life in general I guess. On the one hand, of course programming is an economic activity, but on the other hand the incentives are almost universally bad. code delivered quickly and cheaply is almost universally rewarded regardless of the long term effects


No big argument from me there. I would counter with: have you ever seen software developers that want to rewrite something from scratch, when that is a terrible economic decision?


oh, 100% of the time 🙂


perhaps more often from someone who joined a project later, versus earlier, but also from people who were there from the beginning. In many cases, sure, there are bad technical decisions made in haste, but sometimes it is just a desire to achieve some Platonic ideal of code goodness, d*mn the time/cost to get there.


people also change their minds, what was good then doesn't seem good now


been there, done that, from both sides of the equation 😬


Often based upon lots of new information learned, or changed requirements, in between those two times. It isn't always "wow, I was lazy then".


the sort of interface between programming and business, and in particular how programmers communicate and justify what they do to their managers is, I think, kind of where the rubber meets the road on this kind of thing, and always interesting


or rather, it isn't always "wow, I was really cutting corners then"


I think it is well understood by experienced development managers that lots of engineers are bad at guesstimating how long projects will take. There are other reasons, but at least part of the idea of doing "sprints" is based upon not making people estimate much further than 2 to 4 weeks in advance 🙂


I would connect this to a kind of distinction between what a programmer produces (code) vs. what is experienced by others including those tasked with managing programmers (the effect of the code in the world, uis drawn on screens, emails sent, etc, the artifact).


yeah, beyond timing stuff, just how what is being done and what needs to be done is communicated


(sprints will spin me out in a whole other direction about scrum and consultants, planning poker is wild)


Sorry, not trying to trigger a drawn out discussion on agile practices / hype / consultants, but at least one simple observation is: if someone isn't yet good at estimating how long things take, making them practice it over and over again every 2 weeks is likely to improve their ability to estimate how much they can do in 2 weeks, at least, versus only making them do it every 6 months.


except estimation (from scrum for example) explicitly does not estimate time periods


it estimates "complexity", which, speaking of economic activity, the business doesn't really care about, they care about time periods


> but sometimes it is just a desire to achieve some Platonic ideal of code goodness, d*mn the time/cost to get there. I have a pet theory that this is perfectly rational. "A good codebase" is more immediately an asset for the developers than for their managers. Lack of technical debt means you can make a feature and go home at five.

💯 4