Fork me on GitHub
#clojure
<
2020-03-06
>
hiredman00:03:23

I sometimes even pass a third stop channel as an argument to halt the process

Alex Miller (Clojure team)00:03:41

it is almost always useful to at least allow externally created channels (you can default to creating them) as that gives users the opportunity to set channel/buffer properties that reflect their needs, and also makes it easier to test

Ben Grabow00:03:29

Makes sense. I realize now that it's easy to start with a function that takes in and out as args, then wrap that function with something else that creates the channels, but not easy to go the other way. So that suggests composability dictates taking them as args.

teodorlu08:03:41

Hello! I was looking for a type? predicate in Clojure but couldn't find it. Here's two things I tried. • Am I overlooking a function I should be using from core? • Am i missing test cases where my type-2? predicate will not behave as expected? Thanks!

(defn type? [t]
    (contains? (parents t)
               java.lang.Object))

  (type? (type "123"))
  ;; => true

  (type? (type 99))
  ;; => false

  (defn type-2? [t]
    (= java.lang.Class
       (type t)))

  (type-2? (type "123"))
  ;; => true

  (type-2? (type 99))
  ;; => true

fricze08:03:20

Are you, maybe, looking for instance?

teodorlu08:03:46

It's not clear to me how I would use instance? as a 1-arity function to figure out wheter a value is a type?

fricze08:03:27

(defn type? [t]
    (instance? t java.lang.Object))
?

teodorlu08:03:49

I would be expecting true here:

(defn type-3? [t]
    (instance? t java.lang.Object))

  (type-3? (type 1))
  ;; => false

fricze08:03:46

But with instance? you don’t have to use (type…) . You can just pass value directly (instance? java.lang.Long 1)

ak-coram08:03:20

there's also class? in core I think

ak-coram08:03:03

it's just (instance? Class x)

teodorlu08:03:05

@UFJD2TV46 I've types in my data structure already, so I need to be able to take types as input:

{"name" #{java.lang.Long java.lang.String}}

fricze08:03:09

then maybe isa?

fricze08:03:13

(isa? (type "1") java.lang.Object)

fricze08:03:22

(isa? (type 1) java.lang.Long)

fricze08:03:46

seems to be working that way you want it

fricze08:03:04

(isa? (type 1) java.lang.Object) evals to true

teodorlu09:03:18

@UT9CQ4HSS @UFJD2TV46 (class? t) and (isa? t java.lang.Object) both seem to be doing what I want. Thanks! 🙏

acron09:03:18

What's the proper name given to the practice of allowing a function to use variadic key value pairs? e.g.

(defn foo [& {:keys [a b c]}] ... )
(foo :a 1 :b 2)

teodorlu10:03:25

Variadic keyword arguments?

kirill.salykin11:03:59

hi, I am trying to implement java interface in clojure (based on https://github.com/aws-samples/aws-blog-athena-custom-jdbc-credentials/blob/master/src/main/java/com/amazonaws/custom/athena/jdbc/CustomIAMRoleAssumptionSAMLCredentialsProvider.java)

(def provider
    (reify AWSCredentialsProvider
      (getCredentials [_]
        (BasicSessionCredentials. ak sk token))
      (refresh [_])))
Everything is good so far, but later I need to specify name of the class so it actually can be used, please advice how can I get the name? thanks! UPD: using the defrecord makes seems possible, but builtin ClassLoader fails to load it:
BuiltinClassLoader.java:  581  jdk.internal.loader.BuiltinClassLoader/loadClass
         ClassLoaders.java:  178  jdk.internal.loader.ClassLoaders$AppClassLoader/loadClass
          ClassLoader.java:  521  java.lang.ClassLoader/loadClass
                Class.java:   -2  java.lang.Class/forName0
                Class.java:  315  java.lang.Class/forName
                       nil:   -1  com.simba.athena.athena.utilities.AJUtilities/createAwsCredentialsProvider

rafael14:03:43

Hi. If you need to refer to the java class by name, maybe you need gen-class instead of reify. Check out Chas Emerick's type selection flowchart https://github.com/cemerick/clojure-type-selection-flowchart/

kirill.salykin16:03:39

thanks, found way around

souenzzo12:03:00

How to debug this? => #error{:cause nil, :via [{:type java.lang.ClassCastException, :message nil}], :trace []}

Ben Grabow22:03:44

Can you provide some more context about what you were doing when the error occurred?

souenzzo23:03:05

I call a function and it blow up. have no idea where

Ben Grabow23:03:36

Which function and in what project? It's impossible to help without much more information.

souenzzo00:03:46

I already solve the specific case, but i still wanna know some method to deal with this kind of problem

genekim07:02:28

If anyone encounters problems where no stack trace is generated, which I stumbled into this evening, I found that adding this to JVM opts helped: -XX:-OmitStackTraceInFastThrow Hopefully this might help someone in the future. (I’ve just added this flag to all my REPL sessions — can anyone verify that this is indeed a reasonable thing to do? Thank you!)

4
genekim07:02:10

In my case, it was hato and clj-http libraries where I passed in uri instead of url in the request map. Caused (name scheme to throw exception due to NPE. Both had identical behaviors of generating the #error{:cause nil, :via [{:type java.lang.ClassCastException, :message nil}], :trace []} , with no stacktrace available. I’m using Azul JVM 21 in IntelliJ/Cursive.

Ben Sless08:02:54

I think it's reasonable to disable this option in dev time. Some might argue it should be mandatory

flowthing08:02:12

But yes, I agree that it is a reasonable default in dev.

p-himik08:02:22

You might also find these useful for debugging/profiling: • -Dclojure.compiler.disable-locals-clearing=true-Dvisualvm.display.name=MyProject (helps with finding the process in VisualVM)

👍 2
cjohansen08:02:30

I always add that flag. I can't even understand why this behavior exists. If you have errors in production that are so common that producing their stack trace is a noticeable performance loss, I'd say you have bigger fish to fry 😅

😆 1
Ben Sless08:02:59

You'd think that, but the effects of exceptions, and filling stack traces in particular, on performance, is staggering. Pop quiz - at 10% CPU utilization, what error rate would you need to make a web handler keel-over?

flowthing08:02:58

Yeah, Throwable.fillInStackTrace() has been pretty dominant in many JFR recordings I've looked at.

Ben Sless08:02:54

(time
 (dotimes [_ 1e7]
   (try
     ;; (/ 1 1) "Elapsed time: 11.26919 msecs"
     (/ 1 0) ; "Elapsed time: 10935.099248 msecs"
     (catch Exception e e))))

👍 1
🤯 2
Ben Sless08:02:56

Unless I'm way off, this means at even at single digit error rates, a web handler can completely collapse just from allocating exceptions

Ben Sless08:02:40

Which can happen if clients send bad data, or you have some network problems

seancorfield18:02:56

@U6VPZS1EK You might want to update your CLI version, since it seems likely your version is well over a year old now...

🙏 1
jumar18:02:47

There should also always e at least one instance of the error that has full stacktrace. It might just be little further away in the logs

kwladyka17:02:12

-XX:-OmitStackTraceInFastThrow I use it also in production, because after Java 11 (? I don’t remember version) OmitStackTraceInFastThrow is enabled by default . This cause omit NullPointerException. In consequence in tools for debug like New Relic etc. you see there is a bug, but you have no idea what cause it without stacktrace and you can lose days to figure out this. In my personal opinion this is not worth to have it enabled. If you have many NullPointerException then make catch in right place - this is right call.

kwladyka17:02:43

BTW this flags are my minimum ["java", "-XX:InitialRAMPercentage=70", "-XX:MaxRAMPercentage=70", "-XX:-OmitStackTraceInFastThrow", "-Dclojure.tools.logging.factory=clojure.tools.logging.impl/jul-factory", "-jar", "app.jar"]

p-himik17:02:27

How would putting catch in the right place fix the performance degradation pointed out by Ben above?

kwladyka18:02:29

if you expect NullPointerException as an exception, than do catch NullPointerException in the right place, so you don’t need OmitStackTraceInFastThrow in any case. You want to have stacktrace for NullPointerException, because of bugs. This is what I mean.

kwladyka18:02:53

notice catch Exception vs catch NullPointerException

flowthing18:02:54

Catching NPEs seems suspect. Is it not better to fix the underlying cause instead?

seancorfield18:02:44

The "bug" is typically that you get an NPE that you did not expect so you won't have a catch for it. And that still doesn't address either the need to disable that fast-throw optimization nor the general performance overhead of any exception being constructed.

kwladyka18:02:19

I mean here if you have catch NullPointerException in places where you expect it, then we can assume each unexpected occurrence of this exception is a bug. This imply we need stacktrace to fix the bug. In some cases bug is very not obvious and there is no way to fix it without stacktrace. This also imply fast-throw optimization is not needed, because it would be run only in rare case when there is a bug. At least in my last project (with millions of transactions per hour just to say it was real serious project with thousand of instances) we made decision to turn off this in production. The decision was made because of luck of stacktrace in NewRelic, airbrake etc. so each time we were unable to fix bugs. We have never experienced any performance issue after turning this optimisation off. My personal point of view of course.

seancorfield18:02:52

No, if you expect a particular exception then it isn't a bug, by definition.

kwladyka18:02:33

I think we misunderstood each other at some point.

seancorfield18:02:11

If you slap catch Throwable at the top of your call tree and log that for debugging purposes, fair enough. And, yes, you need a stacktrace -- hence turning OFF that optimization -- and then you pay the price of your app constructing those exceptions with stacktraces which has a definite cost that can bring down a server -- even if you haven't experienced that performance issue yourself.

seancorfield18:02:08

At work, we have occasionally seen bugs that throw exceptions fast enough that the server pretty much grinds to a halt due to constructing thousands of stacktraces. So this is a real problem. It just isn't very frequent (fortunately). But it is the tradeoff that OmitStackTraceInFastThrow is all about.

kwladyka18:02:19

We were monitoring performance very closely. We did canary test first and we didn’t notice even the smallest difference in performance.

kwladyka18:02:13

I see. This make sense too. We didn’t have such problem.

seancorfield19:02:47

We've only seen it rarely. I think turning the optimization off is a reasonable tradeoff. And even with the optimization turned off, there are ways to construct and throw exceptions without a stacktrace if you need to.

👍 1
genekim17:02:19

I learned a ton for this from this thread — thanks, everyone! To summarize: • generating stack traces can make exceptions can be quite expensive (3 orders of magnitude) • for this reason, at some point, the JVM made omitting stack traces the default behavior (JVM 5?) — given the above, there are valid reasons to omit stack traces in production, lest take down your server. But downsides are obvious — you won’t have stacktraces when you need them most. • this makes Clojure debugging quite awful — for that reason, starting in version 1.11.1.1208, Clojure disables OmitStackTraceInFastThrow by default • I should run “brew upgrade clojure” far more often • if you get => #error{:cause nil, :via [{:type java.lang.ClassCastException, :message nil}], :trace []} error, this is the stack trace being omitted — but there was probably a stack trace earlier that you missed. • I learned that catch NullPointerException is a thing

p-himik17:02:33

> starting in version 1.11.1.1208, Clojure enables OmitStackTraceInFastThrow *disables

👍 1
🙏 1
hansen-pansen16:03:35

Hello dear Clojurians, I have a problem dealing with Leiningen and “Multi-Release JARs” (MRJARs). Specifically, I work with Java 11 and Log4J2, but get the following warning upon running or uberjaring:

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
https://issues.apache.org/jira/browse/LOG4J2-2537 reveals that log4j2 ships MRJARs, and you need to leverage those with setting Multi-Release: true inside the manifest. Thing is: I cannot figure out how to set this with Leiningen. Any hints?

hansen-pansen16:03:32

Trying with :manifest {"Multi-Release" "true"}

hansen-pansen16:03:38

… and it works! Sorry!

duckie 4
seancorfield16:03:32

Note that depstar figures this out for you automatically based on what's in the libraries on your classpath.

💯 4
seancorfield16:03:20

And it was the exact same use case -- log4j2, when we switched from JDK 8 to JDK 11 at work 🙂

seancorfield16:03:44

(I had assumed Leiningen would already do that -- but I haven't used lein for nearly five years now)

hansen-pansen16:03:47

Well, the only thing preventing me from switching to clj-new (or rather deps.edn) for all my projects is my inability to comprehend https://ask.clojure.org/index.php/8440/equivalent-of-leiningens-managed-dependencies-in-deps-edn?show=8442#a8442 😉

seancorfield16:03:07

Happy to chat about that via DM if you want to understand more deeply how we use it at work. We have 90k lines of Clojure in a monorepo with thirty subprojects, all managed with deps.edn.

ghadi16:03:48

log4j2 uses MRJARs? or just slf4j 2.0?

seancorfield16:03:15

I thought we encountered it with log4j2 (that's what we use)... Yeah, that performance issue was what bit us and why I added MRJAR detection to depstar.

hansen-pansen16:03:30

@seancorfield I’d love to, I appreciate your offer! But my kids just dropped in, so I will be afk.

seancorfield16:03:12

NP, I'm online "all the time" Pacific TZ.

seancorfield16:03:17

Yup. East Bay, away from the craziness of Silicon Valley 🙂

hansen-pansen16:03:16

Hehe, I am based in Berlin, Germany and used to travel quite a lot to SF. So, thank you very much, maybe in some time!

lmergen19:03:58

what is the idiomatic way to 'amend' an exception thrown using ex-info with additional data ? ideally i want to do something like (try ... (catch Exception e (throw (merge e {:foo :bar})))

lmergen19:03:58

i would have no problems using a library like slingshot for this if it supports it, but i was hoping for a way to do this in vanilla clojure

ghadi19:03:29

a common thing to do is to chain an exception cause:

(throw (ex-info "uh-oh" {:more :context} original-exception))

👍 4
kwladyka19:03:19

there was also something to indicate this exception is thrown because of other exception which you can include, but I don’t remember details to give you an example

👍 4
kwladyka19:03:30

maybe you need this instead of merge

kwladyka19:03:37

merge sounds a little risky

Graham Seyffert19:03:35

Someone on my team has defined a ^:private ^:const vector in one of our namespaces, and I changed the values in it from standard Strings to Jackson’s SerializedString , and now I’m getting an odd compiler error when building the project -

Error:clojure: Syntax error compiling fn* at (a/name/space/a_file.clj:30:21).
java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: next
 at clojure.lang.Util.runtimeException (Util.java:221)
    clojure.lang.Compiler$ObjExpr.emitValue (Compiler.java:4893)
    clojure.lang.Compiler$ObjExpr.emitListAsObjectArray (Compiler.java:4704)
    clojure.lang.Compiler$ObjExpr.emitValue (Compiler.java:4851)
    clojure.lang.Compiler$ObjExpr.emitConstants (Compiler.java:4934)
    clojure.lang.Compiler$ObjExpr.compile (Compiler.java:4612)
    clojure.lang.Compiler$FnExpr.parse (Compiler.java:4106)
    clojure.lang.Compiler.analyzeSeq (Compiler.java:7105)
    clojure.lang.Compiler.analyze (Compiler.java:6789)
    clojure.lang.Compiler.analyzeSeq (Compiler.java:7095)

ghadi19:03:59

take out the ^:const

Graham Seyffert19:03:59

Removing the ^:const on this vector fixes the issue, but I’m just trying to understand what’s going on there

Graham Seyffert19:03:52

presumably only things like Strings / ints / etc should be marked ^:const?

ghadi19:03:15

yeah things that can be emitted as values in the bytecode

Graham Seyffert19:03:19

Just interesting that it was fine when it was a vector of Strings

ghadi19:03:57

vec of constants can be, but the compiler only understands certain constants natively. For others, it attempts some trickery

ghadi19:03:13

look at emitValue() in the compiler

Graham Seyffert19:03:28

Yeah i was digging around in there

Graham Seyffert19:03:03

It hits this call - `

cs = RT.printString(value);
`

Graham Seyffert19:03:08

Which throws the exception

ghadi19:03:21

@souenzzo JVM elides stacktraces in certain cases tldr: run with -XX:-OmitStackTraceInFastThrow

Graham Seyffert19:03:05

Mmm presumably this case actually throws the error -

Graham Seyffert19:03:06

else if(x instanceof IPersistentVector) {
			IPersistentVector a = (IPersistentVector) x;
			w.write('[');
			for(int i = 0; i < a.count(); i++) {
				print(a.nth(i), w);
				if(i < a.count() - 1)
					w.write(' ');
			}
			w.write(']');
		}

ghadi19:03:30

@gseyffert ^:const is overused..

Graham Seyffert19:03:43

Ahaha yeah, agreed…

hiredman19:03:52

I wouldn't say it is overused, I would say people have no idea what it actually does and treat it like some kind of incantation

Graham Seyffert19:03:15

That’s probably more accurate

Graham Seyffert19:03:20

That’s how people on my team use it

ghadi19:03:57

yeah... I find that if you see it once in a codebase, it will appear a dozen more times within a month when people use it without understanding what it does

ghadi19:03:12

^:const can lead to embedding arbitrary objects (like your SerializedString thing) into code. And it can! but the objects have to have accessible constructors

andy.fingerhut19:03:49

Are there any official docs on what ^:constactually does?

andy.fingerhut19:03:27

Or is it better considered an implementation detail that no one wants to promise what it does?

andy.fingerhut19:03:15

I am having trouble even finding it mentioned anywhere outside of RT.java in the Clojure/JVM implementation.

hiredman19:03:05

https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L7325-L7326 is the implementation, you use a name that refers to a var tagged with const, instead of compiling to a deref of the var the compiler compiles the compile time contents of the var as a quoted literal in place of the name

ghadi19:03:32

and the quoted literal goes through that emitValue path we saw earlier. (You run into the same code path when trying to embed objects into macro templates, too)

hiredman19:03:22

yeah, the effect is something like turning

(def ^:const foo 1)

(defn bar [] (+ foo 10))
into
(def foo 1)

(defmacro foo* [] (list 'quote foo))

(defn bar [] (+ (foo*) 10))

hiredman20:03:44

that is all clj specific, it would not surprise me at all if ^:const behaved differently in cljs (because why not)

markbastian21:03:40

I’m writing a macro in which I want to create a local dynamic var in the ns in which the macro is evaluated. When I do this, the macro appears to drop the ^:dynamic declaration but the var can still be treated dynamically. Any idea how to keep the dynamic declaration? Here’s a complete example:

(defmacro create-foo []
  `(def ^:dynamic ~'*foo* nil))
;=> #'user/create-foo
(macroexpand '(create-foo))
;=> (def *foo* nil)
(create-foo)
;Warning: *foo* not declared dynamic and thus is not dynamically rebindable, but its ;name suggests otherwise. Please either indicate ^:dynamic *foo* or change the name. 
;=> #'user/*foo*
(println *foo*)
;nil
;=> nil
(alter-var-root #'*foo* (constantly 42))
;=> 42
(println *foo*)
;42

hiredman21:03:24

you to do

(defmacro create-foo []
  `(def ~(with-meta '*foo* {:dynamic true}) nil))

hiredman21:03:38

if you quote your syntax quoted expression you can see what the reader produces for it

hiredman21:03:58

user=> '`(def ^:dynamic ~'*foo* nil)
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote def)) (clojure.core/list (quote *foo*)) (clojure.core/list (quote nil))))
user=>

hiredman21:03:41

which is the expression that is actually evaluated to create the syntax quoted form, which is rather complicated, and I forget how it is actually built in read, which you would need to know to figure out where your dynamic metadata is ending up

hiredman21:03:23

however, you also don't need a macro to create a var

markbastian21:03:45

I was hoping to use intern, but don’t I need to know the ns for which I am interning a var?

hiredman21:03:35

and if you don't know that, then something is being done in poor taste

hiredman21:03:08

like, if your create-foo macro is being called at the top level, the value of *ns* will be the current namespace

hiredman21:03:37

if your create-foo is not being called at the top level, you are using def not at the toplevel which is gross

markbastian21:03:24

Well, I should clarify… I would, of course, know the ns I am in, but it seems odd to have to declare it as part of the macro. Let me give a complete explanation. I would welcome your comments. A common pattern I’ve seen used (and have used) with integrant is some boilerplate in one ns along the lines of:

(defonce ^:dynamic *system* nil)

(defn system [] *system*)

(defn start
  ([system] (alter-var-root system (fn [s] (if-not s (ig/init config) s))))
  ([] (start #'*system*)))

(defn stop
  ([s] (alter-var-root s (fn [s] (when s (do (ig/halt! s) nil)))))
  ([] (stop #'*system*)))

(defn restart
  ([s] (do (stop s) (start s)))
  ([] (restart #'*system*)))
I’d like to turn this boilerplate into a macro since versioning this snippet across projects gets old. I’ve got this (and it works), but I want to eliminate the dynamic (will use your code to fix):
(defmacro create-system
  "Create a ns-global system. Takes code that evaluates to a system configuration.
  The following items will be created:
    * *system* dynamic variable to hold the system.
    * 'system' function for viewing the current value of the system.
    * Functions start, stop, and restart which will do those actions on the system."
  [config]
  `(do
    (defonce ^:dynamic ~'*system* nil)

    (defn ~'system [] ~'*system*)

    (defn ~'start
      ([~'system]
       (alter-var-root
         ~'system
         (fn [~'s] (if-not ~'s (ig/init ~config) ~'s))))
      ([] (~'start ~'#'*system*)))

    (defn ~'stop
      ([~'system]
       (alter-var-root
         ~'system
         (fn [~'s] (when ~'s (do (ig/halt! ~'s) nil)))))
      ([] (~'stop ~'#'*system*)))

    (defn ~'restart
      ([~'system] (do (~'stop ~'system) (~'start ~'system)))
      ([] (~'restart ~'#'*system*)))))
So, you could, in some ns, do:
(defn config []
  {::web/server {:custom-key "This is a custom key"
                 :host    "0.0.0.0"
                 :port    3000
                 :handler #'handler}})

(create-system (config))
This would give you your single restartable system. Thoughts on this? BTW, I am going to drop the 1 arg system option as this really isn’t something I think it useful for what I am trying to do.

isak22:03:53

I think this is the same feature set as this, no? https://github.com/weavejester/integrant-repl

hiredman21:03:36

my initial though is: this is horrible, use component

hiredman21:03:15

and create-system can't do what you want

hiredman21:03:19

because it is a macro

hiredman21:03:48

and at macro expand time (config) is not the result of calling the config function, it is a list containing the symbol config

markbastian21:03:18

I prefer integrant to component - individual components are very OOish. However, I think at the top level, a component-like approach can be good. Kind of a hybrid solution.

markbastian21:03:37

It actually does work.

hiredman21:03:49

it happens to work

seancorfield21:03:12

Why does everyone say "component is OO" when all of the alternatives seem to just swap global state for maps/records?

hiredman21:03:40

I think someone said it once and it has become standard FUD

markbastian21:03:43

My experience with component was that I was writing a lot of records and it did, indeed, feel very much like I was writing a lot of OOish boilerplate. This isn’t meant in any way to be an insult towards component, it’s just what my experience was. I found integrant to be much more straighforward.

hiredman21:03:49

but there is no accepted definition OOP (Kay's or Stroustrup's) that component is like

lukasz21:03:19

especially that now you don't have to create records if you don't want to

hiredman22:03:32

you didn't have to create records before either, but people are so lazy about understanding the capabilities of their tools

lukasz22:03:50

ah well ,yes - anything can be a component.

lukasz22:03:58

I was referring to the extend-via-metadata bit

markbastian22:03:58

“feels OO” is not the same as “is OO” and I’ll just say that’s how it felt. A prime example is when I needed a db component. Component creates records. Some libraries, e.g. monger, take a bare db connection in the first position. I ended up writing wrappers for a ton of libraries to accommodate them.

hiredman22:03:08

I would just like for people to say "it felt very boilerplate heavy to me" instead of repeating the oo nonsense

lukasz22:03:18

true, although having worked on a pretty big application which used atoms/defonce etc for holding connections and such, I'm not sure if less boilerplate is worth the problems it brings

Mark Marsella22:03:18

What would you consider the best environment for Clojure dev? Tried VSCode with Calva and spacemacs with Cider both seem decent?

seancorfield22:03:11

"best" is very subjective. I use Atom/Chlorine and love it, after years of Emacs/CIDER, but that's a niche option.

seancorfield22:03:33

I'd say: use whichever editor you like best -- the Clojure integration is decent for most of them these days.

seancorfield22:03:21

When folks are just getting started with Clojure, I always say to continue using their current editor, if possible, if it has a Clojure integration.

seancorfield22:03:57

Some people love Emacs (or its variants), some love vim, some love VS Code, some love Atom, some love IntelliJ/Cursive.

seancorfield22:03:01

I sort of wish I could switch from Atom to VS Code (there's Clover, which is the VS Code version of Chlorine, so that part would be the same) but I can't customize VS Code at startup the way I can with Atom so that's a deal breaker for me. I use Cognitect's REBL data browser and all my REBL support code is written in CoffeeScript as part of Atom's startup script. VS Code has no equivalent.

seancorfield22:03:12

Does any of that help @UUQJ356AD ?

✔️ 8
Mark Marsella10:03:09

Yes that is great thanks

hiredman22:03:33

if you are creating your own system framework you should identify strengths and weaknesses in existing frameworks and try to deal with the weaknesses

hiredman22:03:02

you have thrown out component's strengths (proper scoping, explicit dependencies, systems as a first class manipulatable object) and seem to have introduced your own boiler plate problems you are trying to solve by layering on macros

markbastian22:03:07

I suspect the “feels OO” has a lot to do with the fact that you are creating records which implement a declared protocol. Feels like creating a class that implements a declared interface.

hiredman22:03:47

you can use component as is without using records

hiredman22:03:06

you can pass in your own functions that do whatever you want for starting and stoping

hiredman22:03:32

you can use components dependency system without using the Lifecycle protocol at all

hiredman22:03:56

writing a stripped down version of component that just uses maps is super simple https://gist.github.com/hiredman/075b45eaeb01e4b526ce6f8854685487

johnj22:03:12

looks like component and integrant are fundamentally the same but one uses maps/records, the other EDN

markbastian22:03:49

Ok. I apologize if I offended you regarding Component. I just said I like integrant more. Never said I didn’t like Component. I’ve used mount, component, and integrant and I like integrant best. The only issue I’ve had with integrant is the small amount of boilerplate I add to a project to contain my single application map. I know weavejester has written https://github.com/weavejester/integrant-repl and I was just thinking of other ways of doing it. It really isn’t that big of a deal.

markbastian22:03:58

Integrant uses a map with multimethods where Component uses records and protocols.

hiredman22:03:58

I've not used integrant, I am just tired of garbage takes on component

hiredman22:03:14

I've used maps with multimethods with component

hiredman22:03:24

it works ok

hiredman22:03:16

I don't like it has much because it limits your ability to on the fly replace things in tests because defmethods have global effects (the mutate the multimethod)

isak22:03:04

@markbastian Here is another one, seems slightly smarter than integrant, but I have not tried it yet: https://github.com/juxt/clip#defining-a-system-configuration

johnj22:03:26

at what point do you feel to add componet to a project? for ex: as soon as you have a DB and server?

hiredman22:03:51

if I am writing a backend service, I start with component

👍 4
markbastian22:03:46

Any time I need any sort of stateful component (db, web server, etc.) I will use one of the frameworks mentioned.

didibus22:03:45

I don't use them at all.

markbastian22:03:42

I know that’s also a common sentiment. I believe Stuart Halloway also has said he doesn’t use any of them. One thing I like about the Clojure community is that there are a lot of options and we aren’t dictated to use a specific solution (e.g. Spring).

4
johnj22:03:47

javalin, jooby, spark, microanut, jersey... hehe

markbastian22:03:16

True. I guess I should clarify. I feel like with Clojure it’s pretty easy to develop a la carte solutions using whatever you want and switch easily. With heavyweight frameworks like Spring you are all in (For example, you’ve got spring annotations all over your code) and it’s hard to mix and match. Lightweight frameworks (or none) are a lot more flexible in my experience.

isak22:03:40

It avoids the multimethod problem @hiredman mentioned above

markbastian22:03:46

Yeah, I know there’s also duct, which has multiple stages of configuration. I haven’t run across a need for it. Integrant is currently at the sweet spot for me. Thanks for the info on clip, I’ll check it out.

hiredman22:03:59

the critique of these data driven approaches to system building (like clip or integrant) is the same as for xml files for java. application booting can be complicated, so eventually your dsl for describing building a system becomes a programming language

dominicm07:03:01

I'm curious to know what complicated you've encountered in systems?

isak22:03:53

That critique would apply if you are using pure EDN, but that is not needed for either - you can use clojure to define the system map

markbastian22:03:00

My experience (and it seems to mirror many other devs) is that the more data-driven the solution, the simpler it is. For me I see it in Component -> Integrant and Compojure -> Reitit as examples. Again, not saying the former are bad, but I personally like the data driven solutions a lot more.

4
didibus22:03:16

What do you all leverage them for? I've really never felt the need to use any of them

zilti22:03:32

No idea about Integrant, since I use Mount as Component-replacement. But Reitit is sweeeeet.

hiredman22:03:46

for component it holds state

hiredman22:03:03

and it holds the entire system state as a single object

hiredman22:03:17

and has a clear why to starting up and tear everything down

hiredman22:03:36

which makes it super useful for tests

hiredman22:03:29

because the state is mostly all encapsulated in a system object, I can have multiple systems in the same jvm and have tests that check all kinds of distributed system properties

hiredman22:03:01

because the system is described as a map I can manipulate the map, like swapping out a redis pubsub component for a core.async channel, so I can have some tests run without hitting redis

hiredman22:03:41

because the runtime state isn't held in globals (like mount) I can run the same code configured slightly differently

hiredman22:03:11

we had a whole thing at work where we are using braintree for payments processing, and it turned out for some business reason we couldn't use the braintree account we use for credit cards for paypal, and my boss came to me and was like "how long is going to take to add some new paypal support"

hiredman22:03:40

and it was almost like flicking a switch, just add another braintree component configured differently to the system

hiredman22:03:23

(mount is totally gross)

markbastian22:03:41

Any time I need a system with state I use one of the mentioned frameworks (Integrant nowadays). As hiredman has said, it encapsulates yours state so testing, etc. are much easier. Component and Integrant make it easy to have as many systems as you want in your project, so it’s easy to (for example) have a single application system and also have smaller instances that you spin up and tear down as needed for testing. A long time ago we’d do something along the lines of dynamic vars for each stateful item (e.g. a db connection) and it made a lot of things hard.

didibus22:03:05

Ya, never had any of these issues

markbastian22:03:08

The main issue I had with Mount (and why our team tried Component after that) was that it felt kind of fragmented. You initialized different items in different nses vs. having a single system that encapsulated all your state. Integrant and Component both do that.

didibus22:03:18

Doesn't seem useful to me sorry

didibus22:03:07

It seems it's only relevant for tests, but in my tests I either don't mock and do a full integration test, or I mock my side effecting fns

markbastian22:03:08

Again, if you like mount or none of the above I don’t think you should feel bad. Clojure is made for making things that work however you want them to.

didibus22:03:29

Ya for sure. Its just I hear everyone using them and people asking when should you start using them and all. And I've never felt I needing anything else then what Core Clojure offers. So I'm curious what kind of use case others have that these solve more simply

johnj22:03:09

there is a repl benefit too

didibus22:03:59

Like if I have a client for some rest API I just (def cool-client (delay (CoolRestClient.)))

hiredman22:03:33

all the code that uses cool-client referring to it via a global var

hiredman22:03:01

so suddenly when you get a requirements change like "we need to interact with this other cool client" that all gets rewritten

didibus22:03:19

No none of the code will. All the code is written as (defn foo [client ...] ...)

didibus22:03:50

Only the top level orchestrator function that manages the workflow of the request does

zilti22:03:04

Yea right now I e.g. use Mount so the webserver gets restarted when I reload the code

markbastian22:03:18

As an example of something that works well for me with integrant is that I can have a config map for each system I am dealing with. Say I’m working in dev and someone walks into my room and says “We have a production issue.” I can just restart my system with the prod config or launch a second system in the repl with the prod config. As long as my functions take my system components as an argument, I’m off to the races. It’s a very nice, consolidated way to work. If I were using dynamic vars in different nses I might need to go to each and relaunch them or restart my repl.

didibus22:03:16

Hum... Oh so you use it for configuration as well ?

markbastian22:03:16

Do you mean do I use integrant for configuration?

didibus22:03:55

What I do is I'd have a make-client fn for each client. And a global client def that uses that make-client wrapped in a delay. The delay would pull environment specific config from our config and use the info there to figure out how to call the make-client for the current environment. In a repl, if I want to switch to the prod environment, I can just change the "environment" property to prod and reload the def so it'll now constructs a client using the prod config

didibus23:03:17

Something like:

(def config
  {:test {:cool-endpoint "http.x.u.x"}
   :prod {:cool-endpoint "some.prod.comx}}

(defn make-cool-client [endpoint]
  ;; returns a constructed client for given endpoint
)

(def environment :test)

(def cool-client (delay (make-cool-client (:cool-endpoint (get config environment))))

(defn handle-some-request [request]
  (-> (do-a cool-client request)
    (do-b cool-client)
    (etc cool-client)))

didibus23:03:41

Slow to type on mobile phone :p

markbastian23:03:43

Interesting. The way I use Integrant all configuration is pushed to the config map. This gives you a lot of flexibility to do things like restart your singleton system with a different config map, launch additional systems with different maps, merge configs (you can use derived keys to prevent collisions) to have multiple environments’ parts in your system, etc. It’s pretty flexible. It’s also super easy to synthesize what you need for tests.

markbastian23:03:42

I see. The big difference between what you are doing and what Integrant does is that in integrant centralizes the entire config along these lines:

(def dev
  {::web/server   {:host    "0.0.0.0"
                   :port    3000
                   :handler #'handler
                   :datomic (ig/ref ::datomic/conn)
                   :sql     (ig/ref ::sql/conn)}
   ::datomic/conn {:connection-stuff "dev-config"}
   ::sql/conn     {:jdbc-config "dev jdbc string"}})

(def prod
  {::web/server   {:host    "0.0.0.0"
                   :port    3001
                   :handler #'handler
                   :datomic (ig/ref ::datomic/conn)
                   :sql     (ig/ref ::sql/conn)}
   ::datomic/conn {:connection-stuff "prod-config"}
   ::sql/conn     {:jdbc-config "prod jdbc string"}})
You could put them in one map with different keys, though. Something along the lines of (ig/init (all-configs :prod)) vs. (ig/init prod).

didibus23:03:39

Ya, I should check out integrant. If in a way it just can replace my config and give me a few extra bells and whistles that could be worth it.

didibus23:03:10

When I played with component, I wasn't a fan of how it made everything more complex, reminded me too much of Spring and Guice

didibus23:03:18

Mount I actually liked a bit better, since it's more lightweight, but even then, I didn't really see what more it gave compared to what I just manually was doing.

didibus23:03:58

I get that component gives you the same promise that Spring does, but I find in practice I don't care. Like I never need to swap one DB for another except maybe once every 5 year when you choose to migrate from say SQL to Mongo, or Oracle to Postgress. And generally that sort of migration involves more changes anyways, it's not just swapping one client for another, since they rarely have absolutely equal interface and semantics

didibus23:03:33

All I need is to swap from connecting from my prod DB, to my local, to my test, etc.

didibus23:03:54

Similarly, in tests, I mean sometimes I do maybe use an in-memory variant. Like DynamoDB has an in-memory one. But since all my business code takes a client as argument, it's not an issue as well. I just create wtv client the test need and tests the functions that use it.

didibus23:03:10

And for testing an end-to-end request, I generally go the extra mile and use a real DB. And sometimes I'll just use with-redefs if I really wanted to say swap it for a mock or a in-memory one. At the cost of not having parallel tests

didibus23:03:46

Also, I keep all business logic pure

didibus23:03:18

So my requests are like:

(-> (gather-data-from client)
  (transform-data)
  (figure-out-what-to-do-next)
  (case :do-x (perform-x-io client)
          :do-y (perform-x-io client)
  ...)

didibus23:03:30

Something like that

genekim07:02:28

If anyone encounters problems where no stack trace is generated, which I stumbled into this evening, I found that adding this to JVM opts helped: -XX:-OmitStackTraceInFastThrow Hopefully this might help someone in the future. (I’ve just added this flag to all my REPL sessions — can anyone verify that this is indeed a reasonable thing to do? Thank you!)

4