Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-05-21

Channels

#clojure
<
2020-05-21
>
emccue00:05:34

@ramonp.rios If you want to do that you either need to wrap what you are calling into a function at runtime or make that a macro which will take in your code at compile time

emccue00:05:27

(wrap-exception
    (fn [] 
      (-> (persist/update-subcategory datasource body id)
          (handle-result id :update))
    "Subcategory was not updated.")

Ramon Rios08:05:04

Hey, thanks for the tip ! I'm still getting the macro stuff.

emccue00:05:43

so there is a "delay" between when the code is called and when it is evaluated

emccue00:05:58

which is something you can do in any language really

emccue00:05:13

or you make your function a macro

emccue00:05:51

(defmacro- wrap-exception 
  [f info]
  `(try
    (do ~f)
    (catch Exception e
      (log/error "Exception was handled: " (.getClass e) ": " (.getMessage e))
      (throw (ex-info ~info {:status 500
                            :message (.getMessage e)})))))

noisesmith13:05:15

you don't need do inside try btw

emccue00:05:02

which would literally expand to

emccue00:05:35

(try (do (-> (persist/update-subcategory datasource body id)         (handle-result id :update))) (catch Exception e (log/error ...) (throw (ex-info "Subcategory was not updated." ...))))```

mikroskeem09:05:03

hey, has anyone dealt with log4j(2) here before? basically when i send exception back to the client over http i also see passed map's contents with ExceptionInfo, but i can't see that in logger output

mikroskeem09:05:20

11:31:50 [manifold-pool-3-12] E zentria.emperor.core                     - Failed to start server
clojure.lang.ExceptionInfo: HTTP Error: 500
and that's it

mikroskeem09:05:26

seems like when i stringify the error, i'll always see the map; while log4j does its own thing. how can i special case that? 

user=> (str (ex-info "test" {:a true :b false}))
"clojure.lang.ExceptionInfo: test {:a true, :b false}"

valerauko11:05:34

iirc you can't. i ended up doing json structured logs and i put such stuff in the "message" (the part of the json that i could control)

valerauko11:05:27

when i tried i was told using mdc was the "only way" so i just went with all-json instead. sorry if that wasn't what you were asking about

mikroskeem12:05:20

technically could abuse this...

valerauko14:05:22

i figure it's missing because normally libs just log the message and the stacktrace of the exception object -- i'd just catch and log the exception myself in the format i like, though i don't know how feasible that is in your case

mikroskeem19:05:19

honestly could tap into clojure.tools.logging

mikroskeem19:05:47

i'm not using log4j directly, but that library instead

valerauko03:05:19

how are you logging it?

ataggart22:05:31

@U011YV400HX If you are using a pattern layout, it should work if you explicitly use %exception specifier in the pattern string, e.g.: 

%date %level %logger %message%n%exception
Without a specifier to handle exceptions, log4j defaults to %xException, which doesn't print the data map (same for %rException).

💯 4
👍 4
mikroskeem01:05:53

thank you very much. it did the trick. i was using %xEx

colinkahn13:05:34

Is it advisable to call macroexpand type functions within a macro? For example if I want to expand the inner forms before modifying them with my macro?

Alex Miller (Clojure team)13:05:29

I think the short answer is no

Alex Miller (Clojure team)13:05:49

The long answer might also be no but would depend on what you're actually doing

Alex Miller (Clojure team)13:05:18

the macroexpand function is not the same as what actually happens in the compiler

colinkahn13:05:20

I see, it did seem like macroexpand was more for development than production code, but wasn't sure about where the line was drawn. Thanks!

Alex Miller (Clojure team)13:05:38

there are libraries that approximate it more closely but I can't say I've never needed to do so in 10+ yrs of Clojure

dominicm14:05:12

I have an object which I can write to an output-stream, but I need to return an output stream. Do I need to start a thread to call write or something? I am concerned about my output consuming large amounts of memory (i.e. I don't want to use a bytearrayoutputstream)

valerauko14:05:41

(let [my-stream (open-stream)]
  (write-to my-stream object)
  my-stream)
maybe?

dominicm14:05:07

How do you open a stream? I don't think that's a core function :)

dominicm14:05:04

Needs an argument

dominicm14:05:44

Looks like I'm mixed up, I need to produce an input stream, which is easier, because that means I just need a pipedinputstream

noisesmith14:05:05

right - someone can read an inputstream, and will get what you wrote to the other end of the pipe

valerauko15:05:34

i've been trying my hands at using netty to make a http server. i got to the point that i can receive and respond to requests. i'm not doing any java->ring transformations yet because i first wanted to make sure the stuff is solid. however load testing with wrk shows very high (3-4x avg) standard deviation in response time. i'm lost what could cause this. i have an order of magnitude difference between 75%ile and 90%ile. i figure it might be something about concurrency but i can't guess how to pinpoint the problem. any and all hints would be appreciated.

valerauko15:05:40

running pohjavirta with the same handler from the same repl it has much lower std dev, so it's not something hardware. i don't see extreme gc either.

valerauko15:05:26

under supposedly the same conditions aleph: 

❯ wrk/wrk -t2 -c16 -d10s --latency 
Running 10s test @ 
  2 threads and 16 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   296.44us    1.34ms  48.40ms   98.07%
    Req/Sec    45.32k     5.62k   50.56k    91.50%
  Latency Distribution
     50%  127.00us
     75%  203.00us
     90%  350.00us
     99%    3.40ms
  902286 requests in 10.01s, 142.84MB read
Requests/sec:  90148.68
Transfer/sec:     14.27MB
pohjavirta: 
❯ wrk/wrk -t2 -c16 -d10s --latency 
Running 10s test @ 
  2 threads and 16 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   168.65us  618.22us  15.25ms   96.13%
    Req/Sec   109.93k    14.03k  123.63k    91.00%
  Latency Distribution
     50%   54.00us
     75%   92.00us
     90%  108.00us
     99%    2.98ms
  2186950 requests in 10.01s, 294.07MB read
Requests/sec: 218570.44
Transfer/sec:     29.39MB
my stuff: 
❯ wrk/wrk -t2 -c16 -d10s --latency 
Running 10s test @ 
  2 threads and 16 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   458.61us    1.21ms  29.13ms   91.36%
    Req/Sec    76.04k    14.39k  106.41k    66.00%
  Latency Distribution
     50%   63.00us
     75%  161.00us
     90%    1.37ms
     99%    6.15ms
  1517000 requests in 10.04s, 83.91MB read
Requests/sec: 151141.26
Transfer/sec:      8.36MB

lukasz15:05:58

Have you tried Aleph? It provides a web server and is built on top of netty: https://github.com/ztellman/aleph

valerauko15:05:56

i'm familiar with aleph. i started this to learn how netty works so i can maintain a fork of aleph or build my own if the former is too complex and i can get something feasible working

noisesmith15:05:36

just at a surface level: those numbers smell like a throughput / latency tradeoff

noisesmith15:05:49

when you have more requests going through, and less throughput speed, then I'd expect higher variation / more bursty behavior due to context switching overhead

noisesmith15:05:23

if you optimize throughput instead you'll have more predictable behavior, but higher worst case wait / block times for consumers

valerauko15:05:05

what affects that the most? i'd prefer the flatter (predictable) behavior. i think it's easier to tune stuff if it behaves consistently

noisesmith15:05:45

I'm approaching this from a systems background, not really knowing what tuning parameters netty offers

noisesmith15:05:30

but fundamentally you have dials whose behavior includes larger variation in results -it's a complex system and you often can't get the best performance while also maximizing predictability

valerauko15:05:52

thanks! i guess my problem is that i can't find those dials...

noisesmith15:05:12

yeah, reading now, and it's describing precisely the kinds of parameters you'd want to tune

valerauko15:05:34

wow that's awesome time to dig in... thanks!

noisesmith15:05:41

aphyr has some great stuff on distributed systems - and/but I'd be careful not to go to his twitter if coworkers or children might see your screen

lukasz15:05:33

If he's back on twitter that is, he's getting banned pretty regularly

lukasz15:05:41

that's an awesome post btw

valerauko15:05:23

i use mastodon too so that's not a problem haha

noisesmith15:05:24

yeah - it reads like a precise response to @UAEH11THP’s question - he is on twitter currently, but is more active on mastadon now on an explicitly gay themed server

noisesmith15:05:15

(not judging his choice of social media practice, just saying even having his posts on your screen could get you in hr trouble at work)

noisesmith15:05:50

> Many “architecture patterns” are scar tissue around the absence of higher-level language features. brilliant

valerauko16:05:15

i was like, "meh no way i'd need executors and all that for such a simple thing" but then i guess at some point it stopped being simple huh... this rabbit hole is going to be deep

noisesmith16:05:29

yeah, there's nothing simple about distsys - you start out thinking "wow this is weird and difficult" and then after years of study you realize it is weirder and more difficult than you even imagined

valerauko16:05:41

it was the executor all right... thanks!

fingertoe16:05:58

What is the state of the art these days for running Jupyter/Gorrilla Repl style notebooks in Clojure these days? Is there a library that is well maintained with good community prevailing?

kenny17:05:18

Depending on how much structure (or lack thereof) you're looking for, https://github.com/metasoarous/oz could be a good fit.

fingertoe18:05:15

My intention is to build a Sanity Check platform — When supporting legacy apps, when we get unexpected outputs, it would be good to have repeatable troubleshooting steps to look over the input data in question and determine what the source of insanity is.. So I would like to be able to enable a user to run a set of moderately arbitrary queries, Explain their reasoning, then save their work for the next time a similar issue arises.

Eric Scott16:05:01

When using lein deps :tree, suggestion will sometimes be made to "Consider using these exclusions:...". Is there ever a downside to taking this advice?

noisesmith16:05:43

there are libraries (eg. jackson) that will have breaking changes, and which version you exclude will decide which things break for which consumer of the lib

noisesmith16:05:15

luckily, usually, you can find a version that doesn't break the functionality your app needs

Eric Scott17:05:30

So would you say it's a bad practice to automatically make the suggested exclusions, and wait until you suspect there are dependency issues in play before you start fiddling with these things?

noisesmith17:05:11

when there's a conflict it will usually have at least two suggested exclusions

noisesmith17:05:47

and a third option is to explicitly ask for a version before anything with a dependency is asked for (by putting it higher in the deps list)

noisesmith17:05:26

I've often opted to explicitly ask for a version once rather than putting exclusions in 5 different things

Lennart Buit19:05:44

It is worth noting that lein seems to suggest excluding on the version with the highest requirement in its “consider using …” statement. So I tend to not follow its advice blindly

Brieana21:05:50

Hey folks! Quick clarification question- How different is using java on the frontend for desktop or mobile applications the same, or different, as using it for the backend for web applications?

phronmophobic21:05:19

the language itself will be the same regardless, but support from tooling, libraries, and frameworks will be different. clojure on the jvm is most commonly used for backend servers so you’ll probably have an easier time getting started on backend applications. using clojure for desktop and mobile applications is possible, but is far less common. for web applications, clojurescript is really great option and has tons of tooling, libraries, tutorials, and frameworks. clojurescript can also be used to target mobile applications, but it’s less common. It’s hard to give a more detailed response without a little bit more info about your use case.

jjttjj22:05:34

I have an immutable data structure that represents the state of my app and ~10 operations (implemented as functions of this immutable data) to change the state in all ways that are needed. I can put this in an atom and swap! those functions on it and that works good except for that fact that some of the operations, when used in the real "app" call for additional side effects beyond just the state mutation. I'm somewhat stuck on how to do this cleanly/minimally. I'm thinking about having a protocol of all the operations, and then implementing that on an immutable record, and then also on a mutable record which depends on the immutable one in an atom. Any general tips for how to deal with these situations?

emccue00:05:42

I'll second agents. You lose concurrent updates to state in return for sequential side effects, but its probably okay

emccue00:05:00

you can still queue updates concurrently

noisesmith22:05:52

if "fire and forget" with a queue of changes work for your app, you can use an agent and send instead of atom and swap, that won't retry so you can do a side effect as well as altering the data

noisesmith22:05:21

alternatively, you could set a watcher on the atom for the actions, that only gets fired once per change, regardless of the number of retries

👍 4
noisesmith22:05:26

another model is to not use an atom at all, and use iterate, where the function you use is some "driver" that encapsulates all the actions you'd take on the state, plus the dispatch that picks one

noisesmith22:05:41

this likely also needs a queue for events from the outside world

noisesmith22:05:22

the iterate version needs a queue, as unlike agent / send or atom /swap it doesn't have an implicit / invisible queue built in

hiredman22:05:51

sounds fine? like you have an atom and some functions and no problem. the pure functions are swap! on the atom, and the impure functions have to manage the swapping themself

hiredman22:05:34

I've done things like 

(declare ^:dynamic q)

(defn f [x] (update-in x [:x] (fnil inc 0)))

(defn g [x]
  (.put q #(println "Hello World"))
  (update-in g [:g] (fnil inc 0)))

(defn my-swap! [a f & args]
  (binding [q (java.util.concurrent.LinkedBlockingDeque.)]
    (loop []
      (let [ov @a
            nv (apply f ov args)]
        (if (compare-and-set! a ov nv)
          (do
            (loop []
              (when-let [f (.poll q)]
                (f)
                (recur)))
            ov)
          (do
            (.clear q)
            (recur)))))))
which is analogous to the way the stm queues up agent sends

phronmophobic06:05:25

does q need to be a java.util.concurrent.LinkedBlockingDeque or could it be any mutable Deque like data type? it seems like you’re guaranteed that all operations on q will be from the same thread.

hiredman22:05:08

so your functions update the value, but can also queue up side effects to run on a successful swap

notbad 4
didibus04:05:19

Why use this over a watch?

hiredman05:05:03

A watch runs on all changes, you have to split your functions into the pure part, and the part that runs in a watch, and then you would have to look at the old and new value in the watch to determine which f just ran to figure out which side effect to run

didibus06:05:14

Ah okay I see. I guess I can't think of a use case right now where it couldn't be remodeled as such, but I'm sure there are some

hiredman22:05:21

but like, you have a namespace of functions that do stuff when passed data or an atom, you are done

jjttjj22:05:19

Thanks @hiredman & @noisesmith, I think I felt further off from the goal than I actually am after a long day of working on it