Fork me on GitHub
#clojure
<
2018-09-23
>
kulminaator05:09:27

As you noticed the cache interfaces are usually too complicated for people to use in such detail, in higher logic code my stuff looks more like memoize "read from cache with this key or invoke that loader function to get it".

kulminaator05:09:16

we don't iterate over maps by hand when we look up a value for a key, we have a ready made function for us to do that, i don't see why caches should force you to write the if-not-cached-then-load-otherwise-cache-this-and-then-give-me 🙂 that part of the code is pretty universal, there is no point to repeat it everwhere 🙂

seancorfield05:09:03

@kulminaator For the vast majority of cases, the through-cache function is sufficient.

seancorfield05:09:41

The "Cache Stampede" that @orestis mentions is a rare possibility.

seancorfield05:09:16

We use core.cache and core.memoize very heavily at work and we haven't (yet) been bitten by that edge case.

kulminaator05:09:22

indeed, looks similar 🙂

seancorfield06:09:20

To be honest, if you have sufficient contention that a "Cache Stampede" is possible using the most straightforward idiom with core.cache (i.e., through-cache) then you're likely to have sufficient contention that the whole cache-in-an-atom approach is going to cause you problems anyway -- and you would need to move to a mutable cache rather than core.cache's immutable cache approach...

orestis06:09:19

Thanks @seancorfield for looking at that. Indeed, I think that we would hear production horror stories about Clojure itself if atoms where not working in practice.

orestis06:09:16

Are you memoizing things from the db? Commonly used entities? Other things? I’m very curious :)

seancorfield06:09:49

Yes, and that was the reason for adding the ability to core.memoize to ignore arguments -- in particular ignoring the DB connection pool argument that is the first one for all the java.jdbc functions.

seancorfield06:09:45

We have a lot of TTL caches.

seancorfield06:09:54

(and fairly high traffic)

seancorfield06:09:38

You have to have a lot of contention before you get retries on swap! I think. It's definitely a (theoretical) possibility tho'...

seancorfield06:09:30

Also, TTL caches have the added complexity that a lookup can expire a cache item so a lookup can return nil and then you have to retry the whole operation.

orestis06:09:07

Can you share some back of the envelope numbers e.g. requests per second on a single server that you are seeing?

orestis06:09:40

I guess results are heavily situation dependent, but some order of magnitude expectations would be nice.

lmergen08:09:51

"cache stampede" is more properly addressed not in the caching layer, but by adding a so-called 'circuit breaker' in the function(s) that query the actual database (i.e. the ones which you want to back off during stampede)

lmergen08:09:13

and if you want a java api that does this out of the box, take a look at Hystrix => https://github.com/Netflix/Hystrix

lmergen08:09:05

(to answer @orestis' question about patterns)

orestis08:09:07

Oh yes, I have this book on my desk but haven’t got around to reading it yet. Good to know @lmergen

bamdad09:09:56

hi, I’m trying to create a macro that takes a class and create an object, but if I pass the class as variable to the macro it doesn’t work. I’ve asked a question on stackoverflow here. I understand Macros don’t evaluate arguments like functions but I don’t know how can I fix this . https://stackoverflow.com/questions/52457652/a-macro-that-creates-object-in-clojure/

lxsameer10:09:21

hey folks, is it possible to just reload a namespace without reloading its dependencies ?

mfikes12:09:48

That should be the distinction between :reload and :reload-all

awb9916:09:08

does it mean the repl will recompile all functions in a namespace whenever I save the file?

emccue14:09:22

@crankyadmin you can probably make use of laziness for that

emccue14:09:35

for an example:

emccue14:09:59

(imagine some ...s while i write one up)

deliciousowl15:09:35

why not (cycle [1 2 3])?

emccue15:09:34

...because I didn't know about that function

deliciousowl16:09:02

:) clojure is cool like that. Things like (Repeat), (cycle), (repeatedly) make life easy for me. And I only recently found (interleave) and (interpose)

emccue14:09:24

Not sure if that actually helps with your surrounding code, but it does give youthe repeating behaviour

emccue14:09:02

if you can write your problem in terms of reduce or loop/recur then you dont need state (I dont think)

emccue15:09:42

yep ignore me cycle does all that

idiomancy23:09:02

hmmmm. want to write a macro that adds to the docstring of any forms within. that would amount to adding to the :doc entry in the metadata map, right? Is it just me or does that seem like something that might be more trouble than it's worth?

andy.fingerhut23:09:03

Is it very very important to you that you can wrap it around forms within? If instead you wrote a function that modified the doc string of a thing defined earlier, that could be implemented as a fairly straightforward function. As a macro wrapped around things, your macro would need to do some kind of scan over the forms and look for things that can have :doc metadata like def, defn, etc.

idiomancy23:09:57

what if it just assumes everything within can have metadata and I'm fine with it throwing an error if that's not the case?

andy.fingerhut23:09:16

If you restrict what can be inside to something where "the second symbol of every form is assumed to be the name of something being defined, and could have doc metadata -- the macro checks for doc metadata and if present, modifies it" could work with a pretty simple implementation, I suppose. I was perhaps overcomplicating things by imagining the general case of arbitrary code inside the macro form.

seancorfield23:09:30

I'd still write it as a function that I could point at a set of Vars and have it update their metadata. Easier to write and test, and then you could write a macro that wrapped forms and just added a call to that function based on the Vars it thinks are in its body.