Fork me on GitHub
#beginners
<
2018-11-01
>
stopa00:11:30

Hey all, I am looking for a weird kind of cache -- I wonder if something like this exists -- I want an "aync ttl cache" The idea is this: - The cache behaves like a normal ttl cache - But, when the ttl miss happens, it asynchronously runs the expensive operation - While the operation is running, it will still return the old value - When the operation is done, it will return the new value, and re-request it async when the time comes The idea is to be efficient with frequent hit scenarios, and make refreshing the cost "inexpensive" to the caller

stopa00:11:37

as anyone heard of this kind of thing before?

hiredman00:11:10

doesn't that mean it can never expire?

hiredman00:11:49

in order to be able to return the most recent value while the async refresh is running, it must keep the value around

hiredman00:11:28

so maybe that isn't a ttl cache, maybe it is a map with a background task refreshing the data in it every time period

seancorfield01:11:10

There's actually a JIRA ticket open against clojure.core.cache for something that would support this, as I recall.

seancorfield01:11:56

Right now, core.cache doesn't expose enough machinery to make it possible to build it outside the library and I'm not sure I want to build it inside the library...

jaawerth01:11:43

it sounds like you'd need to do something akin to what reagent does when a ratom is deref'd - basically have a macro you use to wrap anything that uses the cache, and when a cache lookup happens, register a handler that re-runs the outer operation 9which would have to be safe to do) when the value is updated, then kick off the async update

seancorfield01:11:14

...hmm, maybe it never got turned into a JIRA issue. Definitely been discussed.

jaawerth01:11:30

or just have the cache accept callbacks

jaawerth01:11:52

that would run immediately and, if TTL is expired, rerun after the async update

seancorfield01:11:20

core.cache supports a lot more than TTL 🙂

jaawerth01:11:39

yeah I'm referring more to the original request

jaawerth01:11:54

it would be a LOT of work to do all that and also be able to do all the stuff core.cache can do

jaawerth01:11:18

but hiredman is right, it wouldn't really be a TTL cache - if you still want to be able to provide the old value after the TTL has expired, you're never actually deleting it

jaawerth01:11:24

kinda defeats the purpose

seancorfield01:11:55

Well, you'd return the old value until the new value had been calculated 🙂

seancorfield01:11:15

And the principle is fine -- for situations where you do an "atomic" read-and-recalculate-if-missing (which is sort of what through-cache is for).

jaawerth01:11:01

but yeah this reminds me a lot of how ratoms work

seancorfield01:11:05

Although, for TTL caches there is an additional complication that reading the value can cause it to expire (and the read can return nil).

jaawerth01:11:02

oh, interesting. I didn't think of that

jaawerth01:11:29

it's almost like you're a major contributor to core.cache or something 😄

seancorfield01:11:13

core.memoize has to workaround that core.cache TTL issue which is where I became overly familiar with that 🙂

dangercoder12:11:14

how can i create a char-array which I can use in Java? trying to use a java librairy which takes in a string like .. "foo".toCharArray()

dangercoder12:11:47

I tried clojures native (char-array "string") but without success

jumar12:11:45

@jarvinenemil char-array should be fine:

(.read (java.io.CharArrayReader. (char-array "HELLO")))
;;=> 72

dangercoder12:11:11

ill try it thanks

pauld12:11:34

If you are getting a constructor error you may need to type hint ^CharArray so the correct constructor is getting chosen. What error are you seeing?

dangercoder16:11:05

I solved it with (char-array) there was an issue further up in the code.

jstaab14:11:43

In javascript I use closures to create modules to inject runtime dependencies, e.g.:

export default config => ({x: param => config.flag ? param + 1 : param -1, y: ..., z: ...})
So this lets me create a module bound to my app's config, which gives me a function (or class, object, component, whatever) that I can then call in my app's context, without my module knowing anything about the app. Is there an idiomatic way to do this in clojure? It looks like defn called in a closure binds a var to the top-level of the namespace, which isn't really what I want, since I want different instances of the defined functions to be bound to different configs. TBH, this looks a lot like a class instantiation. Maybe a macro like (defclosure [config] :x [param] (if (:flag config) (inc param) (dec param)) :y ... :z ...) that would translate the keyword arguments into bindings? Although I do want some shared mutable state in there sometimes.

jstaab14:11:23

I guess the vanilla version of this would be

(defn make-module [config]
  {:x (fn [param] (if (:flag config) (inc param) (dec param)))
   :y ...
   :z ...})
Which isn't so bad, if a little noisy with all the fn calls.

jstaab14:11:09

What if I wanted to define a multimethod in there though?

jstaab15:11:59

Maybe reify? Still, doesn't seem to support multimethods.

lilactown16:11:51

I'm confused what you're trying to do. you posted a JS example using export default, but you're worried about defn setting a var at the top-level?

lilactown16:11:01

how would you do this in JS, first?

jstaab18:11:17

So maybe a bit cleaner example, with multimethods instead of regular functions because that seems to be harder to do. In javascript:

function createMyMulti(config) {
  const dispatch = x => x > 10
  const mm = defmulti('myMulti', dispatch)

  mm.addmethod(false, x => x + config.increment)
  mm.addmethod(true, x => x)

  return mm
}
This allows me to inject some kind of parameter once using config (silly example here), and get a pre-built, complex multimethod out of it. I might for example pass in a path to my global state object as in redux and instantiate multiple pre-build action creators that operate on different parts of the tree. So in clojure maybe:
(defn create-my-multi [config]
  (let [symbol (gensym)]
    (eval `(defmulti ~symbol [x] (> x 10)))
    (eval `(defmethod false [x] (+ x (:increment config))))
    (eval `(defmethod true [x] x))
    (eval `symbol)))

jstaab18:11:46

This is obviously utterly insane, and I don't know if it would work, but I can't think of another way to do it (other than write a non-macro version of defmulit)

lilactown18:11:33

it seems like you could do this easier with regular functions?

lilactown18:11:27

I feel like you’re convoluting the JS example to match the solution you’re exploring in Clojure right now

lilactown18:11:29

e.g. your example in your first post could be written like:

(defn create-context [config] 
  {:x (fn [param] (if config.flag (+ param 1) (- param 1))
   :y ... })

lilactown18:11:00

returning a map of key-vals where the vals are anonymous functions (your “methods”) and other things you want to attach to the context

lilactown18:11:19

if you want to be more formal, you could look into defrecord or deftype which probably offer the dynamism you’re looking for - multimethods do not see to be a good fit for this at all

jstaab18:11:50

It's more an exploration of the problem space; multimethods are a construct I find really useful, but it's a foot gun if used in a closure because it binds global vars. The examples are for sure contrived, but the problem I'm trying to solve is dependency inversion, where the module takes an adapter of some kind (in the examples it's just logic, but it could just as well be a function that makes an api call or executes a query), and uses it without knowing what the parent is doing. I've seen things like integrant and component, but I'm looking for more of a general purpose solution than something oriented to the lifecycle of a system. I guess another way to articulate what I'm trying to do is to define a new namespace at runtime to hold the vars def sprays around.

jstaab18:11:50

Hmm that's an idea. Maybe I could use create-ns and then do (binding [*ns* (create-ns 'x)] (def x 1))?

jstaab18:11:56

Still weird but...

lilactown18:11:39

I think you want to use either plain maps and anonymous functions, or protocols and types

lilactown18:11:50

You should not be using macros for this

lilactown18:11:11

Multimethods are 💯 the wrong abstraction

jstaab18:11:18

Even if I want them open to extension? So 1. call a function that return a pre-built multimethod bound to a namespace, 2. defmethod on it in my application, 3. call it

jstaab18:11:13

Again, it's not really about multimethods in particular, just module factories. Although I guess you've probably answered my question as to whether it's idiomatic at this point 😅

jstaab19:11:24

Anyone else have any input on this thread?

hiredman19:11:42

use component

hiredman19:11:53

pass functions the arguments they need

hiredman19:11:41

don't use eval and defmethod, etc

jstaab19:11:50

Huh, ok. I've looked at integrant, and it looked like you couldn't really have multiple instances of the same component. Is that true/same with component?

hiredman19:11:30

for example, somewhat recently we had a thing at work were we had been planning to bill paypal through braintree, but for some business reason we couldn't link our paypal and braintree accounts, but the braintree sdk can connect directly to paypal, so now we have two instances of our braintree component, one that handles credit card transactions and one that handles paypal

hiredman19:11:57

exact same code, just handed different config at start up

jstaab19:11:11

But you have to do that explicitly at configuration time right? Like, are you able to add/remove (not just start/stop, but re-configure) components dynamically while the system is running?

hiredman19:11:58

component systems are also nestable (I generally recommend against doing that though), so for are messaging system at work, when I want to test if clustering works, I created a system with two components, each one just a copy of the complete message system

lilactown19:11:23

Jstaab I'm still super confused why using maps and functions don't fit your use case

jstaab19:11:26

Huh, ok I'll go play with it and see what I discover. Thanks for the help @U0NCTKEV8 @U4YGF4NGM

hiredman19:11:30

component doesn't explicitly give you a way to reconfigure, but I think it would be possible, it does give you a way to run arbitrary functions in dependency order (which is what its start and stop are built on)

lilactown19:11:41

Multimethods are too static for what you want

hiredman19:11:07

but in general I find starting and stopping to be no-fuss enough that I just do that, no need for a reconfigure

lilactown19:11:02

if integrant doesn't seem flexible enough (or you want to get your hands dirty) I would look up protocols and types - they are I think what you're looking for

jaawerth21:11:51

I've done this on a lower level with both a function that returns a map of functions with the config partially applies, OR a record that stores the config and has protocols wrapping the various dependency functions. You could probably automate this further by defining an EDN/map spec of method->[required function signature of dep with regard to config], possibly even reflecting on the deps in question but that seems dangerously magical

jaawerth21:11:48

But yeah I think passing in the config (which is stored via a key or index) with an adapter function that knows how to apply it to each dep is the most reliable way to do it

jaawerth21:11:12

(def db (configatron9000 db-spec apply-db-spec-to-db-methods), where apply-db-spec-to-db-methods could either be a multimethod that dispatches based on each function in the dep OR a map of method/adapter-fns if they all differ

jaawerth21:11:50

actually.. a protocol is more or less exactly this, isn't it. lol

Logan Powell16:11:22

👋 Hi everyone! Is there a way to "wait" on an atom before derefing it? I have a library that loads a remote resource and reset!s a shared atom, but I have to call it twice for the atom to be hydrated. If possible, I'd like the deref'ing functions to wait until it's been reset! without having to wrap all the functions in an (if.. statement...

Logan Powell16:11:48

perhaps I should use a promise-chan?

schmee16:11:33

yes, that’s exactly when you want a promise or a promise-chan!

dangercoder16:11:11

how can i download a dependency/lib from a master branch? in a lib with leiningen , git pull? and then, any guides?

schmee16:11:40

clone the repo, then lein install and use whatever that snapshot version in project.clj is where you want to use it

dpsutton16:11:17

you can also use checkouts where you just symlink to a local copy from your-project/checkouts

dangercoder16:11:01

clj-http is using an old version of Apache Http-client 🙂

dangercoder16:11:06

in their stable version

dpsutton16:11:05

So you need a local copy of a transitive dep?

dangercoder16:11:38

yeah I need to use their "Master" version in my project.

seancorfield18:11:34

@jarvinenemil You can just add :exclusions on clj-http to exclude the old lib you don't want and then add an explicit dependency on the new version you do want.

👍 4
seancorfield18:11:24

:dependencies [[clj-http "a.b.c" :exclusions [the.apache/library]] [the.apache/library "x.y.z"]]

hiredman21:11:45

it looks like you have something setup to resolve everything to localhost

hiredman21:11:36

what does nslookup say?

awb9921:11:28

.nslookup http://repo.cojars.org Server: 192.168.43.1 Address: 192.168.43.1#53 server can't find http://repo.cojars.org: NXDOMAIN

awb9921:11:21

strange. I can reach http://www.clojars.org without a problem.

hiredman21:11:28

you misspelled the domain

awb9921:11:44

nslookup http://repo.clojars.org Server: 192.168.43.1 Address: 192.168.43.1#53 Non-authoritative answer: http://repo.clojars.org canonical name = http://v.ssl.global.fastly.net. Name: http://v.ssl.global.fastly.net Address: 151.101.197.128

awb9922:11:39

does leiningen read settings from some file?

awb9922:11:52

I guess I need to look into some kind o user specific settings ffiles...

awb9921:11:56

I am sorry. I had a typo before.

awb9921:11:24

I installed leiningen via arch repos, on a fresh machine, without any customization.

genmeblog22:11:13

question: how to make an instance of the nested class which is not static?

genmeblog22:11:59

in Java

class OuterClass {
    ...
    class InnerClass {
        ...
    }
}
OuterClass.InnerClass innerObject = new OuterClass().new InnerClass();

markmarkmark22:11:04

I think that (new OuterClass$InnerClass) should work

genmeblog22:11:29

yes, that works when InnerClass is static

genmeblog22:11:03

but in my case I want to make an instance starting from object

hiredman22:11:14

I kind of forget, but I believe the way javac generates those classes is it inserts an extra argument in all the inner class constructors to take an instance of outer object

noisesmith22:11:17

what does "make an instance starting from object" mean?

genmeblog22:11:40

OuterClass.InnerClass innerObject = new OuterClass().new InnerClass();

markmarkmark22:11:53

I didn't even realize that you could do .new like that.

genmeblog22:11:05

new is called on an object

hiredman22:11:40

so something like (OuterClass$InnerClass. (OuterClass.))

genmeblog22:11:34

ok, checking then

genmeblog22:11:57

thank you very much

hiredman22:11:54

in general, clojure doesn't have java interop, it has jvm interop so in places where the java language doesn't map 1:1 to the jvm, clojure gives something that matches the raw jvm, not something that matches the java language

genmeblog22:11:24

so it's good to know some jvm internals

hiredman22:11:31

so if you want to know how to something in clojure, look in to how that is represented as jvm bytecode

jaawerth22:11:31

interesting. this is my ignorance of javaland speaking, but are there significant parts of the java stdlib that have these? or is it primarily userland?

jaawerth22:11:13

yeah I'm just trying to think of whether I've encountered that directly, and not via a method on a class that uses some other class in a not-so-nested fashion

jaawerth22:11:05

I've spent too much of my career avoiding java so the standard lib is still something of a foreign country to me barring what I've interop'd with from clojure

noisesmith22:11:22

user=> (class (first (doto (java.util.HashMap.) (.put :a 0))))
java.util.HashMap$Node
user=> 

noisesmith22:11:28

just off the top of my head

jaawerth22:11:58

okay yeah I've seen lots of that lol

jaawerth22:11:05

wasn't putting 2 and 2 together

genmeblog22:11:27

I think non-static nested classes are rare

jaawerth22:11:51

I've certainly seen plenty of static ones

noisesmith22:11:56

it's pretty reliable that things with $ in the middle of the name were created by the javac inner class syntactic sugar (which as we know just creates a class with a funny name as inner classes are not a vm feature)

jaawerth22:11:28

yes, I'm embarrassed it admit it took me a sec to put that together after reading the backscroll here 😉

hiredman22:11:38

also methods with $ in the name

markmarkmark22:11:44

I've only ever used non-static inner classes inside the classes they're nested in. and at that point you can just do a regular new to instantiate it.

Jonas23:11:35

Hi everyone, I want to create a website using hiccup, compojure and boot, and I would like my browser to refresh automatically when I change my hiccup code. I'm using boot-http reload to update my server, but I still have to refresh the browser manually. Is there an easy way to add automatic refreshing to my boot project?

chrisulloa23:11:28

I’m sure there are other ways to do this, but have you looked into figwheel? @jonas573

Will23:11:39

In my java spring boot app, I am trying to test a java method, that calls clojure code, against an in memory h2 database. I have my db configuration in my clojure code setup like this

{
    :classname   "org.h2.Driver"
    :subprotocol "h2:mem"
    :subname     "demo;DB_CLOSE_DELAY=-1"
    :user        "sa"
    :password    ""
   }
when running the test the clojure code hangs forever right after this log statement Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource My java code is able to access this databse fine. Anybody know why it appears to have a hard time connecting?

noisesmith23:11:59

why are you initializing c3p0?

Jonas23:11:10

@christian.gonzalez I have, but as far as I can tell, it only refreshes when the cljs and css code changes, not the clj. Can I configure it to also refresh on clj changes?

chrisulloa23:11:31

I think you can specify reloading of clj

chrisulloa23:11:34

:reload-clj-files {:clj true :cljc false}

Jonas23:11:40

That's the lein version. I'm looking at boot-figreload and it doesn't seem to have that option. Lein seems to have much better support for this, but I'd prefer to keep using boot

chrisulloa23:11:35

gotcha, good luck!

Jonas23:11:09

Thanks 🙂

Will23:11:04

Doesn't that create a connection to the database through the clojure.java.jdbc library?

noisesmith23:11:10

c3p0 is a connection pool, I'm not sure if it even works with h2

noisesmith23:11:31

I'm not sure if h2 even has such thing as "connections" given its in memory

hiredman23:11:37

h2 has some kind of nasty behavior for concurrent reads/writes

hiredman23:11:24

so I dunno, but you could be running in to some kind of locking deadlock

Will23:11:14

Ok that makes sense, so I can probably query normally without setting up the connection pool right?

noisesmith23:11:29

right, just providing the h2 config instead of a pool