Fork me on GitHub
#mount
<
2016-04-22
>
hoopes11:04:22

If I have a simple component with a db start/stop, and it contains the conn var - is it good to keep query functions in there as well, or is it best practice that they be in a different namespace?

(defn connect []
  (return-connection-object))

(defstate conn :start (connect))

(defn query [stmt]
    (jdbc/query conn stmt))

hoopes11:04:45

I'm starting trying to figure out unit testing/mocking/etc with rebinding (http://stackoverflow.com/questions/6796488/clojure-database-unit-testing-mocking) so i just wanted to make sure I'm not doing too much in the one namespace, or it's not composable, or it's just not idiomatic - I'm coming from python, fwiw

arnout12:04:02

Binding in the proposed way has its downsides, especially when your code under test is multithreaded. Ideally the conn state can be mocked, and all the other code would use that mock. I'm not sure such a thing is available for JDBC. Using a test DB is not an option?

arnout12:04:21

From a "mount" point of view, your code above looks fine.

arnout12:04:46

Where you put the query function does not really matter, they can be defined in the same namespace.

hoopes12:04:16

arnout: thanks! is there a better alternative for mocking a db for unit tests? rebinding seems to be all i'm finding via google so far. i kinda figured there would be the "here's how to unit test a db-heavy app" tutorial somewhere, but maybe i'm not looking in the right places (insert "programmers are too reliant on google these days" rant)

arnout12:04:01

I either use a test DB, or with Datomic, the Datalog queries can be performed on simple data structures.

arnout12:04:22

(many, not all of them)

hoopes12:04:07

haha, is that new? how did i miss that...thanks to both of you!

hoopes12:04:20

(i mean, that bit in the readme)

tolitius12:04:47

readme is big, easy to miss simple_smile

tolitius12:04:54

as to the query function, the idiomatic way is the one that makes most sense to you

arnout12:04:04

Ah, I thought that you already knew that @hoopes, as that is what I meant with > Ideally the conn state can be mocked, and all the other code would use that mock. I'm not sure such a thing is available for JDBC.

tolitius12:04:31

some prefer to have query in the same namespace as defstate db, which makes query kind of an API for db state

tolitius12:04:02

some just use it in a functional package, usually then query would take db as arg to be stateless

hoopes12:04:00

right, that was what i felt like i read elsewhere - send all the stateful stuff in as arguments, instead of directly using conn in the query function

tolitius12:04:07

but if you are ok with making the function (i.e. query) to close over conn state, this is also ok, as long as you understand the consequences

tolitius12:04:37

yea, there are alternatives

tolitius12:04:23

just try and see what works best for you. all these ways are good given the context

tolitius12:04:03

and shoot more questions as you're doing it if they pop up

hoopes12:04:07

i really appreciate your help - i obviously have a bit more studying to do

tolitius12:04:22

sure, no need to feel alone at it though 😉

aiba18:04:38

Sanity check: is it reasonable to start a core.async/go-loop inside the :start of mount/defstate? e.g. `

aiba18:04:40

`(mount/defstate my-queue :start (let [c (async/chan 1000)] (go-loop [] (try (my-consume (<! c)) (catch Throwable t (log/error t))) (recur)) c) :stop (let [n (count (.buf my-queue))] (when (pos? n) (log/warn "oops, dropped" n "items from queue"))))

aiba18:04:25

(ugh sorry for formatting, apparently that slack setting is per slack team, not global)

aiba18:04:29

(mount/defstate my-queue
  :start (let [c (async/chan 1000)]
           (go-loop []
             (try (my-consume (<! c))
                  (catch Throwable t (log/error t)))
             (recur))
           c)
  :stop (let [n (count (.buf my-queue))]
          (when (pos? n)
            (log/warn "oops, dropped" n "items from queue"))))

tolitius19:04:14

@aiba: the approach is good, but it does not seem you are closing the channel on :stop?

aiba19:04:36

Yeah I wasn't sure what impact closing the channel would have if there are pending takes from the go-loop

tolitius19:04:37

also go-loop returns another channel that will be always "recurring" since the exception is eaten in catch

tolitius19:04:44

I guess it is use case specific, but maybe you'd want to try to exhaust the messages from the channel on :stop

aiba19:04:07

yeah, i'd love to exhaust the messages, how would i do that?

tolitius19:04:10

(but again, this would depend on the use case / need)

tolitius19:04:33

would the producing side be stopped by this point?

aiba19:04:39

great question, i would hope so. i mean, in production where not dropping things from the queue matters, i would hope that this state is never :stopped

tolitius19:04:58

I see. yea it needs a bit more coordination thinking. for example, you can force producer to stop before this consumer by (mount/stop #'your.ns/producer)

aiba19:04:49

ah, you'd call mount/stop on the producer's defstate from within the consumer's defstate's :stop

aiba19:04:54

that's a good pattern to know about

aiba19:04:26

ok i will look into the return value of go-loop and make sure to close the channel. thank you for the thoughts and verifying this is not a totally crazy approach!

tolitius19:04:28

(sorry in the meeting): sure. I would not necessarily call (mount/stop #'your.ns/producer) from within a :stop method. rather call it from the place that shuts things down, whether this is a graceful shutdown or a some sort of last resort exception handler

tolitius19:04:12

but the overall approach of having go-loop/channel as a state is very valid, since this is a stateful resource

tolitius19:04:30

that needs to be started/created and stopped/closed