Fork me on GitHub
#clojure
<
2019-01-29
>
lilactown01:01:12

does anyone have an example of a system (a la component/integrant/mount) whose resources have to be stopped in order?

lilactown01:01:26

even just some hypothetical cases would help

noisesmith01:01:59

@lilactown hypothetical: using zookeeper for service discovery / leader election, you need to withdraw from any roles your instance took on, and do any zk bookkeeping, before shutting down your zookeeper component that manages your connection to the service

noisesmith01:01:38

(of course if it's properly designed the failure case should handle that already, but you might be able to do something simpler if a node explicitly and gracefully leaves)

lilactown01:01:43

nice! does that match what I’m talking about though? I mean that if component A depends on B:

start:
A -> B
B must be started before A. I’m wondering how often it’s important that B also be stopped before A

lilactown01:01:29

in this case, are you saying that you have two components (e.g. one for service discovery, one for the component itself)?

noisesmith01:01:33

oh! - I thought you meant "in order" as in "opposite from starting order" which is how it's done...

lilactown01:01:11

I guess? I actually haven’t used component in a while and it never mattered in my cases.

lilactown01:01:30

if I just stop things in random order, what goes wrong? 😛

noisesmith01:01:32

@lilactown in my example the hypothetical service isn't the only usage of zookeeper, so the zk component is shared by 2 or more components

noisesmith01:01:23

sometimes shutdown of B requires usage of A

lilactown01:01:25

OK, so you’d probably want to withdraw before you shut down the rest of your system so you don’t get incoming requests during your shutdown

lilactown01:01:36

ah good point

noisesmith01:01:46

if there's any tidying up needed - of course we usually try to build things so worst case we don't leave state dangling

noisesmith01:01:14

another hypothetical: there's a lock file on a filesystem where two programs run (eg. an editor), you'd want to shut down the part of your app that wants to write the file before shutting down the part that does file locking and directory watching

noisesmith01:01:44

and on startup you want to open the part that watches and locks files before the part that wants to write them

lilactown01:01:17

this is great!

noisesmith01:01:11

often we avoid this by not splitting concerns, but in a focused enough file management oriented app, I can imagine separate components

hiredman01:01:21

clean stopping is often more important during development then in production

hiredman01:01:51

so you can stop the app, and leave things in a good state were you can start it again without restarting the jvm

👍 5
noisesmith01:01:04

oh, and a much more common / obvious example than the others: imagine if your http server was still running and handling requests, but your db access handler was shutting down

lilactown01:01:35

I think that’s the example I’ve been running with so far, and I don’t really care that much

noisesmith01:01:01

I guess a spurious 500 error isn't hard to handle

lilactown01:01:10

I am thinking about this mainly for a dev POV like @hiredman said though

lilactown01:01:45

so e.g. the file locking example was great, I hadn’t thought about things like that

lilactown01:01:08

not everything is a web app 😛

Vikram04:01:32

Hi everyone, I'm new to clojure and clojure.test so can anyone tell me how to write test cases for any project.

seancorfield05:01:00

You'll find the #beginners channel very helpful -- people there have opted in to helping folks who are new to Clojure @vikram

Vikram05:01:42

Thanks @U04V70XH6 for the suggestion.

Vikram04:01:52

Hey @unni, Its like I have read that link and I've created a project in which I wrote a simple test and after running cider and using C-c C-t p, it is saying that "Running test in all namespaces" in minibuffer but it is not showing the output.

Vikram04:01:23

(ns testcase.core (:require [clojure.test :refer :all] [ring.adapter.jetty :as jetty] [ring.middleware.reload :refer [wrap-reload]] [compojure.core :refer [defroutes GET]] [compojure.route :refer [not-found resources]] )) (deftest addition (is (= 3 (+ 1 1))) (is (= 4 (+ 2 2)))) (defn -main "A simple project to runtest case" [port-number] (jetty/run-jetty (wrap-reload #'addition) {:port (Integer. port-number)})

unni05:01:08

@vikram that is happening because you have this in src namespace and not test namespace from the looks of things.

Vikram05:01:41

Okay thanks.

ska12:01:38

Shouldn't https://github.com/edn-format/edn/wiki/Implementations also mention Clojure's built-in https://clojure.github.io/clojure/clojure.edn-api.html? For anyone trying to use EDN format from Clojure, it'd be really nice to get a pointer which one should use. According to https://groups.google.com/forum/#!topic/clojure/d61ImK2VCag the built-in is just fine.

Alex Miller (Clojure team)13:01:38

Feel free to add it

✔️ 5
Ben Hammond15:01:55

I have a repl with refs to datomic entities defined. Periodically I wish to refresh these to include recent novelty. A macro approach could look like

(defmacro refresh-datomic-entity-as-macro
  "redefine ent-sym to latest version of entity"
  [ent-sym datomic-conn]
  `(def ~ent-sym
     (d/entity (d/db ~datomic-conn) ~(:db/id (deref (resolve ent-sym))))))
but realised that I could alternatively write the function
(defn refresh-datomic-entity-as-fn
  [ent-var datomic-conn]
  (.alterRoot ent-var (fn [e] (d/entity (d/db datomic-conn) (:db/id e))) '()))
normally I would always choose a function over a macro but perhaps not if that required the interop call to clojure.lang.Var/alterRoot How would you choose between them?? are they equivalently awful?

seancorfield15:01:51

As opposed to alter-var-root? @ben.hammond

👍 5
😊 5
Ben Hammond15:01:41

:ahem: hadn't spotted that function

Ben Hammond15:01:05

haha I've never written a macro that I've ended up keeping

seancorfield17:01:10

I didn't know .alterRoot existed so that's TIL 🙂

lilactown16:01:05

I just want to profess the ridiculous feeling of joy and power I have when using Clojure. 200 lines to achieve what in my other proficient languages would be 2,000. I know it’s not a real measuring stick, but every once and awhile I come up for fresh air and look at some of the stuff people are doing, and realize how much more approachable Clojure code bases are. Thank you Rich/Stu/Alex/et. al.!

cider 20
clj 65
parrot 5
borkdude16:01:37

What are your other proficient languages?

lilactown16:01:19

JS, Java, ReasonML/OCaml. in that order 🙂

borkdude16:01:06

I’m curious about your findings about Clojure vs OCaml

lilactown16:01:30

I could probably write a long post about it 😄 one of bigger reasons, honestly, was that we started rewriting our systems at work last year and needed something to standardize on. Clojure was just a better bet back then. Still is

lilactown16:01:52

IMO Clojure(Script) is the best “blue collar” language out there

borkdude16:01:03

I’d be interested in the longer blog post. For now, thanks 🙂

👍 5
5
taylor16:01:54

I came from an OCaml-ish language (F#) to Clojure and wrote something about the transition here https://blog.taylorwood.io/2017/09/15/year-behind.html I also feel ridiculously empowered when writing Clojure. I did C#/Java/C++ for ~15 years before learning functional programming FWIW

Mattias16:01:10

Blue collar probably doesn’t translate across cultures/regions with any precision. I’d be interested in the longer motivation? 🙂

borkdude16:01:05

@UCW3QKWKT I interpret that as the daily work of an average programmer

borkdude16:01:01

@U3DAE8HMG thanks, I already read that one and enjoyed it

5
Wes Hall16:01:32

@lilactown Absolutely love the, "blue collar" programming language metaphor, while I understand the response about it not translating it made deep and intuitive sense to me immediately and will now be stolen. Bravo.

seancorfield17:01:45

I've probably taken about a dozen languages to production -- and I've learned another dozen while doing that (and I came out of university fluent in about another dozen before that) -- and Clojure feels, for me, the most fun and productive of any language I've ever used.

lilactown17:01:30

I can’t take credit for the analogy! I definitely stole it from somewhere 😛

Alex Miller (Clojure team)16:01:41

tell your friends :)

💥 20
lilactown16:01:37

well, I “made” everyone at work switch to it 😛 doing my duty

metal 20
andrea.crotti16:01:33

it already happened a few times that we change something, everything works in the REPL and running tests, but then eventually you run the Uberjar generated and it blows up

andrea.crotti16:01:14

is normally related to silly things like slurp of files not in resources, and similar, but it would be nice to find out earlier on

andrea.crotti16:01:59

one way for example could be to run the uberjar generated for a few seconds and do an HTTP call to see if it started up correctly, has anyone done something like that?

lilactown16:01:49

it’s always useful to test things in a release-like environment before deploying. We use docker at work to do that

lilactown16:01:46

it’s also pretty standard practice to have health checks once deployed

andrea.crotti17:01:46

we also use docker

andrea.crotti17:01:04

so I can run something on the docker image

andrea.crotti17:01:27

was just wondering how to run for example jetty for a few seconds until it's up to try to do a HTTP call

andrea.crotti17:01:56

and we also have health checks, however when there are these issues marathon can't even deploy the changes, and it just loops forever

lilactown17:01:23

are you asking how to do this as an automated test in CI?

andrea.crotti17:01:21

just wanted to know if I was the only one with this problem, and others solved it

chrisulloa16:01:57

i told a friend about clojure and he told me it sounded like i was in a cult

😂 40
Wes Hall16:01:42

Weird. I told my friend about a cult I joined and he told me it sounded like being a clojure developer.

😂 35
💯 5
🔥 15
andrea.crotti17:01:23

writing Clojure with Emacs is a double cult then 😄

manutter5117:01:41

All the best cults write clojure.

mokr17:01:49

Hi, has anyone used com.taoensso/carmine with Redisgraph? I can successfully (car/PING) my running Redisgraph Docker container, and I see four graph commands listed when I run (wcar* (car/command)). But, when I try e.g. (wcar* (car/graph.QUERY "social" "CREATE (:person {name: 'roi', age: 33, gender: 'male', status: 'married'})")), it tells me No such var: car/graph.QUERY. Haven’t used carmine or Redis much previously, but wanted to have a go.

devth17:01:30

👋 any known lib/utils that given a string: 1. tell me if it's a valid uri 2. tell me if that uri refers to an image of any kind

Wes Hall17:01:27

You can use the URI class constructor in the JDK to check for validity. It will throw an exception if it invalid (assuming you mean structurally valid, as opposed to the resource it points to actually existing).

👍 5
Wes Hall17:01:45

Number 2 sounds hard though. http://myserver.com/pictures/myface is a valid URI. You'd have to do a HEAD at minimum to see what's there though.

devth17:01:09

yep, makes sense. thanks for the suggestions

Wes Hall17:01:34

np 🙂 I also could be wrong, it has been known to happen.

devth17:01:56

https://github.com/michaelklishin/pantomime#detecting-mime-type looks pretty good. planning to test it out on some edge cases

noisesmith17:01:00

a warning about java.net.URI: simply creating one will talk to the network

5
Wes Hall17:01:25

Oh yes, I vaguely remember that now. The JDK is bags of fun.

noisesmith17:01:32

or maybe it's equality checks - but what I definitely remember is strange unexpected network interactions

devth17:01:48

i remember it too 🙂 do you even side effect?

nwjsmith18:01:06

@noisesmith do you know where that’s documented? I knew that was the case for java.net.InetAddress, but URI is surprising :S

noisesmith18:01:31

@nwjsmith thanks for making me double check: URL has this behavior, URI is safe(!) > Equality, hashing, and comparison are defined strictly in terms of the character content of the instance. In other words, a URI instance is little more than a structured string that supports the syntactic, scheme-independent operations of comparison, normalization, resolution, and relativization. https://docs.oracle.com/javase/7/docs/api/java/net/URI.html

noisesmith18:01:31

I is only four letters away from L, and looks very similar in most fonts, which is my excuse for my faulty memory

piobaire20:01:26

@manutter51 gives a whole new meaning when you see someone joined #clojure .....

😄 5
Alex Miller (Clojure team)20:01:15

to join, just send me a check for $1000. riches will ensue.

rich 50
manutter5121:01:10

Haven’t even sent in the money yet, and Riches are already ensuing.

chrisulloa21:01:39

shut up and take my money

dominicm21:01:18

Is there a caching system where if the result is not available, then the old one is used until it is? Eg for a variable time api

dominicm21:01:17

Core.cache doesn't seem to have something like this. I feel like I could try something with a futures and deref, but state gets a little complicated, and utilizing someone else's state management seems reasonable.

Alex Miller (Clojure team)21:01:03

if the result is “not available”, then how can there be an “old one”?

Alex Miller (Clojure team)21:01:32

sounds like maybe what you want is a map + a way to trigger a background refresh ?

Alex Miller (Clojure team)21:01:00

I’m not sure I would call that a cache

Karol Wójcik21:01:56

Is there a plan of releasing the new version of tools.deps.alpha with ’add-lib function? It’s needed to add hotload-dependency functionality to clj-refactor., because otherwise I’m unable to add the tools.deps.alpha using it’s hash.

Alex Miller (Clojure team)22:01:18

it needs some more work before it gets added, which is likely not going to happen soon

Alex Miller (Clojure team)22:01:32

so, no there is no plan currently

Karol Wójcik22:01:58

May I please take the code from add-lib branch and create a package out of it? At least for some time until the feature lands to tools.deps.alpha

Alex Miller (Clojure team)22:01:11

It’s open source… just retain the license and copyright

dominicm21:01:08

@alexmiller you're right, I think an atom & a thread is probably good here. My only slight problem is eviction.

dpsutton21:01:58

core.cache has a protocol. one of the functions is evict. maybe you could look at the LRU there and modify it so that evict did not evict but refresh?

borkdude22:01:14

@dominicm I have exactly this problem and solved it with an atom with a map of futures. I need a value for every user of the system. Each user has a future, either running or ready. I do a background refresh, so I don’t throw away the future that’s already done while the new one is running.

borkdude22:01:56

The problem is that futures don’t have a way to say: on-ready, put yourself in this map and swap out the other future. So I’m dereferencing it in the background, with a timeout. When done (or the timeout expires), I swap the old value.

borkdude22:01:18

I’m looking into manifold to see if there’s a better way for that last part.

devth22:01:39

> if the result is not available, then the old one is used until it is i seem to build or want this in every lang or system i work on 😂

Alex Miller (Clojure team)22:01:46

you could give the future the atom and then have it do the swap?

Alex Miller (Clojure team)22:01:32

I don’t know the Guava lib well enough to know off-hand if this is covered, but it has a ton of cache variants and the code is very high quality

borkdude22:01:58

@alexmiller then I would need a way for the future to refer to itself

Alex Miller (Clojure team)22:01:30

it’s waiting for something - when it’s done waiting, do the work

borkdude22:01:00

because I store the futures, not the resulting values. if there’s already a future running (and not cancelled), I wait for that one. I could store the resulting values separately in another atom of course, but I still need the state of those futures.

noisesmith22:01:36

@borkdude a future is effectively a Thread plus try/catch/finally that delivers to a promise

👌 5
noisesmith22:01:05

you could make a new version that lets the code in that Thread access the promise directly

noisesmith22:01:39

of course a clojure future also does some special var binding boilerplate, but it's a relatively small chunk of code

borkdude22:01:05

@noisesmith so instead of storing the futures, I would store the promises?

noisesmith22:01:41

yeah, which I think have the semantics you want - they are deliver-once, and you can tell if they've been delivered too

noisesmith22:01:02

maybe you want a pair of promises - one for the thread itself (has this work started) and another for the result (has it completed / what did it return)

noisesmith22:01:41

I'm a huge fan of delay and promise, you could also use atoms for what they do, but they have the extra guarantee that they never mutate once realized, a one time switch

borkdude22:01:54

I’ll think about it some more, thanks for the idea.

hiredman22:01:08

a pair of promises is almost always a way to get race conditions