Fork me on GitHub
#clojure
<
2020-05-26
>
emccue00:05:32

@roguas So, a large part of your pain is what stuff like system is designed to solve

emccue00:05:05

but if you can manually "start" things, then it should all work out okay

emccue00:05:33

effectively, at the root level you can make a map of your stateful components

emccue00:05:51

which is usually just a database connection, maybe some pooled clients, etc

emccue00:05:21

and pass those in to where they are needed via destructuring the map

emccue00:05:29

so for your config example - assuming your config is just an atom of a map

emccue00:05:23

(defn load-from-path [path]
  (atom ... path ...))

(defn current-value [config]
  @config)

(defn check-for-updates [config]
   ...)

emccue00:05:32

and outside your config namespace

emccue00:05:49

(:require [your.company.config :as config])

(defn make-system []
  (let [config (config/load-from-path)
        queue (create-queue)]
    (start-webserver {:context-for-requests {:config config :queue queue}})))

emccue00:05:53

something like that

emccue00:05:12

so yeah, you end up with some degree of "this ns creates an object"

emccue00:05:28

but an atom isn't the appropriate object usually

emccue00:05:46

so in the case of config, you could imagine maybe reloading it at runtime

emccue00:05:05

and wanting different parts of your code that rely on those values to update themselves

emccue00:05:39

which would necessitate either some kind of restarting of the whole system or of keeping a shared reference to mutable state

emccue00:05:47

which is exactly what an atom is for

emccue00:05:03

but you shouldn't have a ton of atoms

emccue00:05:20

since truly there is only one place where changes can happen

emccue00:05:36

but you should never be creating an atom of the s3 client which you correctly identify

emccue00:05:52

you should start out with something like

emccue00:05:56

(defn create-s3-client [{:keys [whatever values you need from config]}]
  (AWSS3Client. ...))

(... any functions you want to put here, but probably you don't need any if you use the client apis directly ...)

emccue00:05:06

and then up here

emccue00:05:40

(defn make-system []
  (let [config (config/load-from-path)
        s3 (s3/create-s3-client @config)]
    (start-webserver {:context-for-requests {:config config :s3 s3}})))

emccue00:05:36

and bubble stuff down

emccue00:05:00

which again, if you don't need to update stuff at runtime, the atom isn't needed

didibus00:05:03

Ya atom isn't appropriate for that. Better use delay or promise.

didibus00:05:28

Or an actual component library

emccue00:05:42

but if you did then the s3 client as I described wouldn't work

emccue00:05:51

and you would need to do this

emccue00:05:35

(defn create-s3-client [stateful-config]
  {:config-reference stateful-config})

(defn make-put [s3-client stuff]
  ;; What your code now calls s3-client is a stateful object, but not the same as you got from the library.
  ;; So you need to echo through any api calls you want
  (let [current-config @stateful-config
        client (S3Client. current-config)] ;; In this case the client isn't stateful, its dependencies are
    (aws.s3/put! client stuff))) ;; But if you needed to, you could also store the client along, manage deps, etc

emccue00:05:09

so in general - namespaces defining objects and functions in those namespaces being considered "attached" to those structures

emccue00:05:16

not a bad idea

emccue00:05:35

namespaces working themselves like a stateful entity and being singletons

emccue00:05:40

definitely a bad idea

emccue00:05:07

you can make it work somewhat if you use something like mount, which will give you the machinery to mock out namespace vars for testing

emccue00:05:28

but it doesn't help the "hard to understand and reason about" part

emccue00:05:11

the missing part here is how to do this stuff in a REPL, but you can get to that later

emccue00:05:19

since truly you have bigger problems

emccue00:05:50

and yeah you should be appending an extra param to everything

emccue00:05:59

pretend you saw this java

phronmophobic00:05:18

ymmv and it depends on the use case, but I would still prefer a design that separates what from how instead of one where every function has an extra parameter

πŸ‘ 4
emccue00:05:45

public enum S3Client {
  private static final Config config = Config.INSTANCE;

  INSTANCE;

  public void putObject(Object obj) {
    new AWSS3Client(config.port()).send(config.rateLimit());
  }
}

emccue00:05:25

public enum Config {
  private static final Map<String, String> configVals = ...;

  INSTANCE;

  public String port() {
    return this.configVals.get("port");
  }

  public String rateLimit() {
    return this.configVals.get("rateLimit");
  }
}

emccue00:05:52

this is essentially what you get with namespace level atoms

emccue00:05:54

so write your code like you would write java - if you need to make a "new" api for things, give those things their own "class" and "methods" and "constructor"

emccue00:05:06

then compose them together how you need to

emccue00:05:15

don't make singletons

emccue00:05:30

(which i know is moot advice since you already have them, but worth proselytizing)

emccue00:05:51

and if you manage to reduce dependencies on stateful components to be behind an interface, you can always protocol that stuff up for easier testing

πŸ‘ 4
emccue00:05:26

the trick, I think, is to realize that the same advice as java really does still apply (minus paranoia about private)

rmxm01:05:22

Thanks guys, this advice is incredibly helpful. It's a difficult thing to quiz people about since as you can all see a lot of people have same/similar intuitions but different angles and taking a look at those perspectives really supports further thinking.

didibus01:05:35

Ya, it's tricky. The idea is simple: Reduce the surface area of each part of the code to its minimum. But actually doing so is challenging. And generic advice doesn't always help much when dealing with concrete situations

Vincent Cantin04:05:08

TIL:

(merge {} [:a 3] [:b 7])
; => {:a 3, :b 7}

Daniel Tan04:05:06

(into [] {:a 3, :b 7}) reverses the operation

seancorfield04:05:30

@vincent.cantin because (conj {} [:a 3] [:b 7]) is {:a 3, :b 7}

Vincent Cantin04:05:26

yes .. I took a look at the implementation … it allowed a bug to pass unseen, but that was my bad ^_^”

seancorfield04:05:45

@danieltanfh95 seq on a hash map produces a sequence of MapEntry items which look like pairs (vectors).

seancorfield04:05:09

user=> (type (first (seq {:a 3, :b 7})))
clojure.lang.MapEntry

Vincent Cantin04:05:04

my code was (merge context bindings) , where bindings was [:a 1] instead of {:a 1}. It worked for [:a 1] but not for [:a 1 :b 2] .

seancorfield04:05:39

Ah, interesting. Yes...

leetwinski08:05:44

hi guys! i'm looking at this question on so: https://stackoverflow.com/questions/62007445/infinitely-recursive-lazy-sequence-appears-as-empty-sequence-in-clojure

(def primes
  (remove (fn [x]
            (some #(zero? (mod x %)) primes))
          (iterate inc 2)))
so i caught myself not getting what semantic rules are below the fact that primes inside the remove predicate is just the realized part of overall primes sequence. printing it out, like this:
(fn [x]
  (println (realized? primes) primes)
  (some #(zero? (mod x %)) primes))
says > false () > true (2) > true (2 3) > true (2 3) > true (2 3 5) > true (2 3 5) > true (2 3 5 7) > ...

πŸ‘ 4
phillippe22:05:33

have you figured this out? i don't get how the evaluation of primes terminates. just looking at the code i would guess that this would also lead to a stack overflow like the example presented in the SO question

phillippe22:05:24

i would expect for (realized? primes) to be true after the first lazy sequence has been realized. primes is of the form (lazy-seq (cons 2 (lazy-seq (cons 3 (lazy-seq ...))))) , so (realized? primes) just tells you that the first lazy sequence in the chain has been realized.

phillippe19:06:12

i wrote a blog post on this since i was really curious about how it worked

phillippe19:06:46

if you want me to explicitly mention your username in the post, instead of just linking to the post, let me know and i can update it

phillippe19:06:23

thanks for sharing the definition in the stackoverflow post β€” super interesting!

craftybones09:05:24

I’ve not done any Prolog or Datalog, but I’d like a suggestion on dealing with a problem. I have to split a number of people into batches of certain sizes. There are constraints on certain people being in the batch or being in different batches, and other constraints based on other factors. This sounds like something I can use core.logic for. Am I wrong? If not, can anyone provide any pointers on what I should be looking for?

rmxm11:05:16

Hey is me again. I have done some reading on all the things you mentioned yesterday and it seems 2 strategies seem to address this "stateful component" problem. 1. Component lib by Stuart Sierra, 2. Pining the initialization on the upper namespaces. What is the general community take on this? I am pretty sure both aproaches are better than using atoms(I can see how atoms are this giant hatchets for this jungle). To me it feels like component lib introduces some semantics and indirection(which is "not great, not terrible"), but provide clean containment of functions and initialization. Where separating initialization from continuous invocations seems less dev friendly but vastly more explicit.

dominicm11:05:53

You might want to look at integrant and JUXT clip as alternatives with less semantics

πŸ‘ 4
Daniel Tan12:05:30

the state of the application can be mostly contained within databases, which can be thought of a giant atom with good querying ability anyway, no?

rmxm12:05:53

@danieltanfh95 50k loc that doesn't assume this makes difficulties in trying to convert πŸ™‚ trying babysteps and containing stateful parts seems the least radical step

seancorfield15:05:58

We use Component very heavily at work but we were using globals before that. Yes, it's a bit hard to convert, but what you can do @roguas is to start writing components and have the start function set up your globals (instead of how you're setting them now) and then slowly migrate code to use the components directly instead of referring to the globals.

seancorfield15:05:33

We'll still in that process -- we have a 95k loc codebase and we still have a handful of globals left. The refactoring is ongoing πŸ™‚

seancorfield15:05:05

Do I wish we'd started out with Component? Hell yeah! Do I at any point regret switching to Component? Nope.

rmxm16:05:01

thanks Sean, good to hear first hand impressions

didibus19:05:02

That's true, real world code bases are big beasts, it's good to keep in mind while you discuss best practices, a real code base will rarely be able to really keep up with all of them.

rmxm13:05:08

Given all the advice yesterday, I decided to lurk more on github on some production code and found this one: https://github.com/metabase/metabase/search?p=1&amp;q=%22atom+%22&amp;unscoped_q=%22atom+%22 and while a lot of the results are test/* where using atoms may be a little more permissive since its outskirts, it still remains that atom becomes this hatchet for their state requirements

didibus18:05:58

Hum, ya it's hard to say from a quick glance. I think some might be unnecessary and others seems totally fine

lvh23:05:33

An issue I’ve seen newbies get tripped up by a lot is the distinction between maven artifact names and namespaces; is there any canonical documentation or excerpts to tutorials that covers that well already? Otherwise I’m inclined to write something

Alex Miller (Clojure team)23:05:31

no (and this trips up newbie Java users too)

lvh23:05:14

And Python!

Alex Miller (Clojure team)23:05:09

particularly that they are two similar, often overlapping, things with only an arbitrary relationship without even norms much less standards

Alex Miller (Clojure team)23:05:17

Rich and I have actually spent a bunch of time talking about this (he talks about it in Spec-ulation too iirc)

Alex Miller (Clojure team)23:05:54

talking about it in the context of add-lib for example

dpsutton23:05:06

i think i hit on this for a second because git libs take an artifact name in deps.edn but that name is essentially meaningless right?

dpsutton23:05:24

when i made the twitter version i didn't know what should go there at all

Alex Miller (Clojure team)23:05:46

"When artifacts are deployed in a Maven repository, it’s a best practice to use a groupId (the first part of the name) that is something you control (usually via DNS or trademark). In the case where you have neither, you can instead combine the name of a site that establishes identities (like GitHub) with your identity on that site, here github-yourname."

didibus04:05:23

I like this advice

Alex Miller (Clojure team)23:05:42

the second sentence is guidance from Rich and I and not something I've ever seen anywhere else, fyi

seancorfield23:05:54

@lvh FWIW, that's why I decided that clj-new should prefer <org-or-username>/<project-name> and create a folder called <project-name> with a pom.xml that had group <org-or-username> and artifact <project-name> and the main namespace would be in src/<org-or-username>/<project-name>.clj to try to get some sort of sane standardization at least going forward in the CLI/`deps.edn` world.

lvh23:05:42

Absolutely! I think that choice is what made me be conscious about the issue, so thank you!

8