@severed-infinity: websocket handling is pretty easy with aleph - i have a vanilla ring handler which handles websockets, right alongside my yada resources - http://aleph.io/examples/literate.html#aleph.examples.websocket

i might as well ask this question here, but how does juxt's modular relate to danielsz's system ? they appear to be solving the same problem, is it not ?

I think modular predates system. But I'm not sure.

system has changed a lot since I last used it though

ok, thanks

i'm very confused about what the best way to deploy yada services is... it seems like the boot community really likes to run production systems from within the boot environment, rather than using an uberjar ?

@lmergen: you can totally run stuff from an uberjar

yeah, it just appears as if it's a bit more complicated to get working than under lein... or rather, i'm trying to solve errors with the generated .jar for about 3 hours now, without any luck

what you prefer depends on a bunch of factors that might vary from team to team

@lmergen: let's sort it out in #C053K90BR

thanks!

@dominicm: FWIW, I recently started using danielsz/system (only for the reloading + boot features) and it's amazing, like crazy amazing. If it's been a while since you tried it you might want to give it a try again 🙂

@martinklepsch: I used the components from it way back. I opened a couple issues iirc.

Just to show how different system was when I used it 😛

right the components I don't use at all (they still exist)

yes, it seems like the past year, many of these systems have matured a lot

I'm well aware of 'system' but haven't taken a proper look recently. Out of interest, what does system have that edge doesn't?

system is like modular

https://github.com/danielsz/holygrail like system's edge

Yes, but besides prebaked components.

@lmergen: one of the reasons I dislike the current fashion for building/deploying uberjars is the differences that begin to pile up between dev and prod. If it takes you 3 hours to figure out how to run something you were able to run fine in dev then that's what I'm talking about. And if it breaks in prod, who besides the individual that built the pipeline can fix it? I believe we have blindly inherited practices from C/C++ and forgotten the ancient ways of Lisp.

yes, i have been bitten by that in the past, especially when it comes to things like configuration, logging, etc

@lmergen: od course, this is a general point I'm making, completely unhelpful !

so, if i understand it correctly, you are saying that we should just create a prod task in build.boot, and run it from there ?

Yes, boot run -p <profile>

We even have integration wuth systemd, via a systemd script parameterized with the profile.

@malcolmsparks: system gives you a fileset-based reloading workflow, restarting your system when (selected) files change

Yes, you understand it right. However, this is still a 'maverick' choice, the vast majority of Clojure devs do traditional build and CI/CD

@martinklepsch: ah, thanks!

that is actually fairly reasonable

There's a systemd script in the root dir of edge now

We've rum some client projects like this in prod, plus internal stuff. I can't say we've seen any increase in outages as a result.

oh geez, i didn't even see that

The main advantage is support. If there's ever a support issue I can ask a dev to investigate and they're like "fine, I understand this, it's almost exactly like things are on my dev machine"

yes, i really like the idea of this, however, i think the plan falls apart when you're trying to deploy into more elaborate systems (e.g. AWS Elastic Beanstalk)

Yes it does (fall apart). We have some systems on AWS BS via docker and uberjar. I find them a pain to setup and debug but that's a personal opinion. I think the autoscaling feature is mostly oversold, only some systems actually need it. I prefer to build in headroom than scale out, because I don't accept scaling out is ever inexpensive

right

well, that's an entirely different discussion i think

(i also believe the autoscaling feature is oversold, or rather, it's not being sold for the right reasons)

My belief is IT is heavily biased towards ops today, they wield the most influence in large orgs today.

i agree, as someone running a video serving startup, operation (automation) is a fairly big part of our development

been hit by the uberjar/different prod thing quite recently (exacerbated by aot in my case, which somehow made me loose the environnement config). Have been looking at edge, interesting (I still build an uberjar at this point) I like the idea of a task taking a profile as parameter. And the conf file approach so I will probably use aero and steal you approach 🙂 One problem I had was to coerce the values from strings to ints/booleans etc. using clojure.spec though (I had to namespace some keys and reading them in a task made them namespaced under user.boot and when aoting made them namespaced under clojure.core..).

@malcolmsparks: i also think that the influence of ops has become bigger due to the continuous deploy trend

but perhaps you're correct that this is being translated the wrong way -- instead of hardening our production environment using all kinds of awesome bells and whistles, perhaps the best way is to make the production environment more like dev

I’d just like to say that this has been a very interesting thread - lots for me to process. Thx

Ah - turns out I can't use aero in my case since it uses clojure.core.edn to read the file (which is fine), but it does not support auto-namespaced keys.

So I use aero like this now - not sure I'm completely happy with my code but I'll try and see :

(defn ns-tlk
  "keywordize keys. namespaced keys boot.user and clojure.core will have backend.conf as namespace"
  [m]
  (let [f (fn [[k v]]
            (if  (and (keyword? k)
                      (or (= "boot.user" (namespace k))
                          (= "clojure.core" (namespace k))))
              [(keyword (str 'backend.conf) (name k)) v]
              [(keyword k) v]))]
    (walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)))


;; replace read-config from aero to load-file and allow the use of namespaced keywords

(defn read-config
  "Optional second argument is a map that can include the following keys:
  :profile - indicates the profile to use for #profile extension
  :user - manually set the user for the #user extension
  :resolver - a function or map used to resolve includes."
  ([source given-opts]
   (let [opts (merge aero/default-opts given-opts {:source source})
         tag-fn (partial aero/reader opts)
         ;; read the file in here. That makes it easier to coerce (and dev)
         ;; since the namespace is the same
         ;;  pass a string already with correct namespaces
         config (edn/read-string {:eof nil :default tag-fn} (str (ns-tlk (load-file source))))]
     (#'aero/get-in-ref config)))
  ([source] (read-config source {})))

(defn get-conf
  "returns the file configuration"
  [conf]
  ;; TODO memoize ? defonce a {} ? have an env var/record here ?
  (let [edn (read-config (case conf
                           :dev "resources/dev-conf.edn"
                           :prod "resources/prod-conf.edn"
                           (throw (Exception. (str "Not a config " conf)))))
        c (s/conform ::config edn)]
    (if (= :clojure.spec/invalid c)
      (do
        (s/explain ::config edn)
        (throw (ex-info "Invalid spec " (s/explain-data ::config edn))))
      (util/drop-ns-keys c))))

A yada question, how do I do authorization that a user must be logged in?

I'm also not sure roles align with my preferred usage. I think a better idea is to fetch it from the database for every request, this makes banning a user or changing their roles much easier. I'd be interested to know I'd go about setting this up.

@dominicm: i use a yada interceptor / ring handler which checks for a JWT token header, which i generate with buddy.sign