Fork me on GitHub
#polylith
<
2022-02-16
>
Braden Shepherdson05:02:50

I'm considering refactoring a project into polylith, but I'm not certain whether it's a good fit. There's a bunch of tests that run the same set of tests over a set of backends that all share a protocol. It would be cool to split the backends into their own components since they have different dependencies. Does it make sense to wrap those tests into functions in their own component, and then have the tests for each backend call the test suite with an instance of that backend?

tengstrand05:02:55

Hi. Yes, sounds like a good idea. The Polylith codebase uses a https://github.com/polyfy/polylith/tree/master/components/test-helper/src/polylith/clj/core/test_helper that is shared between several projects.

Daniel Gerson08:09:42

Thanks for this discussion. The test-helper code looks quite different from what I imagine a batch of integration tests look like. I.e. Ideally adding a test to the batch should be as simple as adding a deftest to the batch namespace (not maintaining a component interface). I have a similar situation with shared test code for different clients and servers and decided to experiment putting these shared tests in bases\test-client\test\ns\test_client\core.clj although I'm tempted to put these in src instead.

Teemu Kaukoranta06:02:32

I'm migrating our services to Polylith, and a couple of times now my IntelliJ IDEA + Cursive has crashed. I'm using the 1st generation M1 Macbook Pro with 16GB of RAM. Are there any known workarounds to help IDEA cope with the large monorepo, or do other editors fare better?

seancorfield07:02:58

I don't know how "large" your monorepo is but I'll share the size of ours:

Clojure build/config 22 files 378 total loc,
    96 deps.edn files, 1539 loc.
Clojure source 415 files 97126 total loc,
    3836 fns, 994 of which are private,
    597 vars, 34 macros, 94 atoms,
    85 agents, 23 protocols, 66 records,
    799 specs, 23 function specs.
Clojure tests 442 files 25556 total loc,
    4 specs, 1 function specs.
with the Polylith portion of that so far being:
projects: 18   interfaces: 38
  bases:    11   components: 39

loc/test: 33,097 3,743
I use VS Code/Calva day-in, day-out and never have any crashes (and because we don't use Cursive, we can use dependencies with :local/root instead of paths in our :dev alias, which means bricks get to have their (3rd party) library dependencies in their deps.edn and those do not need to be duplicated into the :dev alias).

seancorfield07:02:54

The paths in :dev this is a workaround because Cursive doesn't recognize source in bricks otherwise, unfortunately.

seancorfield07:02:48

(strictly speaking, I use VS Code with Clover for the Socket REPL and Calva for everything else but I'm in a pretty small niche since I want to avoid nREPL etc)

Teemu Kaukoranta07:02:33

Thanks Sean, it's good to hear that you haven't had any issues. We have 16 projects right now, with 11k loc and 6k in tests

Teemu Kaukoranta07:02:08

I'll try adjusting IDEA's memory settings. If nothing else, I can always switch over to VS Code, been interested in trying out Calva anyway 😛

tengstrand09:02:09

I have two MacBook Pros, one with 16 GB RAM and one with 32 GB. The extra 16 GB memory is well spent money.

Teemu Kaukoranta10:02:20

Yeah, unfortunately the first gen M1 had a max of 16GB 🙂 The new ones have more

👍 1
Casey13:02:13

@U04V70XH6 is that first snippet you pasted summarizing the clojure SLOC in detail..from polylith? Or what produced that output?

seancorfield13:02:09

@U70QFSCG2 just a bash script that runs find (and grep) into wc -l for various things.

shields15:02:54

@furkan3ayraktar With the Datomic Cloud upgrade did you have to name your apps differently or anything in the ion-config.edn . For some reason the http-direct function isn't hooking into the API-Gateway. All the lambdas are built correctly but I get a 502 Bad Gateway when calling the Ion API endpoint. error. I was able to deploy and get https://github.com/JarrodCTaylor/ion-cognito-exemplar working. But when I deploy from our polylith app the above problem occurs. Just wanted to get a sanity check from you since I know you went through this upgrade.

shields16:02:42

ion-config.edn

{:allow       [;; whitelist of functions that can be invoked as ions ;
               blvd.blvd-app-api.router/app
               blvd.user.interface/start-consumer
               blvd.user.interface/load-schema
               ;; transaction functions
               blvd.payments.txfn/upsert-transaction]
 :http-direct {:handler-fn blvd.blvd-app-api.router/app}
 :lambdas     {:start-consumer
               {:fn          blvd.user.interface/start-consumer
                :description "Start MSK Consumer"}
               :load-schema
               {:fn          blvd.user.interface/load-schema
                :description "Load Schema"}}
 ;; change this to your app name if playing at home!
 :app-name    "app-name"}

furkan3ayraktar16:02:53

Hmm, it’s been a long while back that I did the upgrade so I don’t exactly remember what I have done. However, latest version of Polylith should work with Datomic Cloud out of the box without any special configuration. Do you get any exceptions in CloudWatch? There could be many reasons you can look for: 1. Datomic instance is not able to start for some reason 2. API Gateway cannot reach to the instance 3. Some other error after the request reaches to the instance

furkan3ayraktar16:02:22

There should be a log group that starts with datomic- in CloudWatch

shields16:02:28

Okay, must be something with our app. Had a https://clojurians.slack.com/archives/C03RZMDSH/p1644890284222339 in the #datomic channel yesterday on this issue. We have no "Exceptions" in the cloudwatch logs and everything else deploys successfully. I was able to clone Jarrod's repo and push it to our system and hit the endpoint successfully. But when I add his exact app into the polylith repo it doesn't connect just like ours doesnt. I'll keep investigating.

shields12:02:44

@furkan3ayraktar are you using the http-direct feature?

furkan3ayraktar16:02:44

Yes, we are using it.