This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-12-01
Channels
- # adventofcode (93)
- # announcements (44)
- # asami (23)
- # aws (1)
- # babashka (48)
- # beginners (112)
- # calva (26)
- # cider (57)
- # clj-kondo (17)
- # cljfx (5)
- # cljs-dev (21)
- # clojure (124)
- # clojure-europe (19)
- # clojure-hungary (40)
- # clojure-nl (3)
- # clojure-spec (7)
- # clojure-uk (3)
- # clojurescript (3)
- # cursive (81)
- # datalog (11)
- # events (21)
- # exercism (1)
- # fulcro (37)
- # graalvm (1)
- # introduce-yourself (8)
- # jobs (1)
- # lsp (1)
- # malli (5)
- # membrane-term (17)
- # minecraft (3)
- # nextjournal (5)
- # off-topic (14)
- # other-lisps (14)
- # polylith (58)
- # reagent (16)
- # reclojure (3)
- # reitit (6)
- # remote-jobs (1)
- # shadow-cljs (55)
- # spacemacs (15)
- # testing (2)
- # tools-build (7)
- # tools-deps (191)
Really enjoying using orchestra to add inline specs to function definitions. I was able to adjust clj-kondo to recognize the macro-redefinition, yet have not found the trick to make this work with clojure-lsp. As a result, several important features of lsp do not work as it does not recognize (defn-spec .. ) as a function definition. Anyone know how to adjust config for clojure-lsp to make this work?
Most likely you need to configure clj-kondo config for lint the macros correctly (clojure-lsp uses kondo under the hood)
How do you count
when using transducers? Is there a better way than this:
(transduce
(comp (filter odd?)
(map (constantly 1)))
+ (range 10))
;=> 5
It's fine with seqs
(->> (range 10) (filter odd?) count)
But where to put it when using transducers?
the length of the vector is known, so I don't know why to waste CPU cycles to specially calculate something that is already calculated
The downside is that it realizes the whole sequence. If you need just the count but not the sequence itself, it's a waste.
But you keep the whole vector in memory
There is also this great lib https://github.com/cgrand/xforms
(enc/qb 1e3 (transduce (comp (filter odd?) (map (constantly 1))) + (range 1e4)))
;; => 262.97 ms
(enc/qb 1e3 (count (into [] (filter odd?) (range 1e4))))
;; => 196.31 ms
Also if (range 1e4)
was instead the source of something larger than ints.
(defn -count
[xs]
(reduce (fn ^long [^long acc _] (unchecked-inc acc)) 0 xs))
This is a better way to countbut it just depends on what we have in the input and how we want to filter it, and what else we want to do with it along the way
(defn x-count
^long [xf xs]
(transduce
xf
(fn
(^long [] 0)
(^long [^long x] x)
(^long [^long x _] (unchecked-inc x)))
xs))
alternatively
(defn count-rf
(^long [] 0)
(^long [^long x] x)
(^long [^long x _] (unchecked-inc x)))
(defn x-count
^long [xf xs]
(transduce xf count-rf xs))
(enc/qb 1e3 (count (into [] (filter odd?) (range 1e4))))
;; => 213.15
(enc/qb 1e3 (-count (into [] (filter odd?) (range 1e4))))
;; => 232.03
a colleague wants to filter anyway and it looks like he wants to use some other transdcuers fn
(time (dotimes [_ 1e3] (count (into [] (filter odd?) (range 1e6)))))
"Elapsed time: 11370.167798 msecs"
(time (dotimes [_ 1e3] (x-count (filter odd?) (range 1e6))))
"Elapsed time: 9113.005152 msecs"
(deftype Counter [^:unsynchronized-mutable ^int i]
clojure.lang.IFn
(invoke [_] (set! i (unchecked-inc-int i)))
clojure.lang.IDeref
(deref [_] i))
(defn counter [] (Counter. 0))
(defn counter-xf
[counter]
(fn [rf]
(fn
([acc] acc)
([acc x] (counter) (rf acc x)))))
(let [counter (counter)]
(into []
(comp
(filter odd?)
(counter-xf counter))
(range 1e6))
@counter)
Terrible 😄this problem has been solved many times by many and this is probably only possible in the clojure world
let's create a new data type to count the number of elements in collection, sounds reasonable 😉
Hi, Not sure if this is the right place to ask - I have a need to build an API adapter that will allow us to integrate to different external APIs that are performing the same job. Are there any libraries/techniques in clojure that make this possible?
Is there a sorted data structure that supports nth
, conj/insert and disj/remove (ie "remove first occurrence of value x"), but which is not a set and can contain duplicates? Is there something better than just a naive binary tree for this? Currently using an avl sorted-set-by
and sticking my values in a tuple with something unique, but this feels a bit hacky
This sounds awfully like a sorted bag or multiset, googling around I found https://gist.github.com/hindol/c92a768f7f784379dc8bcd546a60fa42 and https://github.com/achim/multiset, there’s probably more out there
Not sure if this is the right channel for this q, I didn’t see anything more specific Looking through both real and example projects that use component-like system managers (e.g. mount, integrant, etc.), I notice that during test runs, all of them start and stop the system in a fixture, meaning that the system will be restarted many times during the test run. Is there any particular reason why you’d want to restart the system for each test/namespace? I’d think that most of the components wouldn’t have internal state so there’s nothing to reset, and some of them could be costly to start. I can’t imagine that (for example) starting a jdbc connection pool 100+ times does anything good for your test suite performance. Is there something obvious I’m missing here?
No, you’re perceiving it right. Fixtures are terrible. Fixtures + components more terrible.
If you need like a special mock or stub for a particular test, you can just make one at the location that you need it and pass it in to the fn you’re testing.
They're great: you can inject mock components for testing and since everything happens in memory there's no performance impact. For example, we have components for various API clients and replace them with mocks implementing same protocols but returning fixed responses for different cases.
And if you need a specific version of a component for a particular test you can swap them at any point in time.
If you choose to use fixtures+components: • You gain no new capabilities • Your tests will be slower • You actually type more (because the alternative is to start components at the start of the run)
If you start components at the start of the run: • Your tests will be faster • You lose no capabilities • You type less
I'm not sure we're talking about the same thing - how is (use-fixtures :once (partial test-system/start))
making me type less?
Because some test components have internal state e.g. record all request made against an API (in an atom), so on a per-test namespace basis we can inspect its state. If we preserved the state across whole test suite run it would be hard to make assertions about the request history. That's just one example.
1. Start a base set of components at the start of the entire run. 2. When needed (e.g. mocks/stubs/error checking), start a secondary set of components to test specific fns.
Nothing in component requires you to have only one set running at a time. In fact, this is a major upside of passing things around as opposed to relying on vars.
Depends on how slow your component's startup time is really, fixtures (as in test data injected into the DB under the test etc) have little to do with components. But yes, injecting different versions of a dependency in specific tests is an expected thing to do
No. What I’m saying is it doesn’t depend. Fixtures+components is always slower than re-using components across tests. The difference may not be noticeable right now, but it’s slower than the alternative you could have chosen.
The only possible exception is where every test suite starts its own component system anyways, in which case the two options are equally slow. so timewise, it’s shared components <= fixtures+components
When I get the time I'll measure the time for the biggest test suite, my bet is that resetting Elasticsearch state is way slower than instantiating a record with an atom in it. What sort of components do you have that their start time impacts your test suite's run time? I'm genuinely curious because it's not something I've run into so far
> my bet is that resetting Elasticsearch state is way slower than instantiating a record with an atom in it. I dunno how else to explain this. Nothing is stopping you from using a record w/ an atom in it.
> What sort of components do you have that their start time impacts your test suite’s run time? I mean, anything that sets up a connection pool is gonna need 10s-100s ms startup time. Other components need to initialize caches (by definition this is costly). Do that enough and it adds up. It’s really a function of suite size and number of components.
Our fixtures are "smart" insofar as they start a system (Component) if it isn't already started and they stop it after tests if they started it. So we can run an individual test with a fixture and it will start/stop the system, or we can run a whole namespace, or we can run the test suite. The latter starts the system once and stops it at the end, via a "fixture" built into a wrapper around test-runner
's exec entry point.
Polylith has per-project test fixture points now so it can do the same thing (it starts/stops the system once per project test run -- each project runs in an isolated classloader because they can have different dependencies).
Running per-test-suite "fixtures" (i.e., system start/stop) makes a huge difference to the overall times of our test suite runs, which is why we did this. So I'm mostly "Team @U07S8JGF7 " on this 🙂
Not putting this in the thread because it's not quite related to the fixtures thing, but I'd say, if your component has no state, why are you putting it in a state manager? You can provide mocked behavior in tests with dynamic binding, alter-var-root, or similar, but if a component has no state, it's just a set of related functions. Only use integrant, component, mount, and similar to manage state. Protocols and multimethods provide extension when you need that but without state. Ignore this
Exactly.
What I mean is, your "Only use integrant, component, mount, and similar to manage state" maxim is wrong. Those are perfectly fine solutions for DI.
These libraries are build around wiring together state however. It happens that one kind of state being wired together is dependencies. Dependencies are state when they are created at startup and held.
this dependency injection is a kind of state, so maybe I misunderstood what was meant by "most of these components have no state"
from this > but I’d say, if your component has no state, why are you putting it in a state manager? here you argue that composition of the system is not state, but now you are arguing that composition of a system is state
But my view is that DI can be helpful for constructing systems with different behavior, regardless if the components have state or not. I do find DI a bit awkward in Clojure though
Clearly my understanding of what constitutes "state" in these systems is incorrect or at least differs from the common understanding.
Because I don't see what I'm saying as disagreeing with what either of you has said, but for thinking that dependency injection is a kind of state.
And if dependency injection is a kind of state, then "most of the components wouldn't have internal state" means they have no dependencies to wire, etc. and could be depended on as plain functions, hence, "don't put them in a state library"
Also @U2FRKM4TW you say citation needed, I direct you to the first line of the component readme: > 'Component' is a tiny Clojure framework for managing the lifecycle and dependencies of software components which have runtime state.
Alright. At the same time, neither Integrant nor Clip make the same claim. Mount is somewhat of a outlier because it has inherent issues caused by its ties to namespaces.
It's fair that they don't say that, but at least integrant is designed to be a replacement for component to solve the same problems more or less, but in a more clojure-ey way. And again, it just goes back to "is dependency injection a kind of state" which I have historically considered true.
Hence why I said my understanding is either wrong or at least uncommon
You need to separate DI from instantiated DI assembled systems. The latter is state, former isn’t. A running Component system is state, and it was constructed using DI, but the DI specification isn’t.
New thread since the other two have gotten a bit derailed: if you don’t start your system in a fixture, where do you start it? This is what I went hunting for examples of and didn’t find any
Something like this: https://github.com/stuartsierra/component#entry-points-for-development - you'll need a namespace to manage the running system and ensure it starts once (if that's the approach you're taking)
Yeah, you can start it before you start your test run and alter-var-root
it into a var (or use dyna vars).
Right, but how do you make sure that gets triggered whenever a test is run? Remember that tests can be run both via a test runner or directly in a REPL.
For test runner executions: clojure.test
has no “once per test run” fixture scope, so you’ll have to rely on the test runner to make sure your system gets started. https://github.com/cognitect-labs/test-runner doesn’t have a way of saying “do this before running tests”. You could probably do it in https://cljdoc.org/d/lambdaisland/kaocha/1.0.861/doc/10-hooks, but…
For REPL executions, you need the system to be available, but you also probably want a different system from your dev one. Here, there’s no “test run”, so when do you start/stop the system? You’ll probably also have to watch out for resource conflicts, e.g. webserver ports.
The only solution I can think of that satisfies both is a fixture that starts the system the first time it’s invoked and does nothing on all future runs. Are there other approaches? What do you do on your apps?
wrap the test runner invocation in a fn that initializes the system and then calls test-runner/-main
Ah, so instead of pointing your test alias at something like kaocha.runner
, you instead point it at something like my-wrapper
which does the system init and calls kaocha.runner
? In that case, when does the test system get initialized for running tests in the repl?
do you tend to also start your dev system automatically on repl startup? All the apps I’ve ever worked on required you to run user.start
or similar
got it, makes sense. So you just have to start 2 systems to start dev’ing, that’s not too bad
Oh, for interactive REPL you can just use the same system if you want. Kinda up to you.
I get the sense you might want the systems to be different for dev vs test, for example for dev you probably want to start a real webserver running on a port, but for tests you can just use an in-memory client that calls into the server directly
Cognitect's test-runner
has a -X
entry point that is more amenable to wrapping than its -main
, just FYI. That's what we do at work to start a system component stack at the beginning of a test run.

For REPL use, we can either have a global running system (started via the REPL) or we can specifically run tests with fixtures even when running a single test (using test-vars
I think? I'd have to look at my VS Code/Clover setup since I have a hot key bound to "run test under cursor with fixtures").
I start the system in the first run of the fixture, it uses an atom to check if it is the first run or not.
Hello, i made a Clojure(Script) library called cMQL, for quering MongoDB. It feels almost as if Clojure was the query language of MongoDB, and reduces code up to 3x. cMQL example (playground has 150+ examples) :
(q (= :bedrooms 1)
(= :country.code "GR")
(group {:_id :stars}
{:average-price (avg :price)})
(sort :stars)
(limit 3))
Docs : https://cmql.org/
Playground : https://cmql.org/play/
Github : https://github.com/tkaryadis/cmql-core
Slack : https://app.slack.com/client/T03RZGPFR/C02P0821VJA
Discord : https://discord.gg/zWDzp4B7BfThought in passing. The Clojure website doesn't make it easy to identify the latest stable version number. Not sure if it's intentional but I'd find it helpful when bumping deps on old projects.
that's where you go if you click the Releases page in the top menu bar