Fork me on GitHub

I'm looking for a way to write a service library with self-contained migrations to a given postgres schema, configured at the app level, the app depending on the service lib. It seems like flyway might support this via classpath stuff. Has anyone tried it before in a clojure project?


It is answering partially for your question. It is possible to “manage” flyway from Clojure. deps {com.github.metaphor/lein-flyway {:mvn/version "6.0.0"}}

(ns db.flyway
  (:require [flyway.flyway :as fw]
            [api.db.postgres :as postgres]))

(def flyway-db (fw/flyway {:url postgres/db
                           :locations ["filesystem:dev-resources/db/migration"]}))

(defn restart-db []
  (fw/clean flyway-db)
  (fw/migrate flyway-db))

(defn wrap-test [tests]


👍 1

ah, that's helpful, I think 'locations' could be a hook into it. I would need my own build/deploy scripts around this.

Darin Douglass14:02:46

i’d suggest using migratus migrations are just resources on the classpath


I have an example of using Flyway here:


ah I see, their docs didn't have it but the code can use classpath resources:


fewer layers is nice

Darin Douglass15:02:42

ya i think they’re assuming resources is already on your classpath (it’s a pretty standard top-level directory).


Trying to straddle microservices and monolith by using a shared DB per deploy, with some reusable things packaged up as libs, replacing SQS, SNS, scheduling with just postgres when the scale is small.


I don't intend to add cross-schema constraints, but it could happen someday.

Ngoc Khuat13:02:13 I wrote a simple Mnist reader in Clojure, Could anyone give me some ideas on how to make it faster?

Ben Sless13:02:51

Did you profile it?

Ngoc Khuat14:02:49

I just did a simple time with 2 reader func. How should I profile it?


Sorry, I don’t have time to analyse it deeper, but maybe one of this steps to improve performance will be useful for you.

👀 1
Ben Sless14:02:39

Time does not profile. A tool like clj-async-profiler will let you see where you spend CPU

Ben Sless14:02:25

There are things I can tell you to do which will make it faster but you won't know why

Ben Sless14:02:50

So profile first, understand what's going on, and then you'll be able to ask more relevant questions

👍 1
Ngoc Khuat14:02:01

unfortunately, I couldn’t profile because I’m using mac M1. Tried it with a Linux machine but hit some permission issues setting up clj-async-profiler Will try to find a linux machine tmr 😕


You can also try YourKit or Jprofiler

Ben Sless15:02:40

A few things to look at: Primitive type hints in the loop Avoid using r/map, you can map in place over the array Create a vector directly with vec and not into

Ngoc Khuat04:02:18

So I tried with your suggestion: 1. Primitive type hints in the loop => this adds a bit of performance but not much (from 1564ms to 1445 ms) 2. Avoid using r/map, you can map in place over the array => I’m not sure what you mean by map inplace - is it? 3. Create a vector directly with vec and not into => So instead of doing r/map I used map and mapv . Pretty interesting mapv gives a 30% performance bump Here is analysis with time (still can’t run profiling 😞 )

Read an array of 47.040.000 elements
;; "Elapsed time: 1564.851583 msecs" (r/map without type hint)
;; "Elapsed time: 1475.194334 msecs" (r/map with type hint)
;; "Elapsed time: 1143.0205   msecs" (use mapv instead of r/map)
;; "Elapsed time: 6397.154542 msecs" (use map instead of mapv)

Ben Sless04:02:31

I'll profile it later today and upload the results here

🙏 1
Ben Sless20:02:10

And... mostly GC

Ben Sless20:02:15

what does that tell you? 🙂

Ngoc Khuat15:02:54

Ah, seems like the data allocated is too large compared to the operation I do with it: which is only to-unsigned? This is one minor case where immutable is not shine?

Ben Sless15:02:38

It's okay but done pretty inefficiently, I'll profile allocation, too, see what's dominating it Also not sure if your test was reliable as partition is lazy, maybe doall should be called on it

🙏 1
Ngoc Khuat06:02:56

yep, that partition is definitely lazy.


Is *warn-on-reflection* a ns-specific declaration or does it apply to all required namespaces as well?

Alex Miller (Clojure team)15:02:49

as most commonly used (after the ns form), it's just for that namespace

Alex Miller (Clojure team)15:02:17

(btw, your screen name is a Clojure function homonym - when-some)

nice 2
😄 1

Is there a way to set it for the whole project, or do I just need to annotate every ns?

Joshua Suskalo16:02:27

you can set it from your repl in a project and it'll apply to all nses, right?

Joshua Suskalo16:02:52

The reason it only applies to the one ns is that it gets bound in the require code, right? So the set! only lasts for the binding for that call

Alex Miller (Clojure team)16:02:30

yes, it's bound during the load then that context is popped

Alex Miller (Clojure team)16:02:57

there is not currently a way to turn it on for a whole project. if you set it in your repl, it will apply to all subsequent loads, so you could probably leverage that to check all. I think maybe lein does something like that

Alex Miller (Clojure team)16:02:09

we do somewhere have an ask clojure question or ticket about this (turning it into a compiler setting)

Alex Miller (Clojure team)16:02:31

linters like clj-kondo will also warn you about this too of course


about what?


it doesn't warn on reflection

Cora (she/her)16:02:41

that would be a neat feature


eastwood does it (since it actually runs the code)


there is however an issue to give a warning when you're doing Java interop, to set warn-on-reflection to true:

Alex Miller (Clojure team)16:02:14

maybe I'm confusing it with warnings built into cursive, nvm!


perhaps clj-kondo could do that, but then it has to become as good in following type hints as the clojure compiler, and I don't want to create any false positives... maybe as an experimental linter.


I think I would normally turn it off. In normal Clojure programming, reflection is good for legibility & productivity at no cost to the results. Where reflection is a "legality" problem, Java will let me know.


> at no cost to the results The cost is performance and out of the box GraalVM compatibility


Is this expected behavior, or a bug?

Clojure 1.10.3
user=> (dissoc (sorted-map "a" 1 "b" 2) :c)
Execution error (ClassCastException) at user/eval1 (REPL:1).
java.lang.String cannot be cast to clojure.lang.Keyword
I see a handful of bug tickets concerning thrown exceptions related to sorted-map, so I wonder if this is another.


links to tickets? this is expected behavior, IMO

Joshua Suskalo20:02:40

Yeah, I think it makes sense that if you have a sorted set, you must have only keys that are comparable to each other, and trying to dissoc something that isn't comparable with the keys erroring makes some sense.

Alex Miller (Clojure team)20:02:55

sorted maps/sets are inherently limited to what their comparator can compare, and the default comparator is not a universal comparator.


I can understand after the fact why it would be throwing, based on an incompatible Comparator. But it surprised me that it breaks (in my opinion) the contract of dissoc. Also, there is no mention of this behavior in the docstring for either sorted-map or dissoc

Alex Miller (Clojure team)20:02:25

I'm not sure that the door is fully closed on this, but in general we have not tried to "protect" users from exceptions arising from the use of key values not covered by the comparator


Yeah, I’m not claiming this is a bug, just asking. On the flip side, there is clearly no other option for assoc than to throw an exception. It would seem worse to silently not assoc in the case of a different key type

Alex Miller (Clojure team)20:02:12

there's no way external to the comparator function to know what is allowed, so doing so would require try/catching around use of the comparator and falling back to some "not found" behavior

Alex Miller (Clojure team)20:02:36

I think a note in sorted-set / sorted-map docstring would be justified


Yeah, I agree. Something like “Collection operations performed on the sorted map may throw exceptions if an incompatible key type is provided”.


or whatever 🙂


thanks for clearing things up for me!


Does anyone know of a parallel/concurrent some? Same as some except that done in || and no extra work done as soon as a result is found.

Joshua Suskalo21:02:51

So you could do this by chunking a sequence and making a work queue with all the chunks in it, spinning up a thread pool to take chunks and chew through them with the predicate, and the first one to find something empties the work queue and returns the value to the original thread, but as far as I'm aware this has not yet been implemented and published anywhere.

Joshua Suskalo21:02:19

Although personally I'd make this function return only true and false because multiple runs of it may return different values otherwise.

😃 1
Joshua Suskalo21:02:08

You could so something like this

Joshua Suskalo21:02:25

you could turn that final loop into a go-loop in order to make this return asynchronously as well

Joshua Suskalo21:02:21

if you want the impure "return a value that matches the pred" you could change the cond to save the result from some and return that after setting the work-queue to nil, change :else to return nil, and change the if at the end to be (if (some? res) res (recur ... instead


Doing stuff I don’t know much about, and most certainly not getting interested in calculating primes, but given:

 (let [m (bit-shift-right (long 1024) (long 6))
       n (bit-set 1024 6)
       o (clojure.lang.Numbers/setBit 1024 6)]
using the clojure goes fast clj-java-decompiler I get this (somewhat abbreviated)
public static Object invokeStatic() {
        final long m = 1024L >> (int)6L;
        final Object n = ((IFn)core$fn__14806.const__4.getRawRoot()).invoke(core$fn__14806.const__5, core$fn__14806.const__3);
        final long o = Numbers.setBit(1024L, 6L);
        return null;
So what confuses me here is that it seems like bit-shift-right gets compiled down to simple java operator stuff, whereas neither bit-set or Numbers/setBit get the same treatment. Why would that be?


As in there are clojure intrisics for bit-shift-right but not for bit-set ?


So in order to make @pez win his all clojure prime sieve, could we have some more bit ops intrinsics 🙂


I don't recall there being a jvm instruction for setting a bit


@hiredman You’re absolutely correct.


If you have to generate a mask using shift and then and, or maybe even a second mask to invert the first and then and


But you could imagine (but probs not with todays impl of intrinsics) that you could emit the equivalent of the impl in operators, but I guess the JIT probably just inlines this anyhoop

static public long setBit(long x, long n){
    return x | (1L << n);


the answer is the last thing, the JIT will inline that small call

Sameer Thajudin21:02:44

Just curious, what tool do the members here use for developing in Clojure?


IDEA + Cursive currently, will probably switch to VSCode + Calva at some point.

ghadi21:02:40 Question 16 of the state of clojure survey should offer some insight (Emacs, here)


Emacs (+ Prelude) + CIDER + clj-kondo + clojure-lsp

Joshua Suskalo21:02:52

I use doom emacs, which has cider, clojure-lsp, kondo, and the default way to do structural editing in doom is with lispy, but I really don't like lispy so I use evil-cleverparens.


neovim + conjure + coc

🙌 1

(and clojure-lsp)


I start with the latest git branch of emacs-28+native compilation+xwidgets with Mitsuharu Yamamoto’s patches for MacOS and then add CIDER, clj-refactor, smartparens (and an emacs config evolved over 30+ years). For those who are using clojure-lsp, what does it add over CIDER? how do you deal with overlapping features in CIDER?


these hands (and the same cider setup I had since 2014)


why do people write (:k m) instead of (m :k) for map lookups? by habit, this leads to generic lookups being written as (k m) which invariably blows up when somebody gets clever.


(x :k) - is it :k in map x or function x with parameter :k? (:k x) - it is :k in map x

☝️ 1

instead of (k m) write (get m k) and everything is readable


for me personally readability is the priority


not number of chars

Joshua Suskalo22:02:16

Yeah, so writing (map key) is fine. Writing (:key map) is fine. But in general if you're having generic keys I would prefer (get map key)


(def m nil)
(m :k) ;; java.lang.NullPointerException
(:k m) ;; nil
1. nil punning 2. (:k m) supports a default value, like get, e.g. (:k nil 0) ;=> 0 - edit:`(m :k 2)` also works, as long as m is not nil 3. functions typically operate on data structures, so thinking of keywords as functions that operate on a map seems more intuitive to me than having maps be functions of keywords (even though both make sense mathematically) - there are exceptions though, e.g. (map m ks) looks better than (map #(get m %) ks) or (map (fn [k] (k m)) ks) or ((apply juxt ks) m) edit: oops, just saw there was a discussion outside the thread as well, sorry for repeating most of the points

Ben Sless05:02:57

You could generalize this to a "safe invoke"

(defn call [f x] (if (ifn? f) (f x)))


@UEQPKG7HQ On your second point: all forms of map retrieval allow for a default value:

user=> (def m {:a 1})
user=> (m :b 2)
user=> (:b m 2)
user=> (get m :b 2)


Right, thanks for the correction (I updated my comment)! If m is a proper map but the key you're looking for is missing, my point is moot but if m is nil, you'd get a null pointer exception with (m :b 2) You're right though, in the core of it it's the same argument as nil punning, I just meant to illustrate a different aspect of it; the reader might expect that (m :b 2) and (:b m 2) are equivalent ("if :b is not found in m, return 2") when they're not.


one reason is that (let [m nil] (m :foo)) will throw an NPE whereas (:foo m) will not


yeah, that’s true. lesser of two evils I suppose


there was a good thread a while back talking about this kind of thing. I was kinda parroting the advice from Zach Tellman to use the most concrete bit as possible. Like (contains? some-set value) to aid in readability. But others pointed out that using the flexibility of Clojure has lots of benefits as well. ie, (some-set value) lets you use functions, maps, etc. And leveraging that is a benefit. So the NPE is one consideration, but not the end of the decision to me