Fork me on GitHub

implementing something in clojure and i have a database question... if someone has recommendations please share let's say i have an ever-growing record of support phone calls. each call has a few simple metadata (like call center name), but they also have a bunch of keywords (like "payments", "browser-bug") what database could make querying such data over time easy? like most frequent keyword today i know that i can achieve this with elasticsearch, but i'd prefer something with a smaller resource footprint if possible


Are the keywords fixed, or arbitrary text? Do you need to sub-text-search within the keywords? (ex. expect "browser" to match "browser-bug")?


And... "does it need to be web-scale"? (ie. what performance do you expect of it?)


...b/c without considering performance, almost every db can probably do what you say. Postgres, H2, Datomic, XTDB...


Are you using any db for other things already?


Elastic Search is also a consideration for text searches, as is Lucene.


the keywords are arbitrary, but common. no need for full-text search, exact match is good. it doesn't have to be "web" scale, but i can expect up to 10k new records in a day. i'd prefer a database that has a native way to express a query like "most frequent keywords today" without inner-joining eventually massive tables. so basically i guess what i'm asking is if there is a database (with a smaller resource footprint than elasticsearch) that can do date histogram queries over multi-valued attributes (in this case the keywords) efficiently


A time-series database maybe... like (haven't used).


yeah i was looking at various time-series databases, but i couldn't yet find one that can handle multi-valued attributes


I wonder if timescaledb supports array column types (to avoid joins). Or, maybe just insert each event multiple times.


it supposedly supports every postgres data type, so i'll consider it for sure

Ben Sless05:11:07

I think Apache Druid might have the features you need


@UK0810AQ2 i couldn't find any details about what data types are supported in druid in its documentation. do you happen to know?


The main question for me is whether this is really gonna be used mainly for OLAP. It sounds like an operational database that must be some for some analytical/reporting queries too. I don't have much experience with them, but could a document database fit the bill? It seems that those phone calls are fairly independent (you could say "documents"). I know that many people hate MongoDB (I'm not suggesting to use it per se; and I don't like it much either) but maybe something like this could be used: •


hmm thanks! i've used mongo from clojure before, but i haven't considered it for this use case. i'll look into how it performs and see!


What Is a standard way to validate a large map with string keys? Specifically, I want to make sure that the map contain some mandatory keys and does not contain any key other than a predefined set. As far as I read, I can structurally validate the map with malli using map-of but my requirement is more stringent.


Malli supports "closed" maps.

  [:map {:closed true} [:x :int]]
  {:x 1, :extra "key"})
Or you can programatically close with malli.util/closed-schema Does that fulfill your needs?


Also, you may need to combine with :optional


[:map {:closed true} 
   [:x :int] 
   [:y {:optional true} :int]]


@UC1DTFY1G Why do you specifically want to exclude "unknown" keys? Clojure in general tries to be open to extension and the approach to maps is usually to just ignore additional keys.


Sample data: (on mobile so can’t format properly)

(def data {"k1" 1, "k2" "s", "k5" 4.3, "k23" "ss"}) # I need to validate that "k1", "k2" are present and every key present is in the set #{"k1", "k2", "k3" … "k225"}


Works with string keys as well:

  [:map {:closed true} 
   ["x" :int] 
   ["y" {:optional true} :int]]
  {"x" 1 ;; required
   "y" 3 ;; optional
   "z" 2 ;; not-allowed

👍 1

But if you only want to validate keys and without malli, you could do it manually, (set (keys data)) and using some clojure.set/... functions


Ok I didn’t know you can give string keys in malli schema


(println (clojure.string/escape "a" {\a "b"}))
is returning b , what if I want to return as "b" ?


pr-str returns a string; prn logs like println


@U0CLNM0N6 That is what I wanted, Thank you 🙂

Casey10:11:39 seems to explode when a namespace file that previously existed is deleted, for example, when a namespace is renamed, or just removed. refresh throws a for the oldnsname__init.class What's the proper way to delete a namespace and reload code (without restarting the repl)?

practicalli-john12:11:34 can remove a namespace (but is seems not the clojure.* namespace - which is probably a good safety net )


Although those docs suggest tools.namespace which may have better functions for the purpose described


Although it seems tools.namespace is deprecated.. tools.namespace


If you delete a namespace file, then run refresh, it blows up. If you run remove-ns on the removed NS, then refresh again, it still blows up. Restarting the reply seems to be the only recourse


The API says the namespace is deprecated.


Its seems more a case that things have moved around…. and things have been moved to namespaces under tools. namespace in several cases… not the clearest 😞


I assume if the namespace name & file is changed then refresh removes it from the REPL, but then tries to load it again and goes bang. I guess a hack would be to disable-reload! the old name of the namespace and then reload will not try and reload it.


Or perhaps is a better approach This uses the original suggestion of remove-ns but also seems to update the list of loaded libs (if I am reading correctly)


Sometimes I do need to edit/save the file that is failing to reset, which is not always the namespace I refactored (but something that might refer that namespace. Editing the namespace that cant be found seems to force (reset) to look again. So it seems to be a combination of remove-ns or or editing/saving the file of the namespace that has issues…

Jim Strieter13:11:33

My program does a lot of this:

(loop [my-list' my-list
       results []
       parameter-1 (some-function ...)
       parameter-n (some-other-function ...)]
  (if (empty? my-list')
      (rest my-list')
      (conj results (some-new-result ...))
      (update-parameter-1 ...)
      (update-parameter-n ...))))
The parts that change are parameter-1 through parameter-n. The number of parameters can be anything, and their initializations can be anything. Also the way they are updated from one iteration to the next can be anything. Is there a way I can use a macro so I don't have to write out the parts that stay the same over and over? For example:
(defmacro loop-over-stuff
  "vec-of-params should be a vector of vectors, each containing a tuple, like this:
   [[param-1 init-expr-1 update-expression-1]
    [param-n init-expr-n update-expression-n]]"
  (loop [my-list' (my-list-getter some-record)
         results []
         ;; How to unpack vec-of-params here?
    (if (empty? my-list')
      ;; How to extract update expressions here?


You should be able to do that with a macro. But you might want to try typing out a few real-ish examples of using the new api before you try. From what I'm seeing, it doesn't look like this removes very much duplication. Mostly it just moves the update fns from the recur to the let binding, making it less obvious what your code does. And using such a macro will likely add friction to injecting arbitrary extra steps to massage your data. This looks like a fold. So maybe using reduce could clean up a bit of the boilerplate? (or might only make it awkward and hard to follow).

 (reduce (fn [[results [param-1
                        param-n]] item]
           [(conj results)
            [(update-1 ...)
         [[] [(init-param-1)
              (init-param-n)]] my-list))


Yeah, recommend looking at reduce, which does everything your template does (and potentially more).


Gooood morning. I'm having a terrible time moving from Clojure to Clojurescript. I think I want a figwheel-main project with reagent to make a browser app (new to client-side dev). I'm used to using emacs, cider & leiningen to connect to the repl, but I'm having trouble getting everything to work all at once. I can do the lein fig:build thing which gets me the page & hot reloading, but not a endpoint to connect cider to. If I do lein repl I can connect to the repl via cider, but I don't get the browser connection with the hot reloading. I'm pretty clearly missing an important step, but I sure can't find it. I've despaired of finding a recent walkthrough using all these things. Am I just trying something absurd?


People mostly use shadow-cljs nowadays I guess. I found it easy to use.


Thanks! I’ll look that one up


I work on a base project, might be useful to check (not lein/project.clj, deps.edn)


I’ll take a look bit to be honest I’m not feeling like my main problem getting started is a lack of IDE options


Ok, as you feel. That is just a source code.


@UN16R9L93 I haven't touched emacs+cider, but this looks to be cider stuff for shadow-cljs:

thanks 1

Figwheel-main is a simpler approach to start with, as there are fewer moving parts, especially if sticking with ClojureScript (and html/css) Use shadown-cljs if you want to make use of JavaScript packages npm packages.


When using Cider then cider-jack-in-cljs will start up a REPL, prompting for a few startup options


I tend to use the figwheel hotloading of changes when saving source code files. I mostly use the REPL directly for experimenting with the atom that represents the state of the app. Updating the data in the atom affects the components that are ‘listening’ to that state, via reagent (or rum, react if using those instead)


If you take the Figwheel-main approach, then take a look at this guide for help


If taking the shadow-cljs approach, then study the user guide in detail. It is very detailed and there are quite a few things to be aware of


Thanks! Sounds like I’ll stick with fig wheel-main for a bit. I appreciate the pointers


how much does it matter to shadow core functions with locals? For example name happens a lot to me.

Alex Miller (Clojure team)15:11:54

By design, it does not matter, shadow away


I see. And here I was going through lenghts name destructred keys and such


You will only regret it if you shadow, and then forgetfully use the name inside that scope, forgetting about the shadowing, and wishing it were the clojure.core version

upvote 1
Lennart Buit15:11:39

Especially with name :’)

Lennart Buit15:11:07

The amount of time I’ve had that class java.lang.String cannot be cast to class clojure.lang.IFn


Although Clojure is not concerned about shadowing, those who have to work with the code might not appreciate it. So if it’s meaningless code, then go ahead, but if its supposed to be maintainable code, then consider using more meaningful names.

✍️ 1
💯 2
Tomasz Kontusz16:11:50

Is there any specific reason why vectors are sorted by length first, and only then by elements?


any code sample?


(compare [0 2] [1]) ;; => 1

Tomasz Kontusz16:11:14

(sort '([1 3] [1 2] [1 1]))
;; => ([1 1] [1 2] [1 3])
(sort '([1 3] [1 2 3] [1 1]))
;; => ([1 1] [1 3] [1 2 3])

Tomasz Kontusz16:11:21

I expected the second one to be ([1 1] [1 2 3] [1 3]), and I wonder why it is like it is. Is this a performance optimisation, or is there some common use case where sorting by length first is nicer?

Alex Miller (Clojure team)16:11:55

what else would you do?

Tomasz Kontusz16:11:02

My first intuition was sorting by elements first.

Alex Miller (Clojure team)17:11:25

I don't know off the top of head how that decision was made or the factors that went into it

Alex Miller (Clojure team)17:11:14

certainly it's faster to compare length of two arbitrary vectors than values


Lexicographer order with shorter vectors first as tie-breaker, i.e. dictionary order, would meet some people’s expectations, but it is slower for no -equal length vectors


Easy to write a lexicographic compare function if you need one, of course

Tomasz Kontusz18:11:31

Thanks everyone! I see this is a very old behaviour, for now I'll assume it's not built for some idiom I'm missing but is more of a performance thing.

Ben Lieberman17:11:27

I want to use reduced here but I'm not sure it's quite right?

(reduce-kv (fn [acc k v]
                     (if (some? v)
                       (str acc (name k) "=" v "&")
                       (str acc (name k) "=&"))) "" coll)
I'm building a query string out of a map and want the reduction to terminate in the case where the last key in the map has no value, so that it doesn't have a =& stuck on to the end of the resulting string


A couple points: • there's not really a "last key" in a map (unless you're specifically using something like ordered-map) • your framework probably already includes a function that builds query strings. this implementation doesn't escape keys or values which is a security issue. even if it's ok for the current usage to skip escaping, someone may use it in another context in the future.

Ben Lieberman18:11:04

Thanks for the pointers @U7RJTCH6J. I am doing this because for the moment I am not using a framework, but that's good to know.


if I were trying to implement something like that, I would probably do something like:

(->> {:a 42
      :b :bar
      :c nil}
     (filter #(some? (val %)))
     (map (fn [[k v]]
            (str (name k) "=" v)))
     (clojure.string/join "&"))


even if you're not using a framework, both the java and javascript standard libraries have implementations

Ben Lieberman18:11:09

The particular API I am trying to hit for whatever reason retains keys that have no corresponding value in the query string.


is this clj or cljs?


there is a great library you can trace step by step to see what your form evaluated @U04AYRTELNM

👍 1

hmm, it seems using the java standard lib version is less straightforward than I remember. anyway, there's a library that does just the query parameter part for you,

👀 1

once i saw a trace tool to log each result generate by form but i forget what’s names


see ring.util.codec/form-encode

🙏 1

Is it possible for a function assigned to a Record field to somehow reference the Record it is attached to? (akin to the this arg of a type method).

(defrecord Foo [bar baz])

(def my-foo
  (->Foo "foo"
         (fn [x] (str x (:bar this)))))

Bob B20:11:15

if you think of this as a 'constructor' call, there is no 'this' in the context of constructor arguments, so I don't think so - more generally, I'd say the closest idiomatic thing would be a protocol function (but I'd happily defer to smarter people on that)


You want a record to have a way to include a reference to a function (by being the value of a field in the record), and you want the function to have a reference to that record. That is impossible if all of the references are immutable, because one of those two things must be created before the other, and thus the first one has no way to refer to the second one created.


However, if there is at least one mutable reference in there somewhere, it can be created.


Clojure records are immutable, so without changing their implementation, you cannot have a mutable reference inside of the record.

Alex Miller (Clojure team)21:11:35

why don't you just make it a function on the record? then you'd get the this

Alex Miller (Clojure team)21:11:13

needs to be from an interface, but definterface or defprotocol can do that for you


yeah, I could add even more details to the answer I was starting, but what I was describing is definitely the more involved difficult way 🙂


Thanks. I'm not really sure the best way forward. It's a nesting problem. I'm trying to augment a record by wrapping it with another.

(def A (->Foo [val "a value"
               foo-fn (...)]))

(defn make-b [v f]
  (->Foo [val v
          foo-fn (fn [x]
                   (let [y ((:foo A) x)]
                     (assoc y z v)))]))
But I don't actually want v in the assoc at the end. I want whatever val of the outer Foo is at the time of calling foo-fn.


So I guess I could work this out by making foo-fn part of a protocol? But then I still need to work out how that translates to wrapping one around the other. And can a protocol method also be a basis field at the same time?


how do I install dependencies listed in a deps.edn file ? I'm looking at the documentation and just failing to see how to do this.


Running clj does it - but it also starts a repl


The -P flag will download dependencies in deps.edn, clojure -P for the main dependencies of the project, clojure -P -M:test/run to download the main project dependencies and all the libraries in the :test/run alias


I see it in the --help output under exec-opts now I just missed it


The -P flag also works with -X and -T flag aliases too, so long as the -P flag is first


Typically you don't really need to "install" dependencies, because whenever you start clj/`clojure` it downloads anything listed in deps.edn that is missing. You only need the`-P` flag if you have a specific goal that calls for pre-installing, such as avoiding network usage at a later time, or caching in a Docker image.


clojure -A:build:tests-deps:check-syntax-and-reflections:run-tests:build -Stree -A - all aliases deps which you want to include to install deps


maybe there is other way, than -Stree.


oh yes, I see -P was added 👍


but the point is -A which can be necessary to use with -P