Fork me on GitHub
#clojure
<
2021-04-29
>
Crispin02:04:46

I'm having trouble finding the source of disparate run times in subsequent group-by calls. First call is slow. But then if you evaluate the exact same thing it's fast. Is there some kind of caching in group-by? group-by uses transient collections. Is there internal caching in transient/persistent! ?

raspasov02:04:34

(time …) is not a good benchmark; Use criterium (library), or if you really care/need performance, something like YourKit is essential (it’s a paid product, it’s worth it IMO, I’ve used it)

raspasov02:04:57

Try switching tuples to: (def tuples (mapv vector kmers (next kmers))) … to see if it makes a difference.

phronmophobic02:04:03

fwiw, your example seems like a reasonable usage of time (to me).

raspasov03:04:58

I agree, I didn’t notice the big discrepancy at first (which is very obviously caused by the laziness of map as you pointed out)

👍 3
phronmophobic02:04:54

It's due to laziness. If you add (doall tuples) before the first time measurement, then the times should be similar

👆 3
😂 3
👍 3
3
Crispin02:04:51

thankyou. thats it

👍 3
tlonist06:04:29

Has there been any attempts to do MSA with clojure? I’d like to know if there’s any good sample projects out there in github or else. So far I’ve found one book ‘microservices-with-clojure’ but couldn’t find more, definitely yet to be a mainstream way of developing with clojure. But I think the event driven nature of the architecture suits with immutability and pure function concepts in clojure (or with other fp languages). I’d like to know more the community’s perspectives on this and current situation.

didibus06:04:32

I'm sure you'll find plenty of Clojure wrappers over them by many people, but personally I'd just use them directly with Clojure interop, they all generally just revolve around:

(doto (SomeClient. config)
  (.sendMessage message))

didibus06:04:45

Hum... Actually the AWS ones have native Clojure clients maintained by the company that employs the Clojure core team itself: https://github.com/cognitect-labs/aws-api

tlonist07:04:32

I see your point on not re-inventing the wheel. But having to get acquainted with those tech-stacks in Java will be a little bit of hurdle to jump through.

tlonist07:04:48

Yap, I’ve been using aws-api as well; it was a nice implementation.

tlonist07:04:28

all in all, microservice seems to be very capable with current clojure environment. It is reassuring to know!

didibus07:04:42

Ya, Clojure support for some of those would be good, but the language is too niche, those projects tend to only support the top 10 most used languages like Java, C#, JS, Go, C++, etc.

didibus07:04:59

And even then, they often don't support them first class, normally they use code-gen, but that means they always design their clients in an OO way, and then generate the client code for Java, C#, JS, etc. all from it. So even if they generated a Clojure one, it would probably feel very OO and similar to using the Java one through interop

didibus07:04:15

That said, for the most popular ones, I'm sure you'll find a relatively well maintained Clojure wrapper made by someone else, such as Kafka and ActiveMQ, and all the AWS one are covered by aws-api

👍 3
didibus07:04:21

I'd also have a look at Polylith if I was you: https://polylith.gitbook.io/polylith/ It's not exactly MSA, and it can be used to make micro-services, but it's also different. Kind of let you do a micro-services lite architecture and then decide if you want to deploy things as a monolith or a set of micro-services.

👍 3
tlonist08:04:18

Awesome! thanks for the recommendation, will look into it.

the2bears19:04:58

> A system where services (typically having its own db) communicate with events on pub-sub basis As an aside, I think this is far too narrow a description of micro-services. More important to mention is decoupling and autonomy of services. Communication may be events, it may be a request/response mechanism. Services tend to be small, encompassing a business context. Finally, they lend themselves to independent deployment due to many of the above qualities.

🙏 4
Jim Newton08:04:06

is there a tutorial about how a macro (defined by defmacro) can extract meta data from its list of arguments?

Jim Newton08:04:06

I’d like to invoke a macro as follows: `(my-macro a b ^{some-meta-data-1-as-hash} c d e ^{some-meta-data-2-as-hash} e f)` and within the macro definition figure out that two hashes were given, the first associated with c (which might be any syntactical form, and the second hash was associated with e which again might be any syntactical form.

Jim Newton08:04:06

hmm actually its worse than that. The syntax is

(my-macro ([a b c] ….)
          (^{some-data-as-hash} [a b [c d]] …)
          ...)

borkdude08:04:52

@jimka.issy

user=> (defmacro foo [a] (list 'do (meta a)))
#'user/foo
user=> (foo ^:foo [])
{:foo true}
user=> (foo ^{:foo :bar} [])
{:foo :bar}

Jim Newton09:04:46

couple of things are fuzzy.. meta gets the meta data associated with its given value right? When if I pass the same value twice to the macro, how does meta know which is which?

Jim Newton09:04:46

e.g. (mymacro ^meta-data-1 42 ^meta-data-2 42)

borkdude09:04:11

you can't do (foo ^:foo 1) since metadata cannot be applied to a number

Jim Newton09:04:40

interesting. that means I really don’t understand. I thought meta data was applied to the list (foo 1) with some magic indication that it applies to the 2nd element.

borkdude09:04:58

There is hardly any magic in Clojure. meta just looks up metadata on objects that support it (notably collections and symbols, all implementing clojure.lang.IObj)

borkdude09:04:01

When you write (defn foo [^String x]) what you are doing is you are writing a list with symbols. The symbol x has metadata {:tag String}. And this data is processed by the defn macro (or rather the thing below it, the fn* special form)

Jim Newton09:04:24

so if I make a macro call with the same symbol twice, can I have different meta data on each occurance of the symbol? (mymacro ^meta-data-1 a ^meta-data-2 a)

Jim Newton09:04:18

ok, that means the meta data is not assoicated with the symbol, because a is eq to a

Jim Newton09:04:29

or am I even more confused that I imagined?

borkdude09:04:59

the metadata is associated with a specific symbol. symbols are not interned in Clojure, because they do not have context-free meaning, like keywords or strings

raspasov09:04:39

@jimka.issy so if I’m understanding correctly, you’re trying to associate meta data to a binding form, and have the macro behave more or less like (defn …)

Jim Newton09:04:54

are you asking what i’d like to do after I extract the meta data?

Jim Newton09:04:08

sorry, I don’t understand your question

raspasov09:04:08

Nevermind 🙂

Jim Newton09:04:36

I’m writing a macro which is very similar to fn. fn allows the user to specify several sequences, the first element of each is a lambda-list (an arglist in clojure-speak). I want to optionally associate meta data with some of these lambda-lists

raspasov09:04:18

For writing a macro that’s similar to defn/fn see this: https://github.com/lilactown/helix/blob/master/src/helix/core.clj#L116 (it has been useful to me to look at when writing something similar)

raspasov09:04:05

@U04V15CAJ that’s a neat example 🙂 The short version (if I’m correct):

(defmacro foo-short [a]
 `'~(map (juxt identity meta) &form))

raspasov09:04:45

Took me 5 min to figure out, a small mind bender 🤯 😄

raspasov09:04:55

That triple combo ``'~`

raspasov09:04:41

Is that about right?

borkdude09:04:43

See this example:

user=> (defmacro foo [a] (list 'quote (map (juxt identity meta) &form)))
#'user/foo
user=> (foo ^String x)
([foo nil] [x {:tag String}])

Jim Newton09:04:05

I have to admit that I really don’t understand. but my code works. I have written a macro named dsfn which expands to destructuring-fn as follows:

(dsfn ([x]  'aaa)
      (^{x Boolean}
         [x y] 'bbb)
      ([x y] 'ccc))
expanding to
(destructuring-fn
  ([[x] {}] 'aaa)
  ([[x y] {x Boolean}] 'bbb)
  ([[x y] {}] 'ccc))
here’s the macro
(defmacro dsfn [& forms]
  (letfn [(process [form]
            (let [meta-data (meta (first form))]
              (cons [(first form) (if (nil? meta-data) {} meta-data) ] (rest form))))]
    `(destructuring-fn
      ~@(map process forms))))

raspasov09:04:23

What does destructuring-fn look like?

raspasov09:04:14

Is that another macro?

Jim Newton09:04:11

yes it is another macro.

Jim Newton09:04:30

I hope that link is public. tell me if it is not

raspasov09:04:57

Yes, public.

raspasov09:04:26

That is neat to try to match on the structure/shape of the data.

raspasov09:04:09

What part of dsfn is unclear? It seems to do what it says.

Jim Newton10:04:38

hmm. perhaps it is clear, and I’m just self deprecating.

raspasov10:04:28

That looks like a neat project; Is it part of research or just for fun?

Jim Newton10:04:32

thanks. yes it is both fun and research.

Jim Newton10:04:01

I’m happy to have feedback.

Jim Newton10:04:18

it’s part of a presentation at the European Lisp Symposium in two weeks.

raspasov11:04:24

Nice, any hope this can work in Clojure and ClojureScript?

Jim Newton12:04:27

I have no experience with clojure script.

Jim Newton12:04:47

the theory behind it is that it should work in a large range of dynamic reflective languages

Jim Newton12:04:56

a student of mine wants to investigate javascript

Jim Newton14:04:37

what do functions like int? and integer? and number? double? do in clojurescript?

raspasov15:04:40

@jimka.issy perhaps you want to create a clojure-rte channel here in Slack to discuss this?

raspasov15:04:47

All of the above functions exist in CLJS; there are some gotchas around numbers, but those are all caused by the differences between JS and the JVM;

raspasov15:04:23

Things like this actually run ok in CLJS:

(+ nil 1)
=> 1 
… but would throw an Exception on the JVM

raspasov15:04:32

From here: https://clojurescript.org/about/differences • Numbers • ClojureScript currently only supports integer and floating point literals that map to JavaScript primitives ▪︎ Ratio, BigDecimal, and BigInteger literals are currently not supported ◦ Equality on numbers works like JavaScript, not Clojure: `(= 0.0 0) ⇒ true`

Jim Newton17:04:48

it will be interesting to see how much work will be needed to make it work in clojurescript

Jim Newton17:04:34

There are quite a few places where I depend on the way symbols work and also on the JVM.

Jim Newton10:04:32

How am I suppose to distinguish beween vector-ish and list-ish ? For example, fn distinguishes its semantics based on whether an argument looks like [a b c] or ([a b c] …) ? I had been using the function list? but apparently given (cons 1 (list 2 3 4)) the list? predicate returns false.

yuhan10:04:36

You probably want seq? instead of list? , which is specifically for IPersistentLists

borkdude11:04:33

always use seq? is probably a good rule of thumb

Jim Newton10:04:02

my dsfn macro above calls (cons ….) which produces a non-list 😞

Jim Newton10:04:23

looks like in the code of fn that it calls vector? rather than calling list? to distinguish the two cases.

borkdude15:04:38

Related to something in #beginners : Has there been any discussion around allowing (Class/staticMethod foo bar baz) if staticMethod is a Java method receiving a variable number of things at the end, instead of forcing Clojure users to create an array?

borkdude15:04:34

I guess this is more difficult to support in Clojure because of the dynamic typing? (currently you can look at the type of array to choose the right method)

Alex Miller (Clojure team)15:04:43

yes, fogus has been working on this

Alex Miller (Clojure team)15:04:12

it's complicated of course, but going to take a run at it for 1.11 probably

borkdude15:04:49

ooh really, wow

fogus (Clojure Team)16:04:08

Currently hammock-timing this one. 🙂

borkdude15:04:51

I guess there is an ambiguity when you have varargs of arrays, but this is pretty rare... it's a difficult thing to get right I can imagine

borkdude15:04:09

I didn't know the last one would work as well, but it does (clojure 1.11.0, babashka current).

user=> (defn foo [& {:keys [a b]}] [a b])
#'user/foo
user=> (foo {:a 1 :b 2})
[1 2]
user=> (foo :a 1 :b 2)
[1 2]
user=> ((partial foo :a 1) :b 2)
[1 2]
user=> ((partial foo :a 1) {:b 2})
[1 2]
🤯

Alex Miller (Clojure team)15:04:46

https://ask.clojure.org/index.php/4224/java-method-calls-cannot-omit-varargs for anyone who wants to vote on it (such an old request, jira goes back to 2010!)

borkdude15:04:05

are you adding up votes from ask + the old jira or how does this work?

Alex Miller (Clojure team)15:04:07

it wasn't even jira - that was filed back when we were using Assembla

Alex Miller (Clojure team)15:04:27

primarily using ask, but we look at both

Alex Miller (Clojure team)15:04:00

jira votes did not fully survive the transition into cloud jira (lost any votes from users that weren't migrated)

😞 3
Alex Miller (Clojure team)15:04:59

did not really have a big impact - the highest rated things were still the highest rated things more or less

andy.fingerhut16:04:36

high voter turnout requires high impact issues that fire people's imaginations, not just for Clojure issues 🙂

borkdude15:04:23

voted, thank you

Alex Miller (Clojure team)15:04:38

it has enough votes that it is already on the radar of course, but doesn't hurt

borkdude15:04:42

I am using a similar thing to your ask for some of the issues in my projects, but using github discussions. It makes elaborating on ideas easier while keeping the issue itself more crisp.

didibus15:04:20

Why does defmulti go out of its way to prevent itself from being re-evaluated more than once?

Alex Miller (Clojure team)15:04:44

because you would lose all the methods I think

borkdude15:04:04

can't it update itself if it already exists?

Alex Miller (Clojure team)15:04:38

what problem are we trying to solve?

clj 3
rich 3
borkdude15:04:45

(similar to how vars are updated when you call def twice)

didibus15:04:16

Hum... I can see how someone might have thought that, but I think it would have been less annoying to manage that problem than try to force re-evaluate a defmulti.

borkdude15:04:20

@alexmiller It might be related to a discussion in #clj-kondo where someone suggests that clj-kondo should advice using a var as the dispatch function (to which I didn't agree, but this would solve their REPL problem)

didibus15:04:16

I've just seen a lot of people get burned by that and use tricks to avoid it, like passing it a var to the dispatch-fn, or adding a (def multi nil) above it.

Alex Miller (Clojure team)15:04:16

this is one of those things where repl use has tradeoffs with non-repl use

ghadi15:04:59

someone should say what they're trying to do out loud. It's buried in between the lines

👍 3
borkdude15:04:43

I thought @didibus was referring to what @ddouglass said in #clj-kondo but that might not be the case and maybe it was purely co-incidental

didibus15:04:41

I was wondering if there was any downside to use those trick, since defmulti goes out of its way to not be re-evaluable.

ghadi15:04:53

as someone who has watched rich and alex work for a little while now:

ghadi15:04:21

didibus is asking about "why does defmulti go out of its way not to be re-evaluable"

ghadi15:04:54

I think what they mean is "I want to alter or iterate on the dispatch function at the REPL"

ghadi15:04:16

the stuff about "preventing re-eval" is removed from this, and a presumption ^

didibus15:04:27

Ya, for the Var. I guess adding (def multi nil) above it shouldn't be slower though right?

Alex Miller (Clojure team)16:04:18

no, not sure if that impacts ability of tools to analyze

ghadi16:04:33

changing the dispatch function could change its arity, which would also break expectations on all the extensions

didibus16:04:46

In this case I wanted to know why it prevents itself from being re-evaled, I already know the workarounds at the REPL. Was trying to reason what are the pros/cons of this:

(def multi nil)
(defmulti multi (fn[..] ...))

ghadi16:04:34

keep in mind the extensions could be in any file

didibus16:04:02

I feel it must have been written very early. Because require doesn't force reload by default, and most people at the REPL control what they reload. So I don't see why it would cause issues nowadays, but also I can see how someone thought that it shouldn't.

ghadi16:04:28

what is the "it" in "I don't see why it would cause issues"?

didibus16:04:22

It's an ergonomics issue, if you've defined te wrong dispatch function and want to fix it at the REPL. And it seems maybe the behavior of defmulti is only for the REPL ergonomics as well. In trying to prevent the attached methods from being lost by accidentally re-evaluating the defmulti. So I was wondering if that was all there is to it, or there was like performance, or memory or some other reasons that's not just a REPL ergonomic thing.

didibus16:04:08

And if you bypass it because you personally prefer the other way of working at the REPL, would you risk something doing so

ghadi16:04:02

if you are changing the dispatch function, it is not automatically clear whether the previously extended things are compatible with the new dispatch function

didibus16:04:48

Well, most of the time when you change the dispatch function it's because it was wrong to begin with. So generally either the defmethod already didn't work and you're trying to fix them, or you know you plan to just change your defmulti to work differently and will go update the defmethods too.

didibus16:04:54

Ya, it's a bit of a tradeoff I guess. What is more annoying, reloading the defmulti by accident and having to reload all the other defmethod for it or the other way around.

didibus16:04:15

it = defmulti

borkdude16:04:37

I think you mean, it = making defmulti reloadable

Alex Miller (Clojure team)16:04:43

you may not even know what the defmethods are to reload them

didibus16:04:02

Ya, I see the tradeoff, maybe I feel in practice if you're changing a dispatch-fn, you're probably the owner of the defmulti and your dispatch-fn wasn't working or doing what you want. And if you've got a defmulti from a library, it won't get re-evaluated anyways because require itself doesn't reload lib-spec.

didibus16:04:32

But.. I need to think about that second one more. Like if I vended a library where I expect users can extend the defmulti and my code did:

(def multi nil)
(defmulti multi (fn...))
Would I cause them pain-points at the REPL

matheusashton16:04:49

Hello! I'm trying to run a compojure project with ring-jetty-adapter , and tried to add wrap-reload of ring.middleware.reload so I could run lein run only once while developing, but I'm receiving an exception trying to run:

Caused by: java.io.FileNotFoundException: Could not locate ring/middleware/reload__init.class, ring/middleware/reload.clj or ring/middleware/reload.cljc on classpath.
	at clojure.lang.RT.load(RT.java:466)
	at clojure.lang.RT.load(RT.java:428)
	at clojure.core$load$fn__6824.invoke(core.clj:6126)
	at clojure.core$load.invokeStatic(core.clj:6125)
	at clojure.core$load.doInvoke(core.clj:6109)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5908)
	at clojure.core$load_one.invoke(core.clj:5903)
	at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
	at clojure.core$load_lib.invokeStatic(core.clj:5947)
	at clojure.core$load_lib.doInvoke(core.clj:5928)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$load_libs.invokeStatic(core.clj:5985)
	at clojure.core$load_libs.doInvoke(core.clj:5969)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$require.invokeStatic(core.clj:6007)
	at clojure.core$require.doInvoke(core.clj:6007)
	at clojure.lang.RestFn.invoke(RestFn.java:482)
	at fiis_api.service$eval2080$loading__6706__auto____2081.invoke(service.clj:1)
	at fiis_api.service$eval2080.invokeStatic(service.clj:1)
	at fiis_api.service$eval2080.invoke(service.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:7176)
	at clojure.lang.Compiler.eval(Compiler.java:7165)
	at clojure.lang.Compiler.load(Compiler.java:7635)
	... 96 more
any ideas of what am I missing?

borkdude16:04:02

The dependency?

matheusashton16:04:32

I added [ring "1.9.3"] to project.clj .-.

benwen16:04:03

Hi all: The San Francisco-based Bay Area Clojure meetup is looking for speakers for a May 11 (usu. 8pm PT) zoom event. #clojure-bay-area if you are interested in sharing something or know someone who might fit that description. Past zoom events here: https://www.youtube.com/channel/UCmtrRPVjcEMZrzTfAY3fd0Q Folks from remote locations works as well. Thank you!

noisesmith17:04:09

:/ - (javadoc (.getBytes "hello")) opens a google search for "allinurl:[B.html"

noisesmith17:04:24

maybe arrays should be special cased?

denik17:04:14

what’s the best/fastest way to compare two byte-arrays?

potetm17:04:16

for equality?

denik17:04:34

for sorting, e.g.

(compare (.getBytes "foo") (.getBytes "bar"))
Execution error (ClassCastException) at z1.serve/eval25808 (serve.clj:459).
class [B cannot be cast to class java.lang.Comparable ([B and java.lang.Comparable are in module java.base of loader 'bootstrap')

potetm17:04:08

how do you compare opaque byte arrays?

potetm17:04:30

just the raw byte values (i.e. ignoring any semantic meaning)?

denik17:04:22

they just need to be compared for a db index

potetm17:04:11

Yeah I don’t think that’s built in anywhere

potetm17:04:19

that I know of

denik17:04:03

thank you!

denik17:04:34

for sorting, e.g.

(compare (.getBytes "foo") (.getBytes "bar"))
Execution error (ClassCastException) at z1.serve/eval25808 (serve.clj:459).
class [B cannot be cast to class java.lang.Comparable ([B and java.lang.Comparable are in module java.base of loader 'bootstrap')

denik17:04:12

this works but allocates two vectors 😕 (compare (vec (.getBytes "bar")) (vec (.getBytes "bar")))

Alex Miller (Clojure team)17:04:27

java.util.Arrays has some methods for sorting and equality of byte arrays

Alex Miller (Clojure team)17:04:32

depending on your exact goals, you could certainly write a tight loop over the array for this purpose (in Java for best performance) or there are probably some numerical methods if you convert the byte arrays as numbers

denik17:04:49

thanks alex!

Ronny Li21:04:47

Does anyone know about the general approach behind Roam's multiplayer feature? Wondering if there's any libraries out there that can help me implement something similar 🙂

phronmophobic21:04:11

Not sure what roam is using, but many multiplayer features are implemented on top of CRDTs or Operational Transforms(OTs). Here’s a good video that covers many of the concepts, https://youtu.be/x7drE24geUw

phronmophobic21:04:22

The guy from the video has a crdt library called automerge, https://github.com/automerge/automerge

Ronny Li21:04:40

Thank you! 🙏