Fork me on GitHub
#clojure
<
2022-05-16
>
pinkfrog01:05:47

How can I println to stdout instead of out ? My specific case is, I want to print to the nrepl console instead of the repl.

flowthing13:05:14

Oh, you want to do the opposite.

xji01:05:48

Hi folks, I’m not familiar with Clojure as I program in Elixir professionally (but am quite interested in the comparison between BEAM languages and Clojure, and eager to improve my proficiency in Clojure), therefore sorry if this question doesn’t make much sense or gets something fundamentally wrong: https://bugs.openjdk.java.net/browse/JDK-8277131 is attracting a lot of attention recently. Does it have any implications for Clojure’s core.async? Would virtual threads be a natural fit for the go of core.async? (I mean, in Golang go literally launches a goroutine/green thread). From what I remember, in core.async, when a task/state machine is paused, it’s taken off the thread for another task/state machine to run, while if the execution resumes, it’s put back onto the thread again. Also, if I remember it correctly, the go tasks launched by core.async share a fix-sized thread pool. Then, wouldn’t it make sense to use a pool of virtual threads for it once Project Loom is done? And if so, what would be the implications (e.g. performance or otherwise) to core.async? It would be interesting to see a comparison of core.async vs. BEAM VM’s actor processes in handling massively concurrent workloads, though of course I know such things might not be directly comparable using a single number. (P.S.: One thing I’d like to clarify about JEP 425 is that, will it be a fundamental addition to the JVM itself, or is it more of an “official virtual thread API built on top of conventional JVM threads” kind of thing? AFAIK there were already some libraries providing lightweight threads on JVM e.g. http://docs.paralleluniverse.co/quasar/, which from what I read seems to perform bytecode manipulation. I guess JEP 425 will be something different?)

jjttjj01:05:45

There have been several recent discussions on just this. here's one https://www.reddit.com/r/Clojure/comments/uoo3ad/the_dawn_of_lightweight_concurrency_for_java_and/ but there are more good ones if you use the slack search for "loom"

hiredman01:05:54

There is no answer

hiredman01:05:38

Lightweight threads have not been released yet, and there is no public committed road map for core.async

hiredman01:05:59

Virtual threads do seem like they get rid of the need for the go macro on core.async, however at a low level the channels implementation bakes in callbacks and running them on a threadpool, which is not what you want with virtual threads

hiredman01:05:16

Development on core.async is somewhat slow to materialize (I sort of suspect the devs are playing it slow waiting for virtual threads)

hiredman01:05:14

Something to keep in mind is core.async is just a library, and not baked into the language, and virtual threads to some degree lowers the barrier to creating similar libraries (no need to write a complex macro for code transformation), so it is possible when virtualthreads are generally available we'll see a cambrian explosion of alternative libraries

ghadi02:05:24

re: > One thing I’d like to clarify about JEP 425 is that, will it be a fundamental addition to the JVM itself, or is it more of an “official virtual thread API built on top of conventional JVM threads” kind of thing? yes, it's more than just a library, there are several points of VM assistance that makes this categorically better

👌 1
xji02:05:29

> There have been several recent discussions on just this. Thanks. I was searching using “PEP 425”, though I guess “Loom” would be the right keyword to use, as you mentioned.

xji02:05:04

> virtual threads to some degree lowers the barrier to creating similar libraries (no need to write a complex macro for code transformation), so it is possible when virtualthreads are generally available we’ll see a cambrian explosion of alternative libraries Makes total sense. (And I was gonna post in the original question that I wasn’t able to understand much from a cursory glance at ioc_macros.clj 😅)

xji02:05:52

> however at a low level the channels implementation bakes in callbacks and running them on a threadpool, which is not what you want with virtual threads So I guess with virtual threads you wouldn’t have a “thread pool” in a conventional sense, but they would more be something to be spawned directly on-demand then. Then if core.async is to make use of virtual threads, it will need to change the current low-level implementation of its channels.

hiredman02:05:02

You can have thread pools for virtualthreads, but swapping out the threadpool that is inside core.async for a virtualthread pool doesn't make sense, the threadpool inside core.async is just not needed at all

👌 1
hiredman02:05:04

(arguably the threadpool inside core.async could go away without virtualthreads and it could instead use the default forkjoin threadpool, but virtual threads should get rid of the thread shaped hole in the library)

hiredman03:05:57

So for example, in that reddit thread there is a "benchmark" of go blocks vs virtualthreads, but the way core.async's current implementation prioritizes the needs of go blocks, blocking operations (which virtualthreads use) end up needing 3-4 threads (the two exchanging values via a channel, and 2 in the go threadpool)

seancorfield03:05:17

@U03FHU91HQB I posted the following code fragment in a recent long thread about Loom: https://clojurians.slack.com/archives/C8NUSGWG6/p1652487349607519?thread_ts=1652433406.842079&amp;cid=C8NUSGWG6 but the bottom line is this isn't going to change anything in Clojure (or even core.async) for a long time because backward compatibility across JVM versions is very important (Clojure 1.10 was the first version to not support JDK 7).

👌 1
jumar04:05:56

While I don't know much about this topic this piece specifically caught my attention in the javadoc: https://download.java.net/java/early_access/loom/docs/api/java.base/java/lang/Thread.html > Virtual threads are suitable for executing tasks that spend most of the time blocked, often waiting for I/O operations to complete. > Virtual threads are not intended for long running CPU intensive operations.

seancorfield04:05:46

Yeah, I think we'll have to wait and see until Loom is widely available in a fully-released version of the JDK to see how it benchmarks and what people find is the most suitable use case for it. I have no idea whether a version of core.async based on v-threads even makes sense -- I don't use core.async much -- but I suspect it will take us all a while to get used to a completely different model of multithreaded code in Java/Clojure.

👌 1
Sameer Thajudin04:05:18

What other Clojure frameworks exist that are comparable to Luminus?

Cora (she/her)04:05:17

for full stack ones? there's fulcro

👍 1
Cora (she/her)04:05:12

I'm trying to think of more... coast probably still works?

aratare04:05:25

Depends on what you categorise as “comparable to Luminus”, if you want a fullstack solution there are plenty like @corasaurus-hex mentioned.

👍 1
aratare04:05:49

Just note that fulcro, while being fullstack, is quite different from the rest

Cora (she/her)04:05:44

I guess hoplon? I don't hear much about that one lately

Cora (she/her)04:05:55

there are also example apps for full stack clj(s) development like: https://github.com/souenzzo/atemoia

👍 2
pinkfrog10:05:35

Hi. How to perform split as in shell commands. For example,

"a b c \"a b\""
will be split as
["a", "b", "c", "a b"]

Bart Kleijngeld10:05:24

I think you'll find the answer here: https://clojuredocs.org/clojure.string/split Don't forget to require the clojure.string namespace

pinkfrog10:05:34

That does not work (I mean for the basie #" " re

Bart Kleijngeld10:05:05

(clojure.string/split "a b c \"a b\"" #" ")
["a" "b" "c" "\"a" "b\""]
Works fine here

Bart Kleijngeld10:05:05

Oh, sorry, you want the space between a b to be preserved of course

Bart Kleijngeld10:05:46

I've been trying to play with regex negative lookahead and lookbehind to match only spaces which aren't within the " quotation, but my regex is too rusty. https://stackoverflow.com/questions/40479546/how-to-split-on-white-spaces-not-between-quotes This article seems to address your problem too. Anyways, there might be a better way, I'm curious what others will say (here).

delaguardo10:05:49

one possible approach is to use clojure.edn/read

(require '[clojure.edn :as edn]
         '[ :as io])

(let [st "a b c \"d e\""
      rdr (java.io.PushbackReader. (io/reader (.getBytes st)))]
  (loop [acc []]
    (if-let [el (edn/read {:eof nil} rdr)]
      (recur (conj acc el))
      (map name acc))))

p-himik10:05:44

@U03BYUDJLJF I'm pretty sure you can't use RegEx to parse such things, since shell arguments with their quoting syntax aren't a regular language.

Bart Kleijngeld10:05:31

Just to make sure I'm getting my idea across: instead of matching any space character to split on, I would try to exclude those which have " on either side of them. I tried using #"(?<!.*\") (?!.*\")" but that didn't work. Perhaps because you're correct 😉. Could you elaborate somewhat more? (If this is too off-topic for others you can DM me also)

Alexis Schad10:05:36

I like regexp problems and was able to make it work with group matching:

(defn split-shell-commands [s]
  (->> s
       (re-seq #"\\\"(.*?)\\\"|(\S+)")
       (map #(or (nth % 1) (nth % 2)))))

p-himik10:05:14

You can have arbitrary nesting of ", ', (), and ` - all without quoting. RegEx can only parse stuff that's regular, that can be represented with a state machine without any memory. So, RegEx can't parse arbitrary nesting. Exact same reason why RegEx can't parse HTML.

gratitude 1
Bart Kleijngeld11:05:35

That makes sense. Thanks!

👍 1
p-himik11:05:44

@U01V2D5ALKX Due to the above, your code will not work correctly with any non-trivial nesting. E.g. it fails with "'\"hello there\"' general kenobi".

Alexis Schad11:05:52

nested parsing can be done with extra code over regexp

p-himik11:05:05

But that won't be RegEx. ;)

Alexis Schad11:05:37

nobody said it must be an only regexp solution 😛

p-himik11:05:52

Alright, I'll refine my initial statement - you can't use just RegEx to parse such things. But at that point you would be better off with a regular parser, without any RegEx at all.

Alexis Schad11:05:14

for your example, what's the expected output? \"hello there\" general kenobi or with another parse over \"hello there\" to get hello world

p-himik11:05:47

An easy way to see what arguments should be (assuming they don't have newlines):

#!/bin/bash

for i in "[email protected]"; do echo $i; done
In my example, there are three separate arguments - "hello there", general, and kenobi. The quotes must be preserved. But if you're going to improve your parser, please don't tag me. :) I've spent enough man-months tinkering with parsers - that's not something I enjoy doing.

Alexis Schad11:05:22

so there's no nested things?

Alexis Schad11:05:40

and so regexp works here

p-himik11:05:13

I have sent you a string where your RegEx doesn't work. You can make it arbitrarily complex by mixing all those paired characters, and alternating between some of them does not require quoting. The rest, as I mentioned, I'll leave as an exercise for the reader.

Alexis Schad11:05:46

I just have to manage single quotes:

(defn split-shell-commands [s]
  (->> s
       (re-seq #"'(.*?)'|\\\"(.*?)\\\"|(\S+)")
       (map #(first (remove nil? (rest %))))))
(split-shell-commands "'\"hello there\"' general kenobi")
=> (""hello there"" "general" "kenobi")

Alexis Schad11:05:11

""hello there"" is the string "hello world"

delaguardo11:05:03

and then it will fail on the original string 🙂 "a b \"c d\""

p-himik11:05:38

"Nested parsing" and "parsing of nested structures" are related but different - here we need the latter. Just realized that I was thinking about something completely unrelated. In this case, they are one and the same - because you need to pair every quote, every parenthesis, everything. But RegEx can't do it, because it's regular and nested structures aren't regular. In case you aren't familiar with it, here's a good read that's especially fitting when you start spending too much time trying to use RegEx where it's inappropriate. :) https://stackoverflow.com/a/1732454/564509

Alexis Schad11:05:24

> and then it will fail on the original string 🙂 "a b \"c d\"" It works. Actually we have to be precise for the delimiters. "a b \"c d\"" is the string a b "c d" and so the delimiter is " not \" so the regexp must be #"'(.*?)'|\"(.*?)\"|(\S+)" instead

Alexis Schad11:05:49

Have you used the updated regexp?

Alexis Schad11:05:31

> But RegEx can't do it, because it's regular and nested structures aren't regular. I said the same thing above, when nesting stuff comes, we need extra code over regexp. Regexp can only parse flat.

Alexis Schad11:05:18

Don't get me wrong I never said the regexp is the best solution, I just said it is possible with it.

👍 2
Joshua Suskalo13:05:01

> I'm pretty sure you can't use RegEx to parse such things, since shell arguments with their quoting syntax aren't a regular language. I want to point out that RegEx isn't regular.

Joshua Suskalo13:05:44

Yes this may not be a regular grammar, but RegEx doesn't specify regular grammars, it specifies a superset of regular grammars that includes lookahead and a few other functionalities that allow it to do other things, depending on the implementation that includes nesting.

Joshua Suskalo13:05:01

I also want to note that parsing strings with escaped quotes inside is regular however and requires no nesting. It just requires an alternation clause with a repeat surrounding it where one clause matches complete strings (i.e. sequences of \" or any character but ", surrounded by "), and the other matches text outside of strings.

Joshua Suskalo13:05:55

Full shell handling of nested ' and " however I agree isn't regular and would require a regex engine that supports nesting, and I don't know if the JVM one does.

Joshua Suskalo13:05:41

It appears that the JVM implementation does not support nesting, so a stronger parser is required if you want to do full ' and " nesting.

borkdude14:05:46

@UGC0NEP4Y You can use babashka.process/tokenize to split the characters

👍 2
borkdude14:05:15

user=> (babashka.process/tokenize "foo 'bar' \"baz\"")
["foo" "bar" "baz"]

didibus20:05:33

Curious how you implemented yours @U04V15CAJ I'm guessing its not just a regex 😛

borkdude20:05:08

nope, not with a regex :)

borkdude20:05:45

Thanks to my new #quickdoc output it's pretty easy to find the source :) https://github.com/babashka/process/blob/master/API.md#tokenize

👍 1
shaunlebron14:05:48

Hi 👋 I’m trying to use a java markdown renderer from clojure, and the API requires creating a bunch of classes and class factories to use it. Can someone help me with how to provide these from Clojure? (I looked at the proxy macro that creates instances of an anonymous classes, but it looks like I need to provide the classes themselves.)

Parser parser = Parser.builder().build();
HtmlRenderer renderer = HtmlRenderer.builder()
        .attributeProviderFactory(new AttributeProviderFactory() {
            public AttributeProvider create(AttributeProviderContext context) {
                return new ImageAttributeProvider();
            }
        })
        .build();

Node document = parser.parse("![text](/url.png)");
renderer.render(document);
// "<p><img src=\"/url.png\" alt=\"text\" class=\"border\" /></p>\n"

class ImageAttributeProvider implements AttributeProvider {
    @Override
    public void setAttributes(Node node, String tagName, Map<String, String> attributes) {
        if (node instanceof Image) {
            attributes.put("class", "border");
        }
    }
}

emccue15:05:07

(defn image-attribute-provider
  []
  (reify
    AttributeProvider
    (setAttributes [_ node tagName attributes]
      (when (instance? node Image)
        (.put attributes "class" "border")))))

(let [parser (.build (Parser/builder))
      renderer (-> (HtmlRenderer/builder)
                   (.attributeProviderFactory 
                      (reify AttributeProviderFactory
                        (create [_ _]
                          (image-attribute-provider))))
                   (.build))
      document  (.parse parser "![text](/url.png)")]
  (.render renderer document))

emccue15:05:24

not tested, but roughly this

🙏 1
shaunlebron15:05:42

Would this be equivalent?

class IndentedCodeBlockNodeRenderer implements NodeRenderer {
    public Set<Class<? extends Node>> getNodeTypes() {
        // Return the node types we want to use this renderer for.
        return Collections.<Class<? extends Node>>singleton(IndentedCodeBlock.class);
    }
}
in clojure:
(reify NodeRenderer
  (getNodeTypes [_]
    #{IndentedCodeBlock}))

emccue15:05:50

yeah - so long as you don't need or want a named type

shaunlebron15:05:49

are you talking about omitting the .class at the end of IndentedCodeBlock?

emccue15:05:06

no i mean in the java version you have a NodeRenderer, but you specifically have a IndentedCodeBlockNodeRenderer

emccue15:05:15

there is a named type associated with the implementation

emccue15:05:27

in the clojure version with reify you just get the implementation

shaunlebron15:05:13

oh got it, yeah!

shaunlebron15:05:59

I was mainly worried about all the template parameters, but I guess clojure somehow takes care of the types, and a Clojure Set can be used in place of a Collection Singleton

emccue15:05:21

at the JVM level the types are not real

emccue15:05:27

generics all get erased into returning Object

emccue15:05:56

so clojure/the jvm doesnt see this

emccue15:05:57

class IndentedCodeBlockNodeRenderer implements NodeRenderer {
    public Set<Class<? extends Node>> getNodeTypes() {
        // Return the node types we want to use this renderer for.
        return Collections.<Class<? extends Node>>singleton(IndentedCodeBlock.class);
    }
}

emccue15:05:59

they see this

emccue15:05:13

class IndentedCodeBlockNodeRenderer implements NodeRenderer {
    public Set getNodeTypes() {
        return Collections.singleton(IndentedCodeBlock.class);
    }
}

emccue15:05:32

now later on the actual type of the elements will matter, but not immediately

emccue15:05:59

just when they are being used you'll get ClassCastExceptions if you did like

(reify NodeRenderer
  (getNodeTypes [_]
    #{123 "abc"}))

shaunlebron16:05:54

Thank you! I understand this a little more now. Wolf of Wall Street helps lol