Fork me on GitHub
#beginners
<
2018-08-15
>
Ferdi12:08:26

hello clj community! is there a function like clojure.string/replace-first for the last match? replace-last?

Alex Miller (Clojure team)12:08:45

no, but you can use replace-first with proper regex to do it

Alex Miller (Clojure team)12:08:48

user=> (clojure.string/replace-first "HI FOO FOO" #"(?s)FOO(?!.*?FOO)" "BAR")
"HI FOO BAR"

Alex Miller (Clojure team)12:08:41

5 minutes ago I could not have told you how to do that, but I googled “java string replace last” and took the top stackoverflow hit - https://stackoverflow.com/questions/2282728/java-replacelast

Alex Miller (Clojure team)12:08:00

that page explains the regex in more detail

Alex Miller (Clojure team)12:08:44

Java has a large SDK and an enormous user base - lean on it!

Ferdi12:08:22

ahh cool thanks!

avfonarev12:08:23

@ferdi.kuehne Actually, there is a simpler and more correct way:

user=> (clojure.string/replace-first "aabaabaaab" #"(?s)(.*)(aa)" "$1AA")
"aabaabaAAb"

ben16:08:06

Is there any easy way to get the time, with timezone from a string with clj-time? for instance "2013-02-12T04:30:00.000-02:00"

ben16:08:56

clj-time.coerce/to-date-time seems to get it and convert to UTC, which you could add back with clj-time.core/to-time-zone but this seems a bit fiddly to me

ben16:08:10

Especially as you’d still need to specify the timezone from the string somehow

mikeyford16:08:57

Is this what you're looking for @torvaney?

(require '[clj-time.core :as t])
(require '[clj-time.format :as f])
(f/unparse-local (f/formatter-local "HH:mm:ss") (t/time-now))
=> "17:12:29"

ben16:08:38

Yes that looks like it

ben16:08:55

Thank you @michael.ford

👍 4
mikeyford16:08:38

Hey all, often when I'm working with large nested data strucutres in clj I spend a lot of time trying to find the nested part of the stucture I want and generally just trying to get a mental image of how it's organised. I would love to able to see a tree like representation of the data structure.... Does anyone have any tips or tricks to help with this aspect of clj?

mfikes16:08:41

@michael.ford The main thing I do is pretty-print the data, if it is not too large. The line breaks and indentation help a lot.

lilactown16:08:33

:thinking_face: honestly the experience in CLJS using cljs-devtools or frisk is quite a bit better than what’s available for Clojure

lilactown16:08:57

oo that is nice

mikeyford16:08:11

these suggestions look like just the ticket. thanks!

placeboza18:08:09

datawalk looks interesting

vuuvi19:08:23

where’s the best place for clojure code reviews?

vijaykiran19:08:03

reddit or clojureverse - or even here 🙂

noisesmith19:08:28

#code-reviews has a few people in it

pepas19:08:22

Just found out about Graal and Truffle. Turns out you can compile Clojure using Graal. https://www.innoq.com/en/blog/native-clojure-and-graalvm/

schmee20:08:44

you sure can, head over to #graalvm if you’re interested in this stuff!

Nick Cabral19:08:59

@noisesmith a follow-up to my question yesterday about promises/futures vs. core.async channels: I found an older doc that echoed my exact concern (even mentioning C#'s async/await implementation, which motivated my question, as an example for solving the problem). I'm still reading through it to figure out how it was solved, but it seems the core.async channel implementation is the solution to my concern with promises/delays. I believe the conclusion you and I reached previously was incorrect. Here's the doc I'm referring to: https://dev.clojure.org/display/design/Async+blocks

noisesmith19:08:32

which part was incorrect?

Nick Cabral19:08:23

That derefing a promise/future doesn't busy-wait.

noisesmith19:08:52

it doesn't busy-wait. busy-wait here means consuming CPU while waiting

noisesmith19:08:04

it just blocks until something wakes it up by setting the value

noisesmith19:08:26

you might not want to block a thread for resource or performance reasons, but it's not a CPU issue

Nick Cabral19:08:30

Yeah, it's the "you might not want to block a thread for resource or performance reasons" that I'm concerned with. The async.core channels don't exhibit that problem when awaiting values. I think my original question was confusing because I mentioned "OS threads", wording that I copied directly from the async.core documentation.

Nick Cabral19:08:05

I'm still not convinced that "dereferencing a promise consumes no CPU while waiting". What I've read so far indicates that clojure promises aren't implemented with any mechanism that allows callback-like execution upon the availability of a value. Which implies CPU consumption at some level during blocking. Have you seen something in the clojure codebase or docs that suggests otherwise?

noisesmith20:08:59

deref doesn't loop, it sets a monitor so the thread resumes when the data is delivered

hiredman20:08:11

in general, if your notions of threads and asynchronous whatever come from a runtime without real concurrency (javascript, any of the interpreted languages with a GIL), your intuition about how things work and what happens when is going to match the reality of a concurrent runtime with real threads

hiredman20:08:37

and that context will also color how you understand documentation that you read

hiredman20:08:32

e.g. my reading of that async+blocks page doesn't convey any concern about busy waiting

hiredman20:08:51

a jvm thread is pretty cheap, and depending on your resources you can have a whole lot of them (tens of thousands or more) sitting around doing nothing without you noticing

hiredman20:08:16

and they can do nothing without busy waiting

Nick Cabral20:08:32

"Busy wait" came into the conversation because in my original questioned I mentioned "OS threads" like the async.core page does. That's my fault. Although as far as I know, blocking a JVM thread would indeed block an OS thread. But that's besides the point. The info I was looking for is that "promises/delays block their thread (resources are tied up) when dereferenced, whereas channels don't (similar to async/await mechanisms in .net, JS, and scala)".

hiredman20:08:08

blocking an os thread is not busy waiting

hiredman20:08:00

blocking an os thread generally means the thread is removed from the schedulers ready list until some other thread puts it back

Nick Cabral20:08:50

Right, and all of that involves pushing data around. That's okay too. The point was that it doesn't make sense to spin up multiple threads (which aren't what I'd call "cheap", although that's relative I guess) and have them all block until ready, when the language has an idiomatic way to handle async programming (via the core.async lib).

hiredman20:08:10

they are very cheap

hiredman20:08:20

core.async does spin up multiple threads, by default the threadpool that services go macro callbacks is 8 threads, and provides the thread macro for tasks dispatched to an unbounded caching threadpool

Nick Cabral20:08:08

It does that in order to implement the async behavior (I'm kind of guessing on this part, haven't gotten that far into that code yet)

Nick Cabral20:08:43

Compared to other language implementations where perhaps a state machine is used.

hiredman20:08:34

so the difference between using the go macro, which on parking on a channel suspends the rest of the computation and attaches it to the channel so when another operation on the channel happens it can be re-woken up, and a thread being taken out of the os ready list and waiting somewhere for another thread to re-schedule it is pretty hand wavy

john21:08:51

Looks like the JVM maxes out at some 30 thousand threads

noisesmith21:08:59

For straightforward parallelism core.async won't increase performance, in fact it usually makes things slower. What core.async is good for is making code that does a lot of asynchronous coordination easier to read, and lowering the overhead of code that does massive amounts of context switching

💯 4
Nick Cabral21:08:21

Agreed fully. I think we're on the same page now.

hiredman21:08:25

@john where did you get that number?

hiredman21:08:22

I doubt that is a jvm limit, more often you hit the os default limits, which are tunable

Nick Cabral21:08:58

@john thanks for the link, I had not heard of FIbers. Worth watching.

john21:08:40

The OS is the problem with OS threads... They can't do millions of threads.

hiredman21:08:40

core.async code is almost certainly going to do more context switching then regular threaded code

john21:08:01

Well, on commodity hardware I'm guessing

hiredman21:08:20

but they typically can do enough for whatever you want to do

john21:08:43

I far prefer them over working with core.async

hiredman21:08:48

and you can use core.async with threads

hiredman21:08:03

core.async the library has two parts: channels and the go macro

hiredman21:08:26

the go macro transforms what would be code full of callbacks on channels in to something more easily read

hiredman21:08:04

and channels are a kind of queue

hiredman21:08:27

you can use channels from regular threads

hiredman21:08:48

(core.async uses promises to un-callbackify the api)

👍 4
john21:08:48

Aye, for millions of concurrent clients, a few thousand threads each handling a few thousand clients via core.async is probably the way to go, right?

john21:08:31

Though I guess it's questionable how many threads beyond the number of cores will benefit, when using core.async

Nick Cabral21:08:29

Thanks all. Lot of good info here, some of which is reinforcing my initial suspicions. I've written quite a bit of clojure for a year or so, but haven't delved much into the async-specific corners of the language. Hence my "compare this tool to that tool" question.

Nick Cabral21:08:23

Also glad to hear that just using plain old promise/deref code is common

hiredman21:08:45

it isn't very common

hiredman21:08:59

because threads are cheap, you don't write async code

hiredman21:08:09

so you don't need promises what have you

hiredman21:08:25

you call a function and return the value

Nick Cabral21:08:43

So you're saying you'd actually use Thread interop calls rather than running something with Future?

hiredman21:08:45

so for example, clj-http doesn't return a promise that has the result of an http call delivered to it, it just does the http call and returns the result

hiredman21:08:00

future has nothing to do with promise

Nick Cabral21:08:07

Maybe I'm conflating "future" and "promise" when I shouldn't be.

Nick Cabral21:08:13

They certainly seem to be used similarly

noisesmith21:08:19

not in clojure

noisesmith21:08:35

a promise is a container that blocks reading until it has a value delivered

noisesmith21:08:54

it isn't a thread, doesn't create a thread, doesn't manage threads in any way

noisesmith21:08:05

(aside from blocking)

hiredman21:08:18

future can be thought of as giving a thread a task to do

john21:08:25

If I read the write-up correctly, the new java/fibers interface is going to be similar to the java/thread interface. It'll be interesting to see if a Clojure can just choose either as a backing option

Nick Cabral21:08:02

I pictured promise and delay being used like this to do something asynchronously:

clj
(defn my-async-func []
  (let [p (promise)]
    (future (do-something-expensive)
            (deliver p "finished!"))
    p))
and the caller could deref the result at will (ie. caller wouldn't have to block until they needed to). But it sounds like that's not a common approach in your experience with clojure?

hiredman21:08:36

in that case it would be more common to just use the return value of future over using a promise

hiredman21:08:02

and even more common to just call (do-something-expensive) without a future or a promise

Nick Cabral21:08:21

Your first point totally makes sense; I should have seen that. Second point is surprising, so good to know.

hiredman21:08:50

if you don't need to run lots of do-something-expensive in parallel or something, there is no reason to spin off another thread to do it

hiredman21:08:28

(unless you are using clojurescript)

hiredman21:08:14

all my comments should have a parenthetical "if you are using a language on the jvm which has real threads, does not apply to a similar language on a different runtime without real threads"

👍 4
Nick Cabral21:08:04

"there is no reason to spin off another thread to do it" is exactly what led me to core.async. Desire to (1) minimize new threads if possible, and (2) don't want to block the calling thread (this is sort of just my default approach for long-running operations).

Nick Cabral21:08:12

But if #2 doesn't matter, then yeah I totally get it.

Nick Cabral21:08:11

Async/await mechanisms are so common in other code I've written, I wanted to make sure I understood when these different approaches were best used in clojure.