Fork me on GitHub
#clojure
<
2022-07-13
>
pinkfrog06:07:19

(defn fun ^{:foo 3} [])
(defn fun {:foo 3} [])
What form is preferred? Sounds like people always use the former, e.g., (defn fun ^String []) for return type annotation. If so, then what’s the real use of the second form?

thheller06:07:27

the first one doesn't do the same thing as the second

thheller06:07:45

(defn fun ^{:foo 3} [])
=> #'shadow.user/fun
(meta #'fun)
=>
{:arglists ([]),
 :line 1,
 :column 1,
 :file "C:\\Users\\thheller\\AppData\\Local\\Temp\\form-init2415357942278256113.clj",
 :name fun,
 :ns #object[clojure.lang.Namespace 0x7b178607 "shadow.user"]}

thheller06:07:51

note the missing :foo meta

thheller06:07:05

(defn ^{:foo 3} foo []) would be the same as the second. otherwise the first just annotates the args vector, not the var itself

👆 1
thheller06:07:24

I prefer the second variant because the name comes first and it looks better visually 😉

lsenjov06:07:11

Does annotating the args vector actually do anything/have a use?

Ben Sless06:07:54

Yes, let's assume you call that function, then use java interop on the result. How is the compiler going to know it's a string? You can annotate the call site, or you can annotate the return value

lsenjov06:07:40

To clarify, I mean (defn foo ^{:tag String} []) over (defn ^String foo [])

Ben Sless06:07:03

Ah, sorry The former just adds this metadata to the vector object at read time The latter to the symbol and won't do anything to help with type inference iirc

Joshua Suskalo14:07:22

defn is a special case, both annotate the return type, but on the arg vector is "preferred"

Joshua Suskalo14:07:35

Both are used in clojure core libraries

thheller06:07:27

the tag thing for the return value yes. (defn fun ^String []) is short for (defn fun ^{:tag String} []) IIRC

thheller06:07:07

otherwise I don't believe so no

lsenjov06:07:26

But doesn't that apply to the function metadata, not the args vector?

thheller06:07:23

well technically this all happens at compile time, so it is taken from the fun name symbol or the [] args vector metadata. I believe both places are checked

lsenjov06:07:37

Huh, neat. Thanks!

Martynas M06:07:53

Hey. What is the preferred way to use ReentrantLock on Clojure? I want to use ReentrantLock to be able to inspect later whether the current thread has the lock so I don't think synchronized is what I want: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantLock.html#isHeldByCurrentThread() I found that I should use locking function to lock my code:

(let [my-lock (java.util.concurrent.locks.ReentrantLock.)]
  (locking my-lock
    (println "hi")))
But when I expand the macro I get this (only the locking macro expansion):
(let [lockee__5782__auto__ my-lock]
  (try
    (let [locklocal__5783__auto__ lockee__5782__auto__]
      (monitor-enter locklocal__5783__auto__)
      (try
        (println "hi")
        (finally (monitor-exit locklocal__5783__auto__))))))
So basically this code will run synchronized (my-lock) { } but not my-lock.lock() . Is there a Clojure way of locking via the lock? :thinking_face: Should I avoid ReentrantLock?

p-himik06:07:03

You can use Java interop just fine, (.lock my-lock).

1
Martynas M07:07:44

Yes, the interop would work. But if things are not wrapped into Clojure's core lib then I start to think why. Maybe there is a real difference of some sort. Why don't we have a macro for that? :thinking_face: Maybe synchronized is somehow better? P.S. I found that I can use Thread.holdsLock(obj) (I didn't check that but it looks like it should work): https://stackoverflow.com/questions/61557491/how-to-check-if-thread-holds-the-monitor-in-intellij#61652514 So both could probably work for my needs of "only one thing from these can happen at the same time" as I always carry around that same object. But then I remember people saying "just use ReentrantLock and don't bother with synchronized"... I was always wondering why.

Hermann07:07:56

Because all of Clojure multithreading is built in a way to avoid the need to manually handle locking

p-himik07:07:04

Clojure doesn't wrap a lot of Java constructs. Doesn't mean that those constructs aren't useful. No clue about the "`synchronized` vs ReentrantLock" though - I haven't used locking primitives in years, I'd have to read it all over gain to be able to tell.

Martynas M07:07:30

@U03KLA1A00K I am very happy about Clojure's wrapped primitives but this time I have to lock things myself as I have two in-mem DBs that I have to partially keep in sync in an unusual way. There is no way pmap or core.async can help me this time 😄 I also think STM can't help me here too as I don't control refs myself. @U2FRKM4TW Yes, but then also they wrap some important high-use functions such as subs (`.substring`). So sometimes it makes sense from importance point of view but sometimes it's wrapped simply for supporting special syntax. :thinking_face:

p-himik07:07:36

IIRC, STM has its scope, and not all concurrent things fall into that scope, so you can't just replace all locks with STM. There have been some discussions here about it. Also, STM is not used that much: https://clojurians.slack.com/archives/C03S1KBA2/p1531169303000322

p-himik07:07:37

@U028ART884X Anecdotally, I use subs about 2-3 times a week, while I use locks about once every 5 years. Which one, would you say, deserves its own function more? :)

😆 1
Hermann07:07:51

@U028ART884X Ah I thought you were a beginner just coming fom Java's locking world. (And I have to admit I just skimmed the article and thought it would cover all primitives). It looks like you'll have to create your own macro for that case, though? Is there no way to take a step back and reorder your calls to simplify stuff? Maybe use futures to get the locking effect? Do you have to deal with shared resources? Transactions at DB level?

Martynas M07:07:15

@U2FRKM4TW I didn't know about subs function for a long time and I always used .substring. The way I learned about that function was when I was sent feedback from a home exercise where one of the negative points was that I used .substring everywhere 😄 But I would've put that as a suggestion, not a defect of the code. @U03KLA1A00K Beginner or not the same questions matter. Sometimes a detail can mean something very important. For me locking seems alright but for self-documentation I may want a macro. I mostly try to avoid macros until I really really need them. I also think that by specifying a lock object I increase the understanding that something important is happening. Yes, it's transactions at DB level. They aren't SQL transactions, they're way simpler. I have Primary and Secondary DB. Primary pipes some data via a callback to the secondary DB. Secondary DB can also accept insertions by itself and these insertions have to lock the Primary from writing into Secondary. The difference between the instances is that one is deleted on reboot and the other one is not. Weird-custom-cache-thing.

Hermann08:07:46

That's really some fascinating design 😀 We had a similar issue with a customer-facing API that caused database writes and callback hooks. Some of them would cause a lot of work, but the customer at some point automated a workflow so at regular intervals they bombed us with concurrent request. We had one advantage: the API route always contained an entity ID. We used this to create a map containing per-entity action queues of thunks instead of directly accessing the DB. This performs essentially the same as locking, but was much more predictable in the execution. I don't know if this can be applied to your situation, but sometimes it helps to view a problem from different angles.

Martynas M08:07:23

I'm splitting the DB into two and then I'm doing locking to sync them together. The split is needed to reclaim the performance that I lost by inserting the ephemeral things into the DB as I had one DB before my current refactoring. This refactoring will save-up storage (and network, and insertion-lag) and delete transactions that I don't want to keep (the reboot loses the state of the Secondary node). So I'm not yet sure if I can split the inputs into multiple streams as my DBs have only single-loop when processing txs. Also I don't have prod yet so I don't have so much lag so that this would be useful. :thinking_face: Also probably I wouldn't be able to do it anyway as my data is intertwined very much :thinking_face:

Martynas M09:07:22

Actually I could use a core.async channel. But I want to keep things consistent. At least for now.

Hermann09:07:13

Viewing the updates as input streams would really suggest an async channel, right. Which part of the system are aynchronous anyway? Just the DB updates, or inputs also?

Martynas M10:07:26

Currently I want the nodes to be synchronous as it will make my life easier. The nodes can be made to be eventually consistent with each-other by not having a lock but instead pushing the updates into the second one via a channel. But then the direct inputs into the secondary node will not be possible (easily) and I want to have them. And there are more things that are going on.

Hermann10:07:26

So why not multiplex that channel and input from both?

Martynas M10:07:40

Because both nodes store completely different things and they don't share any data in between. So there is no way to have only one source of events.

Hermann10:07:49

Ah. Well. That's a drawback

Martynas M10:07:52

But that's also the purpose of the split 😄

Hermann10:07:43

I'd still go with the stream-from-multiple-sourcese approach, but instead of passing pure data into it, I'd pass thunks, so the channel acts only for synchronisation

Martynas M10:07:36

What are thunks? Grouped transaction batches?

Hermann10:07:59

Basically it's just a delayed function. You pass the function which has to be evalueted next, in your case the function causing the next DB update. https://en.wikipedia.org/wiki/Thunk

littleli14:07:52

avoid synchronized entirely. It's never better. 1) biased locking is no more 2) synchronized is always worse than ReentrantLock. From JDK19 there will be Virtual Threads and to use ReentrantLock(s) is one of the performance consideration that goes with this JDK addition.

Martynas M14:07:25

Actually in my occasion biased locking would work very well as I know that the majority of the data flow would go between the nodes. Very large majority. And it would also go one way only so only one piece of the code would be running almost all the time.

Joshua Suskalo14:07:34

So I may counter what littleli says here. Yes using core locking constructs is likely to perform better under virtual threads, but that requires that you're going to use jdk 19 or 21 as lts. 21 would be next year and there's time for consideration in the mean time. If you're still on jdk 8 or 11 then you can consider if you're likely to upgrade in the future. Using core's locking construct is perfectly viable, it is reentrant so it likely will act the same way as your reentrant lock would, and it only uses core constructs which may be beneficial for your codebase. On the other hand making a with-lock macro that wraps a code block in a lock is not a very high maintenance burden and is one of the cases where macros aren't just okay, they're encouraged (that is to say: macros are always okay if they introduce useful new flow control constructs that do not obscure the flow of code). I also offer that Clojure not wrapping something is not really grounds to avoid it, as Rich specifically talks about how clojure devs should know and use the java.util.concurrent library, and until clojure 1.11 you were expected to use java.lang.Math directly.

littleli15:07:14

it's really not a counter 🙂 as I basically agree with you. Btw biased locking was disabled with JDK15 but if it helps or not is a subject of measurement. I believe it's under -XX option:

enabled: -XX:+UseBiasedLocking -XX:BiasedLockingStartupDelay=0
disabled: -XX:-UseBiasedLocking

littleli15:07:26

I would just use a future proof way to deal with such cases and we already know what this is. That's all.

👍 1
Joshua Suskalo15:07:55

Yeah it started as wanting to be a counter but after elaboration it just ended up being "i agree with caveats"

👍 1
😂 1
M J11:07:05

Anyone know how can I copy a value to clipboard in clojure?

M J12:07:15

Hey, I'm relatively new to clojure and using VS Code (Calva). There isn't a function that does it without adding to commands or keyboard shortcuts. Im trying to add the function to an "onClick"

reefersleep12:07:31

Sounds like you’re doing it in Clojurescript and not Clojure. I think @U0CKDHF4L’s example is specifically Clojure.

reefersleep12:07:47

I think you need to port something like this javascript example to Clojure: https://www.w3schools.com/howto/howto_js_copy_clipboard.asp

octahedrion12:07:33

ask in #clojurescript

M J12:07:51

Ok thanks anyways guys

reefersleep12:07:17

@U03KB1C9N8M works for me like this:

[:button {:on-click (fn [e]
                         (.writeText js/navigator.clipboard "wow"))}
    "Click me!"]

🙌 1
sheluchin16:07:54

I'm working on a normalized SQL schema to describe clojure code. Does it make sense to keep function and var definitions in separate tables? All functions are vars, but not all vars are functions, and functions have attributes like arity counts and argument lists that other vars don't have. But I don't think clojure makes a distinction. I'd like to keep as close to the model of the language as possible, but not end up with a highly denormalized schema.

borkdude18:07:26

I would maybe make a column for function true/false or so

sheluchin18:07:17

@U04V15CAJ but then all the function-specific attributes should still probably go in another table, rather than introducing a bunch of nullable columns to support the extra function-specific data, yes?

borkdude18:07:39

Not sure. A json column could also work.

gratitude-thank-you 1
Joshua Suskalo16:07:17

anonymous functions are not vars

sheluchin18:07:06

Hmm, I didn't realize this. If a var is used to associate some symbol with a memory address, how does it work for anonymous functions? Is there anywhere I can read about it?

Joshua Suskalo18:07:55

anonymous functions are just that, anonymous. They're not named, but have a body just like any other function. You can't refer to them by name so you either have to just pass them as arguments to other things or call them immediately, or you save it into a local binding.

Joshua Suskalo18:07:40

under the hood there is a name given to these functions, but it's not human readable and it's not saved into a var, just a java class

sheluchin18:07:06

I see. So there is some internal reference for it that clojure maintains under the hood, but that's different from vars which are essentially an interface for humans to be able to name things and keep track of them. Clojure does all the same for anonymous functions (and vars), just without the human-readable name part. Does that sound correct?

Joshua Suskalo18:07:41

it uses the constant pool, yeah, more or less. I'd say it's still important to keep functions separate from vars though because I don't think having phantom "vars" for every anonymous function is a good idea

sheluchin18:07:54

Okay, thank you. Some valuable insights here.

Joshua Suskalo16:07:29

So if you want to support all of clojure, having them be separate tables makes sense.

Joshua Suskalo16:07:18

vars are just name+value+metadata, functions are just a list of function bodies + metadata, function bodies are just arglists + expression lists

hiredman16:07:13

you can also have anonymous vars fyi

2
hiredman16:07:40

but you never really them used

Joshua Suskalo16:07:41

true, with-local-vars. I've used it in a few macros

Joshua Suskalo16:07:23

Most notably defreloadable, a macro I propose as a teaching tool at the end of an unpublished blog article on writing reloadable code.

Joshua Suskalo16:07:44

I really need to get my blog up so I can publish this stuff.

1
hiredman16:07:20

a var is a reference type, like an atom or a ref, but with the ability(but no requirement) to be directly interned in a namespace

hiredman16:07:08

when you call a function like (+ 1 2) you are not invoking the var #'+ , the compiler sees that you are using a name that refers to a var, and generates code to get the var, deref the var (like derefing an atom) and then whatever you got out of the var by derefing is invoked

hiredman16:07:22

but, because being a function in clojure is an interface, vars also implement IFn (as do keywords, maps, sets, and vectors) and can be called directly as a function

hiredman16:07:47

(#'+ 1 2) calls the var as a function, and the implemented behavior for vars when called as functions is pretty close to the code the compiler generates (the var forwards the function call on to whatever value it holds)

Joshua Suskalo16:07:30

considering this is for a linter (context in #clj-kondo) I don't know if that distinction will be relevant.

hiredman17:07:29

I was just reading some code that uses keywords in the :keys destructuring like:

user=> (let [{:keys [:a]} {:a 1}] a)
1
user=>
and I understand it works, but is that sort of like how you used to be able to use the symbol require instead of the keyword :require in the ns macro? Does it just happen to work because the most expedient way to implement :keys has it working, or does it intentionally do that?

☠️ 1
Alex Miller (Clojure team)17:07:13

it intentionally does that but is now vestigial

Alex Miller (Clojure team)17:07:34

originally keywords in the :keys vector was to enable autoresolved keywords to be used (both ::bar and ::foo/bar) but those are both better served now by ::keys [bar] or ::foo/keys [bar]

hiredman17:07:28

ah, of course ::

hiredman17:07:00

gives me the willies to see keywords in :keys like that

borkdude18:07:42

There's an optional linter in clj-kondo to warn about that :)

winsome18:07:34

I've sometimes wished I did it when I'm searching a project for functions that use a certain keyword.

NickName18:07:46

Is is possible to tell cljfmt to end files with newline? Don't see any such option in the docs

Joshua Suskalo18:07:15

#cljfmt might know

👍 1
Jakub Holý (HolyJak)18:07:23

What is the way to ensure that a lazy seq (from map etc) is relized 1 element at a time and not N (whatever the default factor is) ?

Joshua Suskalo18:07:18

The main reason to do this would be for IO, is that your intended usecase?

1
phronmophobic18:07:31

IMO, trying to recover/enforce strict semantics with lazy sequences is a trap.

1
Joshua Suskalo18:07:34

To some extent that may be true but the sequence operations are very convenient for a lot of sequential IO. For this reason I've made a function sequence* which is a transducing context specifically designed for lazily-realized IO.

phronmophobic18:07:53

If you care about when items are produced/consumed, then I would avoid lazy sequences.

Joshua Suskalo18:07:03

(defn sequence*
  "Non-chunking lazy sequence transducer context.

  This acts like [[sequence]], but has characteristics specifically designed for
  working with IO in a transducer.

  "
  ([xf coll]
   (let [rf (xf conj)
         f (fn f [coll]
             (lazy-seq
              (loop [coll coll]
                (when (seq coll)
                  (let [out (rf [] (first coll))]
                    (if-not (reduced? out)
                      (if (seq out)
                        (concat out (f (rest coll)))
                        (recur (rest coll)))
                      (seq @out)))))))]
     (rf (f coll))))
  ([xf coll & colls]
   (let [rf (xf conj)
         f (fn f [colls]
             (lazy-seq
              (loop [colls colls]
                (when (every? seq colls)
                  (let [out (apply rf [] (map first colls))]
                    (if-not (reduced? out)
                      (if (seq out)
                        (concat out (f (map rest colls)))
                        (recur (map rest colls)))
                      (seq @out)))))))]
     (rf (f (cons coll colls))))))

🤯 1
Joshua Suskalo18:07:14

Here's a transducing context for lazy IO, use it if you like

❤️ 1
Joshua Suskalo18:07:24

I'm not aware of a good reliable way to make a lazy sequence non-chunked (and to stay that way) otherwise.

Jakub Holý (HolyJak)18:07:04

I do

(->> (fetch lot of stuff)
     (filter yet-unseen?) ; depends on an atom
     (map run-some-expensive-check-and-update-seen) ; update the atom
     ...)
I know it is kind of ugly to use a mutable variable like that but it is just a quick and dirty script

Joshua Suskalo18:07:37

I'm not sure how exactly this would prevent unwanted io unless the idea is that you'll actually specify mutably that you want no more stuff inside that map. Seems prone to error.

Jakub Holý (HolyJak)18:07:35

Well (filter yet-unseen?) checks the incoming stuff against a set (stored in an atom) and filters out stuff already checked. Then in the next step we run the check and if the results are relevant, we add that stuff to the set so that we do not check it again so step 3 feeds back into step 2.

Alex Miller (Clojure team)18:07:55

If you have a requirement of user control of realization, don't use lazy seqs

👍 1
Joshua Suskalo18:07:09

right. It makes sense, it just feels a little dangerous is all.

Joshua Suskalo18:07:00

@U064X3EF3 I can appreciate the sentiment but a lot of library code has been written to work on lazy sequences. Lazy IO is a footgun, but I don't think it's a stretch to say that sometimes the foot's worth less than the deadline and that's a lot of code that doesn't have to get re-written for some bespoke thunk mechanism.

Alex Miller (Clojure team)20:07:12

the dangerous part is why it makes sense. lazy seqs make no guarantees about when or how much is realized.

Joshua Suskalo20:07:56

that's fair. Working with the lazy-seq macro like this is technically relying on an implementation detail.

phronmophobic21:07:30

You can use all the transducer stuff like map, filter, etc without lazy sequences.

Joshua Suskalo21:07:08

that's true, and doing side effects in an eager transducing context is generally going to be more correct

Ed22:07:19

Wouldn't switching to transducers fix the original problem, regardless of laziness because it would remove any buffering between the filter and the map?

Joshua Suskalo23:07:04

No in theory because it's undefined if calling first and rest on a lazy seq realizes one elt or one hundred (and it being undefined is why chunked sequences comply) and so lazy-seq may create a sequence witch realizes an amount different from expected. No in practice because clojure.core/sequence realizes the first argument prematurely from a pragmatic standpoint. @U0P0TMEFJ

Joshua Suskalo23:07:51

so in practice you need a sequence impl that doesn't do that and which relies on a concrete type that specifically realizes one element per call to first/next, which in practice lazy-seq with a single cons call in it whose rest is a recursive call (like in my sequence* impl above) does this, but that relies on an implementation detail of lazy-seq so could more correctly be implemented with a reify.

Ed15:07:23

I may have misunderstood, but I thought that the original problem was because there was a chunked sequence being returned

(->> (fetch lot of stuff)
     (filter yet-unseen?) ; depends on an atom - returns chunked seq
     (map run-some-expensive-check-and-update-seen) ; update the atom
     ...)
so I thought that the problem was that run-some-expensive-check-and-update-seen updated the atom, which changes what the filter yet-unseen? call would have returned if it hadn't produced a list of 32 things in a chucked seq. However, using a transducer instead of ->> to create the pipeline, like
(sequence (comp (filter yet-unseen?) (map run-some-expensive-check-and-update-seen) ...) (fetch lot of stuff))
would mean that for every truthy return from yet-unseen? a corresponding run-some-expensive-check-and-update-seen would run before the next invocation of yet-unseen? meaning yet-unseen? is always working with the "latest" view of the value from the atom because filter isn't returning a chucked sequence before calling map and the transducing context is providing lazyness from the outside ... is that right?

Ed15:07:07

either way, it might be better to join the filter and map operations using keep so the read and writes are joined together??

💯 1
Jakub Holý (HolyJak)15:07:09

You are right Ed, transducers solved my problem here

Jakub Holý (HolyJak)15:07:50

I am sure I will have this question again so I recorded highlights form this conversation at https://holyjak.tumblr.com/post/689730176615120896/can-i-un-chunk-lazy-sequences-to-realize-one 🙂

Ed16:07:26

cool ... glad you got it sorted 😉

shaunlebron23:07:47

is there a prescribed method yet for displaying all the function args in a stacktrace? I’m debugging something pretty nasty and this would be useful

shaunlebron14:07:58

Thank you 🙏

jpmonettas18:07:59

it was created specifically for that kind of debugging

shaunlebron18:07:00

@U0739PUFQ wow great work, thank you brother

jpmonettas18:07:15

thanks! I hope it helps, if you have any questions let me know

shaunlebron19:07:56

exactly what i wanted

jpmonettas19:07:15

any feedback and ideas are welcome

shaunlebron19:07:08

@U0739PUFQ the right half of this screen makes me wonder if there's opportunity for interplay with Portal: https://github.com/djblue/portal. either in borrowing ideas on how data is displayed, or exporting data to be displayed there somehow.

jpmonettas19:07:08

yes, that is my TODOs already

jpmonettas19:07:43

I thought about portal integration (which you already kind of have via the def button)

jpmonettas19:07:15

but I think I'll write that functionality (copy portal) so you don't need to have too many windows open

jpmonettas19:07:03

the latest version allows you to define any value you are seeing for the repl, then you can tap> it to portal

👌 1
shaunlebron19:07:08

my favorite thing about the demo is seeing all the ways you added to navigate an overwhelming amount of breadcrumbs

shaunlebron19:07:55

kept pulling more and more things out of your sleeve lol

jpmonettas19:07:25

my idea with FlowStorm is for it to be a debugger but also a "reverse engeneering" tool

jpmonettas19:07:34

reverse engineering in the sense of creating a mental model of a sometimes unkonwn codebase, and being able to do that from execution traces instead of reading code, which is not that easy with dynamically typed langs

💯 1
shaunlebron19:07:42

I’ve been endeavoring to study the clojure compiler, and there's not too much written about it, so reverse-engineering is kind of the task is

jpmonettas19:07:16

yeah I'm also on the same path

jpmonettas19:07:35

I would love to use FlowStorm for something like that, but it is written in Java

jpmonettas19:07:29

but if you are into understanding the ClojureScript or the ClojureDart compiler something like FlowStorm should help a lot

shaunlebron19:07:49

oh, I missed that point that it couldn’t reach the java parts, so I see why you chose the cljs compiler in the demo

jpmonettas19:07:21

yeah, because FlowStorm can only instrument Clojure code

jpmonettas20:07:19

I don't think something like FlowStorm can be created for most languages, Clojure has a pretty special combination of immutability (you need for tracing), being expression based, and easy to instrument by code rewriting

👏 1
jpmonettas20:07:33

maybe you are interested

shaunlebron20:07:33

RSVP’d, thanks 👍