Fork me on GitHub
#clojure
<
2022-06-22
>
jumar12:06:56

Is there a difference between using this explicitly in defrecord's method body vs using the name of the field directly? That is

(defrecord simple [s]
  protocol/EventSender
  (send-user-event [this u e] (prn "s: " (:s this))))
vs.
(defrecord simple [s]
  protocol/EventSender
  (send-user-event [this u e] (prn "s: " s)))
I first thought that maybe the second version captures the value of s at the time when the record was created but that's not the case "
(defrecord simple [s]
  protocol/EventSender
  (send-user-event [this u e] (println "s: " s)))

(def ss (->simple "ahoj"))

;; this prints: s:  hello
(protocol/send-user-event (assoc ss :s "hello") 1 {:a :b})

jumar12:06:39

From the docstring: https://clojuredocs.org/clojure.core/defrecord > Note that method bodies are > not closures, the local environment includes only the named fields, > and those fields can be accessed directly.

Alex Miller (Clojure team)12:06:37

It's better and faster to access the field directly

jumar12:06:08

What are the cases when is using this actually useful or necessary? I found my defrecords' methods bodies don't need this at all.

Alex Miller (Clojure team)12:06:42

Passing this on to another function

👍 1
jumar12:06:02

I actually found one case in my code: (proto/get-subscription this subscription-id) below

(defrecord FakeSubscriptionService [plans addons subscriptions]
  proto/SubscriptionService
  ...
  (get-subscription [_this subscription-id]
    (find-by-id subscriptions subscription-id))
  (update-subscription-custom-fields [this subscription-id custom-fields]
    (when-let [subscription (proto/get-subscription this subscription-id)]
      (merge subscription custom-fields))))

Kelvin15:06:33

The replies give more details as to why using the field name directly is better

Richie15:06:06

Why can't I unform a map? I see that the map keys aren't really conformed but the values are conformed. Is there a workaround?

(s/def ::test-spec
  (s/map-of (s/or :keyword keyword? :string string?)
            (s/or :keyword keyword? :string string?)))

(s/unform ::test-spec (s/conform ::test-spec {:kw "val"}))
;; nth not supported on this type cljs.core/Keyword
(s/conform ::test-spec {:kw "val"})
;; {:kw [:string "val"]}
(s/unform ::test-spec (s/conform ::test-spec {"kw" "val"}))
;; {"kw" "val"}

Richie15:06:03

clj and cljs

Alex Miller (Clojure team)16:06:55

Did you read the docstring of s/map-of ?

Richie17:06:30

Oh no. Thank you!

zimablue19:06:55

Hi, this is not a request for help but a technical understanding-related question so please ignore unless interested. I /think/ that there's a close relationship between code transformations that enable >! <! to work or async/await in other languages and continuation passing, and pythonesque generators with 'yield'. I see that people have written libraries like this one https://github.com/mszajna/await-cps to add 'yield' like semantics, could that be done with the 'go' macro instead? Or if not directly then by hacking it somehow? it feels like the work that 'go' does in code transformation must enable synchronous generators by effectively creating continuations (the state machine encapsulates the continuation?) but there's an extra layer that forces us into 'async-land'. I'm not confident of this.

Alex Miller (Clojure team)19:06:31

They definitely share the aspect of analyzing and transforming Clojure code to support “pausing”

Alex Miller (Clojure team)19:06:31

The go code has a different goal so not sure you can make it so that (without hacking it a lot)

Alex Miller (Clojure team)19:06:31

Both of these mimic the analysis done in the Clojure compiler and we've considered for a long time making that more amenable to external use for things like that

zimablue19:06:59

thanks so much for your reply, as someone non-comp-sci pondering and trying to learn it's incredible to have people like yourself around

hiredman19:06:27

https://gist.github.com/hiredman/5644dd40f2621b0a783a3231ea29ff1a does some code transforms like the go macro to support delimited continuations. It has more comments and links to docs on some concepts involved. It explicitly does the transformation as a kind of cps, where the go macro is more ssa/basic blocks

zimablue19:06:45

that's interesting

zimablue19:06:52

is it very complete/tested?

zimablue19:06:03

could one implement the go macro on top of it?

zimablue19:06:15

is there anything related I can read? if nothing else, how did you learn to write this sort of code?

seancorfield19:06:29

@U63D7UXJB You might also want to look at #missionary which I think can provide generator-like semantics as well as controlled async operation.

hiredman19:06:31

The bottom of the file has some playing around defining a go macro work alike in terms of the delimited continuation operators shift and reset

seancorfield19:06:18

(it does some fascinating code transforms)

hiredman19:06:26

The comments in the file have links to papers and presentations with related work

hiredman19:06:53

And describe to some degree what the code is doing when it analyzes and transforms

zimablue19:06:52

btw if people find this interesting they might find this interesting which i bumped across in trying to google this question

ghadi19:06:06

there will be no need for code rewriting / CPS transform after Project Loom lands in the JVM

💯 1
zimablue19:06:09

someone trying to make a theory and people having arguments in comments, where clojure is referenced

ghadi19:06:12

also, async/await is considered problematic technology, despite new langs still unfortunately adopting it

ghadi19:06:18

cooperative coroutines are harder to compose

zimablue19:06:55

a reason for async/await to be problematic was described in this blog article that I read once: https://lucumr.pocoo.org/2020/1/1/async-pressure/ which I think in clojure language translates somewhat saying "we need both >! and <!" but maybe also that channels should have a pollable atom to describe business. Again not confident of this

ghadi19:06:42

see the article "what color is your function?"

zimablue19:06:43

@U050ECB92 I use clojurescript so loom won't help me I guess

zimablue19:06:28

I've read that article, but tbf it could be applied to clojure's go channels equally? since they have the same contagion property, especially in clojurescript where you can't <!!

ghadi19:06:09

yeah it could, but after Loom <! and <!! could be equivalent

hiredman19:06:40

A lot of spilt ink for unbounded queues being bad

seancorfield20:06:18

Re: Loom - https://clojurians.slack.com/archives/C8NUSGWG6/p1652487349607519?thread_ts=1652433406.842079&amp;cid=C8NUSGWG6 (variants of go and go-loop that use virtual threads so you can use <!! inside them "without blocking").

seancorfield20:06:04

(and needs no code transformation)

seancorfield20:06:22

Missionary does work with ClojureScript FWIW.

zimablue20:06:43

@U0NCTKEV8 tbf pedantry can help people like me new to the topic

zimablue20:06:00

sometimes blogs can be so concise that I don't grok them

hiredman20:06:43

the thing is, queues are everywhere, and controlling them is something every multithread system deals with either implicitly or explciitly

hiredman20:06:41

the reason that blog post is so "oh my god! here is this thing you might have a problem with!" is python code is famously single threaded

hiredman20:06:32

where the jvm hasn't had that limitation

hiredman20:06:07

so, for example, on the jvm you can just spin up threads to do things, where doesn't use await/async or any kind of async library

hiredman20:06:26

but doing that in an uncontrolled manner is bad

hiredman20:06:26

because those threads take up resources and sit in the os scheduler's run queue

hiredman20:06:16

and if you run away spinning them up you will bog your process down and eventual hit soft or hard limits

hiredman20:06:03

so you can create an executor with a bound threadpool

hiredman20:06:25

well that executor has a run queue which placing things in also takes up resources, etc etc

zimablue20:06:11

also js, single-threaded

zimablue20:06:23

the python guys are trying to un-single-thread it, if you google "gilectomy"

hiredman20:06:12

sure, but looking to them for advice about it is in some cases very one eyed man leading the blind

hiredman21:06:59

if you are just interesting in the mechanics of continuations you can also use "the continuation monad" to make it possible to play with that without doing a full compiler in a macro like core async does

zimablue22:06:25

@U02F0C62TC1 thanks for this, working down my reading list of reading lists

zimablue22:06:42

Also @U04V70XH6 thanks for the link to missionary, that and cloroutine underneath it seem close to what I was asking originally

1
Volodymyr Vizovskyy21:06:12

Most idiomatic way of putting up an asynchronous blocking queue? Need to delete a bunch of files N seconds after the processing occurs

Joshua Suskalo21:06:42

Use java.util.concurrent.BlockingQueue

ghadi21:06:42

assuming you don't care about cross process durability, use a java DelayQueue, launch a thread to poll it

Joshua Suskalo21:06:00

Oh yeah, that'd be better for encoding the delay

Volodymyr Vizovskyy21:06:02

Thanks for the suggestions! Will check both solutions:+1: are there any clojure "connectors", or do I just have to go full interop?

Joshua Suskalo21:06:34

Generally java.util.concurrent stuff is reasonably well designed. I've not looked for clojure wrappers because they feel pretty comfortable to use, given you want mutability.

Joshua Suskalo21:06:25

(the one notable exception to this imo is ConcurrentLinkedQueue not supporting casing the tail, which was a big enough gripe I wrote my own queue in clojure)

Cora (she/her)23:06:24

@U5NCUG8NR do you have a link to your queue impl?

Cora (she/her)23:06:32

I'm interested 😄

Joshua Suskalo23:06:08

No problem, lmk if you have any questions.

Joshua Suskalo23:06:38

It's based on the same whitepaper as the java util concurrent concurrent linked queue, but you can cas the tail

Cora (she/her)23:06:03

and I will, thanks

Cora (she/her)23:06:25

do you know the name of the paper, off-hand?

Joshua Suskalo23:06:46

no, but it is in the javadoc for that class

Cora (she/her)23:06:54

ahhh ok thanks!

Cora (she/her)23:06:01

this is really great, btw. I have need of a queue with similar properties

Cora (she/her)23:06:07

it's nice to see how

Joshua Suskalo23:06:14

Yeah, i was really surprised that not only was it not in the stdlib, I couldn't find an implementation for it in java or clojure as a library either

vlad_poh22:06:48

Struggling to convert/translate the following code from C# to clojure

Cora (she/her)22:06:35

what do you have so far?

Cora (she/her)23:06:06

you might have luck using buddy for some of this https://github.com/funcool/buddy

Cora (she/her)23:06:55

and what issue are you seeing with your implementation?

Cora (she/her)23:06:39

also I think SHA256withRSA already does hashing of the input, so you're double-hashing it?

Cora (she/her)23:06:50

I might be wrong

vlad_poh23:06:33

The service errors out with the following on their end

System.Security.Cryptography.CryptographicException: ASN1 corrupted data

Cora (she/her)23:06:25

nothing jumps out to me right now except the it's hashed twice

Cora (she/her)23:06:22

lines 47-50 you SHA256 the message, then within the function called on 53 you're using SHA256WithRSA which I believe will SHA256 the message again

Cora (she/her)23:06:34

it's worth a try just skipping the first SHA256 to see if it works

vlad_poh12:06:50

@corasaurus-hex that got me one step further Learned c# has unsigned bytes and java has signed bytes. I’ve narrowed it down to just the following lines Getting different results. comment lines in c#

;; var rsa = new RSACryptoServiceProvider();
;;                 rsa.ImportRSAPrivateKey(source: Convert.FromBase64String(privateKey),
;;                     bytesRead: out int _ );
(defn get-private-key [s]
  (.generatePrivate
   (KeyFactory/getInstance "RSA")
   (PKCS8EncodedKeySpec. (decode64 s))))
(def rsa (get-private-key privateKey))


;; var rsaFormatter = new RSAPKCS1SignatureFormatter (rsa);
;; rsaFormatter.SetHashAlgorithm (algorithm);
;; var result = rsaFormatter.CreateSignature (signingHash);
(def rsaFormatter (java.security.Signature/getInstance "SHA256withRSA"))
(.initSign rsaFormatter rsa)
(.update   rsaFormatter signingHash)
(def result (.sign rsaFormatter))

tstout13:06:05

I have been doing something similar with this code: https://github.com/tstout/conceal

vlad_poh12:06:50
replied to a thread:

@corasaurus-hex that got me one step further Learned c# has unsigned bytes and java has signed bytes. I’ve narrowed it down to just the following lines Getting different results. comment lines in c#

;; var rsa = new RSACryptoServiceProvider();
;;                 rsa.ImportRSAPrivateKey(source: Convert.FromBase64String(privateKey),
;;                     bytesRead: out int _ );
(defn get-private-key [s]
  (.generatePrivate
   (KeyFactory/getInstance "RSA")
   (PKCS8EncodedKeySpec. (decode64 s))))
(def rsa (get-private-key privateKey))


;; var rsaFormatter = new RSAPKCS1SignatureFormatter (rsa);
;; rsaFormatter.SetHashAlgorithm (algorithm);
;; var result = rsaFormatter.CreateSignature (signingHash);
(def rsaFormatter (java.security.Signature/getInstance "SHA256withRSA"))
(.initSign rsaFormatter rsa)
(.update   rsaFormatter signingHash)
(def result (.sign rsaFormatter))