Fork me on GitHub
#clojure
<
2020-05-02
>
hiredman03:05:08

Redis has a thing that cosplays as transactions on the weekend

potetm03:05:00

Does it seriously not work?

potetm03:05:58

It’s hard for me to imagine that’s the case in solo-server mode. The whole thing is single-threaded. They’d have to actively work to not give the guarantees they say.

potetm03:05:53

Or are you referring more to durability guarantees (which, I agree, are pretty much zilch)?

hiredman03:05:47

No rollbacks either

didibus04:05:28

So..., is it me or the use of xf and rf is kind of confusing and interchangeable? I used to think of xf as transducer and rf as reducing function (of the shape meant for transducible process to use).

didibus04:05:54

But if you look at some code say:

(defn dedupe []
  (fn [xf]
    (let [prev (volatile! ::none)]
      (fn
        ([] (xf))
        ([result] (xf result))
        ([result input]
          (let [prior @prev]
            (vreset! prev input)
              (if (= prior input)
                result
                (xf result input))))))))

didibus04:05:22

Here xf is used to indicate a reducing function. Seems a bit confusing that it isn't named rf.

didibus04:05:35

Oh, actually I think it's only wrong in the transducer guide on http://Clojure.org. I think many people get confused by this looking at the guide.

andy.fingerhut05:05:51

You could consider creating a PR on the http://clojure.org site to fix it, if you have signed a CA (or are willing to): https://github.com/clojure/clojure-site

didibus07:05:27

My current work contract doesn't allow me to sign the CA. Which is why I can't make the PR unfortunately. At least from my understanding

andy.fingerhut07:05:31

You could create an issue describing the change you think ought to be made, and someone else who signed the CA might make the change.

👍 4
💡 4
Olical09:05:22

Anyone know the main differences between swapping namespaces with in-ns compared to ns? in-ns seems to have a very bad time if the ns doesn't exist yet, so I'm leaning towards the latter. (context: Editor REPL tooling so all of this happens passively)

Olical10:05:20

Ah, in-ns doesn't mess with *1 *2 *3 but ns does :thinking_face:

dominicm10:05:26

I think ns might lose your requires too... I might be making that up

dominicm10:05:43

I guess looking at the expansion will be educational

Olical10:05:58

It won't clear them as far as I can tell, but I does mess with the captured historical results

Olical10:05:20

In-ns seems like the better tool, but if the user forgets to eval their ns form and the file isn't loaded it'll throw weird errors

Olical10:05:25

Because clojure.core isn't around

Olical10:05:31

When the NS is essentially nil

dominicm10:05:49

ns messes with metadata it would seem

dominicm10:05:56

Looking at the source anyway

dominicm10:05:40

The first thing ns does is call in-ns, so I'm a little confused to how it works.

dominicm10:05:54

You shouldn't be having a bad time if it doesn't exist, the docstring specifies it will create it if necessary

dominicm10:05:18

However, I'd always require a ns before calling in-ns on it

dpsutton14:05:58

it will create the ns but it won't have clojure core. you'll get errors like def is undeclared and its not a great experience

Olical14:05:03

Ah okay! That's annoying :thinking_face: wonder if I could detect lack of core and require it

dominicm14:05:37

It's possible to exclude clojure core though :)

noisesmith22:05:58

an idiom I use in my repl: (doto 'my.ns require in-ns)

noisesmith22:05:10

your editor tool could do something like (if-let [n (find-ns 'my.ns)] (in-ns n) ...)

dpsutton22:05:34

I love that one. (doto 'my.ns require in-ns)

noisesmith22:05:14

I learned it from Phil Hagelberg, and haven't stopped using it since

Olical09:05:47

That's worryingly cool, thank you! I'll have a little look at it's behaviour now.

Olical09:05:02

Ah the problem with that being you might be editing a file that has a ns name that doesn't match up with the path in any way. So you can still tell Clojure to load-file it. I think require will assume too much about what you're doing, hoping you're doing it "correctly" if that makes sense. Really cool but won't work for my use case I'm afraid.

Olical09:05:09

The find-ns makes more sense really. I could leave it on user if it's not defined :thinking_face: or use ns as a one off.

dominicm09:05:23

One problem with this is that it will throw an exception if the namespace isn't already on the classpath

dominicm09:05:50

Ugh, threads and not updating 🙅

Olical09:05:03

Yeah, I'm not going to do it. Will leave it as is for now and add an FAQ or something, I don't want to assume.

Olical09:05:32

I suppose I could try to find and evaluate the whole ns form on buf enter or something, but again, you might not want that and that might throw errors. So, yeah. It's complicated, I'll leave it simple until I can think of a good solution.

dominicm09:05:41

I think fireplace does this, and occasionally I have to run create-ns

dominicm09:05:08

I like your load idea tbh. Except it'll probably not be good in scratch buffers with lots of side effects.

dominicm09:05:00

(try (in-ns) (catch (load "buffer") (in-ns))

dpsutton14:05:20

That will also make the source macro not work. Not sure if that’s a big deal or not

dominicm14:05:18

Nrepl load might work though

nick10:05:22

In ~90% of my previous(non-Clojure) projects there was some kind of job/message queue(most recently AWS SQS & RabbitMQ). How can I approach the same task in Clojure ecosystem? It is rather hard to find open source examples of real projects. That's an example that I could find for a RabbitMQ consumer: (defn message-handler [ch {:keys [content-type delivery-tag] :as meta} ^bytes payload] (println (format "[consumer] Received a message: %s, delivery tag: %d, content type: %s" (String. payload "UTF-8") delivery-tag content-type))) (defn -main [& [port]] (let [conn (rmq/connect {:uri amqp-url}) ch (lch/open conn) port (Integer. (or port (env :port) 5000))] (println (format "[main] Connected. Channel id: %d" (.getChannelNumber ch))) (lq/declare ch qname {:exclusive false :auto-delete true}) (lc/subscribe ch qname message-handler {:auto-ack true}) (jetty/run-jetty (site #'app) {:port port :join? false}) (.addShutdownHook (Runtime/getRuntime) (Thread. #(do (rmq/close ch) (rmq/close conn)))))) Is it acceptable to co-host such a consumer in its own thread along with the main web server? Not sure how it may affect the performance of a web server. Or perhaps I should explore Immutant because it has some kind of messaging infrastructure built-in?

lukasz16:05:34

@nfedyashev yes, it's common to spin threads to host your consumers, web server and other "background" stuff. Once you have to coordinate many stateful things like that I'd suggest looking at Component (or something equivalent)

👍 4
potetm12:05:22

@nfedyashev I’m unsure what you’re asking. None that has anything to do with Clojure per se.

potetm12:05:56

> Is it acceptable to co-host such a consumer in its own thread along with the main web server?

potetm12:05:28

Depends on your hardware specs, processing constraints, etc

👍 4
potetm12:05:13

> Or perhaps I should explore Immutant because it has some kind of messaging infrastructure built-in? I’ve used queue’s a ton. I’ve never used or even seriously considered Immutant.

👍 4
nick12:05:00

Got it. Would you mind telling what did you use for queues? I've noticed that Kafka is very popular in Clojure community for some reason

potetm12:05:05

I’ve not used Kafka. SQS, Rabbit, JMS, and Redis

potetm12:05:21

(Redis takes a little more work)

potetm12:05:54

I’ll get you an easy pattern to get started

🙏 4
potetm12:05:47

Are you familiar with component

potetm12:05:46

(defn consumer [q]
  (while true
    (let [v (receive-message q)]
      (do-work v)
      (ack v))))

(defrecord QueueConsumer [n-threads queue ^ExecutorService exec]
  component/Lifecycle
  (start [this]
    (if exec
      this
      (assoc this
        :exec (let [e (Executors/newFixedThreadPool n-threads)]
                (dotimes [_ n-threads]
                  (.submit #(consumer queue)))
                e))))
  (stop [this]
    (if-not exec
      this
      (do (.shutdownNow exec)
          (.awaitTermination exec 5 TimeUnit/MINUTES)
          (assoc this :exec nil)))))

(defn produce-events [q]
  (doseq [event (fetch-events)]
    (send-message event)))


(defrecord QueueProducer [queue ^ExecutorService exec]
  component/Lifecycle
  (start [this]
    (if exec
      this
      (assoc this
        :exec (let [e (Executors/newSingleThreadExecutor)]
                (.submit #(producer queue))
                e))))
  (stop [this]
    (if-not exec
      this
      (do (.shutdownNow exec)
          (.awaitTermination exec 5 TimeUnit/MINUTES)
          (assoc this :exec nil)))))

potetm12:05:11

So I just kinda slapped that together – may not run, but it shows the general pattern if you’re familiar with Component

potetm12:05:38

receive-message and send-message and ack are obviously going to be queue-dependent. (e.g. SQS will be .receiveMessage .sendMessage and .deleteMessage respectively)

nick12:05:23

Got it. Thank you!

👍 4
paul a14:05:01

i would like to define a function like this, which returns a function that takes some initial value and successively applies a function (`emulate-step`) to it (think (f (f (f ... (f initial-value) ... )))):

(defn emulate-n [n]
  (->> (repeat emulate-step)
       (take n)
       (apply comp)))

paul a14:05:12

the problem of course is that this results in a stackoverflow for large n

paul a14:05:16

is there an idiomatic way of writing that higher-order function such that it would have the same stack performance as loop/`recur`, and yet which would not require me to manage my own step counter?

paul a14:05:20

does iterate use the previous result as the input for the next result?

paul a14:05:46

oh neat, perfect

alekszelark16:05:56

Hi! If my clojure project contains some java files, is there a way to compile them so one who uses my library will be able to use it with tools deps through sha. Now I compiled them by lein and pushed to the repo. My deps.edn looks like

{:paths ["src" "classes"]
 :deps {org.clojure/clojure {:mvn/version "1.10.1"}}}
But maybe there is a better way.

Alex Miller (Clojure team)16:05:30

in short, no not easily. if you want to publish a lib with compiled java files in it, it's probably best to deploy it to clojars and use it via maven version

alekszelark17:05:38

thanks, I thought so

nikolavojicic23:05:45

This runs indefinitely: (partition-all 0 [1]) Also, for negative n... Is it a bug? ------------------------------- (partition 0 [1]) :=> () (partition -1 [1]) :=> ()

nikolavojicic23:05:33

On Clojure 1.10.1