Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-06-20

Channels

#clojure
<
2020-06-20
>
Adam Helins13:06:54

What do you think about this interval tree implementation? Any way to make it a bit less fragmented? http://clj-me.cgrand.net/2012/03/16/a-poor-mans-interval-tree/ Looks a bit like a hack, but a convenient and multiplatform one. Rather surprisingly, it looks like there isn't any efficient clj/cljs interval tree out there.

marciol16:06:34

Hi all, after reading this post: https://medium.com/helpshift-engineering/achieving-graceful-restarts-of-clojure-services-b3a3b9c1d60d I started to think about how Clojure deliver to us not only a simpler way to deal with threads and asynchronous primitives from Java in general, but also a easier way as well. But all this can be undermined when one needs to handle the threads lifecycle. Is there some kind of framework a la Stuart Sierra Components to help us with this? Something like a mini Erlang OTP framework for Clojure?

miro17:06:39

If you are looking for more complex use cases then ultimately you may want to manage your own thread pools, either directly in java or using Claypool https://github.com/TheClimateCorporation/claypoole Having said that even for lots of complex use cases I was always fine with just using a combination of async (just always make sure you set proper size of async thread pool) and Stuart Sierras component (e.g. here https://github.com/mikub/titanoboa/blob/master/src/clj/titanoboa/server.clj#L135 )

lukasz22:06:04

I was inspired by the blog post that you've shared and wrote a tiny utility namespace to declare and install shutdown hooks: https://github.com/nomnom-insights/nomnom.utility-belt#lifecycle-hooks - we use this approach across 10+ services, and guarantee safe shutdowns of components, threadpools etc with this approach. It's somewhat simpler than what HelpShift folks did as Component determines the order of start/stops so we didn't have to worry about it.

❤️ 3
kwladyka16:06:08

@seancorfield hey, you wanted me to notify you if I will find alternative to wkhtmltopdf to generate PDFs in Java / Clojure. I am still during research, but probably /Applications/Google\ Chrome --headless --disable-gpu --print-to-pdf=foo.pdf html-to-pdf.html is the best. Simple, can be easy run in docker. No extra dependencies. I can test things in Chrome easy. It works like expected so far. It will be funny if this solution after more testing will be the right one. No reason to use pupeteer or selenium or whatever. Probably 😉

👍 12
😳 3
kwladyka16:06:50

oh and this is x100 faster

seancorfield17:06:04

Oh, that is an interesting way to do it! Thanks!

kwladyka17:06:57

--headless --disable-gpu --run-all-compositor-stages-before-draw --timeout=3000 --print-to-pdf=foo.pdf html-to-pdf.html I added a few parameters, but not sure if they are really useful

💯 3
deactivateduser17:06:38

You are doing ${deity}’s work here, @U0WL6FA77!

👍 3
deactivateduser17:06:26

BTW this is a conceptually similar approach as I’ve used with OpenOffice / LibreOffice (albeit for format conversions), where we would spin up a pool of OO / LO background processes, and then communicate with them via sockets (they have a primitive socket-based API) to perform conversions.

deactivateduser17:06:11

That said, OpenOffice / LibreOffice weren’t really designed to run as a headless “microservice”, and they’re pretty unstable when used in this way. I would expect / hope Chrome would have better operational characteristics, especially as you’re re-running it each time (rather than leaving it up and running as a daemon).

kwladyka18:06:50

thank you, the hardest part was to realize that. Whole google saying to use pupeteer, wkhtmltopdf and all other stuff which add extra layers on chrome which today are not really needed. I don’t even want something to effect my HTML. Especially to generate PDF from template. Go outside the trap of using additional libraries is so hard 🙂

☝️ 6
kwladyka18:06:22

calling chrome directly is super fast comparing to for example wkhtmltopdf

kwladyka18:06:40

so even when re-run this each time it looks fine

deactivateduser18:06:35

That’s great! I wouldn’t have expected Chrome to start up so fast!

deactivateduser18:06:57

But if it’s fast enough to just shell a new Chrome process each time, that’s a great KISS solution! 😉

deactivateduser18:06:59

I wonder if headless Chrome binaries are available in most PAAS runtimes? :thinking_face: (no need to answer that btw - it’s something I can research when I have the need 😉)

kwladyka18:06:42

I don’t know. I will make all of this in Dockerfile. I just tried to install chromium in clojure:openjdk-14-tools-deps-1.10.1.502-buster and it looks good

💯 3
kwladyka19:06:15

chromium-shell which is headless throw exception, because of no X… it doesn’t make sense. I will just use standard chromium then.

kwladyka19:06:14

to be precise *chromium - free open version easy to install from command line *Chrome - google product based on chromium

kwladyka19:06:25

After a few years of deal with * PDF, finally 🙂

❤️ 3
deactivateduser19:06:42

Yeah OO/LO require X too, and IIRC there’s a headless “noop” X server you can install to provide that. But if there’s a chromium package that doesn’t need X at all, all the better. 😉

Ben Sless06:06:36

Have you tried pandoc?

kwladyka08:06:10

no, but I need PDFs with precision to each mm, so any markdown will not give me that

kwladyka18:06:11

there is only one issue with this. I need rotate90 and today it doesn’t work good in chrome. There are differences in position, small, maybe about 1mm or 0,5mm but because of that I have to use something extra to rotate90 all pages in PDF after generate.

kwladyka18:06:19

Do you know the simplest way to rotate all pages in already existing PDF file?

kwladyka18:06:46

some really small library or something?

kwladyka19:06:48

for now I used pdfbox library

kwladyka19:06:11

(defn rotate90-pdf [path]
  (let [document (PDDocument/load (File. path))]
    (try
      (doseq [page (.getPages document)]
        (.setRotation page -90))
      (.save document path)
      (finally
        (.close document)))))

kwladyka19:06:28

I will public final solution later

Daniel Tan16:06:24

anyone use arcadia?

Daniel Tan16:06:36

i need to know how to use dependencies with it

noisesmith14:06:25

my understanding is that with all of clojure-clr, the current best practice is to use the native clr package manager, with arcadia you might want to use the package managment features of unity

Daniel Tan08:06:48

So, the suggestion is not to use core async and use what the CLR provides?

noisesmith14:06:45

no, the suggestion is that if you use core.async, use the clr package manager to acquire it but as far as I know there's no way to use core.async with clr, as it hasn't been ported to that vm

sveri17:06:51

@seancorfield You wrote a few times (well, at least twice) in the last week that you just evaluate forms to the REPL during development. That's basically my workflow too, but I never was able to do that when changing rout definitions with compojure (I think I tried bidi once and that did not work either). So for a changed route I have to reload the namespace and then my components. Do you have a setup where you can evaluate your routes in the REPL and have them changed instantly without a reload?

seancorfield17:06:09

Use #' on the various symbol references, e.g., #'app instead of plain app, so that you have an extra level of (Var) indirection, and then you can eval a new version of the function and it will be picked up immediately.

👍 6
kwladyka17:06:59

nice trick, how this exactly work?

kwladyka17:06:07

I saw this with alter-var-root but not alone

kwladyka17:06:36

oh it is with alter-var-root, sorry

miro17:06:54

This is quite cool and useful trick actually, thanks Sean!

seancorfield17:06:04

@U0WL6FA77 Not just with alter-var-root. The #'app syntax is short for (var app) which yields the Var itself that app is bound to. The Var then holds the reference to the current definition. So when the Var is passed around, it is dereferenced when its value is needed, e.g., in a function call.

👍 3
seancorfield17:06:19

We use it in production code, even tho' it introduces an overhead due to the extra indirection, because it's very convenient when developing and we don't want to have different code in dev/test vs production.

kwladyka18:06:08

hmm I have to try this and see this in action

kwladyka18:06:55

but hmm sounds very good, why I didn’t know this before 😉

dominicm19:06:58

I measured the performance, it's negligent.

sveri17:06:18

I see, thank you. I was always kind of hesitant to do that as I am afraid it has runtime implications in the production environment. Does that have any drawbacks for production?

kwladyka17:06:56

One of the alternative can be integrant if it makes sense to use it

kwladyka17:06:26

I use trick mentioned by @seancorfield with alter-var-root in user ns. So I have something like below and can restart the server from the REPL 

(ns user
  (:require [api.core :as core]
            [server.core :as server]
    ;; ring
            [org.httpkit.server :as httpkit]
            ;[ring.middleware.reload :refer [wrap-reload]]
            [ring.middleware.stacktrace :refer [wrap-stacktrace]]
            [ring.middleware.session :refer [wrap-session]]
            [ring.middleware.session.memory :refer [memory-store]]))

;;; ring

;(def mem (atom {}))

(defn wrap-debug [f]
  (fn [{:keys [request-method] :as request}]
    (println "request" (pr-str request))
    (let [response (f request)]
      (println "response" (pr-str response))
      response)))

(def app
  (-> server/app-stateless
      (wrap-stacktrace)
      (wrap-debug)
      #_(wrap-session {:store (memory-store mem)})))

(defonce app-server nil)

(defn app-ring-start [_]
  (httpkit/run-server app {:port 8080}))

(defn app-ring-stop [server]
  (when server
    (server))
  nil)

(defn app-start []
  (alter-var-root #'app-server app-ring-start)
  :started)

(defn app-stop []
  (alter-var-root #'app-server app-ring-stop)
  :stopped)

;;; refresh

(defn dev-start []
  (app-start))

(defn dev-stop []
  (app-stop))

(defn dev-restart []
  (dev-stop)
  (dev-start)
  :restarted)

(comment
  (dev-restart)
  (refresh)
  (refresh :after 'user/dev-start)
  (httpkit/run-server (wrap-reload #'user/app) {:port 8080}))

kwladyka17:06:07

but with current project I use integrant so I don’t need this anymore. Probably I like more integrant approach

kwladyka17:06:59

another way which I use to reload handlers is to reload ns containing handles. It works too without reloading the ring. But you have to do it manually, even if change only files which are dependencies. You have to remember to reload ns with handlers. It can be tricky sometimes to remember.

seancorfield17:06:41

That code above won't pick up changes to the middleware or the app function or app-stateless, without stopping the server, reloading the code, and starting the server again. By using #' references on those symbols instead, you can update any of those functions while the system is running. That's what we do at work -- we can develop from the editor, via a REPL, into a continually running server app.

seancorfield17:06:52

@sveri It does introduce a little overhead because of the additional indirection, but we haven't seen it as a problem in production code (accessing the database and/or third-party HTTP services completely overwhelms the small overhead of a few Var indirections!).

thosmos18:06:14

Is there existing tooling that can do either of the following refactors? I need to move a file like `app.ui.foo` to `app.foo` and have all dependent requires updated. I also need to move individual fns from one namespace to another like `app.foo/myfn` to `app.bar/myfn` and have the tooling change all refers from the old ns to the new one.

seancorfield18:06:32

@thosmos I hear folks talk about Cursive having some sophisticated refactoring. There's also some refactoring stuff in the CIDER stack for Emacs. I don't use either so I can't provide details. There are #cursive and #cider (and #emacs ) channels that might help you get more detailed information about them.

kwladyka18:06:25

In Cursive you can change ns name, but only on the same level. So a.b.c can be changed to a.b.d, but not to a.b. So far I am doing this job manually.

kwladyka18:06:07

To be precise: you can refactor only the last part after . in ns (unless there is secret way which I don’t know)

sveri18:06:07

@seancorfield Thanks, nice to hear that. I will try that out.

lilactown21:06:27

is it completely untenable to embed a JVM inside of an iOS app? like is it just completely disallowed - either by app store rules or some attribute of the platform?

dominicm21:06:32

@lilactown my recollection is that code execution is completely blocked. Nothing dynamic. Except JavaScript, that's cool.

orestis08:06:53

In particular the built in JavascriptCore runtime. I think this is what Replete uses?

Nico23:06:38

what's an easy library for plotting charts of data in clojure? I don't need an entire statistical computing system like incanter, and I want something that's easy to use at the repl

💡 3
Nico23:06:04

I just want to be able to (for example), plot the output of frequencies on a seq as a histogram at the repl

phronmophobic00:06:40

there currently aren't any built in plotting functions, but this would be simple to do with a library I've been working on, https://github.com/phronmophobic/membrane: 

(defn histogram [freq]
  (let [plot-height 300
        max-freq (reduce max (map second freq))
        bars (for [[k n] freq
                   :let [height (* plot-height (/ n max-freq))
                         offset (- plot-height height)]]
               [(ui/translate 0 offset
                               (vertical-layout
                                (ui/filled-rectangle [0 0 1 0.3]
                                                     20 height)
                                (ui/label k)))
                (let [val-label (ui/label n)]
                  (ui/translate 0 offset
                                val-label))])]
    (apply horizontal-layout
           bars)))

(let [nums (repeatedly 50 #(rand-int 50))
      freq (->> nums
                frequencies
                (sort-by first))]
  
  (ui/run
    #(histogram freq)))
which looks like:

phronmophobic00:06:30

if you're interested, I can add a few basic plotting functions to try out

Nico01:06:51

this seems really cool, thanks

phronmophobic02:06:46

what plots would you be most interested in?

jumar06:06:00

Oz is pretty cool

☝️ 6