Fork me on GitHub
#membrane
<
2023-02-27
>
jlmr09:02:09

I’m working on an idea for the application that incorporates a zoomable and pannable canvas (like apps such as Figma and Miro have). However, I’m struggling with bounding boxes. A scale of 2.0 seems to “compress” the bounding box of a view: clicks at the middle of a [100 100] rect are reported as being at a position near [100 100] while it should clearly be something like [50 50]. Clicks in the right side and/or lower half are not detected as clicks. What am I doing wrong? Or is this expected?

👀 2
phronmophobic21:02:29

@U56R03VNW, that was a bug in membrane 😬 . It should be fixed and available in the latest version:

com.phronemophobic/membrane {:mvn/version "0.11.1.2-beta"}

jlmr07:02:14

Yes that fixed it, thanks!

genekim20:02:37

@smith.adriane I’ve been making some graphs in vega/vega-lite and used darkstar to create SVG images. • Do you have any recommendations on how to display an SVG image using skia backend? • Alternatively, I’m currently trying to convert SVG to PNG image using com.formdev/svgSalamander — first cut will be a PNG file. If I can create a PNG or some other bitmap in memory (BufferedImage?), would I be able to display that using ui/image? Thanks! PS: experimented over the weekend with trying to modify darkstarto emit a PNG file, but couldn’t get that to work. Couldn’t get View.toCanvas to work, and didn’t have a CLJS REPL to really get to the bottom of what was going wrong.

phronmophobic21:02:41

> If I can create a PNG or some other bitmap in memory (BufferedImage?), would I be able to display that using ui/image? Yes, you should be able to pass a byte array to ui/image .

phronmophobic21:02:06

still looking at the options for rendering SVG more directly...

genekim21:02:02

I’ve tried both batik and svgSalamander. • Batik hung during transform • svgSalamander: having same problem when rendering to BufferedImage. I’ve never used BufferedImage before, so trying to find another way to render to something else…

phronmophobic21:02:04

oh, interesting. I know AWT can collide with GLFW, but I think it depends on which APIs are used. I guess they're using some API that is incompatible. It's probably not buffered-image per se, but something else loaded by awt.

phronmophobic21:02:46

how many SVG's are you converting? It seems like it's not too much work to add direct SVG support to membrane.skia, but it does require a change to skialib. Is there a workaround that's possible until SVG is more directly supported?

phronmophobic21:02:36

If you need something ASAP, then the alternative is to create the image in a subprocess.

genekim21:02:23

Not urgent at all! My plan was to create a graph every time a simulation is run, when the sim-state is updated. If it could keep up, I’d even consider updating it everytime the “frame” changed, so I could change color of dot, based on the current turn. (I actually have PNG files being generated working using rsvg-convert command line utility, but wanted to do this inside program, so it didn’t require an external brew install, etc. But surprisingly not as straightforward as I had hoped. This way, anyone downloading repo could run it. vs. having to install external utilities. Or showing blank window if not available.)

👍 2
phronmophobic22:02:37

I'll try to get the SVG renderer added soonish.

genekim20:02:28

Wow, having all sorts of problems on macOS — (def bi (BufferedImage. 1200 240 BufferedImage/TYPE_INT_ARGB)) hangs due to issue described in this article and many more. https://stackoverflow.com/questions/46793769/bufferedimage-causes-a-program-freeze-on-macos-but-not-on-windows (It works if I include JVM argument -Djava.awt.headless=true, but that causes Skia window to not appear!) Rethinking this whole SVG -> PNG approach. Anyone have any ideas?

phronmophobic20:02:29

I think BufferedImage should be fine to use with skia, but if you have an image in byte form, you can pass that directly to ui/image

genekim22:02:13

Hopefully quick question: Is there a way to get skia window to show up if I pass this arg to JVM? “-Djava.awt.headless=true” (so that I can do BufferedImage.createGraphics() without hanging.)

phronmophobic22:02:03

I have no idea. If I had to guess, then I would say no.

phronmophobic22:02:47

awt is pretty gnarly on its own. Trying to figure out how it interacts with glfw and mac osx APIs seems very difficult.

genekim22:02:50

Woof. There be dragons down here. Ejecting myself from this area of problem-solving! 🙂

phronmophobic22:02:07

I think you can use BufferedImage alongside skia, but maybe not it. My intuition is that some operations are ok and some operations will load conflicting code.

phronmophobic22:02:27

That's also a can of worms though.

phronmophobic22:02:01

This seems to work:

(import 'java.awt.image.BufferedImage
        'javax.imageio.ImageIO
        'java.awt.Color
        'java.io.ByteArrayOutputStream)
(require '[membrane.skia :as skia])

(defn get-image [fname]
  (with-open [is ( fname)]
    (let [image-stream (ImageIO/createImageInputStream is)
          buffered-image (ImageIO/read image-stream)]
      buffered-image)))

(defn get-image-bytes [img]
  (let [baos (ByteArrayOutputStream.)]
    (ImageIO/write ^BufferedImage img "png" baos)
    (.toByteArray baos)))

(def bi (BufferedImage. 100 100 BufferedImage/TYPE_INT_ARGB))

(System/setProperty "java.awt.headless" "true")
(def g (.createGraphics bi))
(.setColor g Color/BLACK)
(.drawString g "hello" 0 20)

(skia/run (constantly
           (ui/image
            (get-image-bytes bi))))

phronmophobic22:02:09

I'm not totally sure what "java.awt.headless" does. I'm not sure if you have to load membrane.skia before setting java.awt.headless to "true", but maybe that helped?

phronmophobic22:02:10

Also fyi, trying to mix AWT and glfw (ie. membrane.skia) can be trouble and even if it works sometimes, it might eventually cause the JVM to hard crash.

phronmophobic22:02:12

or maybe it's fine.

genekim23:02:21

Thank you, @smith.adriane !!!! https://github.com/realgenekim/simulator-couch-paint/blob/41593527248527d742b395b0bbecbe86cf09a88b/notebooks/s05_from_adrian.clj#L43 Obviously, some sorcery and knowledge needed of arcane aspects of AWT and GLFW to make this work. Much appreciated!!!

phronmophobic23:02:33

charts and plots are definitely a "missing piece" in membrane. thanks for leading the charge!

phronmophobic23:02:31

If you have a retina monitor, I might recommend drawing the the chart twice as large and then scaling down.

🙏 2
genekim23:02:46

BTW, I think this is super cool — this allows using vega/vega-lite in membrane apps! vega-lite -> SVG -> PNG (Sometimes requiring writing to filesystem, but I think that can be addressed via ByteArray somehow. I haven’t used that before beyond blind copy/pasting from SO. 🙂

🎉 2
phronmophobic00:02:40

got this working locally:

(skia/run
  (constantly
   (skia/svg (slurp "/Users/adrian/Downloads/Clojure-Logo.wine.svg"))))
Just need to do some bookkeeping to make it ready for release.

phronmophobic02:02:04

@U6VPZS1EK, just pushed a release with SVG support:

(skia/run (constantly (skia/svg (slurp ""))))
skia/svg accepts either strings, bytes, or a java.io.File. It requires updating membrane and skialib:
com.phronemophobic/membrane {:mvn/version "0.12.1-beta"}
com.phronemophobic.membrane/skialib-macosx-aarch64 {:mvn/version "0.12-beta"}
I've tested it with a few SVGs, but it could use a few more pokes. Let me know if you run into any issues.

genekim03:02:03

Amazing!! I’ll try it later tonight — I can’t wait to pass a bunch of Vega graphs. I’ll record it!

clojure-spin 2