Fork me on GitHub
#off-topic
<
2022-11-27
>
Ben Sless12:11:42

Weird stuff: • msgpack serialization more compact than JSON • after compression, same size as JSON • msgpack jackson factory way slower than hand-rolling a packer when used with jsonista So, got any recommendations for a cross platform serialization format without schema which is more efficient than json and smaller than json when compressed?

Ben Sless12:11:44

a. how is transit more efficient than json? (not denying it's better) b. what about other runtimes? other languages should be able to read it, too

Ben Sless12:11:51

c. good morning 🙂

Ben Sless12:11:11

The size of the compressed data in transit ( 🙂 ) is a significant factor here

ghadi12:11:16

David Nolen has some benchmarks somewhere showing it to be smaller than JSON, despite being built upon JSON

ghadi12:11:54

It does caching for map keys, iirc

Ben Sless12:11:12

Things get interesting when you apply compression, though

Ben Sless12:11:23

[
 (count (compress-bytes json-bytes))
 (count (compress-bytes msgpack-bytes))
 (count (compress-bytes transit-json-bytes))
 (count (compress-bytes transit-mpack-bytes))
 ]
;; => [585072 548639 588981 552158]

Ben Sless12:11:38

And I can't recreate David's results, at best I get the same size after compression, which is nice, but my current goal is specifically decreasing compressed message size with schema-less formats

ghadi13:11:48

What is the data?

Ben Sless13:11:21

Lots of maps which look pretty much the same. values can be anything which can be found in JSON. Medium is Kafka

Ben Sless13:11:11

I think there's some asymptotic gain which compression can achieve, as things like encoding repeated keys in a table is pretty much what compression algorithms already do, but without regards for structure

phronmophobic18:11:45

Just to throw out some other options for comparison: • https://github.com/Datomic/fressianhttps://en.wikipedia.org/wiki/Bencode

hiredman18:11:52

(I guess that would count as having a schema though)

Ben Sless19:11:10

Fressian is not suitable because of the various runtimes I have to support and the curious property that it works better for batches Bencode should asymptotically perform the same as json when compressed

borkdude12:11:08

lol, twice on the front-page of https://news.ycombinator.com/news with something clojure-related, that's a first for me :)

nice 7
🎉 12
2
clojure-spin 4
borkdude 9
jjttjj14:11:12

Are there any linux desktop environments (or programs?) out there particularly well suited for "programmability" of desktop settings (ie getting and setting monitor config, keyboard shortcuts, themes)? I'm imagining something like a rest api for all desktop settings. I know there's dconf on gnome. Is this about what you get on all desktops, or is there one that is particularly nice to use? I've experimented with NixOs a bit recently and that is sort of adjacent to what I'm thinking about but I'm wondering if there's any other interesting takes that focus more on configuration than package management. Or are they just inseparable? https://clojurians.slack.com/archives/C8NUSGWG6/p1669501470950189 made me think of it. I just want full control over my computer from a repl!

jkxyz15:11:19

I think you can set all of these things with Sway using swaymsg. So maybe also with i3? I have a daemon running which watches for displays connecting/disconnecting and it uses swaymsg to modify the display configurations dynamically

2
jjttjj15:11:53

Awesome that looks great

sheluchin15:11:38

Sway is awesome.

skylize16:11:54

Any choice of window manager will inherently lead to more customization than a "desktop" because a window manager really offers only the feature of managing windows. Or in the new world of Wayland, the primary value is actually compositing, with window management as a necessary feature on top of that. Any other tools or features are totally separate apps. So you go around picking and choosing the tools you want from all around the ecosystem. The downside, of course is the same. If you don't install a feature yourself, you won't have it. You can expect more friction getting tools to look and/or act cohesively. A Desktop instead provides a curated collection of tools, already massaged into a cohesive experience, and probably an API for developers to make apps that slot in well with everything else. If going the window manager route, you still likely want to pick a single desktop API to prefer over others (such as the GTK API from Gnome) because many apps defer the drawing of window chrome to these APIs.

skylize16:11:24

I recently switched from Sway to https://hyprland.org/. It is notably less mature than Sway but offers more advanced compositing features, like animated window moves. Both are built on the same wlroots compositing library, so the vast majority of apps that work with one also work with the other, without any changes. From what I've seen in Sway's Github Issues, Sway seems to a have general stance of aiming to implement no more or less than the same features as i3, but on Wayland. There's some advantage to that, in giving a clear target for success and an easy way to avoid bloat. But unfortunately that also means I have zero expectation to see any further innovation from the project.

👀 1
Martynas Maciulevičius17:11:12

What innovation would you expect in a window manager? What do you expect and what do you get from hyprland?

skylize17:11:51

I don't know exactly. But Wayland is a completely different beast from X11, with wildly different costs and benefits to trying to implement certain features. i3 is culmination of trying to make a highly-customizable window manager in X11, so why should Sway limit itself to the design decisions of someone working on a different platform. I can't remember now what the request the first time I saw a feature get rejected outright, with zero consideration, simply because it was not in i3. I do remember my silent reaction of "well that's dumb". As I said earlier, the current primary selling point of hyprland over Sway is more advanced desktop compositing. Subtle animations when things get moved around the screen makes things more pleasant to me. It also has autotiling, such that new windows are generally split in the direction that makes sense with the current layout, without me needing to specify. Otherwise there is been very little impact from switching. Just a different config syntax. But I also support the willingness to try things and make their own decisions, instead of pinning themselves to a 13 year old X11 app.

Martynas Maciulevičius17:11:52

> feature get rejected I use i3 and I'd like to use sway (but my laptop doesn't allow this). But IMO it's good that sway doesn't want to be everything at the same time. You already have Gnome and other environments that are accomplishing this. Do you have a keyboard-only workflow with hyprland or is it more of a gnome-like config? What did you do with it? Did you write some layer on top to make it usable with keyboard?

skylize17:11:54

Config was very similar to writing a Sway config with a lot of similar settings and features, but with a different syntax. I don't think Hyprland is trying to do anything wildly different from Sway, just offering an alternative with different design decisions on some places, including the choice to make the GPU actually do a tiny bit of work.

skylize18:11:10

Worth noting that window managers in Wayland currently do a lot more than their X11 counterparts. Because of the Wayland security model, and lack of standalone solutions around it, a window manager must do its own compositing and control setup of and access to inputs and displays. So decisions by the developers reach deeper into your total experience.

Martynas Maciulevičius18:11:39

> lack of standalone solutions around it, Yeah... For me I have to have reverse prime to be able to use Wayland. I use it in X11 just fine but it's not implemented in Wayland. So for me trying new compositor is more about "does reverse prime work already or not?" Edit: No, hyprland and sway still don't work :thinking_face:

jkxyz18:11:36

Is reverse prime related to multiple GPUs? Apparently you can specify to Sway/wlroots which GPU is the primary one to use for rendering. There’s also Xwayland which has the usual X features for compatibility. I wouldn’t say there’s much reason to switch to Wayland though if X is working fine for you right now. I needed it for the per-display scaling options when I got my new laptop.

Martynas Maciulevičius18:11:39

> Is reverse prime related to multiple GPUs Yes. I have AMD integrated one and Nvidia dedicated in my laptop. > which GPU is the primary one to use for rendering Yes, I tried it but my integrated monitor isn't connected to the Nvidia card so it works only on one. The other one can't be enabled. So either integrated with integrated monitor, or external. But when I'd disconnect the external it would still not work on the integrated one. So it's not a working setup. > much reason to switch to Wayland though if X is working fine for you right now I tried it on my previous laptop and I noticed that refresh rate was smoother all over the place. Maybe I misconfigured something. But on the other hand the log-in/log-out is unstable and it's not "secure" in the same way. I'd also like to use the scaling options as well.

skylize18:11:29

Xwayland is just for compatibility. There's no reason to use Sway if you want primarily X11. That is just adding Wayland barriers in the way of controlling your own system.

Martynas Maciulevičius18:11:50

On my old laptop it was smoother so this is why I wanted to use it. Even with the lock issues. The scroll in firefox was way nicer. Also there was no tearing when switching windows. i3 does some tearing where it shows contents of RAM or something.

skylize18:11:17

Yeah, Wayland is generally smoother, but X apps usually don't benefit much, if at all.

jkxyz18:11:01

Drew says something about multiple GPUs here: https://drewdevault.com/2018/10/20/Sway-1.0-highlights.html > You can also now run sway on multiple GPUs. It will pick a primary GPU automatically, but you can override this by specifying a list of card names at startup with WLR_DRM_DEVICES=card0:card1:.... The first one will do all of the rendering and any displays connected to subsequent cards will have their buffers copied over.

Martynas Maciulevičius19:11:55

I already tried it with this bad boy and my motherboard isn't wired the way it should so it can't work without "reverse prime":

# This one:

WLR_NO_HARDWARE_CURSORS=1 WLR_DRM_DEVICES=/dev/dri/card1:/dev/dri/card0 sway --unsupported-gpu

# And this one but it still renders on only one.

WLR_NO_HARDWARE_CURSORS=1 \
  __NV_PRIME_RENDER_OFFLOAD=1 \
  __NV_PRIME_RENDER_OFFLOAD_PROVIDER="NVIDIA-G0" \
  __GLX_VENDOR_LIBRARY_NAME="nvidia" \
  __VK_LAYER_NV_optimus="NVIDIA_only" \
  WLR_DRM_DEVICES=/dev/dri/card1:/dev/dri/card0 \
  sway --unsupported-gpu
I'll try the card0:card1 syntax too but I'm pretty confident it won't change much Edit: Yep, doesn't work:
Failed to open device: 'card1': No such file or directory

😔 1
jpmonettas21:11:42

I used to be a StumpWM user (many years ago) https://github.com/stumpwm/stumpwm, which has the nice thing that it gives you a REPL into your WM, and it is written in CommonLisp

jkxyz21:11:11

Similarly there’s also exwm https://github.com/ch11ng/exwm

Martynas Maciulevičius21:11:52

Hm... babashka should also be possible to use in a setting like this :thinking_face: Hmmmm

Annaia Danvers15:11:52

I used Awesome back in the day, which is almost entirely scripted in Lua. https://awesomewm.org/

russmatney15:11:08

I've been building up a tool like this - it's a bit of a hack and a never-ending project, but for me it's worth it to configure window-management code (keybindings and workspace/client behavior) in a clojure repl, then run keybindings/commands via babashka. https://github.com/russmatney/clawe It's not quite mature yet, but I've been using it/refactoring it for a few years, and I love it - these days I'm trying to rework things to separate my own config from the project itself, and add more wms (it supports awesomewm and yabai right now, but i3 and bspwm impls would be reasonable with the wm-protocol). The recent osquery stuff is intriguing - I'm considering refactoring some of my namespaces to use it, as I'd expect it to be more broadly applicable as well. One of my favorite hacks (though very AwesomeWM specific) is https://github.com/russmatney/clawe/blob/b0e37b2e0a417aae1659212001f7322669932221/src/ralphie/awesome.clj and ralphie.awesome/fnl macro, which lets you write syntax-quoted fennel directly in clojure.

🆒 1
russmatney15:11:33

One thing that was a nice win (besides babashka itself) was babashka adding support for -x, which made it easy to execute any function in the project - that's how most of the bindings work, and you can get pretty far on just that.

jjttjj16:11:17

I never really thought about, until now, how coupled the "things that manage keyboard/mouse input" and "things that set the rules for the shaping/sizing of windows on your screen" tend to be.

jjttjj16:11:36

It seems like they could be separate concerns but they all seem to be packaged together

russmatney16:11:54

for sure - digging into bspwm and yabai, those things tend to get separate executables/tools. i ended up reaching for sxhkd (which bspwm uses) on linux to have something decoupled to fallback on

jjttjj16:11:54

nice! clawe looks really cool, and seems like something i've always wanted to do as a side project but never started. I'm going to have to dig into it a bit

russmatney16:11:11

thanks much! it's a bit of a monorepo after I got annoyed moving between projects and repls, but it might deserve to have some things broken out at some point - feel free to ping me about whatever, I'd love any feedback!

👍 1
skylize16:11:27

Input and window display are not inherently linked. But for windows managers in Wayland to provide a reasonable share of the expected features, the only way (currently) to get sufficient permissions on inputs is to own them.

skylize16:11:11

I expect someone will figure out how to design a side-channel for input devices, that window managers can tap into, at some point in the future.

1
skylize16:11:49

In the past, window managers were built on Xserver. And X couples everything in your system to everything else, including control of input devices. X window managers simply rely on X for trafficking input events around. When you pull X out the equation, and add on the stricter security model of Wayland, you are left with window managers needing to step in and create their own input management... until someone builds a standalone Wayland input manager with a nice API for sharing with other applications.

MIJOTHY16:11:44

did you try https://github.com/nix-community/home-manager as part of your nix exploration? it’s a collection of modules for user-level system configuration. i would say it is more geared towards config management than package management