Fork me on GitHub
#off-topic
<
2023-03-05
>
p-himik12:03:10

Can someone recommend a relatively simple log stash platform that: • Supports structured logging (preferably EDN but JSON or anything else would also work) • Supports plain HTTP log reporting without a strict need to install agents and other cruft like that (with maybe optional plain libraries) • Tracks user interactions in a browser and reports them along with any errors • Knows about JS sourcemaps and applies them to stack traces Rollbar comes close but their telemetry is not useful without me attaching a unique ID/class to every single UI element. Otherwise, it looks like "A user clicked on button.rc-button.btn.btn-default". I've seen a lot of praise for New Relic but no idea whether it has support for the last 2 items and I definitely do not want to use that monster. Maybe if I ever hire a separate devops engineer, but not by myself.

lispyclouds13:03:13

Do you have a need to be cloud agnostic and whats the kind of budget you're looking for?

lispyclouds13:03:01

Also what's the kind of analytics you're expecting to do on the logs?

borkdude13:03:18

I guess you've already looked at sentry right? At my previous job we've had good experience with it and it supports all of the above

p-himik13:03:04

Cloud agnostic - yes. Budget - not my concern, so any. Analytics - none, it's just for logs. Sentry - I have heard about it but didn't know that it supported the last 2 points. Will check it out, thanks!

lispyclouds13:03:57

i could recommend Splunk too here. Quite the industry standard and good analytics. Bit pricey though

lispyclouds14:03:23

and https://www.graylog.org/ bit less on the fancy side, but quite used a lot, has things like the HTTP api and cheaper

lispyclouds14:03:48

i like splunk's query DSL more

p-himik14:03:25

At the superficial level, Sentry definitely gets a fat + for being open-source.

Daniel Gerson14:03:11

My last contracting job used https://logz.io/ and they supported structured logging. Had great support with someone from the company taking us through the tool. Can't really speak to many of your other criteria (it was a Java job). Thought I'd just mention it.

thanks3 2
Daniel Gerson14:03:43

(Oh, and that it was much more pleasurable to use than Splunk)

lispyclouds14:03:19

i've heard good things about logz too

kwladyka16:03:21

google support JSON logging in google cloud run I use https://github.com/kwladyka/consistency-clj/tree/master/src/consistency/logs/google and it get JSON structured logs from standard output of the app. So with google I use default, standard logging without extra tools, which make things simpler. But this is only for BE. For FE Sentry or to measure user behaviours hotjar (although I was working with this years ago) Maybe there are other interesting tools for FE today. My last research was years ago. For BE I do everything to keep things simple and not use tools like New Relic. I use New Relic and other tools in work. But when I have choice I try to use as less tools as possible. Clouds native logs are good enough when log to them correctly. Although I like much more google readability vs AWS for logs.

jkrasnay00:03:34

Doesn’t do anything on the browser side, though, afaik

orestis12:03:17

We have Sentry working for both server and client, with CLJS source maps etc. I haven't found a way to get better browser telemetry without unique identifiers, but I haven't tried either.

orestis12:03:15

Server-side we used the log4j integration to get started quickly, and client-side we use their JS client (opted for the low-level one, not the React version)

👍 2
kwladyka16:03:32

How would you compare clojure vs lua vs ocaml? I was trying to read about them, but I didn’t find easy way to understand differences and use cases in reasonable time. clojure is on JVM and interop on top of Java clojusrcript in on js and interop on top of javascript What about lua and ocaml?

2
p-himik16:03:24

Lua has its own VM, it's more similar to Java in this regard. But that VM is also tiny and easily embeddable - that's why many programs use Lua as a scripting language. With that being said, it's a very low-level language.

kwladyka16:03:04

Is Lua on top of other packages (like JVM, C etc.) ot there is huge gap for libraries?

p-himik16:03:08

No clue what you mean by "on top of other packages". Lua is a thing on its own. But you can find or somewhat easily write an interpreter that would be hosted on JVM or even on top of Clojure, since Lua is so tiny.

p-himik16:03:27

If you're not sure whether you need Lua, you probably don't need it. ;)

kwladyka16:03:10

I mean if for example I will need to code something with crypto currency I can find libraries in Java or JavaScript. As I understand with Lua I would have to write everything from 0. Is it correct?

kwladyka16:03:37

I stay with Clojure, but I want to update my knowledge about other languages

p-himik16:03:52

Lua is primarily intended for embedding. It's not the main language, usually.

p-himik16:03:37

If you really want to update your knowledge, I'd suggest simply reading Wikipedia articles on the languages. Such dialogs with lots of back and forth will end up spending more of your time than just reading a few articles.

kwladyka16:03:47

What really mean “ntended for embedding” in this case? Sorry, english is not my native language.

kwladyka16:03:18

yeah, I was trying to read articles, but I didn’t find what I wanted in reasonable time.

p-himik16:03:51

Suppose you have an app that uses some compiled language but that needs to be easily extensible by users. That's where you use embeddable languages. You embed some scripting facilities into your app. Browsers are such apps - they embed JS runtimes.

👍 2
p-himik16:03:05

Everything that I have said so far is literally in the first sentence in the Wiki's Lua article. :D Plus the first sentence from the "Implementation" section of that article.

kwladyka16:03:03

yeah I know, but on the end I still don’t really understand the Lua use cases and this is very not clear for me if it will be possible / painful to interop Java or other libraries.

p-himik16:03:48

It will be possible and it will be painful.

👍 6
lispyclouds16:03:43

one of the frequent use case for lua is to write the ai or connecting code in game engines. the main code is C++ but lua makes it easy to write the embedded logic. for instance you want to write some hooks or custom logic

👍 2
lispyclouds16:03:11

also editors like neovim uses lua as its config lang by embedding it.

kwladyka16:03:25

What about ocaml?

lispyclouds16:03:39

like its mentioned, lua as the main app lang is quite cumbersome but its great for wiring things together

kwladyka16:03:22

> like its mentioned, lua as the main app lang is quite cumbersome but its great for wiring things togethe like glue code in Java, Python and C?

lispyclouds16:03:12

yes, more used in native langs like C, C++, Rust. due to its great native interop

lispyclouds16:03:11

OCaml is its own lang, similar space as Rust/C/C++/Go etc. Suitable to write the main app in but has its own ecosystem. Used a lot in finance or compiler tech due to its high perf

kwladyka16:03:39

Does OCaml has huge gap for libraries?

lispyclouds16:03:50

not really, depends on what you want

kwladyka16:03:25

for example crypto or generating PDF etc.

lispyclouds16:03:56

should be there. its not as widely used like the others but its general purpose

kwladyka16:03:13

I mean in js or Java you can find almost everything. Expect AI, because AI is in Python.

kwladyka16:03:53

*There is AI, but Python has more.

lispyclouds16:03:57

yeah, high performace like high frequency trading, implementing compilers etc is OCaml's thing

kwladyka16:03:09

So use cases for OCaml are like in C / Rust. This is really about the same, but just a different language? Perhaps OCaml is not good for hardware - I don’t know this part.

lispyclouds16:03:31

> crypto or generating PDF there's no specific lang good for those. all can handle it, you need to choose based on what youre comfortable with and how much time youd like to spend building.

kwladyka16:03:21

> there’s no specific lang good for those. agree, but this is good example because it is relatively new things and I think JS has libraries for all of them. Java should be ok. But if for example OCaml has nothing to support this and there is no way to interop is shows clearly… how to name it… when to use it and when not.

lispyclouds16:03:07

well ocaml is as old as java/js and lua is older 🙂

kwladyka16:03:13

so on the end can I think about OCaml like about Rust? But just a different syntax?

lispyclouds16:03:29

and very different way to write code.

kwladyka16:03:40

*for use cases

lispyclouds16:03:53

its much more functional like haskell

kwladyka16:03:18

ok, thank you. I think it is much more clear now for me.

mpenet17:03:37

lua has great ffi, so there is more than just the pure lua stuff as ecosystem

☝️ 2
mpenet17:03:55

but it's mostly used/designed to be embedded

mpenet17:03:56

one example is the wide-spread use of libcurl when needing an http client

mpenet17:03:45

also the full interpreter is ~250kb irc, so while you cannot "compile" lua programs per say, shipping the compiled interpreter with your app is nothing, you can end up with full programs taking <1mb all included easily. Also in terms of performance it's can be very impressive in some (not all) contexts.

mpenet17:03:36

+ you get to use fennel lang, which a is very clojure lookalike syntax for lua (a transpiler)

mpenet17:03:52

then lua is a strange, but beautiful beast. Everything revolves around tables. It's really worth exploring imho, even if you don't use it (it doesn't take a lot of time either, it's tiny as a language).

mpenet17:03:05

I wouldn't qualify it as low level, it's more akin to js

mpenet17:03:37

about ocaml, beautiful language but imho the big issue is the ecosystem, it's not as widely used as the others, there's a lot of dying libs, fragmentation, the tooling is/was...not very user friendly and libs are generally not as good quality as what you'd find on the jvm. At least that was my exp using in professionally ~5-6 years ago. There are some outliers, some companies producing high quality libs for it, but that doesn't make a gigantic community like we have on the jvm. About the use of ocaml to write compilers, I am not sure it's because of its perf characterisics in particular, I suppose mostly for correctness guarantees & help in design since most people doing that kind of work usually rely/ied on these kind of languages (either academically or otherwise).

mpenet17:03:55

... but again, that's my experience from years ago, I guess that could have changed

kennytilton23:03:30

Anyone have a good idea for how to present a web framework tutorial in which an application is evolved in a series of incremental elaborations? One crazy idea is a branch per elaboration. Or maybe a tag? A tricky feature will be being able to revise a elaboration after the fact. Branches would be good for this, but if I want later branches to inherit this work...ugh. Ideas welcome! 🙏

mjw00:03:11

Another option I’ve seen (and used) would be a top-level directory for each elaboration. The difficulty I’ve found with the per-branch model is issuing corrections after the fact: checkout each relevant branch, and add a separate commit for each. With a top-level directory for each elaboration, fixes can be issued in one go. This approach also simplifies comparing iterations side-by-side without relying on things like git worktree. One downside of this approach, however, is that searching through the entire codebase generates a lot of noise and duplication, making it harder to pinpoint the right file you’re looking for. This is annoying, but IMO not terribly so.

🙏 4
kennytilton01:03:09

Wow, ugit looks like an amazing tutorial. Thx for the pointer. 🙏

sheluchin01:03:50

yw @U0PUGPSFR. I haven't looked at this in detail, but maybe you could get something out of it https://github.com/motform/clogit I've had these things in my notes forever because I've long thought that tutorials should be written as iterative git patches that readers could apply. Wanted to try making something for it, but never got around to it.

👀 2
Rupert (All Street)11:03:48

I agree that folder per iteration could work well - means that git details aren't embedded into the tutorial. Also you can make bulk edits easily across all versions (e.g. find and replace a typo across all files). Regardless of the storage mechanism - I think a tutorial where the content is mainly additive vs editing is easier to consume. E.g. prefer to add new files/functions at each revision than to edit/revise files/functions. Obviously this means structuring the code in an appropriate way to make this possible (dependency injection could help).

kennytilton12:03:26

Hmm. I am not understanding the folder/top-level dir solution. Is the key that it is also a namespace root? OK, that explain the codebase search concern. It certainly solves the problem of going back to enhance an elaboration. And a nice win is that it let's folks see before-after simultaneously....ha-ha, OK, all that was in @UGTAV6LR2’s remarks! This is a great example in comprehension, btw! :rolling_on_the_floor_laughing: So I think NS-per elab works, unless I go with my latest idea: forget fixing past elaborations. Let the record stand. Treat this more as "the making of". Now I have to go find out what git worktree does... :rolling_on_the_floor_laughing: Thx all! 🙏

👍 4
mjw14:03:27

@U0PUGPSFR In short, git worktree creates a different local repo from a specific branch, so you can work on multiple branches of the same repo simultaneously in different directories. Here’s a short overview: https://spin.atomicobject.com/2016/06/26/parallelize-development-git-worktrees/

kennytilton14:03:42

That helps a lot; the git help threw me with "multiple branches". Sounded like the worktree would have two branches going simultaneously. 🙂 But I get it, and have def done things like taking a whole new clone in desperate circumstances. Thx! 🙏

Daniel Jomphe18:03:23

This is probably too futuristic vs your present needs, but if you review all mentions of "`stack`" on the following https://engineering.fb.com/2022/11/15/open-source/sapling-source-control-scalable/, you might be tempted to try something. I spent some time imagining myself using it 😂 and it felt like it would make working with a commit tree the easiest and most satisfying way possible. They really seem to have nailed something there. Not sure about how exposing your work to end users would work well or not. Edit: oops, I'm late to the party!

kennytilton19:03:12

np! I am still dilly-dallying with polishing the beast, but wearying of that and ready to just announce it and get on with the instructional bit. So...perfect timing! But, yeah, seems like they just have a client ready to share: https://github.com/facebook/sapling Thx for the input! 🙏

🙂 2