Clojurians Log v2

What are the chances a pattern matching macro will be more readable in this case?

I’ve been pretty happy with https://github.com/clojure/core.match maybe this is what you’ll need?

No, I don’t think pattern matching is a good fit here. The input is all strings. I want let behaviour since I need to destructure results from re-matches in some cases, but I want to do so on demand. For now I have gone with better-cond after @borkdude suggested it.

Not sure just how related this is, but I made a macro ages and ages ago I just put into a gist that may be connected: https://gist.github.com/IGJoshua/09c0e4fcd7e496a611bc8724fce96e0c

It allows you to make a cond expression to match some expression against a list of regexes in order, while creating local bindings for the capture groups.

The regex operator in meander does that

oh cool, I didn't know about that library.

Oh hmm, it might not have existed when I wrote this macro

I wrote this for a discord bot I wrote back in late 2017

meander is pretty cool 🙂

maybe the unix tool bat (a cat with syntax highlighting etc) would work? https://github.com/sharkdp/bat/releases

I use jq for my JSON logs:

aws logs tail /some-log-group --follow | cut -f 3- -d ' ' | jq -R '. as $line | try (fromjson) catch $line'

over the years I've learned to decouple test coverage from test suite runner, they will be different steps in CI (perhaps performed in parallel) else you'll find Cloverage intricacies dictate how your test suite runs. It's worth emphasizing that cloverage instruments that code, to date that isn't issue-free, so there's value in running your tests in a raw/uninstrumented way

https://github.com/kiranshila/cybermonday

this didn't show up in my google searches, but I recalled there had been something new like this released recently! ty

np! @me1310 is occasionally around on slack if you have questions too.

we’re building https://github.com/nextjournal/viewers/tree/main/modules/markdown and using it in Clerk

https://github.com/nextjournal/viewers/blob/main/modules/markdown/src/nextjournal/markdown/parser.cljc

neat!

what's the motivation & tradeoffs you're considering? i.e. why not use cybermonday?

besides not being aware of cybermonday our lib builds on markdown-it and uses graal on the JVM to deliver consistency across runtimes

goals are similar in getting data out including ToC and handing a bunch of extensions. Because we build on markdown-it we could integrate with our nextjournal editor built on prosemirror eventually https://prosemirror.net/examples/markdown/

if I wanted to use this in the JVM, I'd need to use Graal so I could load markdown-it (which is a JS lib)?

no, we use graals Nashorn replacement, works with any JVM https://github.com/nextjournal/viewers/blob/main/modules/markdown/deps.edn#L6

also available here https://clojars.org/io.github.nextjournal/markdown

interesting

maybe I could do the same thing for org mode. a lot of the Java and Clojure org parsers seem pretty low quality

it's an insane task so I don't consider it a slight against any of the authors 😄

yep, that was our thinking and why we built on the state of the art js impl.

if the browser isn't a target, pandoc is also worth checking out

enjoyed this podcast on the subject recently https://overcast.fm/+Y-HXW34kI

I think pandoc is really what I want. that's a great suggestion

just have to figure out how to integrate it so I can get an AST

maybe via @suskeyhose’s https://github.com/IGJoshua/coffi 🙃

this is certainly turning into a 🐰🕳️

hehe

@mkvlr interesting using markdown-it and nashorn for jvm consistency. It took me quite a bit of time to get an IR that matches flexmark and remark

I found remark’s ast much nicer than markdown-it’s fwiw

we went with markdown-it because that’s what prosemirror settled on

Ah

and are currently looking at making io.github.nextjournal/markdown compatible with https://pandoc.org/MANUAL.html#pandocs-markdown

there is also this implementation used by logseq - not technically a clj impl though https://github.com/logseq/mldoc seems to be used from cljs here: https://github.com/logseq/logseq/blob/e0888b896f525465b99dff9c28806d8001fc2d37/src/main/frontend/format/mldoc.cljs

Glad to have the mention of #coffi 😉

@lilactown in cybermonday, you can get an ast with the md-to-ir fn in the ir namespace, if you wanted to give it a try

yeah that's really nice. if I give up on pandoc I'll probably try cybermonday next

I really want to support more than just md - md is a good starting point tho

Yeah Pandoc works with org too, iirc. @djeis went down this rabbit hole a bit at one point

I've been using https://github.com/vsch/flexmark-java (same parser cybermonday uses) directly and it's worked well for me.

@mkvlr that podcast is great!

i guess the best way to use pandoc is to shell out? there are some wrapper APIs but they're out of date and/or I think they simply shell out regardless

this was just asked this morning in #tools-build :)

👀

there are cases where your paths are not exactly what you want to compile but I think it would be a reasonable default and I'll probably look at it later today

Great - thanks!

Not 100% sure. From the behavior of = I would say it is designed to differentiate between integers, decimals/fractions and floating point numbers:

(= 1 1/1) ; => true
(= 1N 1). ; => true
(= 1 1.0) ; => false

I think one of the reasons that = returns false is that it is probably extremely difficult to implement hash to be consistent with = if = behaved like == for numbers.

This article I wrote mentions that deep in the middle somewhere, but the article is mostly about the "what" of = and ==behavior, because the "why" answers I can only really make educated guesses about.

https://clojure.org/guides/equality

The N suffix is for Java’s BigInteger. So it is an integer type, and can be safely compared with other “precise” number types. Numbers suffixed with M are Java’s BigDecimals and therefore not as precise as integers. Doing math with decimals could involve rounding and therefore introduce errors. Therefore = differentiates between these “classes” of numbers.

to compare numbers it ensures the “categories” are the same. And that lookup is here:

static Category category(Object x){
	Class xc = x.getClass();

	if(xc == Integer.class)
		return Category.INTEGER;
	else if(xc == Double.class)
		return Category.FLOATING;
	else if(xc == Long.class)
		return Category.INTEGER;
	else if(xc == Float.class)
		return Category.FLOATING;
	else if(xc == BigInt.class)
		return Category.INTEGER;
	else if(xc == Ratio.class)
		return Category.RATIO;
	else if(xc == BigDecimal.class)
		return Category.DECIMAL;
	else
		return Category.INTEGER;
}

(i had never seen this stuff so interesting to me)

Thank you both!

The answer is rather complicated. But two things need to be true: variables inside expanded code bodies need to be stable because they are captured in persistent storage. Also, I need to produce stable hashes from the AST. So gensyms ruin that. Common lisp allows you to dynamically bind *gensym-counter* , but that counter is in a private Java field in Clojure. I even tried setting the private id counter in clojure.lang.RT. which is used by autogensym, but that does not seem to affect gensym .

It does not seem to affect gensyms generate in clojure.core macros, I mean. Doing alter-var-root on clojure.core/gensym or replacing the id field in clojure.lang.RT, using for example org.apache.commons.lang3.reflect.FieldUtils has no effect.

you are correct that core functions are linked in a different way that cannot be overridden

you could write a codewalker that looks for gensyms and strips the number off of them. It wouldn't work in cases where the macros genuinely want to have a number as the end of a local, non-auto-gensymmed symbol, but those cases would be rare.

Thanks, @ghadi

np, are you doing some content-based hashing? @aneil.mallavar or a database of forms?

@suskeyhose, yes I think I'll have to do that. I guess I can rely on the guarantee that gensym vars always end in numbers.

@ghadi - I'm capturing execution state of functions and need the variables to be stable.

trying to do continuations?

yes

good luck 👍

java is getting virtual threads Soon™

🙏

You could look at core.async for a CPS transform that it does

but the reality of current continuation libraries (and maybe just a reality of libraries that don't try to modify bytecode on the fly?) is that they only work by CPS transforms, so you can't just save a continuation in any function, only in the ones that go through this transform.

I'm looking forward to project loom though, and I'm hoping they expose the internal continuations used for that.

although I doubt they will

I hope they don't.

@suskeyhose @ghadi great pointers. Thanks!

I guess I hope they do in an entirely academic sense. I have no desire to write code for production that uses continuations, and I have no desire for the issues in libraries that exit continuations bring as evidenced by scheme.

@aneil.mallavar If you need a reader for fn literals that produces stable results, I have one.

Not sure if your problem is beyond that.

user=> (require '[edamame.core :as e])
nil
user=> (e/parse-string "#(foo % %2)" {:all true})
(fn* [%1 %2] (foo %1 %2))
user=> (e/parse-string "#(foo % %2)" {:all true})
(fn* [%1 %2] (foo %1 %2))

Even if they don't expose the continuations, for all the "fun" stuff we can just do --add-opens

I don't think anyone using a custom continuation library would flinch at that

yeah that's fair

@borkdude - hey, thanks, that's kind of you. I think I solved it using @suskeyhose's suggestion. I'm just doing a tree walk. (It wasn't so bad after all.)

ok

in case you need it next time, it's here: https://github.com/borkdude/edamame

try using clojure.repl/dir-fn instead

my hero!

Not a bug. The CLJ version uses .toString internally because its argument might be any subclass of CharSequence which is not a string itself. By passing a number there, you're using it wrong. The fact that it works is accidental.

I understand this usage is wrong, but should this function throw an exception?

Such an approach would require checking every single argument of every single function - IIRC there has been an explicit decision at some point to not do that. But there are optional function specs. No clue whether the built in ones include that particular check though.

Yes, this is the result of 'garbage in, garbage out'. Passing a number is basically the equivalent of undefined behavior in C++. It happens to do something reasonable-ish, but it's not intended.