clj-yaml

Ingy döt Net 2024-05-18T19:04:08.347379Z

Has anyone ran into this with clj-yaml?

Error: org.snakeyaml.engine.v2.exceptions.YamlEngineException: The incoming YAML document exceeds the limit: 3145728 code points.

Ingy döt Net 2024-05-18T19:05:26.347129Z

I know clj-yam uses snakeyaml, not snakeyaml-engine but same author

Ingy döt Net 2024-05-18T19:05:37.475399Z

I'll try it out myself 🙂

borkdude 2024-05-18T19:06:31.132669Z

have you tried grepping the clj-yaml repo for that number? that might give you some hints: https://github.com/search?q=repo%3Aclj-commons%2Fclj-yaml%203145728&type=code

Ingy döt Net 2024-05-18T19:19:16.620819Z

https://github.com/search?q=repo%3Asnakeyaml%2Fsnakeyaml+%22The+incoming+YAML+document+exceeds+the+limit%22&type=code https://github.com/search?q=repo%3Asnakeyaml%2Fsnakeyaml+getCodePointLimit&type=code Can't find that number in the code but: https://www.google.com/search?q=codePointLimit+3145728&oq=codePointLimit+3145728 Is interesting. This is using clj-yaml under the hood:

$ ls -lh big.yaml 
-rw-r--r-- 1 ingy ingy 8.4M May 18 15:02 big.yaml
$ ys -e 'slurp("big.yaml").yaml/load()'
Error: The incoming YAML document exceeds the limit: 3145728 code points.

Ingy döt Net 2024-05-18T19:27:35.479439Z

$ ys -pe '1024 * 1024 * 3'
3145728
🙂

Ingy döt Net 2024-05-18T19:28:05.032249Z

not sure what inspired that limit for snakeyaml

Ingy döt Net 2024-05-18T19:30:03.252929Z

I'm working on integrating a C++ parser (to replace snakeyaml-engine) into yamlscript and was testing on bigger inputs...

borkdude 2024-05-18T19:36:05.174839Z

The link I posted gave the answer though

borkdude 2024-05-18T19:36:11.113299Z

it's configurable

borkdude 2024-05-18T19:37:18.190849Z

not sure if you understood that, just double-checking

Ingy döt Net 2024-05-18T20:01:53.602329Z

Thanks. I missed that. I wouldn't have thought to look in the clojure code since it's a snake error. Makes sense though. Added less than a year ago it seems https://github.com/clj-commons/clj-yaml/commit/ab980163accdaf679090dc5ee93e7858e18768f0 @pitalig++

lread 2024-05-18T21:03:56.711269Z

Yeah clj-yaml is a very thin wrapper over SnakeYAML. It does not translate SnakeYAML exceptions.