Fork me on GitHub
#ring-swagger
<
2019-09-09
>
borkdude14:09:59

@niklas.collin I believe cheshire can do this lazily

Empperi14:09:32

Yeah could be. I ended up actually writing JSONL format which suits better in my use case

Empperi14:09:56

And for that jsonista works just fine since I write a new JSON object per line

borkdude14:09:13

There's also a command line tool which supports streaming: https://github.com/borkdude/jet

Empperi14:09:00

nice. Doesn't support Windows though, at least the installer doesn't 😜

borkdude14:09:27

is JSONL just writing json objects, one on each line?

Empperi14:09:33

And apparently it does graalvm native compilation

Empperi14:09:38

Yeah, it basically is

Empperi14:09:59

It's like "better CSV"

borkdude14:09:06

that's right, no pre-built Windows, but if you build one on your own machine it may work in Windows. Also you can maybe use it from the JVM, but in that case, I would just look at how streaming was implemented and copy it. But probably jsonista already supports it like that (passing a reader, read objects until you hit EOF).

Empperi14:09:09

and actually on windows powershell can do that stuff out of the box excluding edn/transit stuff of course

borkdude14:09:28

that's nice. there may also be a jq for Windows

Empperi14:09:41

I guess there is but can't see why I would use one 🤷

Empperi14:09:48

when native tools work just fine

borkdude14:09:28

right. can you show me an example of how you use it in Powershell? PS also runs on Linux and Mac nowadays

Empperi14:09:20

well, it of course totally depends on what one is doing...

Empperi14:09:24

but a random example:

Empperi14:09:07

(irm ).somePathInJsonObject.andAnother | select key1,key2

Empperi14:09:17

would handle json which would be of format:

Empperi14:09:14

{"somePathInJsonObject": {"andAnother": [{"key1": 1, "key2": 2, "key3": 3}, {"key1": 4, "key2": 5, "key3": 6}]}}

Empperi14:09:37

and naturally objects could have more keys and in this example it would retrieve the values in the array but only keys key1 and key2

Empperi14:09:04

and if one passes the result into Format-Table then it gets rendered as a nice table

Empperi14:09:18

but that is not very usable anymore since then data isn't objects anymore

Empperi14:09:30

then one would have to go to the *nix way of string manipulation 🙂

borkdude14:09:44

Ah. Just for fun I tried this with jet:

$ jet --from json --keywordize --query '[:somePathInJsonObject :andAnother (map (select-keys [:key1 :key2]))]' <<< '{"somePathInJsonObject": {"andAnother": [{"key1": 1, "key2": 2, "key3": 3}, {"key1": 4, "key2": 5, "key3": 6}]}}'
[{:key1 1, :key2 2} {:key1 4, :key2 5}]

borkdude14:09:21

thanks for showing the example

borkdude14:09:29

but tl;dr: lazily consuming elements of a top-level array is something cheshire supports, jsonista, don't know

ikitommi14:09:11

jsonista is 1:1 wrapper for Jackson Databind, which is not by default, streaming. Databind builds on top of Jackson Streaming could be exposed in jsonista. PRs most welcome.

borkdude14:09:57

yeah, I guess the most common use case is top level objects streaming which you now can already do with an inputstream or reader

ikitommi14:09:02

JsonFactory f = mapper.getFactory(); // may alternatively construct directly too

// First: write simple JSON output
File jsonFile = new File("test.json");
JsonGenerator g = f.createGenerator(jsonFile);
// write JSON: { "message" : "Hello world!" }
g.writeStartObject();
g.writeStringField("message", "Hello world!");
g.writeEndObject();
g.close();

// Second: read file back
JsonParser p = f.createParser(jsonFile);

JsonToken t = p.nextToken(); // Should be JsonToken.START_OBJECT
t = p.nextToken(); // JsonToken.FIELD_NAME
if ((t != JsonToken.FIELD_NAME) || !"message".equals(p.getCurrentName())) {
   // handle error
}
t = p.nextToken();
if (t != JsonToken.VALUE_STRING) {
   // similarly
}
String msg = p.getText();
System.out.printf("My message to you is: %s!\n", msg);
p.close();

ikitommi14:09:03

yes, true that. Does that apply also to writing a lazy sequence into outputstream?