Clojurians Log v2

when?

Seems to be slated for 28

Status: Integrated

great

unbelievable it took so long

Quite unlike the class mapper style like the other external libs, so quite nice

oh yeah, the mapping classes "problem". how are they proposing to do it?

They don't have it as far as I can see, just the flow style methods on a JsonValue

I guess it's an exercise left to the user or just use the existing object mapper libs. Don't think it matters much to Clojurians?

Maybe people have to implement a JSONValue interface or so ?

They call out Python and Go for their similar approach

We often, however, just need to perform simple tasks such as extracting some data from a JSON document. The Python or Go code to accomplish such tasks is simple; the Java code should be equally simple.

Clojure has classes too, e.g. PersisntentHashMap. I guess you can manually convert these into JSONValue or perhaps they offer some extension mechanism?

Don't see any extension mechanism, they just map to the standard j.u.Map and List, so technically can work with Clojure things

(defprotocol ToJson
  (->json ^JsonValue [x]))

(extend-protocol ToJson
  nil                        (->json [_] JsonNull/NULL)
  Boolean                    (->json [b] (if b JsonBoolean/TRUE JsonBoolean/FALSE))
  String                     (->json [s] (JsonString/of s))
  clojure.lang.Keyword       (->json [k] (JsonString/of (name k)))
  clojure.lang.Symbol        (->json [s] (JsonString/of (str s)))
  Number                     (->json [n] (JsonNumber/of n))   ; check the overloads, see below
  java.util.Map              (->json [m]
                               (JsonObject/of
                                (persistent!
                                 (reduce-kv (fn [acc k v]
                                              (assoc! acc (if (keyword? k) (name k) (str k))
                                                      (->json v)))
                                            (transient {}) m))))
  java.util.Collection       (->json [c] (JsonArray/of (mapv ->json c))))

(defn write-str [x] (.toString (->json x)))

something like that

Perhaps extension is a later feature. Current things are a good feature to get out sooner I think

yeah using a clojure protocol you don't need the extension, we can do it using their small API

I did some benchmarks using the above implementation. It seems 1.5x slower than clojure.data.json

I guess they can speed it up in future JDKs

Yeah seems like a small incremental incubating release

But progress!

I don't like the API of the builtin one - and not for the same reason I think most people do

I wish they provided an API for encoding/decoding (like my library does! Wah wah wah! https://GitHub.com/bowbahdoe/json - but seriously)

you know what. maybe clojure.core should just add clojure.data.json in core. it's already tiny and pretty fast

Did clojure.data.json ever fix its read ahead problem?

remind me with a repro

On phone, but it read from whatever data source in chunks. So if you read from a file that makes it faster, but it you want to read off a socket it causes a deadlock

I know because I based my lib heavily on clojure.data.json and students couldn't use it for their Socket communication homework until I changed that

https://github.com/clojure/data.json/blob/94463ffb54482427fd9b31f264b06bff6dcfd557/src/main/clojure/clojure/data/json.clj#L164

Doesn't seem like it

We switched to https://github.com/mpenet/oda to get out of the Jackson pile. So many things had strong opinions about the Jackson version that it became a pain in the neck. It’s been running in production for a couple of months, with only one complaint (since fixed upstream).

@emccue Is there already a JIRA for this?

I forget

I think this would be the simplest possible repro:

(let [w (PipedWriter.)
      r (PushbackReader. (PipedReader. w) 64)]
  (.write w "{\"n\":1}\n{\"n\":2}\n")
  (.flush w)
  (println (json/read r))
  (println (json/read r)))  ; hangs on 2.5.1, prints {n 2} on 1.1.0

public class HangingReaderTest {
    @Test
    public void hangingReaderTest() throws IOException, InterruptedException {
        var read = Collections.synchronizedList(new ArrayList<Json>());

        int serverPort = Math.max((int) (Math.random() * 10000), 1000);

        try (var serverSocket = new ServerSocket(serverPort)) {
            var serverThread = new Thread(() -> {
                try {
                    var client = serverSocket.accept();
                    var is = client.getInputStream();
                    var os = client.getOutputStream();

                    var reader = Json.reader(new InputStreamReader(is));
                    var writer = new OutputStreamWriter(os);

                    while (true) {
                        var ping = reader.read();
                        if (ping != null) {
                            read.add(ping);
                            Json.write(
                                    Json.objectBuilder()
                                            .put("pong", "yes")
                                            .build(),
                                    writer
                            );

                            writer.flush();
                        }

                    }

                } catch (IOException e) {
                    e.printStackTrace();
                    throw new UncheckedIOException(e);
                }
            });
            serverThread.setDaemon(true);
            serverThread.start();

            Thread.sleep(100);

            try (var client = new Socket("0.0.0.0", serverPort)) {
                var is = client.getInputStream();
                var os = client.getOutputStream();

                var reader = Json.reader(new InputStreamReader(is));
                var writer = new OutputStreamWriter(os);

                for (int i = 0; i < 5; i++) {
                    Json.write(
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            writer
                    );
                    writer.flush();
                    var pong = reader.read();
                    read.add(pong);
                }
            }

            assertEquals(
                    List.of(
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("pong", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("pong", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("pong", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("pong", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("ping", "yes")
                                    .build(),
                            Json.objectBuilder()
                                    .put("pong", "yes")
                                    .build()
                    ),

                    read
            );
        }
    }
}

here was the unit test I have to test it

https://clojurians.slack.com/archives/C02165ZDU9E/p1791294616329219

yeah here is the thread where i brought it up last https://clojurians.slack.com/archives/C03S1KBA2/p1730645579973689?thread_ts=1730644448.196279&cid=C03S1KBA2

String body = ... REST response body, which is a JSON document ... ;
JsonValue json = Json.parse(body);
json.get("properties").get("periods").asList().stream()
    .mapToInt(j -> j.get("temperature").asInt())
    .average()
    .ifPresent(IO::println);

{
  ...
  "properties": {
    ...
    "periods": [
      {
        "number": 1,
        "name": "Today",
        "temperature": 54,
        "temperatureUnit": "F",
        ...
      },
      {
        "number": 2,
        "name": "Tonight",
        "temperature": 48,
        "temperatureUnit": "F",
        ...
      },

The Java JSON API doesn't support streaming at all btw

what do you mean?

did you read the rest of the convo?

i did. i’m just posting the proposed API from the jep. so confused what you mean the JSON API doesn’t support streaming

yeah, one of several complaints. I don't mind not supporting streaming values for efficiency, but not being able to read forms off a Reader means theres a lot of student things it can't be used for

ah gotcha. stream input

@dpsutton to color my other complaint, in the json lib I have the above example would be this:

Json json = Json.read(body);
field(json, "properties", field("periods", array(int_())))
    .stream()
    .mapToInt(Integer::intValue)
    .average()
    .ifPresent(IO::println);

since I reify the Json -> X procedure as a JsonDecoder and the X -> Json procedure as a JsonEncoder/`JsonEncodable`

which lets you do things like this

Json json = Json.read(body);
List<Integer> periodInts = field(json, "properties", field("periods", array(int_())));
List<Duration> periods = field(json, "properties", field("periods", array(int_().map(Duration::ofMillis))));

its slightly more awkward for the common case but much more composable for the less common ones

then the ways to handle null in their proposed api stink, etc.

When 28 has this in preview my goal is to have students try it to gather evidence for my case

You have minions students? That’s very handy.

I have a stream of them via some coding help discords

Ah, cool. Sorry, off topic.

(god the degree to which AI lobotomizes them)

@henrik we are in #C03RZGPG3

Hang on, this is Russell’s barber paradox, isn’t it? A person is only on topic of they’re off topic, something something.

Stop being swedish and enjoy the chaos

All hail Eris Hail Discordia

It fares better on the Desired and Admired graph: https://survey.stackoverflow.co/2026/technology/data#admired-and-desired

Off-topic to the off-topic: Does the "Lisp" (sitting at 3% vs 1% for Clojure) in the survey refer to "Common Lisp" or some collection of Lisps?

If it is just Common Lisp, that may better reflect the ability of certain communities to "rally the troops" to complete the survey. With a few hundred respondents each, it wouldn't be hard to skew the results. I would be surprised if Common Lisp is actually more widely used than Clojure nowadays.

This might be too heavy weight for you, but maybe you can get some inspiration from Leslie Lamport's TLA+. It's based on simple mathematical notation, describing valid states and state transition. The /\ and \/ correspond to logical AND and OR

Trans(from, to) ==
  /\ InTD(state, from)
  /\ state' = to

Init == state = "LogOut"

Next ==
  \/ Trans("LogOut", "Main")
  \/ Trans("Main", "Settings")
  \/ Trans("Settings", "Main")
  \/ Trans("LogIn", "LogOut")
  \/ Trans("LogIn", "Report1")
  \/ Trans("Report1", "Report2")
  \/ Trans("Report2", "Report1")
  \/ Trans("Reports", "Main")

Spec == Init /\ [][Next]_state

there's a lot more to TLA+ than just notation, but I'm just suggesting learning a little bit about it and see if using predicates on states and transitions could be useful in your case

That is a pretty long winded way to write a map between states. It depends a lot on what properties of a state machine are important to emphasize. If you want to say something fsm like, you can do a map with keys as pairs of states x inputs to outputs (or nested maps)

There is https://en.wikipedia.org/wiki/Turtle_(syntax) for expressing graph-like data. The "T" in Turtle stands for "Terse" but it might not be sufficiently terse for this application. You could even use the https://github.com/quoll/raphael to parse Turtle if you want to build a DSL in Clojure to define the state machine.