clj-otel

grzm 2024-12-18T20:04:36.564139Z

Following up on https://clojurians.slack.com/archives/C034UN5273N/p1732131903644569, it appears I can indeed use autoinstrumentation, disable instrumentation for jetty, and then "add back" http server metrics using the clj-otel helpers. Here's fork that shows what I did https://github.com/grzm/clj-otel/tree/grzm/autoinstrument%2Bmanual-jetty.

clojure -M:otel:otel-no-jetty -m example.rpg-service.main
I also added grafana and friends because I wanted to confirm that as well.

steffan 2024-12-20T19:27:33.923229Z

Thanks for trying that out, it's useful to know that's a viable option 👍🏼

grzm 2024-12-18T23:55:03.635029Z

An issue I ran into today when using steffan-westcott.clj-otel.api.trace.http/wrap-server-span : When unit testing, we create a ring handler outside of the context of jetty (so we have to spin up an http server and worry about port conflicts and the like when running unit tests), and pass request maps directly to the handler. We don't populate the request map with :scheme or a "host" header, so server-request-attrs throws: (re-find #"^(.*?)(?::(\d*))?$" host) and (name scheme) both throw when those values are null. https://github.com/steffan-westcott/clj-otel/blob/7ead19eb1f8bfaaba230b328e6b00d252d5c6c57/clj-otel-api/src/steffan_westcott/clj_otel/api/trace/http.clj#L47-L63

(defn- server-request-attrs
  [request captured-request-headers]
  (let [{:keys [headers request-method scheme uri query-string remote-addr]} request
        {:strs [user-agent content-length host forwarded x-forwarded-for]} headers
        [_ host-name host-port] (re-find #"^(.*?)(?::(\d*))?$" host)]
    (cond-> {HttpAttributes/HTTP_REQUEST_METHOD (str/upper-case (name request-method))
             UrlAttributes/URL_SCHEME        (name scheme)
             ServerAttributes/SERVER_ADDRESS host-name
             UrlAttributes/URL_PATH          uri
             UrlAttributes/URL_QUERY         query-string
             ClientAttributes/CLIENT_ADDRESS (client-ip forwarded x-forwarded-for remote-addr)}
      user-agent      (assoc UserAgentAttributes/USER_AGENT_ORIGINAL user-agent)
      content-length  (assoc HttpIncubatingAttributes/HTTP_REQUEST_BODY_SIZE
                             (parse-long* content-length))
      (seq host-port) (assoc ServerAttributes/SERVER_PORT (parse-long* host-port))
      captured-request-headers
      (merge-headers-attrs "http.request.header." captured-request-headers headers))))
I'm not sure what an appropriate resolution to this would be: from the standpoint of testing our application, :scheme and and the host header aren't interesting, so I don't want to add them just to satisfy monitoring. otel metrics expect SERVER_ADDRESS (recommended) and URL_SCHEME (required) to be populated (https://opentelemetry.io/docs/specs/otel/semantic-conventions/#in-development-reserved-attributes, https://opentelemetry.io/docs/specs/semconv/http/http-spans/#http-server-semantic-conventions), so making them conditional in the server-request-attrs function can silently fail those expectations. Currently I've made the inclusion of wrap-server-span in the our middleware configurable, and that should keep for now. And perhaps this last is the expected solution.

steffan 2025-06-19T16:20:58.438229Z

@grzm Update: I've just made a https://github.com/steffan-westcott/clj-otel/commit/29d8adbc0e1242307242482d6a45d22db9efcadd to improve the processing of potentially incomplete Ring request maps. This is intended to support use cases like yours outside of production.

grzm 2025-06-19T16:40:14.808539Z

Nice! Thanks!

steffan 2024-12-20T18:32:55.593609Z

According to the https://github.com/ring-clojure/ring/blob/master/SPEC.md, :scheme is required on all requests. According to the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Host on MDN, Host must be present for valid requests.

grzm 2024-12-20T19:14:58.869399Z

That makes sense. I'll update our tests to fill in these values. Cheers!