If you look at the languages used in proof assistants that use dependent types they need something like this for type checking

Even if it were possible, it's probable that it would add a lot of bloat in terms of memory and efficiency

What you are talking about is some kind of value based equality, instead of reference equality

functions are already immutable

We're not talking about deciding whether two functions with different definitions have the same behavior, are we? CS theory tells us that is an undecidable problem, and you are likely aware of this, so I am guessing that is not what you are hoping for here.

The Clojure compiler doesn't currently save the S-expression used when defining a function in memory anywhere. If it were saved somehow with the function definition, then of course comparing those S-expressions for value-equality like other nested collections could be done.

Defining exactly what equality for "lexical environments" means seems like it could get tricky pretty quickly, perhaps, depending on what you wish for there.

For example, if direct linking is enabled in the Clojure compiler, then I'm pretty sure that defining a function that mentions another function by name f inside of it, then later redefining f in the REPL, then later defining a different function using the same expression as the first, would leave you with equal S-expressions, but different lexical environments.

However, if you did the same sequence of operations with direct linking disabled in the Clojure compiler, then I think the lexical environments of the two might be equal.

If you are interested in this topic then Unison language may be interesting to you https://www.unison-lang.org/learn/the-big-idea/

Unison was the inspiration for this question 😉

I don't believe that would work out for Clojure, that maintains a very strict idea of backwards compatibility (and it's a good idea to do so), but there is nothing to prevent this to become norm sometime in a Lisp far far away.

Is that the correct way to invoke require? What if you do (require 'my-app.extended)

require accepts the same arguments as for ns :require attribute. So it is ok to call it with '(my-app.extended)

there are few useful functions to inspect current state of defmulti, which is afaik mutable construction defined as defonce: methods , get-method , remove-method You can check if your multimethod have desired implementation during the test

sometimes it is useful to set defmulti var to nil and reload everything once again Normally I put (def multi-name nil) right above defmulti definition to reset its state via evaling whole namespace

For unknown reasons my test works when using @UE72GJS7J suggestion 😮 and stops working as soon as I add the parans again I used the parans because that is what the first example on https://clojuredocs.org/clojure.core/require tells me to:

;; Require  and call its file function:

user=> (require '())
user=> ( "filename")
#<File filename>

so, I guess there is a difference somewhere 😅 It isn't the first time these multimethods gives me the impression of being a bit fickle @U04V4KLKC thanks for sharing your tricks @UE72GJS7J thanks for the suggestion

The example on ClojureDocs is wrong, but it works because is already loaded. The docstring says that libspecs should be vectors or symbols, and lists are used for namespace prefixes. (require '()) doesn't do anything because there are no symbols or vectors after the prefix. But you could require it like (require '(clojure.java io)) where clojure.java is the prefix. For example using clojure.set which is not loaded already by the REPL:

Clojure 1.10.3
user=> clojure.set/rename-keys
Syntax error (ClassNotFoundException) compiling at (REPL:0:0).
clojure.set
user=> (require '(clojure.set))
nil
user=> clojure.set/rename-keys
Syntax error (ClassNotFoundException) compiling at (REPL:0:0).
clojure.set
user=> (require 'clojure.set) 
nil
user=> clojure.set/rename-keys
#object[clojure.set$rename_keys 0x1d035be3 "clojure.set$rename_keys@1d035be3"]

I updated the ClojureDocs example

It is getting better everyday... thanks for taking the time to not confuse noobs

also worth to file a question at http://ask.clojure.org. I think it is a bug in require to return normal result when args are badly shaped.

So the parens vs not parens is actually known behavior, and it would work as you want if you used a vector instead of a list.

The reason is that a list in a require like this is considered a prefix list, which is an obscure version of the require form that treats namespaces as hierarchical, which is why they are no longer encouraged to be used, since you are encouraged not to think of namespaces like this.

(require '(clojure.core
           [walk :as walk]
           [zip :as z]))

sometimes you want to extend a few protocols to the same type, sometimes you want to extend the same protocol to a few types

Neither is preferred - which you should use depends on the situations as described by @U0NCTKEV8

Java 19 Virtual Threads! My new favourite thing!

19? Let’s say Java 17 max 🙂

but you made me curious

I would use some form of java.util.CompletableFuture and have a list of them and invoke them all and wait for all of them

with virtual threads, id use structured concurrency. virtual threads are built exactly for this purpose:

(defn structured
  []
  (let [scope      (StructuredTaskScope$ShutdownOnFailure.)
        task1      (.fork scope
                          (fn []
                            (Thread/sleep 1000)
                            10))
        task2      (.fork scope
                          (fn []
                            (Thread/sleep 1000)
                            42))
        start-time (System/currentTimeMillis)
        _ (.join scope)
        _ (.throwIfFailed scope)
        end-time   (System/currentTimeMillis)]
    {:result  (* (.resultNow task1) (.resultNow task2))
     :time-ms (- end-time start-time)}))

all the tasks here wait in parallel and since its all IO bound in your case, they dont consume a whole thread each

in my case this IO operations are actually pretty heavy. They copy thousands of millions of rows from CSV to postgresql. I am actually not sure how much CPU it needs. I didn’t profile this yet.

this is hobby project….

Why java.util.CompletableFuture instead of some of clojure solution like promise or pmap?

id say still the network and waiting time would dominate over the CPU time. is its all in a single postgres transaction?

I don’t use transactions for this

but yes, probably postgresql is a bottleneck

clojure future does give you a completablefutue. id not use pmap as this is a chain of sideeffects and pmap has laziness and chunking; not a good combo with side effects i think

with doall of course

if its a lot of INSERTs highly recommend a transaction, both from reliability and perf reasons

Can you say more about that?

my idea was to have each functions to operate only on 1 table. So to 1 table will inserting only 1 function at a time.

There will be no situation where many thread will inserting to 1 table

Do I still need transaction to have good performance?

My intuition saying me it is the best simple solution, but it can be harder, than I think 🙂

okay so whats the max level of concurrency could you posisbly have?

I don’t know yet. This is a beginning of this project.

right now 3

i was saying transactions as each insert is wrapped in a transaction if ones already not started and if theres 1000s of inserts, batching them all into on txn is faster

But I think I will not cross 10 unless I will find a way to to make it faster and still keeping simple

yes, I do 2000 row insert in 1 query

that should do it too

so its 3 tables and 3 big insert queries is what i understand?

Do you have some experience in choosing the most performant number of rows in 1 insert query?

expectation if it should be 100 / 2 000 / 20 000

yes, right now 3 tables and each table has own feeder

I don’t think inserting to 1 table from 10 threads will make if faster. Maybe even slower. But I can be wrong.

not sure if theres a right number here. depends on the network bandwidth and data marshalling efficiency

Because if inserting to 1 table from 10 threads is faster, than inserting to 1 table from 1 thread, then I will do it in different way

yeah for me too i think being able to batch most into a single query/txn should be the most efficient then concurrent queries

Alternative approach - download the file, then import it from postgres

id say in this case since the question is about how much a query can fit rather than concurrency, try profiling each query first?

> Alternative approach - download the file, then import it from postgres I have to modify it a little before import

bummer

yes I will, but still the question about concurrency is valid

I mean all of this options are valid to run functions parallel

Just trying to think if I should prefer one on another for some reason

overthinking probably ;)

Few other things to consider - batching your inserts, and doing everything in one transaction

but start with just trying to run it single threaded and see how it holds up before you parallelize

I already did it

Did you time each section of execution?

no, it is proof of concept stage right now heh

so the way id approach is if the follwoing is true: • 1 table per csv • all rows can fit into a single insert without choking newtork have a fn which takes in data and runs the big insert make this fn return a future have a list of these futures and call (run! deref futures)

I think the most important part before doing any sort of optimization is measurement. Should give you pretty good indication where you should focus your efforts

and when you start multithreading you run into other issues, such as bottlenecks and contention

big +1, id time each query and find the network or marshalling bottlenecks

agree, the postgresql is side topic of this thread which became main topis heh

For example, if the files are large, you might be chocked on GC if you parallelize downloading the parsing them

I do lazy parsing CSV

So I'd say it's beneficial to understand the characteristics of each part of your pipeline and their resource requirements

otherwise OOM

That's good, but the downside is you hold on to all those delicious bytes you got off the network

so multiple files - more garbage, more OOM

Unless you stream the bytes in from the client

Which you can do with the builtin http client with LineConsumer (or something)

heh of course not. This are ZIP files which I have to unarchive.

Joy

but at least you can stream that part

@U7ERLH6JX why (run! deref futures) vs pmap or async or (Thread. ...). I am asking because I don’t have a stron opinion which one to use.

I have this feeling (doall (pmap ...)) do all what I need and care about number of thread, but hmm just looking strange

the problem is that different problems should be bound by different numbers of threads

im always a bit iffy against pmap and side effects as the paralleslism is uncontrolled and you never know

e.g. downloading IO bound, unzip and process is CPU bound and ingest is postgres bound

I would use three core async pipelines with different parallelism here

yes, but from I saw usually when software developers try to control number of threads by themselves it perform worst, than pmap. And machine can change in any moment, the CPU can be better / worst.

Classic ETL

so we dont want to control the threads per se but the number of things in parallel

but with (run! deref futures) it is as uncontrolled as pmap right?

its controlled by the count of futures

same as what Ben is saying with pipelines

it will be the same situation with pmap, because [f1 f2 f3 ....] will control thread by number of functions

almost the same like number of features

but here are functions

> I would use three core async pipelines with different parallelism here Can be, but not a overkill?

and again I have to control number of parallel N after change a machine. pmap will change it by automate.

well I think I will start from whatever. It shouldn’t make any difference in performance.

Thank you for your input

> Can be, but not a overkill? Possible, but I'm used to it

also see https://www.grammarly.com/blog/engineering/building-etl-pipelines-with-clojure-and-transducers/

well this one from Ben themselves tries to address similar feelings i have for pmapping side effects 😄 https://bsless.github.io/mapping-parallel-side-effects/

the But Why Not pmap? section

yeah the pmap is specific… although in work 1 software developer were trying to do async to increase performance for a long time… After that I removed all this code and I used pmap achieving better performance.

so while this is true pmap do things by automate and you don’t have control of it does it good

and usually this is not a problem you can’t control it

not always of course

> After that I removed all this code and I used pmap achieving better performance. probably the bottleneck wasnt in the code but outside like network or io

async is just hard and complex

well, nice plug to my first message: java 19 virtual threads, solution of all of this 😛

What Ben is talking about as async pipelines is formally done in the structured example i showed

but I have also another hobby project which is for synchronising bookeeping + warehouse + estore and there async is have to to strictly control side effects

I think while it is not the case (and usually is not) pmap if perfect

to overuse our favourite quote: pmap here is easy not simple.

hmm I see it as easy and simple

As long as software developer understand what pmap does (side effects) it is safe to use

not simple from the fact that its hiding a lot of machinery and isnt decoupled enough for reasons like error handling etc

heh I see it simple exactly because it does all this machinery under the hood and as a software developer you don’t have to write it yourself

thats the easy bit there

yes, but also I can do things in 1 line which make it simple

Exactly. pmap is easy but it is not "simple" in the Clojure definition of things -- you are conflating them.

pmap is easy but it is nearly always the wrong solution for production code. Learn to use the underlying platform's executors / thread pool stuff where you have a lot more control over individual pieces -- and you can tune them as a result of profiling/benchmarking.

Hmm when I can do a thing in 1 line it is simple for me. What make it complex in your opinion?

That has been explained repeatedly in this thread. You've been given a lot of good advice here but you just don't seem to want it...

> where you have a lot more control over individual pieces Agree, but this is not always needed.

pmap = laziness + lack of control over concurrency. You can get both too much concurrency in some situations and not enough in others, and you're relying on a whole bunch of machinery to maybe do the right thing (or maybe not). And your use case is about concurrency over two or three completely different processes, only one of which is likely to be CPU bound. They need different approaches.

oh I know why maybe we name pmap differently. It is complex function but it is simple to use. That is what I was meaning.

"It is complex function but it is easy to use." -- fixed that for you.

so far I didn’t see async which was simpler, than pmap in any way. But maybe I didn’t see good async project.

just as a summary which helps me: • pmap: okay for parallel unordered CPU bound tasks. hardly the case in a lot of prod code. not to be used for IO • everything else: a specific tool for a specific problem

@U0WL6FA77 In case you are wondering why there is some specificity being used here between "simple" and "easy", the following talk by Rich Hickey is considered one of the "core concepts" of Clojure https://youtu.be/LKtk3HCgTa8

writing correct async code and reasoning about it is very very hard. no doubts on that one. hence theres a lot of effort to address it, specially on the JVM side

let’s reverse it: show me simpler code for async, than pmap but NOT for a thing which you have to strictly control side effects like for example bookkeeping + warehouse + estore.

But your measure of "simpler code" is that it is easier for you to understand -- instead of learning how to do this properly.

no, it is the measure of things which I have to keep in my head during coding and maintenance this code later

like “what is happening in this code” make it complex too (but not in the sense of core functions which I don’t know, but just a complex code with complex flow)

You asked for advice. We gave you advice. If you don't want to take that advice and learn how to use the things we're suggesting, that's up to you. When you have the choice between Thing A that you already know how to use and Thing B which you don't, A is always going to seem easier. If you learn how to use B, it becomes easier for the future. I'm dropping out of this thread now.

sure, thank you

edit: (deleted, misread)

Clojurists mean very specific things when they use the terms "simple" and "easy". Most are happy to explain the distinction, as several people have already done. And if those explanations weren't clear enough for you, then watch the video @UG00LE5TN linked above. Trying to argue their meanings does not help you or anybody else, because it is shorthand used unanimously by the community for important concepts at the core of Clojure. Just learn it like you learn the name of a function, so you can get the most out of people trying to help you. There is nothing wrong with choosing something "easy" for expediency. Just realize that you are likely setting yourself up for future headaches, because you didn't choose "simple".

@U90R0EPHA To cut the topic off: Can you write the definition in a few sentences?

"Simple" describes a low level of actual complexity of a thing. "Easy" describes difficulty in getting a thing to do exactly the thing is was designed for. The two are not necessarily mutually exclusive. But easy is often the result of complex infrastructure built up underneath. So as soon as you want to do something ever-so-slightly different from the original intent, that complexity rises in front of you putting up roadblocks.

so what is complex in pmap for use case when you don’t have to control side effects order?

by your definition

I'm basically a beginner to Clojure as well. I have not had any reason yet to use pmap, so cannot really comment. But @U04V70XH6 is one of the most experienced Clojure devs you will ever get to talk to. So I suggest paying close attention to his take on it. > and you're relying on a whole bunch of machinery to maybe do the right thing (or maybe not)

Agree, @U04V70XH6 is a very good developer. I am not questioning it.

Stop @-ing me, both of you!

I said I was dropping out of this thread -- stop dragging me back in.

np

@U90R0EPHA I agree with that, but now one could give me any example or explanation what is complex in pmap for the use case where you don’t need to control side effects order and juice 101% of performance. That make a discussion less educative.

with all other cases I totally agree, it is even almost impossible to do things with pmap.

So in my feeling it is more about repeating common trues, than really focus on use case. But maybe I am the one who misunderstand what they wanted to say.

All in all I think it is more about misunderstanding both sides.

I got the distinct impression everyone was very focused on trying to understand your exact usecase, because the answer to your original question is "it depends".

And it sounds like you're pretty happy with pmap for now, so go for it. But, I guess you've been warned. Probably should bookmark this thread for when your usecase outgrows the "easy" of pmap. 😉

yeah, it is even too early for this code to optimise it in that sense to get the best performance. it is proof of work and even I don’t know what will be final need for it.

I think the main argue is: in my opinion pmap is simple or complex depends on use case, while all others opinion is pmap is always complex.

But again definitely not for all use cases. I hope I made it clear. For many uses cases pmap is a very bad choice and I would never use it.

If it's IO, number of processors doesn't matter, so definitely you shouldn't be using pmap. You can just use a bunch of futures on their own, unless you have like hundreds+ of them you're trying to do concurrently, then you probably want a queue, some executor services are good here or core.async.

But postgress itself will have a limit. Like how many concurent inserts to postress you think your DB can handle? Parallelizing on your side won't help you go faster then the single writer which is Postgress. In fact, you might slow it down further if you overload it. Once you figure that out, and I expect it's not that much, you can experiment with just futures. Like try with 30 futures, and see the load on postgress, and if it's too high go down to 20, or if not high enough go.uo to 40.

Basically, think about it, Postgress is like a highway entrance, trying to send too many cars to enter it at ounce you'll just get congestion. You want to saturate the lanes but not try to jam more cars then there are lanes.

we mix topics and there is misunderstanding here. Let’s make the thread end.

A namespace alias?

Is there a self namespace alias or something?

Like ::http/routes #(deref #'self/routes)` ?

just do the same thing you would do if you weren't using var-quote

the name in var-quote is resolved the same way as the name without var-quote

Let me be a bit more explicit: I wanna merge this namespace / file: https://github.com/pedestal/pedestal/blob/master/samples/hello-world/src/hello_world/service.clj With this namespace / file: https://github.com/pedestal/pedestal/blob/master/samples/hello-world/src/hello_world/server.clj

You can't alias the "self" namespace. But you can move that code to a common namespace and alias it instead.

You can see that the server.clj example uses an alias to service namespace

user=> clojure.reflect/reflect
Syntax error (ClassNotFoundException) compiling at (REPL:0:0).
clojure.reflect
user=> #'clojure.reflect/reflect
Syntax error compiling var at (REPL:0:0).
Unable to resolve var: clojure.reflect/reflect in this context
user=> (require '[clojure.reflect :as r])
nil
user=> clojure.reflect/reflect
#object[clojure.reflect$reflect 0x12a160c2 "clojure.reflect$reflect@12a160c2"]
user=> #'clojure.reflect/reflect
#'clojure.reflect/reflect
user=> #'r/reflect
#'clojure.reflect/reflect
user=>

It just seems dumb to do:

(ns sample.service.core
  (:require [io.pedestal.http :as http]
            [sample.service.core :as core]))

In order to do: ::http/routes #(deref #'core/routes)

Why do you even need to have the same exact code in that lambda, character for character? Let one place have core/routes and the other just routes if you don't want to move routes to a different ns.

First of all is just an example that I bumped into and using that I'm trying to have a more deep understanding of clojure namespaces and symbols 🙂 The whole POC is this:

(ns sample.service.core
  (:gen-class) ; for -main method in uberjar
  (:require [io.pedestal.http :as http]))

(defn hello-world
  [request]
  (let [name (get-in request [:params :name] "World")]
    {:status 200 :body (str "Hello " name "!\n")}))

(def routes
  #{["/greet" :get `hello-world]})

(def service {:env                 :prod
              ::http/routes        routes
              ::http/resource-path "/public"
              ::http/type          :jetty
              ::http/port          8080})

;; This is an adapted service map, that can be started and stopped
;; From the REPL you can call http/start and http/stop on this service
(defonce runnable-service (http/create-server service))

(defn run-dev
  "The entry point for dev"
  [& _]
  (println "\nCreating your [DEV] server...")
  (-> service ;; start with production configuration
      (merge {:env :dev
              ;; do not block thread that starts web server
              ::http/join? false
              ;; Routes can be a function that resolve routes,
              ;;  we can use this to set the routes to be reloadable
              ::http/routes #(deref #'routes)
              ;; all origins are allowed in dev mode
              ::http/allowed-origins {:creds true :allowed-origins (constantly true)}})
      ;; Wire up interceptor chains
      http/default-interceptors
      http/dev-interceptors
      http/create-server
      http/start))

(defn -main
  "The entry point"
  [& _]
  (println "\nCreating your server...")
  (http/start runnable-service))

In this POC, the run-dev function uses the deref #' trick to make routes reloadable in pedestal. The problem is that this does not work, because if you run the run-dev function from a dev repl that starts in the user namespace it tries to resolve the routes symbol in the user ns as user/routes, but I want it to resolve to the sample.service.core ns as sample.service.core/routes

The obvious answer is to change #(deref #'routes) to #(deref #'sample.service.core/routes) I was just wondering if there is a better way (given I want to keep everything in the same namespace)

I do not intent to keep it all that way, its mostly for learning purposes

if you want to use code is core, why is dump to require core?

Sorry the require core is mistype from before

Corrected

#(deref #'core/routes) is equivalent to #(do core/routes)

or (fn [] core/routes)

Ok still, how can I avoid core self reference/alias

once you start wrapping it in a function, you don't need var quote to ensure you get an updated value

not sure what you mean by self ref/alias

are you just trying to avoid a cyclic dependency in your namespaces?

Sorry in the end I may not understand the problem correctly

I'm trying to solve an exception when running the run-dev function

what is the exception?

That seems to be resolved If I replace ::http/routes #(deref #'routes) with ::http/routes routes

Gimme a sec

like, I haven't used pedastal, but they might only allow a single layer of function indirection

so if routes is a function, and your function returns its value, then you have a function returning a function instead of a function returning routes

But this is from the official hello world pedestal example: https://github.com/pedestal/pedestal/blob/master/samples/hello-world/src/hello_world/server.clj

sure, but I don't know what edits you might have made

but what is the exception you are getting?

I literally I just moved the def/defn s from here: https://github.com/pedestal/pedestal/blob/master/samples/hello-world/src/hello_world/service.clj to here: https://github.com/pedestal/pedestal/blob/master/samples/hello-world/src/hello_world/server.clj

Exception:

Error processing request!
Exception:

clojure.lang.ExceptionInfo: java.lang.NullPointerException in Interceptor :io.pedestal.http.route/router - 
{:execution-id 1, :stage :enter, :interceptor :io.pedestal.http.route/router, :exception-type :java.lang.NullPointerException, :exception #error {
 :cause nil
 :via
 [{:type java.lang.NullPointerException
   :message nil
   :at [java.util.regex.Matcher getTextLength "Matcher.java" 1770]}]
 :trace
 [[java.util.regex.Matcher getTextLength "Matcher.java" 1770]
  [java.util.regex.Matcher reset "Matcher.java" 416]
  [java.util.regex.Matcher <init> "Matcher.java" 253]
  [java.util.regex.Pattern matcher "Pattern.java" 1133]
  [java.util.regex.Pattern split "Pattern.java" 1261]
  [java.util.regex.Pattern split "Pattern.java" 1334]
  [clojure.string$split invokeStatic "string.clj" 225]
  [clojure.string$split invoke "string.clj" 219]
  [io.pedestal.http.route.prefix_tree$partition_wilds invokeStatic "prefix_tree.clj" 52]
  [io.pedestal.http.route.prefix_tree$partition_wilds invoke "prefix_tree.clj" 47]
  [io.pedestal.http.route.prefix_tree$contains_wilds_QMARK_ invokeStatic "prefix_tree.clj" 84]
  [io.pedestal.http.route.prefix_tree$contains_wilds_QMARK_ invoke "prefix_tree.clj" 80]
  [clojure.core$some invokeStatic "core.clj" 2718]
  [clojure.core$some invoke "core.clj" 2709]
  [io.pedestal.http.route.map_tree$router invokeStatic "map_tree.clj" 55]
  [io.pedestal.http.route.map_tree$router invoke "map_tree.clj" 51]
  [io.pedestal.http.route$eval11808$fn__11809$fn__11810 invoke "route.clj" 473]
  [io.pedestal.interceptor.chain$try_f invokeStatic "chain.clj" 54]
  [io.pedestal.interceptor.chain$try_f invoke "chain.clj" 44]
  [io.pedestal.interceptor.chain$process_all_with_binding invokeStatic "chain.clj" 171]
  [io.pedestal.interceptor.chain$process_all_with_binding invoke "chain.clj" 146]
  [io.pedestal.interceptor.chain$process_all$fn__10681 invoke "chain.clj" 188]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [io.pedestal.interceptor.chain$process_all invokeStatic "chain.clj" 186]
  [io.pedestal.interceptor.chain$process_all invoke "chain.clj" 182]
  [io.pedestal.interceptor.chain$enter_all invokeStatic "chain.clj" 235]
  [io.pedestal.interceptor.chain$enter_all invoke "chain.clj" 229]
  [io.pedestal.interceptor.chain$execute invokeStatic "chain.clj" 379]
  [io.pedestal.interceptor.chain$execute invoke "chain.clj" 352]
  [io.pedestal.interceptor.chain$execute invokeStatic "chain.clj" 389]
  [io.pedestal.interceptor.chain$execute invoke "chain.clj" 352]
  [io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15075 invoke "servlet_interceptor.clj" 351]
  [io.pedestal.http.servlet.FnServlet service "servlet.clj" 28]
  [org.eclipse.jetty.servlet.ServletHolder handle "ServletHolder.java" 799]
  [org.eclipse.jetty.servlet.ServletHandler doHandle "ServletHandler.java" 554]
  [org.eclipse.jetty.server.handler.ScopedHandler nextHandle "ScopedHandler.java" 233]
  [org.eclipse.jetty.server.handler.ContextHandler doHandle "ContextHandler.java" 1440]
  [org.eclipse.jetty.server.handler.ScopedHandler nextScope "ScopedHandler.java" 188]
  [org.eclipse.jetty.servlet.ServletHandler doScope "ServletHandler.java" 505]
  [org.eclipse.jetty.server.handler.ScopedHandler nextScope "ScopedHandler.java" 186]
  [org.eclipse.jetty.server.handler.ContextHandler doScope "ContextHandler.java" 1355]
  [org.eclipse.jetty.server.handler.ScopedHandler handle "ScopedHandler.java" 141]
  [org.eclipse.jetty.server.handler.HandlerWrapper handle "HandlerWrapper.java" 127]
  [org.eclipse.jetty.server.Server handle "Server.java" 516]
  [org.eclipse.jetty.server.HttpChannel lambda$handle$1 "HttpChannel.java" 487]
  [org.eclipse.jetty.server.HttpChannel dispatch "HttpChannel.java" 732]
  [org.eclipse.jetty.server.HttpChannel handle "HttpChannel.java" 479]
  [org.eclipse.jetty.server.HttpConnection onFillable "HttpConnection.java" 277]
  [org.eclipse.jetty.io.AbstractConnection$ReadCallback succeeded "AbstractConnection.java" 311]
  [org.eclipse.jetty.io.FillInterest fillable "FillInterest.java" 105]
  [org.eclipse.jetty.io.ChannelEndPoint$1 run "ChannelEndPoint.java" 104]
  [org.eclipse.jetty.util.thread.QueuedThreadPool runJob "QueuedThreadPool.java" 883]
  [org.eclipse.jetty.util.thread.QueuedThreadPool$Runner run "QueuedThreadPool.java" 1034]
  [java.lang.Thread run "Thread.java" 829]]}}
 at io.pedestal.interceptor.chain$throwable__GT_ex_info.invokeStatic (chain.clj:35)
    io.pedestal.interceptor.chain$throwable__GT_ex_info.invoke (chain.clj:32)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:57)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:44)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:171)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:146)
    io.pedestal.interceptor.chain$process_all$fn__10681.invoke (chain.clj:188)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1990)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1990)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:186)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:182)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:235)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:229)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:379)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:352)
...

...
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:389)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:352)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15075.invoke (servlet_interceptor.clj:351)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:799)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:554)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:233)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1440)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:188)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:505)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:186)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1355)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:127)
    org.eclipse.jetty.server.Server.handle (Server.java:516)
    org.eclipse.jetty.server.HttpChannel.lambda$handle$1 (HttpChannel.java:487)
    org.eclipse.jetty.server.HttpChannel.dispatch (HttpChannel.java:732)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:479)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:277)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:311)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:105)
    org.eclipse.jetty.io.ChannelEndPoint$1.run (ChannelEndPoint.java:104)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:883)
    org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run (QueuedThreadPool.java:1034)
    java.lang.Thread.run (Thread.java:829)
Caused by: java.lang.NullPointerException: null
 at java.util.regex.Matcher.getTextLength (Matcher.java:1770)
    java.util.regex.Matcher.reset (Matcher.java:416)
    java.util.regex.Matcher.<init> (Matcher.java:253)
    java.util.regex.Pattern.matcher (Pattern.java:1133)
    java.util.regex.Pattern.split (Pattern.java:1261)
    java.util.regex.Pattern.split (Pattern.java:1334)
    clojure.string$split.invokeStatic (string.clj:225)
    clojure.string$split.invoke (string.clj:219)
    io.pedestal.http.route.prefix_tree$partition_wilds.invokeStatic (prefix_tree.clj:52)
    io.pedestal.http.route.prefix_tree$partition_wilds.invoke (prefix_tree.clj:47)
    io.pedestal.http.route.prefix_tree$contains_wilds_QMARK_.invokeStatic (prefix_tree.clj:84)
    io.pedestal.http.route.prefix_tree$contains_wilds_QMARK_.invoke (prefix_tree.clj:80)
    clojure.core$some.invokeStatic (core.clj:2718)
    clojure.core$some.invoke (core.clj:2709)
    io.pedestal.http.route.map_tree$router.invokeStatic (map_tree.clj:55)
    io.pedestal.http.route.map_tree$router.invoke (map_tree.clj:51)
    io.pedestal.http.route$eval11808$fn__11809$fn__11810.invoke (route.clj:473)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:54)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:44)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:171)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:146)
    io.pedestal.interceptor.chain$process_all$fn__10681.invoke (chain.clj:188)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1990)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1990)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:186)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:182)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:235)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:229)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:379)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:352)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:389)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:352)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__15075.invoke (servlet_interceptor.clj:351)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:799)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:554)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:233)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1440)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:188)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:505)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:186)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1355)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:127)
    org.eclipse.jetty.server.Server.handle (Server.java:516)
    org.eclipse.jetty.server.HttpChannel.lambda$handle$1 (HttpChannel.java:487)
    org.eclipse.jetty.server.HttpChannel.dispatch (HttpChannel.java:732)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:479)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:277)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:311)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:105)
    org.eclipse.jetty.io.ChannelEndPoint$1.run (ChannelEndPoint.java:104)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:883)
    org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run (QueuedThreadPool.java:1034)
    java.lang.Thread.run (Thread.java:829)

Context:

{:response nil,
 :cors-headers
 {"Access-Control-Allow-Origin" "",
  "Access-Control-Allow-Credentials" "true"},
 :io.pedestal.interceptor.chain/stack
 ({:name :io.pedestal.http.impl.servlet-interceptor/ring-response,
   :enter nil,
   :leave
   #function[io.pedestal.http.impl.servlet-interceptor/leave-ring-response],
   :error
   #function[io.pedestal.http.impl.servlet-interceptor/error-ring-response]}
  {:name :io.pedestal.http.impl.servlet-interceptor/stylobate,
   :enter
   #function[io.pedestal.http.impl.servlet-interceptor/enter-stylobate],
   :leave
   #function[io.pedestal.http.impl.servlet-interceptor/leave-stylobate],
   :error
   #function[io.pedestal.http.impl.servlet-interceptor/error-stylobate]}
  {:name
   :io.pedestal.http.impl.servlet-interceptor/terminator-injector,
   :enter #function[io.pedestal.interceptor.helpers/before/fn--10888],
   :leave nil,
   :error nil}),
 :request
 {:protocol "HTTP/1.1",
  :async-supported? true,
  :remote-addr "127.0.0.1",
  :servlet-response
  #object[org.eclipse.jetty.server.Response 0x193edc94 "HTTP/1.1 200 \nDate: Wed, 02 Nov 2022 21:03:34 GMT\r\n\r\n"],
  :servlet
  #object[io.pedestal.http.servlet.FnServlet 0x4c3dbb8c "io.pedestal.http.servlet.FnServlet@4c3dbb8c"],
  :headers
  {"origin" "",
   "sec-fetch-site" "none",
   "sec-ch-ua-mobile" "?0",
   "host" "localhost:8080",
   "user-agent"
   "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36",
   "cookie"
   "visited-index=true; admin-auth=Basic%20YWRtaW46OUxjdjNSOUVxMkZ5ak02OHVaVm4h; initial-tab=; current-applet=23FA4143-22B4-49D7-BDC5-3ED54EDF69CB; wguser=anonymous; connect.sid=s%3AWi8UbBRQ7OA_MhqSjhDDAu9Bj4WHSVnG.4lqLECNJOlRXUOhkR7wNCcApZ13HieNY75Ld7ytmJM0",
   "sec-fetch-user" "?1",
   "sec-ch-ua" "\"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\"",
   "sec-ch-ua-platform" "\"Linux\"",
   "connection" "keep-alive",
   "upgrade-insecure-requests" "1",
   "accept"
   "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
   "accept-language" "en-US,en;q=0.9",
   "sec-fetch-dest" "document",
   "accept-encoding" "gzip, deflate, br",
   "sec-fetch-mode" "navigate"},
  :server-port 8080,
  :servlet-request
  #object[org.eclipse.jetty.server.Request 0x252fc77f "Request(GET //localhost:8080/)@252fc77f"],
  :path-info "/",
  :uri "/",
  :server-name "localhost",
  :query-string nil,
  :body
  #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x409c2b3b "HttpInputOverHTTP@409c2b3b[c=0,q=0,[0]=null,s=STREAM]"],
  :scheme :http,
  :request-method :get,
  :context-path ""},
 :enter-async
 [#function[io.pedestal.http.impl.servlet-interceptor/start-servlet-async]],
 :io.pedestal.interceptor.chain/terminators
 (#function[io.pedestal.http.impl.servlet-interceptor/terminator-inject/fn--15050]),
 :servlet-response
 #object[org.eclipse.jetty.server.Response 0x193edc94 "HTTP/1.1 200 \nDate: Wed, 02 Nov 2022 21:03:34 GMT\r\n\r\n"],
 :servlet
 #object[io.pedestal.http.servlet.FnServlet 0x4c3dbb8c "io.pedestal.http.servlet.FnServlet@4c3dbb8c"],
 :servlet-request
 #object[org.eclipse.jetty.server.Request 0x252fc77f "Request(GET //localhost:8080/)@252fc77f"],
 :io.pedestal.interceptor.chain/execution-id 1,
 :servlet-config
 #object[org.eclipse.jetty.servlet.ServletHolder$Config 0x5753cf44 "org.eclipse.jetty.servlet.ServletHolder$Config@5753cf44"],
 :async?
 #function[io.pedestal.http.impl.servlet-interceptor/servlet-async?]}

(broke it to 3 messages in order to fit)

are you in a fresh repl?

Yep

what is the value of routes?

Ah I just found out it must be some problem with the hello world example in the pedestal repo

It seems that other examples wrap it in route/expand-routes

https://github.com/pedestal/pedestal/blob/master/service-template/src/leiningen/new/pedestal_service/server.clj

I'm reporting this upstream, sorry for your time

#(route/expand-routes (deref #'service/routes)) is equivalent to #(route/expand-routes service/routes) if you have direct linking off, which if you are redefining things you likely want off anyway

and I think it is equivalent in the direct linking case too

(direct linking only effects direct function calls, and that is not one)

Thanks for your help, I'll look more into it!

Thanks also for your patience towards a clojure newbie!