I looked at compojure.api.sweet it looks like the api function returns a record (like a hash map) so I'm a bit puzzled about how that's supposed to work (I've never used compojure-api).

You'll have to find someone who's familiar with that library. The maintainers are in Europe I think so when they come online they might be able to help you (right now it's 4:30 am for them I think).

okay, thanks for the response, will wait for them

@ikitommi I get that api already has JSON middleware, but the post api that I want to make is not supposed to have pre-defined body structure other than some fixed key terms, and this piece of code is giving me an exception when I try to make a post request, is there a way to handle this case without specific body structure

(ns dice_api.handler
  (:require [compojure.api.sweet :refer :all]
            [ring.middleware.json :refer [wrap-json-response wrap-json-body]]
            [dice_api.schema :refer [roll_value]]
            [dice_api.dice :refer [roll_dice]]
            [schema.core :as s]
            [ring.util.response :refer [response]]))

(defn post-handler
  ""
  [req]
  (let [n (get-in req [:body :n])
        s (get-in req [:body :s])]
  (response (roll_dice n s))))

(def api-routes
  (api
    {:swagger
     {:ui "/"
      :spec "/swagger.json"
      :data {:info {:title "Dice-api"
                    :description "Compojure Api example"}
             :tags [{:name "api", :description "some apis"}]}}}
    (context "/api" []
      :tags ["api"]

      (GET "/roll/:n/:s" []
        :path-params [n :- s/Int
                      s :- s/Int]
        :return roll_value
        :summary "Rolls :n dice of sides :s"
        (response (roll_dice n s)))

      (POST "/roll" request
        :summary "roll with body params"
        :return roll_value
        (post-handler request)))))

(def app
  (-> api-routes
      (wrap-json-body {:keywords? true})
      wrap-json-response))

The record implements IFn with 1 & 3 arities, works just like a ring handler.

Ah... OK. Definitely not clear from the docs. Any idea why @puru072’s example doesn't work?

Indeed, the same level can also occur on different 'branches'

partition-by maybe?

I am thinking that perhaps using reduce in some way to produce a 'generic tree structure' from the input, where a new element [level other-stuff] is added as a new right sibling to the last node that was seen with the same value of level , if there was any, and if not, I am guessing the new level value must be one greater than the previous value, and becomes a child of the previous node. One could implement a sanity error check that every value of level was either one more than the previous one, or less.

this makes sense. I was thinking reduce might be the way I have to go.

Maybe, reduce and check the last element of the accumulator and depending on the relation between levels, push it as a child or as a sibling. Edit: I think that's what you were saying @U0CMVHBL2 - I am still waking up. Thanks for the input.

Yes, at that level of detail, that is what I was thinking. Code is more specific than English, of course, but sounds like you have the same basic idea.

If you always have a correct tree structure for the N-1 elements seen so far, and if you have an efficient way to "walk the rightmost branches" down "level" steps deep and add a new leaf there, then that is what each single reduce step could do.

Sounds like a candidate for 4clojure

From the gist I posted - is the example of using recursion "by function name" (for a lack of better words?) ... not idioimatic / or potentially problematic? I didn't quite get the solution, but I got close. I'm not super familiar with recursion capabilities of clojure, but I recall folks saying that certains means of doing it are supported while other's aren't. (clj-kondo flagged one of my usages as not correct).

If you know that your recursion depth is limited to a reasonably small amount, e.g. a few hundred deep or so, then I wouldn't worry about it. If your recursion depth is one per element of a sequence/list/vector, and you want your code to be able to handle sizes of those things that are tens of thousands of elements long, then you will likely exceed default JVM stack depth limits.

loop/recur does not have that limit

Got it. Thanks again!

Does some combination of line-seq and drop do what you need?

I'll take a look at that, thanks!

In general, the beginning of line N in a file could be anywhere from N bytes from the beginning of the file, to any arbitrary position later.

Unless someone has a pre-built index of the starting position of some of the lines of the file, I do not see how you can avoid reading the entire file up to the point you want.

Or if you somehow know that all lines have a particular length in bytes.

Oh I see. You need to parse the whole thing so until you've counted the correct number of \n's.

Gotcha.

Right, unless you know more about the structure of the file than simply 'it is a text file'

If you have control over the way the columnated data is generated, you might be willing to assume that all lines have X bytes, but other than that you have to assume the line lengths could be anything.

Sounds good.

There's a reason why people prefer databases for large data 🙂

If you want 'random' access to lines of a text file, and your file system gives you a quick way to jump to byte offset X within a text file, as I think most file systems do, then building and maintaining an index like "line 100 begins at byte offset 39104, line 200 begins at byte offset 77025, etc." would let you drastically reduce the number of bytes that you would need to scan to find the start of the line you are interested in.

You can trade off the number of lines you keep that info about in the index, versus the worst case scan time needed to find the line you want.

Java has support for every version of this you might want, either via streaming Readers, RandomAccessFile, or memory mapped files with MemoryMappedBuffer

Clojure does not really wrap any of that stuff but it's trivial to use via interop

Yes, people prefer databases for large data

RandomAccessFile sounds like the thing I would want

Andy & Alex - thank you for taking the time to answer! I appreciate it 🙂

Hey you mention in the beginning of the video that there are classes of queries in sql that are impossible or very hard to write. Do you have any example of those? I'm also interested in shortcoming/trade-offs of Datalog? I find it particular helpful to know about the downsides of a technology and always appreciate hearing from experts which challenges they have met, using those.

the reason for io/resource is to allow treating files on disk during development the same as resources inside a jar when deployed, without conditionals in your code

it's meant for the kind of thing that the app carries with itself

thanks, that makes sense!

future-cancel is not reliable, it's opt in, you need to ensure anything that should be cancellable do one of the things that future-cancel can effect

(sleep or wait on IO, more or less)

otherwise, the task can check if the thread it is in is cancelled and exit gracefully

but there's no reliable way to simply kill it from the outside

Ahhhh. Thanks! That sounds exactly like it would cause the issue. The subtasks I'm running are process-heavy, not sleep/io.

you can add a check of (.isInterrupted (Thread/currentThread)) to the loop / process https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Thread.html#isInterrupted()

future-cancel will make that return true

Hi! Thanks for pointing that out! 🙂

I actually got here by learning "group-by"

But you're right, its not sorted by month.

So say you come across an interesting repo on github or whatever. Do you clone it and start up a REPL (REBL too?) instance in your editor to start exploring it? Just reading the code on github hasn't worked for me because I can't understand the inputs. (please see below)

When someone's asking about a library on Slack, I'll often run clj -Sdeps '{:deps {the-library {:mvn/version "RELEASE"}}}' which starts a REPL with just that library as a dependency and then I'll play with it to see if I can answer their question. I might also add -A:rebl-11 to that (an alias from my dot clojure file) so I get REBL started up -- Cognitect's data/code visualizer. I agree that reading code on GitHub isn't always enough -- but with practice it definitely gets easier over time. It can be a hard slog when you're getting started...

I like that strategy. That one liner definitely makes it seem a lot more frictionless in getting my hands dirty quickly with an unfamiliar repo

I just can't form a mental picture of what the function inputs are like I can with a type annotated language. Just today, when looking through various projects I was interested in, I am seeing function parameters like `request` (which I know with web dev now usually means a hashmap), `req` (same), `params` , `rules`, etc.

None of these functions have doc strings which I've found is quite common. It leads me to believe that I just don't have the understanding that other Clojure users do about what these parameters probably look like.

t's honestly caused me to leave Clojure a few times. I always come back because it is by far my favorite language to actually write code in (and this slack community has been my favorite so far, tbh). I just can't read anybody's code (including my own sometimes months later)!

There are some conventions that help here. But because clojure is so liberal with the shorthands and shortcuts it lets you take in writing code, an undisciplined codebase can become unreadable.

I often end up resorting to capturing the data inside the function call, and then investigating it in the same manner @seancorfield describes above

Which I get. I know the common ones like m for map but even in the most popular Clojure projects like Ring I don't really know exactly what dir-path is for an input

I'm assuming I have to get better (using pattern recognition?) at looking at the let bindings or destructuring to parse what that input probably was?

the pattern often looks like: • change (defn f [arg] (frob (munge arg))) to (defn f [arg] (def f-arg arg) (frob (munge arg))) • run the code that calls f • run something like (-> f-arg type) • based on that, either (-> f-arg keys) or (-> f-arg first type etc. • iterate on thes calls, adding different functions at the end of the -> chain until I understand what I'm seeing in context

reading the code definitely helps too, but the data in the running app can't mislead you in the ways that sloppy code might

I think that is the direction I have been leaning. Stop expecting to just read code on github and understand it. Actually get my hands dirty in the REPL (well, using my editor) and maybe trying this REBL thing out.

I think I need to do the same

also, there are some idioms of function composition and data destructuring that one simply learns through seeing them repeatedly

It's really not just the "sloppy" code I am struggling with. It is "good" code from well respected Clojure members. I think I just need to work on understanding what the function is doing with the inputs to actually see what the input is if that makes sense. It's just different than when I use other languages and can see ok, well this is a vector of strings.

if you find a question like this for a library, file an issue so the authors can improve the docs. it's totally reasonable to ask for a docstring or more info.

eg. (fn [[k v]] ...) is an idiomatic function on a key/value pair of a map

Yeah I think upping my destructuring skills will help in a huge way. I'm almost wondering if I should just assume the input is a map until I figure out otherwise.

https://clojure.org/guides/destructuring

@U9J50BY4C I definitely have encountered the problem you’re describing, even more so with some JavaScript projects (like React Native’s metro); downloading the project and inspecting it locally definitely helps; familiarity with new language syntax definitely helps but up to a point; understanding certain fn paramaters might only come with a “bigger picture” understanding of the whole project; this is a long-winded way of saying that sometimes there’s just no easy shortcuts to understanding a big project, no matter how experienced you are 🙂

Now that I'm looking back at the docs from my original question: http://pedestal.io/api/pedestal.service/io.pedestal.http.html#var-create-server

I see the map is described in better detail down the page at default-interceptors

all things being equal, the average Clojure project is more understandable to me, because I think most Clojure projects are fundamentally simpler and better structured than the average JavaScript project

I guess I just thought there would be some kind of clojure spec or something in the http namespace codebase

So in what you just posted, William, I am seeing some of the inputs are obj and it just says this will do something to an obj. But what is an obj?! I think I need to get more comfortable with the generic dynamism (?) going on here.

The param I'm referring to is service-map

on the jvm, Object is the superclass of (nearly) all values, most clojure functions are technically type Object ... -> Object

in context - "edn-response" - obj would be any datum that edn can turn into text

Does clojure spec not provide a way to say that service-map is going to be map with these keys

it provides a way to say that, spec is inconsistently used and pedestal as a project predates spec by years

ahhh ok gotcha

I just assumed it would be used there if it could be since pedestal is used in tutorials everywhere 🙂

if they added a spec they'd have to account for the previous ways it perhaps accidentally worked

Thanks for the clarification!

I'm glad I didn't steal everyone away from your question William! Thanks for the help everyone. I'm going to keep plugging away

@U015JG247UZ Re: Pedestal -- I've been building web apps and server processes with Clojure for about a decade and I've never used Pedestal. I get the impression it is not particularly beginner-friendly.

What would you recommend?

Depends what you're trying to build -- I don't do any ClojureScript stuff at all (we looked at it about six years ago and decided to go with JS for our front end back then -- a lot has changed since, and if we were starting over we would probably try it again).

Our web apps are mostly just Ring / Compojure (one is Ring / Bidi), running mostly on Jetty (one runs on Netty).

For SSR (server-side rendered) apps, we use Selmer for templating.

Cool I'm writing microservices on jvm primarily right now

We don't even bother with compojure-api, but if we had to generate Swagger/docs we probably would.

We just try to keep our stack nice and simple, so it's easier to reason about and easier to debug.

I'm rewritting an existing api with the goal of simplicity/readability being a high priority.

Right now it's a dropwizard app and requires hunting down docs for days to try and change any of it's core config/behaviors

You kind of just get used to it. You develop a kind of tacit intuition about it. And like other said, you read the implementation code, and try it in the REPL

Function signatures in Clojure can be way more powerful then in other languages. They almost become like mini-dsl of their own. So definitely do expect to spend a bit more time with them

I keep running into !!ERROR: Use of undeclared Var circles.components/circle

If I need to publish a repo so its easier to debug, lmk

there is no namespace circles.components defined here

there's circles.components.circle which has a var circle in it

so you could (:require [circles.components.circle :as c]) and then [c/circles] should work

Ahh I see, that seems to have worked :thumbsup:. Thank you

Are there any best practices around component organization? Or does it vary across projects / organizations

sorry just seeing this. don't fret about it too much. see what issues you run into. i think best practices when you're just starting out can often impede making sure