Fork me on GitHub
#yada
<
2016-12-30
>
vinnyataide02:12:06

@malcolmsparks care to explain how asciidoc is compared to orgmode?

martinklepsch08:12:34

I'm having some troubles serving resources from the classpath. I tried new-classpath-resource as discussed here: https://clojurians-log.clojureverse.org/yada/2016-02-11.html but a request to / returns a 405.

["" (yada/yada (cpr/new-classpath-resource "public"
                                           {:index-files ["index.html"]}))]

dominicm10:12:08

@martinklepsch does "public/" work? Nvm, misread the source

dominicm10:12:19

There may be a bug in the code, just based on a cursory read. Not entirely sure how path-info works, but we first remove #"^/" (as in the / request...) and then only load the index file if the path ends in /.

dominicm10:12:55

A good test for this bug would probably be checking if this works when nested.

malcolmsparks11:12:24

this has been reported before, Jannis Pohlmann contributed this code

martinklepsch11:12:21

I read the subresource docs but didn’t see my issue.

martinklepsch11:12:43

Somehow the issue solved itself

malcolmsparks11:12:28

hmm. don't like issues that solve themselves...

malcolmsparks11:12:37

there must be something odd going on

martinklepsch11:12:42

I’ll ping back if it comes up again. Might have been related to my routes as well

malcolmsparks11:12:06

I'm not completely happy with the way things are with classpath resources right now. Bidi has this thing called ResourcesMaybe that allows the pattern matching to continue if a resource isn't found

malcolmsparks11:12:28

but bidi's ResourceMaybe isn't as complete as a yada resource

malcolmsparks11:12:21

ideally it should be possible to get the best of both worlds, but needs some thought. The separation of bidi and yada is quite a key principle for me

malcolmsparks11:12:09

btw. there's lots in yada that I'm not happy with, and I'm slowly improving yada but being much more careful to preserve compatibility for existing users

malcolmsparks11:12:29

@vinnyataide really needs a blog article, but in summary they are similar at the back-end and both can produce excellent configurable PDF output, xrefs, auto-numbering, toc, etc.

malcolmsparks11:12:18

that's if you decide to go with TeX (rather than FOP) for the backend, which I believe it still unmatched in various areas (like hyphenation rules)

malcolmsparks11:12:42

At the front-end, they are different obviously - orgmode's biggest disadvantage, imo, is that it's informally defined, org really relies on Emacs as an editor (although you can edit it with something else), and it's a bit of a black art

malcolmsparks11:12:58

The key benefit of asciidoc is that it has an extra tier in the pipeline - DocBook XML . This makes it very configurable - you can use XSLT to customize any existing behaviour - XSLT is a wonderful thing, very under appreciated

malcolmsparks11:12:30

Asciidoctorj does a good job of rendering to HTML natively (and dynamically if needed, we do that on our http://juxt.pro website)

malcolmsparks11:12:21

But also asciidoctorj does a good job of turning AsciiDoc into DocBook XML (I use DocBook XML 5 because it's all XML, having shed the last vestiges of its SGML history)

malcolmsparks11:12:45

DocBook is very mature, has a bigger community than Emacs/org-mode

malcolmsparks11:12:00

O'Reilly and other publishers use it too, so it's well trodden

malcolmsparks11:12:55

AsciiDoc and DocBook are isomorphic, that's the point - so you get the power of DocBook without needing to hand-author XML, which was the real pain with DocBook authoring

malcolmsparks11:12:00

Best of both worlds

malcolmsparks11:12:34

For those who aren't familiar with DocBook - http://docbook.org/ -it's been around forever and came out of the SGML world for publishing technical books - it has great support for the features you need when writing technical documentation

malcolmsparks11:12:57

No idea what any of this has to do with yada, apologies if you're not interested in this stuff 🙂

malcolmsparks11:12:35

But if anyone is interested, the PDF publishing toolchain is in the yada repo under doc/ (or atleast will be soon when I've pushed to github)

martinklepsch12:12:49

Now when doing curl localhost:4000 I get the 405

malcolmsparks12:12:41

The problem with new-classpath-resource is that when there is no 'path-info' (the part of the path that remains after the rule has been matched), then the sub-resource logic is not called - you just get the top-level :methods decl which is empty, hence it's a 405 (no such method)

malcolmsparks12:12:51

this has always been a problem with this code

malcolmsparks12:12:07

the workaround is to place a redirect at "" which targets some index file which can be served by new-classpath-resource

malcolmsparks12:12:25

there's yada/redirect, or bidi has a redirect capability, or just a ring thing

malcolmsparks12:12:49

new-classpath-resource can't do redirecting to index files for you, at least not at the root

malcolmsparks12:12:13

or we can just fix new-classpath-resource now to do the right thing

malcolmsparks12:12:47

which would be to redirect to the value of :index-files by default

malcolmsparks12:12:23

but I suggest the workaround for now because currently I'm trying to get quite a big amount of holiday yada work pushed

malcolmsparks12:12:25

I don't think an empty path is valid, one will always be appended to a URL by your browser or user-agent

malcolmsparks12:12:34

GET / HTTP/1.1

malcolmsparks12:12:45

you obviously cannot have the following HTTP request

malcolmsparks12:12:56

that would violate the HTTP standard

malcolmsparks12:12:08

So what's going on here is that there's a / matching

malcolmsparks12:12:36

the "" of your new-classpath-resource rule matches, and "/" becomes the path-info

malcolmsparks12:12:20

that means we avoid the bug where new-classpath-resource have 0 methods defined for the case where there is no path-info, and duly creates a sub-resource for handling the path-info of "/"

malcolmsparks12:12:40

You can group the first three routes together

malcolmsparks12:12:51

["/" [ [] [] [] ]]

malcolmsparks12:12:08

remember, when you do this, you need a vector-of-vectors on the right-hand-side

malcolmsparks12:12:30

it's not like pedestal where you can have ["/" [] [] []]

malcolmsparks12:12:03

that's really the main thing to remember with bidi - in the right-hand-position it's a handler, or vector-of-vectors

malcolmsparks12:12:51

but I think you've found a nice workaround for the new-classpath-resource issue - nice!

martinklepsch13:12:17

Thanks Malcolm, that’s helpful. I’ll try to internalize "the right hand should be either a handler or a vector of vectors”.

malcolmsparks13:12:09

@martinklepsch good luck - shout if you need more help

martinklepsch13:12:22

In the Swagger example given here https://juxt.pro/yada/manual/index.html#hello-swagger the hello-routes are available at / and under /api/ — is there a practical reason for this?

martinklepsch13:12:41

Or are the routes under /api different in some way?

malcolmsparks13:12:45

No, you could just use the /api ones

malcolmsparks13:12:52

I see your point, it's a bit confusing

martinklepsch13:12:25

I’m new to Swagger so it might be more confusing to people like me than to people more familiar with Swagger

malcolmsparks13:12:33

No, the docs definitely need improving here. If you want Swagger, use it, otherwise avoid it

martinklepsch14:12:30

I don’t know if I want it but playing around with the integration won’t hurt 🙂

malcolmsparks14:12:34

didn't quite understand you 😉

martinklepsch14:12:04

Well, the Swagger support or integration or whatever you want to call it 🙂

malcolmsparks14:12:22

the swagger integration needs a bit of work -it's hard because swagger doesn't always follow http and does its own thing - it's a bit of a tension between compile-time declarative and run-time dynamic (which REST prefers)

malcolmsparks14:12:04

e.g. if you compute your 'produces' variants at runtime, how can swagger descriptions be produced ahead of time?

malcolmsparks14:12:58

ring-swagger gets round this by firing tracer bullets into your resources - I'm not sure I want to do that - it's a bit of a hack - you don't want to launch the missiles when you think you're only creating a Swagger description

malcolmsparks14:12:52

these days, APIs actually do pretty severe things, perhaps some even launch missiles who knows

malcolmsparks14:12:57

(i'm personally not a fan of people launching missiles at other people)

malcolmsparks14:12:47

I'm currently finishing work splitting yada into a minimally small working core and a set of extensions - users shouldn't notice any difference but it paves the way to add non-standard extensions and contribute things to yada that normally wouldn't be considered core

malcolmsparks14:12:12

extensions include json, transit, json-html, swagger, multipart

malcolmsparks14:12:41

it will also allow usage of yada where dependency-hell is an issue

malcolmsparks14:12:55

the minimal yada library will only have 5-6 dependencies

ikitommi14:12:59

have you @malcolmsparks considered using the HAL spec? should suit better for REST-style api-docs. There is also a nice ui for that (https://github.com/mikekelly/hal-browser)

malcolmsparks14:12:55

@ikitommi Hi Tommi - I have never heard of HAL before - thanks for the tip!

ikitommi14:12:41

Spring Data REST uses that, so should be widely adopted. Haven’t used it myself.

vinnyataide14:12:42

@malcolmsparks thank you sir, I really appreciate the explanation

jannis14:12:41

@malcolmsparks @martinklepsch Hey, sorry the code I contributed causes problems. Shall I try to fix new-classpath-resource to make up for it? I just need to understand the problem first and the behaviour we want.

jannis14:12:04

It sounds like @martinklepsch’s problem was that “localhost:4000” lead to an empty path info, which then didn’t fall back to :index-files (the if conditional in new-classpath-resource isn’t very good, I admit that).

malcolmsparks14:12:16

@jannis Hi there - no complaints really, thanks for the contribution - it's been useful. The only issue is that people expect different behaviour when it is called without a path-info (e.g. the 'root'). They expect a redirect to :index-files but instead get a 405 (because there's :methods {} in the parent resource)

jannis14:12:50

Would a check for an empty path info -> then fall back to :index-files be enough or is that only a partial fix? (Or no fix at all)

malcolmsparks14:12:43

A simple redirect would work {:methods {:get (fn [ctx] (throw (ex-info {:status 303 :headers {"location" (-> ctx :request :uri (str index-files))}})}}

malcolmsparks14:12:34

@jannis yes, that would work - only if there is no path-info there is no sub-resource called, so the resource has to work on its own

malcolmsparks14:12:07

so you can't check for an empty path-info, yada does that already - if there is a non-empty path-info it will create a subresource

martinklepsch14:12:21

Do I understand right that the sub-resource can’t work because sub-resources only work if there’s a path-info?

malcolmsparks14:12:48

I think it might be possible to implement the use-cases needing subresources (file directories, etc.) using dynamic properties functions - the subresources thing adds additional complexity to yada, I don't think it's warranted

malcolmsparks14:12:58

@martinklepsch yes - that's how it's implemented

malcolmsparks14:12:21

sub-resources were a bit of an experiment to make it possible for yada to serve directories of files (at first)

malcolmsparks14:12:31

not sure they work that well in practice

malcolmsparks14:12:44

path-info, however, is very useful, and could form part of a better approach

martinklepsch14:12:56

Essentially instead of :methods {}. There would need to be an implementation that returns index-file?

malcolmsparks14:12:26

well, there needs to be an entry in :methods {}, i.e. :methods {:get ...}

malcolmsparks14:12:46

and I think a 30x redirect is probably what is needed here

jannis14:12:05

Hmm. I don’t understand the logic around subresources yet, so I’m not sure where the {:methods {:get …}} redirect would be placed.

martinklepsch14:12:39

right, that’s what I meant

malcolmsparks14:12:09

the problem is that if the path-info is empty ("") then the function in :sub-resources isn't called

jannis14:12:44

Ah. And :methods would be used as a fallback?

malcolmsparks14:12:50

the idea of sub-resources is having a parent resource that looks after a whole collection of resources, each 'sub-resource' is targetted with a different path-info - subresources are dynamically created for each request

martinklepsch14:12:09

Why would a redirect be better than the actual response? If I understand correctly a redirect would change the URL to contain the path displayed in e.g. a URL bar to index-file. That may not be the desired (?)

malcolmsparks14:12:13

I think you've added an empty one to get past the schema check 🙂

jannis14:12:33

Haha, could be. It’s been a while 😉

jannis14:12:29

Although I think I adopted it from jar-resource, which was the old classpath-resource IIRC.