Fork me on GitHub
#cljdoc
<
2022-04-09
>
Cora (she/her)00:04:30

I suppose any query language is bound to clash with the various characters allowed in symbols

lread02:04:18

Heya @corasaurus-hex, do you mean like allow searching for curse words? simple_smile But seriously… I’m not sure exactly what you are referring to yet. Maybe search syntax?

lread03:04:02

Maybe a comparison with libraries search will help? The libraries search (which I am still working away at btw) offers no explicit syntax (you can contrast this with https://github.com/clojars/clojars-web/wiki/Search-Query-Syntax, I considered this, but I think we are doing the right thing for cljdoc). Cljdoc libraries search finds libraries matching group-id, artifact-id and blurb (a shortened description, this refinement is a wip). A library is scored (and sorted in results) by some factors: • an exact match on group-id and/or artifact id should guarantee the match appears first in the list (refinements here are a current wip). • we otherwise match text anywhere in a token (this is also a wip) so a search for corf will match seancorfield. • libraries that are more popular will typically appear before the less popular ones (popularity (a wip) is via clojars download count over the last year or so). • matches in the blurb are given much less weight than all other matches I think (wip, tbd) I will also: • as a rule, show any field we are searching in result (e.g. we’ll start showing the blurb) • be highlighting matched search terms in result

Cora (she/her)03:04:49

right, the lucerne query syntax was what I was referring to, something like that

Cora (she/her)03:04:05

I undertook an effort to compare various client-side search libraries. it's all very spotty and uneven

Cora (she/her)03:04:54

i'm only part-way through the comparisons. I really like flexsearch but it doesn't have anything that highlights results

lread14:04:37

My opinion: Unless you sense a very clear advantage of a search syntax, I'd avoid it, at least for initial versions.

Cora (she/her)20:04:40

I think lunr.js combined with mark.js is going to be a workable solution

lread22:04:55

But… if you decide some search syntax is warranted, lemme know! Would be nice to be consistent for libraries & docset search.

Cora (she/her)22:04:19

I think what's weird about the whole thing is that search for docsets needs to go beyond just full text and needs to include some atypical characters

Cora (she/her)22:04:40

for example with foo->bar as a function name if the person types exactly foo->bar then you'd want that to match over matching, say, a function named foo-bar. you typically don't match on non-word characters with full-text search

Cora (she/her)22:04:29

so tokenizing needs to be customized, as well as searching since if the user searches foo->bar we'd also want foo-bar in the list of matches just further down the list

Cora (she/her)22:04:53

and it gets even more sticky when it comes to the docstring searches since prose will also contain valid symbol characters that are not word characters except they will have different semantic meaning, they're not necessarily symbols

Cora (she/her)22:04:34

and this needs to be weighed against memory usage and search time

lread22:04:44

Interesting! Yeah, I see similar, but less complex issues for libraries search. For the foo->bar situation, libraries search weighs an exact match on group/artifact very heavily so those will appear at the top. But it will also match items containing foo and bar. For the free text prose in the library blurb, I’m not doing anything special/custom, but do see your concerns for docsets.

lread22:04:15

I am getting a little stumped with TypeScript and images 🧵

lread22:04:49

I’m experimenting with showing a wee clojars or maven-central icon in search results.

lread22:04:22

So I whipped up a couple of svg images and plunked them under resources/public

lread23:04:24

But have so far been unsuccessful in referencing them from search.tsx.

lread23:04:07

I do see parcel’s build plunking my svg in the expected spot:

resources-compiled/public/out/clojars.0eb7cf5b.svg                          2.44 KB    327ms
But my current attempt to add the following to search.tsx
import clojars from "../resources/public/clojars.svg";
And then later reference via
<img src={clojars}></img>
Results in sadness. I get the following in my browser console:
Uncaught SyntaxError: import.meta may only appear in a module
I also tried (I’ve just been googling around, I don’t know TypeScript yet) adding a custom.d.ts file:
declare module "*.svg" {
  const content: any;
  export default content;
}

lread23:04:54

I wonder @corasaurus-hex, what someone who actually knew TypeScript would do. simple_smile

lread23:04:23

This is probably very easy once you know what to do.

lread23:04:26

(this could also be more of parcel.js issue, dunno that tech well at all either).

Cora (she/her)23:04:15

can you try clojars.meta.url?

Cora (she/her)23:04:48

I think the static files stuff is misconfigured. I can look when I'm home but it'll be a while. in the meantime you could skip the import and just try "/dist/clojars.svg" or "/clojars.svg" or some other plain string in the image tag src

Cora (she/her)02:04:54

ok, I figured out what you need to do

Cora (she/her)02:04:58

stick the file in resources/public/static and it'll get copied over to resources-compiled/public/out

Cora (she/her)02:04:50

and then you can just refer to the path like

<img src="/file.svg" />

Cora (she/her)02:04:53

I suspect you may also put it in resources/public and then import it but I'm not 100% sure

lread03:04:52

Thanks @corasaurus-hex very much appreciated! I was thinking the generated hashy filenames are important for... caching? So much to learn... If I'm not screwing things up by going the static route, I'm happy to do that.

Cora (she/her)04:04:18

yep, it's for caching

Cora (she/her)04:04:48

add a parcel.d.ts file in the root with:

declare module "*.svg" {
  const src: string;
  export default src;
}

Cora (she/her)04:04:22

change this line in tsconfig.json

"include": ["js/**/*"]
to be
"include": ["js/**/*", "parcel.d.ts"]

Cora (she/her)04:04:32

then you can do an import like

import cactus from "../resources/public/cactus.svg";
and then use that in an img tag like
<img src={cactus} />

Cora (she/her)04:04:54

it'll use the hash as part of the filename and have the mtime as ms since epoch as the query string

Cora (she/her)04:04:10

this way if you change the file it'll change the url it hits

Cora (she/her)04:04:23

I can add that to main, actually

lread12:04:19

Thanks so much @corasaurus-hex! I did try the equivalent of your parcel.d.ts in a js/custom.d.ts(see thread above) and did not have success with that. But maybe I had a typo or something.... I'll try it again today. (I'll try your PR too). I'm curious as to what is making a seemingly simple thing so awkward. Perhaps just my level of knowledge TypeScript and parcel.js. And, I guess it is more about cache busting, yeah? A new filename gets generated for each release.

Cora (she/her)13:04:44

the main difference was yours had any for the default export type, it seems, but the working one needed string

Cora (she/her)13:04:23

but the whole "simple thing is super awkward" part is just javascript in general. the whole ecosystem is lumpy like that

lread14:04:57

Ah! I did not spot that difference, thank you!

lread01:04:55

And it stopped working. But moot point. I’ve abandoned the svg experiment for now because I found the icons more confusing than helpful (for the end-user of cljdoc). I am still super thankful for all your help!

Cora (she/her)02:04:58

the clojure server proxies to the dev server?