This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-05-21
Channels
- # announcements (17)
- # babashka (2)
- # beginners (51)
- # calva (14)
- # clojure (30)
- # clojure-europe (12)
- # clojure-uk (3)
- # clojurescript (22)
- # cursive (10)
- # defnpodcast (1)
- # etaoin (1)
- # fulcro (7)
- # graalvm-mobile (3)
- # music (1)
- # obb (4)
- # podcasts-discuss (1)
- # sci (2)
- # shadow-cljs (37)
- # spacemacs (3)
- # xtdb (16)
What is your favorite/recommended webscraping tool in Clojure? Something practical and comprehensive (like Scrapy in Python if you know it)?
https://github.com/cgrand/enlive is a very useful library that is often used to scrape the web and transform it into useful data structures in Clojure
Thanks, @U05254DQM! Will check it out now.
playwright is a useful library for puppeting real browsers, and seems less fussy than setting up selenium
I mean the main advance part I want is the ability to go to sub-pages by clicking on links, all done by the scraper.
Htmlunit is a sort of headless browser written in java, it can execute JavaScript and you can programmatically click things etc
@hiredman how do you suppose you would implement the format command in a C/C++ hosted language without re-implementing all of printf?
Are you trying to avoid using an existing C/C++ implementation of printf?
@U036UA9LZSQ Is this for jo_lisp? I think I would expect it to behave like the C/C++ formatting when there are differences...
The core behavior is similar enough I think? (been a while but all the basics are the same, right?)
I think it be nice to omit printf like formatting, and maybe go straight for something like this: https://openjdk.java.net/jeps/8273943 ?
@U0K064KQV Not relevant to what Jon is working on I suspect -- implementing Clojure in C++ as a hosted language there.
I think he's concerned with balancing compatibility with Clojure vs amount of work involved. I wonder how ClojureCLR deals with format
?
Oh, I thought he was only inspired by Clojure and diverging. Arguably, if that JEP does happen, Clojure might inadvertently support it form Java 😛
Yeah, I'd expect interop in jo_lisp to open up whatever C/C++ has to offer in terms of other string formatting systems 🙂
Mostly, C doesn't have a mechanism to construct a variable argument list to pass to printf
In that case, I consider format one of the things I'd expect to be of the host. But I'd expect cl-format to be consistent.
Is cl-format
widely used? I think I've only ever encountered one code base that used it?
Probably not widely, I use it though, it's a great template language once you know it, very powerful. In CL, it also supports inline variables, which I wish Clojure's implementation also had though, apparently they're open to it, just no one ever finished it.
@U036UA9LZSQ You could write a dispatch on the number of arguments, up to a reasonable maximum, and support just that "subset" of available calls?
Even though its implemented in Clojure itself?
(defn printf
"Prints formatted output, as per format"
{:added "1.0"
:static true}
[fmt & args]
(print (apply format fmt args)))
So the issue only occurs when you need to forward variable arguments to a C function
My google does show that C's format is variadic though 😛, but my familiarity with C is low, and I have no clue how you're implementing Clojure to compile to C
Hum, ok I think I understand the issue now having looked more at the C printf. Sean's idea seems maybe best. But I wonder if you could use vprintf instead. The use of vararg means you'd statically know the length, though I'm not sure how you'd apply over it.
yeah, the problem with that approach, from my research, is there is no way to construct a va_list.