This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-11-16
Channels
- # announcements (11)
- # beginners (184)
- # calva (91)
- # cider (68)
- # cljdoc (42)
- # cljs-dev (44)
- # clojure (228)
- # clojure-dev (1)
- # clojure-europe (3)
- # clojure-italy (4)
- # clojure-losangeles (6)
- # clojure-nl (9)
- # clojure-spec (73)
- # clojure-uk (19)
- # clojurescript (61)
- # core-async (6)
- # cursive (2)
- # datomic (11)
- # fulcro (28)
- # hyperfiddle (16)
- # leiningen (2)
- # luminus (3)
- # off-topic (19)
- # om-next (1)
- # re-frame (2)
- # reagent (12)
- # reitit (4)
- # ring-swagger (5)
- # shadow-cljs (14)
- # slack-help (6)
- # spacemacs (2)
- # tools-deps (40)
- # vim (15)
- # yada (4)
Hi all, I trying to implement web scraper
(System/setProperty "clojure.core.async.pool-size" "1024")
(def to-fetch-url (async/chan 1000))
(def to-fetch-attributes (async/chan 1000))
(def to-persist (async/chan 1000))
(defn fetch-url [loc out]
(when-let [url (url loc)]
(async/>!! out (assoc loc :url url))))
(defn fetch-attributes [loc out]
(async/>!! out (attributes loc)))
(async/pipeline-async 500 to-fetch-attributes fetch-url to-fetch-url false)
(async/pipeline-async 500 to-persist fetch-attributes to-fetch-attributes false)
both fetch-url and fetch-attributes doing web scraping
the problem: it invokes fetch-attributes
only once
please advice what is wrong?but I do:
(async/go-loop []
(async/<! to-fetch-url)
(async/<! to-fetch-attributes)
(async/<! to-persist)
(recur))
everything works, which seems weird…it turns out that out
chan should be closed