Fork me on GitHub
#perun
<
2017-12-10
>
bhagany18:12:47

@richiardiandrea I have finally had some time today for looking at these things. Your second question is easier (adding things to global metadata), so I'm going to start with that. Here's a task I've written to include recent posts in global metadata:

bhagany18:12:51

(deftask recent-posts
  "Adds the `n` most recent posts to global metadata as `:recent-posts`"
  [n num-posts NUMPOSTS int "The number of posts to store"]
  (with-pre-wrap fileset
    (let [options (merge +recent-posts-defaults+ *opts*)
          global-meta (pm/get-global-meta fileset)
          recent (->> (pm/get-meta fileset)
                      (filter post?)
                      (sort-by :date-published #(compare %2 %1))
                      (take (:num-posts options)))]
      (perun/report-info "recent-posts" "added %s posts to metadata" (count recent))
      (pm/set-global-meta fileset (assoc global-meta :recent-posts recent)))))

richiardiandrea18:12:32

How to generalize that now :) because this can apply to any "global" properties you want to pass around in all website's pages

bhagany18:12:32

A few thoughts - I think you should be able to modify this to get tags from the entries instead. Also, I think maybe that your use-case should be an automatic feature of the tags task.

richiardiandrea18:12:16

Yep basically I need a task that let me append to global starting from the entries (no filtering)

richiardiandrea18:12:23

Will start from the above

richiardiandrea18:12:36

Maybe contribute back if I like what I am doing :)

bhagany18:12:41

that sounds great!

bhagany18:12:17

for your other question about paginating the results of assortment - it is possible to write a grouper function that would do what you want, but it is not as easy out of the box as I'd like. I'm still thinking about how best to improve assortment, but in the meantime, would an example of a grouper function help?

richiardiandrea18:12:32

@bhagany I think I will not paginate in the first version so low priority ;) I have a grouper function that does that..was just wondering whether I was missing something

bhagany18:12:55

okay, sounds good - you aren't missing anything πŸ™‚

richiardiandrea18:12:24

It was such a nice experience to work with perun especially using Deraen's boot-livereload

richiardiandrea18:12:55

The only thing is that I am compiling less at every watch so it is kind of a slower refreshing experience

bhagany18:12:39

ah, hrm. it compiles even without a css change?

bhagany18:12:17

I think perun should be able to help with that, if you wouldn't mind posting that less task

richiardiandrea18:12:58

Yes no well, I rolled my own less task because I am using the js less compiler..but I haven't implemented the check for previously changed files

bhagany18:12:41

okay, you may want to look into using io.perun.content-task - it is basically an abstraction of the file-watching pattern that you just need to plug a rendering function into

richiardiandrea18:12:02

so maybe I should chain the two somehow

bhagany18:12:28

nice! looking

richiardiandrea18:12:35

@bhagany in the above, is post? a perun function?

bhagany18:12:54

ah, no sorry -

bhagany18:12:56

(defn post?
  [{:keys [path]}]
  (.startsWith path "public/posts/"))

bhagany18:12:16

it's pretty specific to my particular usage

richiardiandrea18:12:18

can some meta on the fileset obj can be used as well? do we have any available?

bhagany18:12:54

yep, you can call io.perun.meta/get-global-meta in any task to get the current global metadata

bhagany18:12:16

(it takes the fileset)

bhagany18:12:08

(and I just corrected that namespace, sorry)

richiardiandrea18:12:57

so there is no metadata on each individual entry of the fileset, only "global" am I right?

bhagany19:12:44

there is metadata on each entry

bhagany19:12:19

my post? filter is using the :path key of that metadata, on each entry, for instance

richiardiandrea19:12:26

so if I had a :post entry I can just read that one

bhagany19:12:27

yeah, assuming you've put it there by some means, like metadata at the top of a markdown file, or meta arguments passed to perun tasks

richiardiandrea19:12:54

yep that was the plan πŸ˜„

richiardiandrea19:12:30

but I guess I will stick to your task for now, I have a folder for the posts as well

bhagany19:12:46

sounds good πŸ™‚

richiardiandrea19:12:37

and here you go, here is the other simple one:

(deftask all-tags
  "Adds all the found tags as :all-tags global-meta"
  []
  (with-pre-wrap fileset
    (let [options (merge +recent-posts-defaults+ *opts*)
          global-meta (pmeta/get-global-meta fileset)
          all-tags (->> (pmeta/get-meta fileset)
                        (filter post?)
                        (mapcat :tags)
                        (vec))]
      (pcore/report-info "all-tags" "added %s to metadata" all-tags)
      (pmeta/set-global-meta fileset (assoc global-meta :all-tags all-tags)))))

bhagany19:12:00

looks great!

richiardiandrea19:12:03

I actually need dedupication there πŸ˜„

bhagany19:12:12

ahhhh. right

bhagany19:12:27

hrm... does lessc report the files it writes?

richiardiandrea19:12:46

I don't think it does...

bhagany19:12:03

okay, I'll see if I can work around that

richiardiandrea19:12:13

I was checking content-task and I saw that it needs more things in there...

bhagany19:12:15

yeah, my intuition says that you'd be able to use other things that perun defines for those other arguments, but I'm not absolutely sure

richiardiandrea19:12:27

also the all-tags seems not to work because it needs to happen after rendering the posts...

richiardiandrea19:12:30

uhm, actually even :recent-posts does not display in the individual posts

bhagany19:12:08

yeah, my needs allow me to call recent-posts after the initial rendering

bhagany19:12:49

but I'm assuming you need the tags on each page, in a menu or sidebar of some sort?

bhagany19:12:58

okay, I have a solution for you. well, actually, an example

richiardiandrea19:12:02

I tried just now to render twice, I get a frankestain, but I see the tags and the posts

bhagany19:12:22

ah, you're on the right track

bhagany19:12:43

I'm calling render twice there, but I'm using two different render functions

bhagany19:12:05

the second one basically wraps the content produced by the first one

bhagany19:12:04

in your case, you could do the initial render of the content that's unique to each page, then collect metadata, then render each again

richiardiandrea19:12:21

let me bend my mind for a sec, not sure I get this πŸ˜„

bhagany19:12:38

no worries, I'm here to help πŸ™‚

richiardiandrea19:12:31

so my second renderer would render only the menu/sidebar?

bhagany19:12:38

yes, right

bhagany19:12:21

if that doesn't appeal to you, I think there's another way too, if you'd like to hear it

richiardiandrea19:12:03

I'd like to, just to see if it is less code change

bhagany19:12:34

okay. is the metadata you need to generate your sidebar at the head of your markdown files?

richiardiandrea19:12:57

yes, basically :tags + :recent-posts for now

bhagany19:12:33

okay, if you take a look at the markdown task, you'll see that it's the composition of two simpler tasks: yaml-metadata and markdown*

bhagany19:12:19

that would allow you to split up the work that markdown does, so that metadata is collected, then you can run your custom tasks, and then markdown*

richiardiandrea19:12:49

yep that would work for :tags, not for :recent-posts if I understand correctly

richiardiandrea19:12:11

unless I only display :title and :headline

bhagany19:12:24

I'm not sure I follow

richiardiandrea19:12:57

uhm ok let me try first your suggestion

richiardiandrea19:12:12

checking those tasks..

bhagany19:12:29

okay, let me know if you have any questions

richiardiandrea19:12:41

thanks a lot πŸ˜„ really appreaciate!

richiardiandrea20:12:21

@bhagany that first solution does not seem to work for me...I basically call a page function already...which wraps...but moving the sidebar is not enough, the content also changes...rendering twice I thought would replace the file but for some reason it does not

bhagany20:12:04

Ah if the content also changes, then yeah, that wouldn’t work

bhagany20:12:17

I’m confused about the file replacement not happening though

richiardiandrea20:12:42

yeah me too...I am investigating

richiardiandrea20:12:26

it looks like the rendered the second time around has the content of the first render

richiardiandrea20:12:27

yes so :content now contains the whole html

bhagany20:12:47

Ah, yes, sorry I wasn’t clearer about that

bhagany20:12:08

It’s going to progressively build up the html, from the inside out

bhagany20:12:46

You can take a look at the rendering functions for http://nicerthantriton.com to see

richiardiandrea20:12:12

ok now I see why that works πŸ˜„

bhagany20:12:26

You’re fine, go ahead :)

richiardiandrea20:12:30

but it is not hmtl I see now in :content πŸ˜„

richiardiandrea20:12:43

I see a string containing hiccup

bhagany20:12:51

After which task?

richiardiandrea20:12:12

now I do:

(perun/render :out-dir posts-dir :renderer ' :filterer post?)
   (recent-posts)
   (all-tags)
   (perun/render :out-dir posts-dir :renderer ')
  

richiardiandrea20:12:02

i see you are doing an into the second time in page

richiardiandrea20:12:19

so I was expecting hiccup in there...

richiardiandrea20:12:59

so the first time :content is set to whatever the first render returns

richiardiandrea20:12:06

is it converted to string?

bhagany20:12:31

yes, I think it'll be the file contents as a string

bhagany20:12:40

I'm trying to understand why I have that into though...

richiardiandrea20:12:28

yeah it looks like your return html5 stuff from post...

bhagany20:12:15

I think maybe that's just leftover from an earlier iteration of this code...

bhagany20:12:45

before I modified render to work this way, I used function composition to achieve what I wanted, and in that case, I think that content was a hiccup data structure

bhagany20:12:57

and now that it's a string it kind of accidentally works

richiardiandrea20:12:14

if my first renderer wrap thing with hiccup.core/html5, then I receive hiccup as data in the second render

bhagany20:12:34

in the :content key?

bhagany20:12:44

that's really odd...

richiardiandrea20:12:05

so this works:

(defn render-content [{global-meta :meta entries :entries entry :entry}]
  (html5 ;; I should not need this, maybe perun bug?
    [:div.twelve.wide.column
     "<!-- Post -->"
     [:div.ui.right.floated.basic.segment
      [:div.ui.compact.segments
       [:div.ui.segment
        [:a.ui.medium.image {:href (:canonical-url entry)}
         [:img {:src (or (:image entry) "")}]]]
       (when-let [sponsor-img (:sponsor-image entry)]
         [:div.ui.right.aligned.basic.secondary.segment
          [:div "Sponsored by"
           [:a.ui.tiny.image {:href (:sponsor-url entry)}
            [:img {:src sponsor-img}]]]])]]
     [:div (:content entry)]
     "<!-- End Post -->"]))

(defn render-page [{global-meta :meta entries :entries entry :entry}]
  (let [all-tags (or (:all-tags entry) (into #{} (mapcat :tags) entries))
        latest-posts (or (:latest-posts entry) (take 3 entries))
        active-tags (->> [(:tag entry)] (keep identity) (into #{}))]
    (base/page
     global-meta
     [:div.ui.centered.main.stackable.reverse.grid.container {:role "content"}
      (:content entry) ;; will receive the html from render-content above
      "<!-- Sidebar -->"
      (base/sidebar all-tags latest-posts active-tags)
      "<!-- End Sidebar -->"])))

bhagany20:12:13

thanks, was just going to ask for that πŸ™‚

bhagany20:12:30

oh, that works

bhagany20:12:39

I missed that part

bhagany20:12:57

so yes, that's what I would expect - you were expecting not to have to call html5?

richiardiandrea20:12:10

yep in render-content

bhagany20:12:50

okay, yeah, render isn't tied to any particular html renderer, so it's up to you to figure out how you want to return html

bhagany20:12:13

but the return of your render function needs to be the string that you want written to the file

bhagany20:12:27

(also, it doesn't have to be html)

richiardiandrea20:12:45

so :content just needs to be a string

bhagany20:12:55

yes, right

richiardiandrea20:12:05

let me try one thing

richiardiandrea20:12:56

@bhagany fun!

(ns 
  (:require [clojure.edn :as edn]
            [com.andrearichiardi.blog.base :as base]))

(defn render-content [{global-meta :meta entries :entries entry :entry}]
  (pr-str
   [:div.twelve.wide.column
    "<!-- Post -->"
    [:div.ui.right.floated.basic.segment
     [:div.ui.compact.segments
      [:div.ui.segment
       [:a.ui.medium.image {:href (:canonical-url entry)}
        [:img {:src (or (:image entry) "")}]]]
      (when-let [sponsor-img (:sponsor-image entry)]
        [:div.ui.right.aligned.basic.secondary.segment
         [:div "Sponsored by"
          [:a.ui.tiny.image {:href (:sponsor-url entry)}
           [:img {:src sponsor-img}]]]])]]
    [:div (:content entry)]
    "<!-- End Post -->"]))

(defn render-page [{global-meta :meta entries :entries entry :entry}]
  (let [all-tags (or (:all-tags entry) (into #{} (mapcat :tags) entries))
        latest-posts (or (:latest-posts entry) (take 3 entries))
        active-tags (->> [(:tag entry)] (keep identity) (into #{}))]
    (base/page
     global-meta
     [:div.ui.centered.main.stackable.reverse.grid.container {:role "content"}
      (edn/read-string (:content entry)) ;; will receive the html from render-content above
      "<!-- Sidebar -->"
      (base/sidebar all-tags latest-posts active-tags)
      "<!-- End Sidebar -->"])))

bhagany21:12:38

hehehehehe, so that outputs the hiccup as a string, then?

richiardiandrea21:12:54

yes pr-str outputs clojure data

richiardiandrea21:12:29

I feel better to have data there πŸ˜„

richiardiandrea21:12:26

this could be a topic for my new blog πŸ˜„

bhagany21:12:59

it's interesting, it will definitely be more flexible for subsequent renders

richiardiandrea21:12:58

you could serialize and pass a function over πŸ˜„

richiardiandrea21:12:42

other question...sometimes yaml errors out with:

richiardiandrea21:12:54

ll; mapping values are not allowed here;  in 'string', line 4, column 87:
                                                  ...  in a GitHub blog? The answer is: of course you can! At the end  ... 
                                                                                      ^
        problem: "mapping values are not allowed here"
    problemMark: #object[org.yaml.snakeyaml.error.Mark 0x5cf45f80 " in 'string', line 4, column 87:\n     ...  in a GitHub blog? The answer is: of course you can! At the end  ... \n                                         ^"]

richiardiandrea21:12:16

ah maybe a character that means something