Fork me on GitHub
#clojure
<
2016-03-31
>
base69800:03:46

I got code shamed about my documentation today so was looking to improve it. JSDoc format seems ok.

base69800:03:02

I like the documentation of clojure.jdbc simple_smile

hiredman00:03:12

if you are talking about what seancorfield linked to before, those are prose documents, any blogging platform / wiki will let you write those, they aren't generated from docstrings or comments in the code

base69800:03:36

I noticed the source had mostly no documentation and small functions simple_smile

base69800:03:39

self documenting

hiredman00:03:37

shrug it depends who you ask

hiredman00:03:17

I have been interviewing, around and had a 200 line answer to a coding challenge (200 lines including some function docstrings and tests) dinged for not including documentation

hiredman00:03:31

174 lines actually

base69800:03:55

Where have you been looking?

seancorfield01:03:20

@base698: Yes, I used to run the SF Clojure meetup (for a couple of years). Kind of a pain since I live out in the East Bay and it’s a hike into SF at rush hour.

hiredman01:03:31

all over, I've been looking for a remote position

seancorfield01:03:59

As for the prose docs I linked to, they’re maintained by a team of folks and use Jekyll to generate static HTML from Markdown, as I recall.

hiredman01:03:47

the market in remote clojure positions has not been what I thought it was

base69801:03:03

People seemed to be discussing it a lot in jobs

seancorfield01:03:22

World Singles is 100% remote. We might (may be, possibly) be hiring a junior Clojure dev later this year.

hiredman01:03:39

what is a junior clojure dev?

hiredman01:03:57

I have five years of experience writing clojure professionally (and remotely)

seancorfield01:03:00

Far too junior for you @hiredman — sorry 😞

seancorfield01:03:43

I know your background! Unfortunately I think we’re more likely to hire fresh out of college this time around and grow someone into the team.

seancorfield01:03:25

(so we may well look for someone who is interested in FP, played with Clojure, but has pretty much zero actual workplace experience)

seancorfield01:03:36

We haven’t fully decided yet.

seancorfield01:03:18

I’m a bit surprised there aren’t more remote senior roles out there, to be honest.

seancorfield01:03:40

Remind me where you’re based?

hiredman01:03:57

senior roles, non-senior roles, whatever

seancorfield01:03:13

Ah. You’ll be at Clojure/West in a few weeks?

richiardiandrea01:03:40

oh cool I am looking forward to meeting all of you guys 😉

seancorfield01:03:56

I may bend your ear about that java.jdbc reducers ticket you opened… simple_smile

seancorfield01:03:20

Esp. after the unfold post on the list.

richiardiandrea01:03:36

I read that too nice

hiredman01:03:58

I'll try to wear something loud simple_smile

base69801:03:01

where did that unfold post go? @hiredman posted it here earlier

hiredman01:03:36

I made a proposal of adding unfold as a function to clojure.core after our discussion yesterday

base69801:03:09

Thanks for all your help today! See you guys tomorrow.

zcaudate01:03:45

@hiredman, @base698: one problem with blogging platforms are that once the source code changes or the api changes, the tests have to be rewritten

zcaudate01:03:05

as well as the documentation.

zcaudate01:03:43

It’s trival to change tests… but it’s a bitch to change documentation because you don’t even know where it broke

zcaudate01:03:42

the most awesome thing with clojure was that code is data

zcaudate01:03:17

I don’t know how it would have been possible with java

zcaudate01:03:42

anyways… so by having documentation examples that are tests, the documentation can be updated very easily

zcaudate01:03:51

just as tests are done

roberto01:03:38

@zcaudate: is hydrox compatible with clj.test, or only with midje?

zcaudate01:03:07

@roberto: it is… though it needs a bit of work

zcaudate01:03:41

but that’s only the docstrings part

zcaudate01:03:05

there’s two parts:

zcaudate01:03:24

- a way to manage docstrings from tests

zcaudate01:03:34

- a way to generate testable docs

zcaudate01:03:17

I’m biased towards midje because of the => syntax

zcaudate01:03:46

and tend to do things on a by need basis

zcaudate01:03:41

but I haven’t actually hit all possibilties apart from simple tests… 😶

roberto01:03:22

yeah, I don’t use midje. I don’t like adding more deps than I need to. And when I was learning clojure, clj.test was easier to pick up.

zcaudate01:03:43

ultimately, it’s not a replacement for comprehensive testing

zcaudate01:03:02

it’s a way to manage metadata around the function

zcaudate01:03:21

by using a test framework to verify the correctness of input

zcaudate02:03:36

so:

^{:refer example/hello-world :added \"0.1\"}
(deftest hello-world-test  
   "Sample test program”
    (is (= 1 1))
    (is (identical? 2 4)))

zcaudate02:03:27

will get put into your docstring as:

zcaudate02:03:15

"Sample test program  
  1 
   => 1
  
  (identical? 2 4)
   => true"

roberto02:03:34

yeah, that is nice

zcaudate02:03:35

I still prefer the midje syntax

roberto02:03:37

and if the test fails?

zcaudate02:03:55

then when you run the test runner, it will fail

zcaudate02:03:11

and then you will know to change your documentation

zcaudate02:03:39

otherwise I would have no clue

roberto02:03:47

oh, so, is the documentation still outputed for failed tests?

zcaudate02:03:17

when you run lein hydrox import

zcaudate02:03:42

it will take your tests and import all tagged tests into your functions

seancorfield02:03:49

@zcaudate: have you seen Midje-readme? Runs the code / output in your readme as Midje tests.

zcaudate02:03:33

so if you don’t run that command import, then no, the docstrings will remain the same

seancorfield02:03:36

I had never heard of it until I saw someone sent a PR for clj-time.

zcaudate02:03:56

no! though I had something very similar

zcaudate02:03:11

but I didn’t bring it in to hydrox for a few reasons

zcaudate02:03:15

- I wanted to keep my documentation together

zcaudate02:03:40

- there is only so much that can be done with one file

zcaudate02:03:51

and it gets messy really quickly

zcaudate02:03:18

@base698, @roberto, @seancorfield: once a project reaches a certain size (when readme.md starts getting really heavy), the testable documentation becomes invaluable.

roberto02:03:53

it is what I find attractive about hydrox

zcaudate02:03:51

there is no way I could have produced (and maintain) something like this: http://docs.caudate.me/hara/hara-event.html

zcaudate02:03:46

without being able to write examples that are verifiable through a test framework

zcaudate02:03:31

… and that’s only 1 out of 21 libraries in the hara project

roberto02:03:21

is it compatible with cljs?

roberto02:03:27

I mean hydrox

zcaudate02:03:33

… unfortunately no

roberto02:03:55

okay, I think devcards could be an adequate equivalent in cljs

zcaudate02:03:15

I haven’t seen devcards in action

zcaudate02:03:44

what does it do?

zcaudate02:03:25

this was a while back… http://z.caudate.me/midje-%60lite%60-for-clojurescript/ the clojurescript documentation was generated: http://purnam.github.io/purnam/

roberto02:03:29

I haven’t used it but it looks interesting

roberto02:03:57

you can create interactive docs with code that live updates

zcaudate02:03:15

oh that’s nice

roberto02:03:32

I just think it was not sold appropriately, it took me a while to understand its benefits

roberto02:03:34

and use case

zcaudate02:03:45

i’ll definitely have to take a look

roberto02:03:03

most of the demos were focused on demo candy, and I couldn’t figure out the why and when

zcaudate02:03:43

can you generate a seperate documentation file with it?

zcaudate02:03:08

I’m sure that’ll be trivial.

adamkowalski02:03:44

hey whats the best way to send initial data to the client? if I have a server should I wait for a websocket on-open event and then send the initial data over transit?

rauh09:03:06

Is there some kind of macro toolkit/helpers? In particular, is there libraries that find all possible return values of form? (By walking into common structures like case/cond/when etc...).

zcaudate09:03:40

it’s probably overkill

zcaudate09:03:08

but it’ll definitely have everything you need

rauh09:03:48

@zcaudate: Thanks. I'll take a look!

rauh09:03:51

Problem is, it is actually for a cljs macro.

bronsa09:03:22

Not sure I'd suggest using tools.analyzer.js, it depends on an ancient version of clojurescript and hasn't been updated in a while

lwhorton12:03:42

what I want to be able to do is something like ...

(ns my.module
  (:require [com.somelib :as somelib]))
...
(defn myWrapperAPICall []
  ... modifies args, other stuff
  (somelib/APICall modifiedStuff))

// another ns
(ns another.module
  (require '[my.module :as m]))

(m/myWrapperAPICall ...)
(m/someLibAPICall ...) <- not explicitly wrapped/exposed
But I'm not sure how to provide only a subset of enhancements without wrapping the entire api? For example, if I were wrapping ring, I wouldn't want to provide my own implementation for each and every function, 99% of which would just refer to the actual ring implementation. How can I require a lib and all it's functions, then "expose" it as my own with just a few overrides?

mpenet12:03:04

potemkin and a few other libs allow to do this (at your own risk)

lwhorton12:03:46

Is this not an idiomatic thing to do?

dm312:03:57

usually you redefine the vars in these cases

dm312:03:07

so you don't really have your own API

dm312:03:26

and it's a "hack"

lwhorton12:03:31

How do I override behavior of a library or "shield" myself from its own changes (at some point int the future)?

mpenet12:03:10

lwhorton: as long as you freeze the version of that dependency you should be fine

lwhorton12:03:14

for e.g., in the javascript world, apis change every other day and frameworks go in wild directions that I dont necessarily agree with.. rather than fork /maintain my own, I usually bring the module into my project but maintain my internal app-specific api

dm312:03:39

I haven't had the need to do that in Clojure

dm312:03:56

you use the version that you started with, as long as you don't upgrade - everything is fine

mpenet12:03:57

in your case you can provide a facade to it, yes

lwhorton12:03:18

thinking of React, for example - 13.1/2/3/4...15.1/2/3 etc.. They all provided some much-needed performance updates / bug fixes, but they also introduced non-trivial api changes with regards to object creation, etc. If I didn't have a facade in my internall app api, I would spend hours every week just trying to keep up.

lwhorton12:03:59

So this facade has to manually provide each and every api that I might need in the app, even if it's a direct call to the lib api?

dm312:03:06

yeah, you have to think of proper abstractions then, like Reagent

dm312:03:40

in my experience having dumb wrappers doesn't help much

mpenet12:03:54

you can automate this, a few libs used to do this as I mentioned

mpenet12:03:30

but it's a bit hairy tbh

mpenet12:03:06

here the top ns is a huge bag of all functions in sub-nses, but this would work with other dependencies as long as they are in the classpath

mpenet12:03:25

you can overwrite stuff as long as it's done after the aliasing

lwhorton12:03:14

This is helpful, thanks @dm3 and @mpenet. To give you some larger context (if you care at all), my plan was to use component [https://github.com/stuartsierra/component] as a IoC container for my app, and to inject my own implementation of necessary libs so I could override/enhance with ease. Something like:

(ns app-reframe.core (:require [reagent.core :as reagent])

(defrecord Reagent []
  component/Lifecycle

  (start [component]
    reagent)

  (stop [component]
    component))

(defn init [] (->Reagent))

dm313:03:18

to be honest, I've never found much need in trying to shield myself from libraries

dm313:03:39

choose wisely and embrace simple_smile

dm313:03:11

again, this is coming from Clojure/-script perspective

dm313:03:17

where things are pretty stable

lwhorton13:03:27

Mm, I suppose. Javascript has made me so weary of everyone else's code but my own.

dm313:03:59

I'm currently using Mount (instead of Component) and Hoplon (instead of Reagent/Om)

dm313:03:11

total lines of code are extremely small

dm313:03:52

and I'm quite sure I could maintain them if the original maintainers went a way I didn't like

lwhorton13:03:14

"Shielding" aside, just using the IOC pattern above ^ it would be really great for testing purposes to be able to stick in a mock via injection. What stinks is having to someone expose the entire api of reagent inside that (start [component] reagent/a /b /c ... etc) (not sure the syntax to do this even)

lwhorton13:03:46

Interesting, I'll have to take a look at Mount. I have investigated Hoplon but it just didnt fit my usecase as neatly as reagent, I don't think.

dm313:03:09

curious, what would that be - if you feel like it please share on #C08BDAPRA

lwhorton13:03:40

Well, the major weak-point with Hoplon is there's no react-native option... so when things like https://github.com/ptmt/react-native-desktop start to appear, I just feel disadvantaged not having react around.

dm313:03:00

if you're planning on doing that - absolutely simple_smile

dm313:03:54

(on going native on osx/android, I mean)

lwhorton13:03:52

The value prop is just too tempting. Taking 1-2 weeks to make some modifications/fork to support an entirely different platform or take months to learn a whole new stack and write parts of your app twice.

lwhorton13:03:24

I just peaked at Mount, thanks for that tip btw... it actually seems like a better deal right from the get-go.

dm313:03:48

then again, you can run a web app in electron or something

lwhorton13:03:50

not everything is method-object-ey like in component, and there's no full-app buy-in

dm313:03:58

haven't actually looked in that direction much

lwhorton13:03:23

Yea me neither, I've always wanted to poke around with electron/nwjs but just haven't had the time.

andfadeev13:03:24

hi everyone! I have this problem with compojure and multiple middlewares I use components

:site-handler (handler-component (:site-app config))
:http (jetty-server (:http config))
:site-endpoint (endpoint-component site-endpoint)
:api-endpoint (endpoint-component api-endpoint)
with deps
:http [:site-handler]
:site-handler [:site-endpoint :webhook-endpoint :api-endpoint]
:site-endpoint {:db :db
                :input-chan :notification-worker-in-channel}
:api-endpoint {:db :db
               :channel :notification-worker-in-channel}
in site routes I have something like this:
(defn site-endpoint [{{db :spec} :db input-chan :input-chan}]
  (-> (routes
       ;; more routes
       (POST "/send-message/:channel-id" [channel-id :<< as-int :as request]
         (message-route/send-message db input-chan channel-id request))
       (resources "/"))
      (wrap-routes (fn [handler]
                     (-> handler
                         (wrap-access-rules {:rules rules :on-error on-error})
                         (buddy-middleware/wrap-authorization backend)
                         (buddy-middleware/wrap-authentication backend)
                         (defaults/wrap-defaults defaults/site-defaults)
                         )))))
and in api
(defn api-endpoint [{{db :spec} :db channel :channel}]
  (api
   {:swagger {:ui "/api-docs"
              :spec "/swagger.json"
              :data {:info {:title "PushToTelegram Public API"}}
              }}
   (context "/api" []
     (GET "/send/channelMessage" []
       :return {:channel-id s/Int
                :recipients-count s/Int}
       :query [message ApiChannelMessage]
       (api-route/send-channel-message db channel message))
     )))
and then in handler component they are combined with (routes api-routes site-routes) in post route I have ring antiforgery exception, because of api-middleware, and the only way I can fix it is to wrap api routes with some context, e.x (context "/my-api" [] (api ;; etc)) but then I have swaggers docs broken with wrong api urls What is the right way to split api and site routes in web app?

ikitommi14:03:30

@andfadeev what kind of antiforgery-exception do you get? Have put all the static routes too under api. wrapping into undocumented makes them not to appear in swagger-docs.

ikitommi14:03:58

api-middleware uses ring-middleware-format, which might cause problems as it consumes the body (which is the mutable part of the request).

andfadeev14:03:09

I get Invalid anti-forgery token in site post routes, recently I did not use compojure-api, it was compojure route with (context "/api" [] ...) wrapped with api-defaults

ikitommi14:03:43

If you could create a non-working minimal project, I could poke with it.

ikitommi14:03:41

but - creating the API with a separate function and mouting it to /api context works I believe?

ikitommi14:03:37

e.g. (routes (context "/api" api) site-routes).

andfadeev14:03:56

yes, with context in works, but then in swagger docs urls are "/api/method" but with context they really are "/api/api/method"

andfadeev14:03:08

or I miss something?

ikitommi14:03:10

you can say :basePath "/api" to swagger

ikitommi14:03:23

... and remove the api-context from inside the api.

andfadeev14:03:01

oh, that is what I was looking for, I will try now

mpenet15:03:42

dm3: component is not really "method-object-ey", you can just use maps if you dislike records and has the merit of being mutation free compared to mount

base69815:03:00

Good morning! Anyone know what :>> is? I see condp uses it, but that's the first i've been aware of it

mpenet15:03:09

base698: it will apply the function after :>> to the result of the predicate

base69815:03:27

is it used anywhere not condp?

mpenet15:03:18

base698: tbh it's the first time I see it (I have been using clj since early 2010)

isak15:03:23

> Note :>> is an ordinary keyword.

josh.freckleton16:03:15

I'm trying to "rotate" a matrix, and if each column doesn't have the same number of items, that column goes missing, but I'd like to keep it. Here's my approach:

(def m [[1 2 3] [4 5 6] [7 8]])
(apply map vector m)
I've thought about padding shorter rows with nil, but I don't want to artificially change the matrix for just processing reasons. Thougts?

josh.freckleton16:03:39

For example, even this returns an empty list:

(apply map vector [[1] [2] []])

turbopape16:03:49

so you’d basically keep only the vectors of the maximum length ?

josh.freckleton16:03:11

I want to keep all vectors, so in the second example, I'd like [1 2]

turbopape16:03:17

rotating only [1 2 3][4 5 6] ?

josh.freckleton16:03:09

not quite, I'd like [[1 2 3] [4 5 6] [7 8]] to turn into [[1 4 7] [2 5 8] [3 6]]

turbopape16:03:11

ah, you’d have : [1 4 7] [2 5 8]

turbopape16:03:18

ah o I get it

turbopape16:03:00

let me think a little ...

turbopape16:03:53

you’d need an fn that takes the firsts of seqs + nils..

josh.freckleton16:03:42

the problem lies with map, it maps until one of the collections is "exhausted", or out of items. But I want it to keep going...

josh.freckleton16:03:00

do I need to roll my own map? I think there'd be an easier way...

turbopape16:03:29

look at this:

turbopape16:03:31

(loop [remaining [[1 2 3][4 5 6][7 8]] result []] (if (not (every? (comp not empty?) remaining) ) result (recur (map rest remaining) (conj result (mapv first remaining)))))

turbopape16:03:39

it gives you

turbopape16:03:17

[[1 4 7] [2 5 8]]

turbopape16:03:03

If you pad with nil the shortest one, you couldn’t get any better

turbopape16:03:08

like you said.

turbopape16:03:31

so just pad it with :NA

josh.freckleton16:03:02

😞 If I pad, I'll need to pad + unpad over many iterations. It seems wrong to modify the data like this to allow it to process correctly, no?

turbopape16:03:02

you’re probably right

josh.freckleton16:03:57

@turbopape:

(loop [remaining [[1 2 3][4 5 6][7 8]]
       result []]
  (if (every? empty? remaining) 
    result
    (recur (map rest remaining)
           (conj result (remove nil? (mapv first remaining))))))
This works. Check it out!

josh.freckleton16:03:08

you inspired it simple_smile

josh.freckleton16:03:43

or more generally:

(defn rotate [mat]
  (loop [remaining mat
         result []]
    (if (every? empty? remaining) 
      result
      (recur (map rest remaining)
             (conj result (remove nil? (mapv first remaining)))))))

(rotate (repeat 5 (range 5)))

turbopape16:03:43

Cool ! that’s how building on community’s knowledge works man simple_smile

turbopape16:03:51

I’m glad I helped it happen simple_smile

josh.freckleton16:03:31

actually, I think that flips the matrix around a diagonal, this should rotate it:

(defn rotate [mat]
  (loop [remaining (map reverse mat)
         result []]
    (if (every? empty? remaining) 
      result
      (recur (map rest remaining)
             (conj result (remove nil? (mapv first remaining)))))))

josh.freckleton16:03:52

(I map reverse over the input matrix before processing it

turbopape16:03:42

or you can into result, I think… let me try...

turbopape16:03:06

nope, forget it,

turbopape16:03:18

into will flatten every thing, so...

jr17:03:55

@josh.freckleton: I usually stick to higher level functions to do the legwork of transforming a data structure

(defn- keep-each
  "Applies f to each seqable in coll with keep to eliminate nils.
   Runs until every seq is exhausted"
  [f coll]
  (when-not (every? empty? coll)
    (lazy-seq
     (cons (keep f coll)
           (keep-each f (map next coll))))))

jr17:03:32

where rotate now becomes

(defn rotate [matrix]
  ;; Take the first element of each seq producing a new seq
   (keep-each first matrix))

jamesleonis17:03:49

It sounds like you’re writing a transpose matrix function, but with some data missing. If you’re willing to pad (and then remove, of course) with nil

jamesleonis17:03:08

Here’s a non-general idea

(def items [[1 2 3] [4 5 6] [7 8]])

(->>
 ; pad with nils
 (for [[x y z] items] (vector x y z))
 ; do the transpose
 (apply interleave)
 ; remove padding
 (filter some?)
 ; Resort back into partitions
 (partition-all 3)
 ; Convert back into vectors
 (map vec)
 ; Convert entire list back to vector
 vec)

jamesleonis17:03:08

Have you looked at core.matrix?

josh.freckleton18:03:08

@jr: awesome! Ya keeping it higher level is a good idea, I like what you did.

josh.freckleton18:03:24

@jamesleonis: the padding is a good idea, just a little expensive of an operation to do/undo many times. And I looked at core.matrix, but transpose isn't quite the same as rotate, right? Maybe it has something else I'd need though, and I like how it can use BLAS, sounds fast.

adamkowalski18:03:06

@josh.freckleton: as far as your “rotate” function is concerned if you want [[1 2 3][4 5 6][7 8]] to turn into [[1 4 7][2 5 8][3 6]] it might be easier to first write a function which extracts a column from a matrix by going over each row and taking the nth item from it. then you could map that function over your matrix using a range from 0 to the number of columns in your matrix - 1. by then end you should have a set of vectors representing each column in the matrix effectively rotating it 90 degrees

adamkowalski18:03:01

(def mat [[1 2 3] [4 5 6] [7 8]])

(defn column [matrix index]
  (->> matrix
       (map #(get % index))
       (filter (complement nil?))))

(defn rotate [matrix]
  (let [columns (apply max (map count matrix))]
    (for [c (range 0 columns)]
      (column matrix c))))

(rotate mat)

adamkowalski18:03:27

=> ((1 4 7) (2 5 8) (3 6))

base69818:03:23

Anyone have experience getting a jenkins build up for lein?

pbostrom18:03:37

hmm, I have not used that, we just always have latest lein installed on the jenkins workers

base69818:03:25

ah, with the shell executes?

base69818:03:27

seems easy even

pbostrom18:03:55

yes, most of our builds involve multiple calls to lein among other things so we just wrap everything in a bash script and have jenkins call that

josh.freckleton19:03:51

@adamkowalski: Ah, I hadn't heard of Neanderthal, that looks awesome, thanks

afhammad19:03:05

This has happened to me twice now: I’m in the repl, I run a rethinkdb query that throws an exception that renders the repl unresponsive. If i kill the repl (and any running java processes) and try to run lein repl again it keeps timing out. Any ideas? (Im on Mac)

manutter5120:03:30

@afhammad: Is the REPL’s port still in use by any chance?

afhammad20:03:34

@manutter51: as far as i can tell, no.

manutter5120:03:10

Other than that, all I can think of is lein clean

ghadi20:03:26

afhammad: it might be a naughty rethinkdb library

danielcompton21:03:42

@afhammad: where is the exception coming from? Is it thrown by the library, or by your code?

danielcompton21:03:18

Are there any side effects that are being run on lein repl startup, e.g. connecting to RethinkDB?

afhammad21:03:06

@danielcompton: exception from lib, no nothing explicit happening on repl start

afhammad21:03:52

FYI exception was due to user error not a bug in rethinkdb

jasonjckn21:03:39

in my project.clj

:ring {:handler search.api/handler
         :init search.main/init
         :nrepl {:start? true :port 7804}
         :auto-reload? true
         :configurator
         ~(fn [jetty]
            (doseq [connector (.getConnectors jetty)]
              (.setHeaderBufferSize connector 8388608)))

jasonjckn21:03:50

i get the error no reader function for tag function

jasonjckn21:03:00

while invoking uberjar

jasonjckn21:03:44

obviously it's the ~(fn ... ) part, not sure how to write this proper

weavejester22:03:23

As far as I know, Lein-Ring doesn’t support the :configurator option, @jasonjckn

jasonjckn22:03:54

@weavejester: you wouldn't know by chance how to solve HTTP 413 FULL head would you?

jasonjckn22:03:16

i'm passing

:adapter {:request-header-size 8388608}}
but no lock

jasonjckn22:03:50

fixed it, upgraded jetty