This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-01-06
Channels
- # architecture (17)
- # beginners (118)
- # boot (22)
- # cider (9)
- # clara (38)
- # cljs-dev (4)
- # clojure (213)
- # clojure-austin (4)
- # clojure-greece (1)
- # clojure-italy (4)
- # clojure-russia (5)
- # clojure-spec (7)
- # clojure-uk (5)
- # clojurescript (3)
- # cursive (21)
- # datascript (3)
- # datomic (31)
- # duct (12)
- # emacs (9)
- # graphql (5)
- # gsoc (1)
- # hoplon (9)
- # leiningen (2)
- # off-topic (37)
- # om (2)
- # onyx (2)
- # parinfer (5)
- # perun (4)
- # reagent (2)
- # spacemacs (1)
- # specter (25)
I’m looking forward to seeing a talk on the what/why of this. I don’t see how this does anything but confuse the clojure tooling landscape, or at least that’s how my jaded “oh cognitect is just on the maven/clojars/leiningen hate train again”. I’m quite sure this is largely my lack of ability to see the big picture
Did you read the blog post?
That is the what and why
Yes. I’m just not seeing it. I guess I’ll wait for the community to absorb it and explain why this makes my life better
There is no hate here. Leiningen has served a critical role in making Clojure tractable for many many people. It solved problems with the tools at hand and opened up the maven ecosystem. But in the meantime we lost sight of some of the original benefits of Clojure.
Many projects don’t need Leiningen or artifacts or Maven or Clojars for people to use them. clj and git deps dramatically lower the bar for creating and using those projects.
There's a bunch of stuff (beyond dep management) routinely difficult and unnecessarily tangled in Leiningen. A couple I recall: - how do I invoke this lein plugin's code from a regular Clojure ns (my workaround: fork, remove/adapt lein ns references) - how do I create a simple script that runs with my project's dependencies loaded? (workaround: lein-exec)
Thanks. I won’t argue. I’m all for progress. This doesn’t look like progress to me, but I’m 100% willing to be shown how it will be, if it is. Thanks for you work!
I wasn’t trying to raise specific concern - I was expressing my desire for a talk that goes into more depth about the motivation so I could more fairly evaluate it.
Pretty excited about git/local/jar deps as they are such a pita to get working with Lein
git dependencies are presumably just an occasional lifesaver, else you can end up like this https://thenextweb.com/dd/2016/03/08/cocoapods-popular-basically-took-five-github-servers/
Git deps are cached locally so unlikely to be an issue
They could be a problem on CI though. Not sure if CI providers transparently proxy/cache git requests; you need that extra layer as machines are constantly created from scratch
I don’t think what we are doing here is anything like the link above
Say my project has 10 git deps, wouldn't 10 git clones be performed with each push to CI?
probably. how is this different than downloading Maven artifacts on every CI push?
I may be mistaken, but Maven artifacts are simpler/faster to cache/download/cdn? Single files vs. directory cloning
it’s not obvious to me that these are significantly different
I’m not saying you’re wrong, just that these are assertions without evidence
try it out and let me know :)
there is nothing preventing you from making/using artifacts when that’s appropriate
that’s for sure
sounds like more than regularly :)
> there is nothing preventing you from making/using artifacts when that’s appropriate I based my assertions in the fact that Maven downloads consists of the discrete transmition of 2-3 files, which maybe can be a "dumb" http request
a git clone is definitely more heavyweight in terms of both IO and CPU load than a maven fetch, but I think it's hardly the case that we'll ever be a problem for github
git’s pretty smart :)
when it becomes an issue, I’m happy to look at it :)
the main issue with the cocoapod case, if I recall, was it was all in single repo, so all the users of cocoapod were constantly hitting that one repo
I have a couple open source library ideas. Is there a place where i could post the ideas and get feedback on what looks like it has the most promise and usefulness. I also know there are probably lots of great projects that need help to, i feel like helping a project is somewhat harder then making your own (assuming its a small enough endeavor).
Awesome to hear! Probably #clojure / #clojurescript are good for questions in the form "is there currently a good library for X?" #off-topic may be better for "I have X idea, wdyt?", although that could as well be asked here just my impression
Thanks Vemv. I’ll just pester for advice here then. Probably tomorrow.
(jcuda.cudnn.cudnnTensorDescriptor.)
(cudnnTensorDescriptor.)
the second works fine; the first gives ne am error of: clojure.lang.Compiler$CompilerException , java.lang.ClassNotFoundException@alexmiller that's a great workaround for missing ones, thanks
@drewverlee clojureverse might be a good place to put it, long term eyes on it.
anyone knows how to generate something like
<script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&callback=initMap" async defer></script>
with Hiccup?
I haven’t tried it, but I would expect something like [:script {:src "..." :async "async" :defer "defer"]
? Matching the html5 convention of <tag attr>
being the same as <tag attr="attr">
I tried a few things but nothing works
I tried this as well http://html2hiccup.buttercloud.com/
hallo everyone 🙂 is there a dedicated channel for discussing the engineering and design of Clojure? For instance, I'm particularly interested in how the for
macro implementation works -- but I have dozens of other questions as well 🙂
but it gives something that doesn't make sense either
@goomba clojure implementation gets discussed in this channel a fair bit, so I’d say just fire away 🙂
Okay great 😄 how the hell does the for
macro implementation work?? 😄 the fact that it can accept any number/order of :when:
, :let
, and :while
keywords is pretty astounding
when I macroexpand
the code it's pretty intimidating
have you looked at the source?
I have -- it's some pretty advanced macro work
it works on the bindings in terms of the current pair, then all the rest. with let / when / while it takes the current pair to construct the proper form, then nests all the rest inside the proper construct
it's one of the simpler things that for is doing actually
I guess it's a bit trickier than all that, but the basic structure to look for is "process this binding pair; then put the rest inside it"
okay but let's say I have (let [x (range 10) y (range 10) :let [res (+ x y)]] res)
... how on earth does it carry along the bindings in the proper order
I assume the outer let should be for
subtle but nuanced point as well is when you carry a symbol in a macro depending on how you write your macro you can lose the original value associated with the binding
it processes them one binding pair at a time
you can shadow, yes
okay so you're saying that it does something like
so, it starts with x, (range 10)
(mod-pairs here is empty) and treats the rest as just next-groups
no I'm still losing it
unless they're using a special construct that I don't know about
and since that pair doesn't start with :let or :while or :when it is going to emit a let form that expands the next-groups
recall that this is a macro, all we are doing here is constructing the lists thatwill create the code we actually want to run
(for [x (range 10) y (range 10 :let [res (+ x y)]] res) ;;=>
;=> pseudocode option #1
for x in range 10:
for y in range 10:
res <- (+ x y)
yield (???) res
I don't know if htat helps you, that doesn't help me at all
in fact it actively confuses things because for
doesn't do a for loop
it isn't a loop, it's a generator
I'm trying to understand in a way I can wrap my mind around what the generated code looks like
if you take away the chunking and just do one at a time
reading the macro-expanded code is a weird place to start, to me - why not look at the source? it uses emit-bind
to process each binding pair
in that context, it figures out what kind of form to generate, and then puts all the other bindings inside a recursive call to emit-bind
inside that form
I found the source pretty intimidating haha
and I couldn't find any articles discussing it 😛
the source is less intimidating than the macroexpansion
okay let's annotate it then and see if we can't gain some better understanding
(assert-args
(vector? seq-exprs) "a vector for its binding"
(even? (count seq-exprs)) "an even number of forms in binding vector")
that's pretty much just error checking I thinklet's do this in a thread if you don't mind
not at all
(let [to-groups (fn [seq-exprs]
(reduce1 (fn [groups [k v]]
(if (keyword? k)
(conj (pop groups) (conj (peek groups) [k v]))
(conj groups [k v])))
[] (partition 2 seq-exprs)))
okay so let's see what is that doing
I see we're splitting up the k v pairs in the seq-exprs
so it's figuring out whether the left side of the binding is a keyword
the k / v is binding, value
(conj (pop groups) (conj (peek groups) [k v]))
what do you suppose is going on there?
with a vector, pop efficiantly removes the last item, and peek efficiently fetches the last item
that idiom is the efficient way to update the last item in a vector
sure, but I mean, why are we doing that differently for keyword than a non-keyword
I'm not sure I can visualize the underlying data structure
because keywords don't introduce a new collection to iterate
so instead of introducing a new iteration nesting, they introduce something which acts in the last context - thus they get added to the thing before it
what do you suppose is the data structure of groups
? a vector?
this is exactly what you were asking about - how to make :let and :when etc. do the right thing
okay it starts out as an empty vector
it's a reduce, so we know for a fact it's a vector
right
that's why I started with "ith a vector, pop efficiantly removes the last item, and peek efficiently fetches the last item"
let's say we have [x (range 10)]
as the first pair
implicitly, "this is a vector"
that will make groups <- [[x (range 10)]]
no - groups is empty
if the next is :when (< x 3)
k is x
right and then it does
(conj groups [k v]))
right, next step OK
now groups
is [[x (range 10]]
, k
is :when
, and v
is (< 3 x)
so [[x (range 10)]]
is groups, and k
is :when and v
is (< x 3)
so it's going to make that into
[[x (range 10) :when (< 3 x)]]
?
because it applies (conj (pop groups) (conj (peek groups) [k v]))
?
right
gotcha, fascinating
okay I think I understand that part
now this part
anything starting with a keyword, we know that goes inside the thing before it
err (fn [& msg] (throw (IllegalArgumentException. ^String (apply str msg))))
... that's just an error handling function
emit-bind is where things get interesting
yeah... do you have time to look at that?
I don't want to take up your whole day haha
if you skip to the bottom, it calls to-groups on the seq-exprs to generate a list of binding contexts, then for each one it uses emit-bind to create the next nested form
well, emit-bind recursively creates each nested form itself that is
by the way the reason I'm working on this is I'm doing my level best to create Clojure hosted on Python
it's going pretty well but the for macro is killing me
okay let me unpack what you just said
you're talking about this part
`(let [iter# ~(emit-bind (to-groups seq-exprs))]
(iter# ~(second seq-exprs)))))
heh even this part is pretty difficult to implement
(fn emit-bind [[[bind expr & mod-pairs]
& [[_ next-expr] :as next-groups]]]
right- putting the stuff in that top let block in context, how it actually gets used
gotcha
big picture, emit-bind is going to take each of those groups made in the to-groups reduce call, and make a sub-form out of it
(the guys at cogintect are very very clever)
okay, tracking
so it will take something like
and then it will self-call to do the same thing to the next - making it a nested form
[[x (range 10) :when (< 3 x)]]
and produce some kind of form out of it
right
now this part
we can probably come back to but I feel like there is a very important reason they're doing this
giter (gensym "iter__")
gxs (gensym "s__")
so they're kind of indirectly calling the autogensym forms instead of the forms directly
we need to create new names for each iteration
ohhhhhh
if it just used iter# every level of self-call woudl get the same name
so for instance if we just used x
it would get shadowed?
well x is not even valid as a binding inside `
so it needs to be x#
but then x# would shadow, yes
and x# is a shorthand for (gensym "x")
- so they are just doing that same thing but in a more controlled way
gotcha
alright so now we're at do-mod
f*ck that formatting is bad
deleted that block we both can just look at the source
that's what I've been doing this whole time, easier to do it side by side
skipping down to the end I think now I see where the major form that gets emitted is
the false branch of (if next-groups ...)
line 4641
oh I'm using the output of (source for)
- I could open github though
okay so the two "loops" being setup
oh you aren't even looking at master :P
oh what??
that's alpha-14
we're looking at the same thing now though
oh haha
I just dug up master
which should we look at?
master would work for me, I like my editor more than the github UI
okay, I'm looking at master
in master, line 4590 is the start of for in core.clj
so 4632 and 4640 (in master)
the actual body being emitted in a loop inside a lazy-seq on line 4655, which is inside a let in the else-branch of next-groups
right
okay so a lot of this has to do with chunking results
that's the difference between do-cmod
and do-mod
I'm sorry brb I need another coffee and my dogs are going crazy, be back in 5
okay sorry back
so if we're ignoring chunked sequences, we're close to being done
which is fine, for me, conceptually
so at 4625
either we have another groups or we're done with that groupses
ohhhhhhhhhhhhhhhhhh
I get it now
wow, that's very cool
so this is really a transformation
yes - every macro is right/
from [x (range 10) :when (< 3 x)]
to
(let [x (first groups)
xs (rest groups)]
(if (< 3 x)
(cons x res) then continue macro
res then continue macro
very very interesting
okay, thanks sir! That gives me what I need
REALLY appreciate your help
cool - it's also helpful to me to actually go in and read in detail
obviously my pseudocode isn't very accurate but it gives me enough to implement
thanks @noisesmith!!! you were right looking at the source code was a much better idea
I can’t get Ultra’s nice stack traces to work. my profiles.clj looks like this:
{:user {:plugins [[venantius/ultra "0.5.2"]]
:ultra {:repl {:width 250}
:stacktraces true}}}
and the pretty-printing of data structures in the REPL works fine, but stacktraces are as ugly as ever. Anything else I need to do to activate stack traces pretty-printing?Seems that Ultra loads correctly and overwrites pst
as it should:
user=> clojure.repl/pst
#<Fn@9429385 io.aviso.repl/pretty_pst>
I solved my immediate problem by doing this:
(defn refresh []
(let [e (clojure.tools.namespace.repl/refresh)]
(if (instance? Throwable e)
(clojure.repl/pst e)
e)))
Still feel like this shouldn’t be necessary though…the problem , I bet, is that refresh deletes and replaces the things that ultra is updating
so one alternative is to initialize ultra after calling refresh, another would be to exclude the things ultra updates from being refreshed