Fork me on GitHub
#clojure
<
2018-06-11
>
danielsz00:06:22

@rplevy The dependency hell problem you mentioned was never solved in the Java world, where things get even hairier because uberjars are sometimes used as dependencies, and now you need to resort to class relocation.

danielsz00:06:35

@rplevy Thank God we are spared this since in Clojure our dependencies are non AOT, source-only.

dpsutton00:06:31

There's even a utility called Mr Anderson which will copy the source into your own namespaces to prevent version conflicts

danielsz00:06:13

The Maven's shade plugin has more options to include/exclude dependencies than Boot/Leiningen. https://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html

danielsz00:06:00

Which leads me to plug in the new build tool I released today: Meyvn.

danielsz00:06:42

It uses the Shade plugin.

dpsutton00:06:50

Oh I saw that on Twitter. That name is gonna be confusing in speech

john14:06:52

One idea: Having a standalone executable is cool. And it might be how I would actually use the tool day to day. But you could also have a pure clj -m meyvn.core deploy, for instance, for folks that don't want to mess around with the system's path.

john14:06:40

ah, I see you can pretty much accomplish that under the hood

dpsutton00:06:10

Also should add more information about what that plugin sends home

dpsutton00:06:20

I looked into what info it sent home

danielsz00:06:39

It sends home the Group id.

danielsz00:06:34

It says that in the README, but I could publish the code.

danielsz00:06:06

The presentation video teaches you how to spell the name! 😜

danielsz00:06:32

Basically, pronounce it Mayvn.

danielsz00:06:56

BTW, the capsule project makes some claims about better conflict resolution, but I haven't the chance yet to verify those claims.

danielsz00:06:34

But if it is true, then it could be incorporared in Meyvn.

danielsz00:06:28

This is the rationale of Meyvn, leveraging Maven and its practices.

dpsutton00:06:30

ok. i just remember reading your readme. it said you might reach out to them to offer (require?) the commercial version. it struck me that it might be interpreted that companies might lose access to this tool without the upgrade

danielsz00:06:24

Nope. It's open source. And I want to be able to continue to write open source software. So I'm going to reach out to companies and offer them a commercial license. It's up to them if they want it or not.

dpsutton00:06:33

yeah. but it's dual licensed. just any worry about build tooling can cause people to worry. was just thinking if you made the language clearer, like "I might reach out to you to offer more services but you can always use the free community version".

dpsutton00:06:06

but this looks like a nice complement to the clj tooling

danielsz00:06:27

Sure, that's a good advice. I'm going to to take it, too. Thank you.

dpsutton00:06:36

looks good to me!

danielsz00:06:43

Awesome. Thanks.

lockdown-00:06:25

if you do (count (range 1e8) are the items of the seq being garbage collected as they are counted?

danielcompton00:06:37

@danielsz that section could be better further up, or at least mentioning that there are metrics collected further up. Also, calling out explicitly what is reported would be good.

danielcompton00:06:53

And it looks like these metrics get sent to https://paste.ee/, not you directly?

danielsz00:06:45

Yes, private pastes to http://Paste.ee.

petterik08:06:59

@lockdown- yes, or the items can be garbage collected as long as you don't hold on to the head of the collection. See the difference:

clj -J-Xmx128m
user=> (let [r (range 1e8)] (count r) r)
OutOfMemoryError GC overhead limit exceeded
user=> (count (range 1e8))
100000000

petterik08:06:56

Also, if you do want to count ranges, if you pass a long as the argument to (range) it's counted in O(1) instead of O(n), because LongRange implements Counted.

user=> (time (count (range 1e8)))
"Elapsed time: 5691.129185 msecs"
100000000
user=> (type 1e8)
java.lang.Double
user=> (time (count (range (long 1e8))))
"Elapsed time: 0.041133 msecs"
100000000

pesterhazy09:06:17

neat head-holding demo using clj -J-Xmx128m!

pesterhazy09:06:43

the new tool makes things so much easier

vale13:06:27

I've been trying to figure out specs but I can't find any info on a certain case: How do I validate hashmaps where keys aren't keywords? I have to expect hashes that have strings for keys (like "@language"). How can I create specs to validate this case? Example: { "@language" "en", :type "Entry", :attachments [] }

vale13:06:10

The JSON parser can convert it to a keyword :@language and I can hack around it like this:

(def language (keyword (str *ns*) "@language"))
(s/def language string?)
But if I try to use it like (s/valid? language my-map) I get an error "Unable to resolve spec: :user/@language"

mpenet13:06:43

s/def takes keywords not symbols

mpenet13:06:06

common mistake, and the other way around with s/fdef

mpenet13:06:48

it's a bit sad that spec doesn't guard against this, given that it ... validates things

vale13:06:40

that's not the case

vale13:06:58

>k must be namespaced keyword or resolvable symbol

vale13:06:07

i give it a resolvable symbol

vale13:06:15

the symbol resolves to a namespaced keyword

vale13:06:29

and from the error it's clear that it's trying to look that up

mpenet13:06:33

odd, never seen that used

vale13:06:53

i have to hack it like this because i can't just type ::@language that gives a syntax error

vale13:06:36

it's really weird because it acts like it registered the spec alright, yet it fails to resolve the keyword it accepted for registration

mpenet13:06:37

ah yeah, you're right

vale13:06:20

I feel really stuck here. I don't know how to use string keys in specs, and I can't seem to use keywords.

mpenet13:06:33

seems like a bug

vale13:06:47

very much so, sadly

mpenet13:06:17

user> (def foo ::asdf)
#'user/foo
user> (s/def foo string?)
user/foo
user> (s/valid? foo "asdf")
Exception Unable to resolve spec: :user/asdf  clojure.spec.alpha/reg-resolve! (alpha.clj:69)
user> (s/valid? ::asdf "asdf")
Exception Unable to resolve spec: :user/asdf  clojure.spec.alpha/reg-resolve! (alpha.clj:69)
user> (s/get-spec ::asdf)
nil
user> (s/get-spec foo)
nil

mpenet13:06:06

not sure I see the link with string keys?

mpenet13:06:25

you can use map-of, but yeah s/keys is a keywords only thing

vale13:06:30

The map comes in in JSON and the parser converts the keys to keywords. I guess I could tell it to keep the keys strings if that'd make spec work but I guess it wouldn't.

mpenet13:06:03

if the parser converts to keywords you're good to go normally

vale13:06:28

The problem is, as outlined above, that some of the keys contain the sign @

vale13:06:40

which apparently murders the spec macros

mpenet13:06:53

I thought you could make get by with eval, but no

vale13:06:28

If you know of a way please tell me

mpenet13:06:42

actually it works

mpenet13:06:43

(eval `(s/def ~(keyword "user" "@asdf") string?))

mpenet13:06:51

it's horrible but heh

vale13:06:03

haha what

mpenet13:06:14

with ns it would be something like

(eval `(s/def ~(keyword (str *ns*) "@asdf") string?))

mpenet13:06:29

@alexmiller should that be filed as a bug (docstring or lib bug)? Since s/def docstring states :

Given a namespace-qualified keyword or resolvable symbol k, and a
  spec, spec-name, predicate or regex-op makes an entry in the
  registry mapping k to the spec

vale13:06:25

You just saved the day

vale13:06:11

user=> (def context (keyword (str *ns*) "@context"))
#'user/context
user=> (eval `(s/def ~context string?))
:user/@context
user=> (s/explain (s/keys :req [context]) {context "asd"})
Success!
nil

gfredericks15:06:13

should libraries list a org.clojure/clojure dependency? should it be in the :provided profile?

hiskennyness16:06:39

Am I the last one to give up on ordering my code bottom to top so I do not have to be forever declaring subroutines? The Pascal nature of Clojure beats another top-downer into submission? Scroll to the bottom to find the big picture? #justsayin

hiskennyness16:06:38

I write code top-down, why not organize it caller over callee? It’s OK, I will live. Nice callback to when Pascal went toe-to-toe with C for micro primacy.

hiredman16:06:52

there is a hackernews thread where rich discusses this. if I remember the gist of it is he wants things to work the same way in the repl and in files

hiskennyness20:06:58

Thanks! That is a great insight into his thinking. I thought there was something about Java that necessitated the declares. The repl rationale is odd, though. Who writes more than a tweet at the repl? I never use the REPL that way, been Lisping for twenty years. Thx again.

russ76720:06:11

I don't have a cool video, but I write a lot of stuff at the repl too. But who am I to argue with someone else who remember Pascal.

gfredericks21:06:15

I guess it's worth emphasizing that there's a style where you eval forms from a text editor directly into the repl when doing those sorts of things, it's simpler to keep in your head that what you're doing is very close to what actually happens in production

hiskennyness09:06:07

Yes, and right at the start of that video we discover that what people mean by REPL is “dynamic development”. But once we recast it that way and I am actually working in a text editor with REPL evaluation a keychord away, it is no longer relevant that lower level subroutines must be defined before higher. And this brings us to the compilation units Rich mentioned. I have a 90kloc Lisp program that builds as one compilation unit thanks to that feature being offered by AllegroCL. Most Lispers use ASDF, which is unbearable for me to use precisely because it lacks that skill.

hiskennyness09:06:11

Might I add that dynamic development began with BASIC (trying to top my Pascal reference.)

hiskennyness09:06:31

Heh-heh — was toggle switch programming dynamic? Missed that one.

gfredericks10:06:57

> it is no longer relevant that lower level subroutines must be defined before higher. I'm not sure how this relates to the things before it

hiskennyness10:06:28

Rich’s rationale for the ordering of function definitions is that that is how Lispers work, entering forms sequentially at a REPL, building up the dynamic environment bottom to top. Of course it would be a hard way to start each day, so we have source files. Once we have source files, we can have main at the top and sub-sub-sub down at the botton out of the way. If we had a source file treated as a compilation unit.

hiskennyness10:06:11

The video was making a pitch to folks accustomed to compiling the whole thing before being to able to witness the smallest change, with these same folks not accustomed to interrogating a running application.

gfredericks11:06:50

is your argument basically that "only the dynamic environment part of this is important, and that doesn't preclude compiling whole files"?

hiskennyness11:06:20

Not sure I follow your last. Of course we must deal with whole file compilation when re-booting our apps. Indeed, that is the nit I am picking: I have to sprinkle declares all over because CLJ does not even recognize a single file as a compilation unit. It treats a source file as if I were sitting literally at a repl typing in each form one by one. But as Rich said, hey, use declare. Full circle. 🙂 Me, I am going to just re-order my source the Pascal way.

gfredericks11:06:59

I was trying to understand your argument; maybe you were trying to make a distinction between using a repl to introduce new code vs using a repl to interrogate a program?

hiskennyness19:06:26

I am getting a kick out of the fact that every guess you make at what I am saying makes me first understand your guess before I can comment. I detect a slippery slope! 🙂

gfredericks19:06:10

oh well ¯\(ツ)

hiskennyness19:06:15

But here are my points. No, “the Lisp Way” does not justify a single source file being ordered callee first. Second, I doubt a single source file being treated as a compilation unit gets into craziness. And we already have requires, so there would be no need for CUs to cover anything more.

gfredericks19:06:29

but what do you think of the "works the same way as the repl" attribute? that it's not valuable enough to justify the awkwardness?

hiskennyness19:06:48

Third, I started this because I found messing with declares annoying. So, fourth, I am now ordering my code callee first. Life is good! 🙂

hiskennyness20:06:14

Again, nobody sits at a repl writing interesting code. Why, when we have Cider? And as you noted, that is not like typing source into a command line of a REPL.

hiskennyness21:06:20

Oh, I do not merely eval unless I am learning syntax, debugging, mebbe testing a chunk of code But if I am building a complex function that will call a few that call a few, I type in all the source and eval a snippet, generally as a clojure test.

gfredericks23:06:08

so when you referenced cider you meant that you use it for things like running tests, and documentation, and whatnot?

gfredericks11:06:14

how many units of compilation is (def f (fn [] (def g (def h)) (def i (fn []))))? In either the functions case or the vars case

gfredericks11:06:54

(or would you say some aspect of that should be prohibited?)

sophiago11:06:32

I think six?

sophiago11:06:36

I've confused myself now, though...

sophiago11:06:59

I'm pretty sure the answer is two, but then I have to refine my understanding of this.

sophiago11:06:53

I suppose what I actually care about is what ends up being a Java class since that means the compiler emits a chunk of bytecode for it. I'm not sure if that strictly makes that the compilation unit. If so, then it's actually a different concern from what Kenny is talking about.

gfredericks12:06:34

if I had to bet five dollars on it, I would predict that rich would say the compilation unit is a single top level form i.e., for my example it would be just one

hiskennyness14:06:48

In Lisp-land anyway a compilation unit decides what things are visible to the compiler during one compilation. The Lisp standard says that a source file is a compilation unit, so caller and callee can appear in any order in one source file. AllegroCL lets me stipulate that an entire collection of source files is one compilation unit, so it will not consider a callee undefined until it finishes compiling all 80kloc across dozens of files. So multiple CUs in one expression is different. 🙂

gfredericks14:06:30

this seems highly relevant to that question: https://technomancy.us/143 So this doesn't compile: (do (defn foo [] (inc bar)) (def bar 12)) What does that say about whether it's one or many compilation units?

sophiago16:06:23

I'm not sure. I wouldn't have expected that to compile.

sophiago16:06:10

Oh, he just says it: "Clojure, like many Lisps before it, does not have a strong notion of a compilation unit."

sophiago16:06:39

"Strong" semantically speaking. Practically the answer is "top level forms," which doesn't seem to map to how the compiler emits bytecode at all.

sophiago18:06:11

Now that I've had some coffee (I'm on a bizarre sleep schedule, which today amounted to 4-5 hours this morning)... I think the answer to, "What does that say about whether it's one or many compilation units?" is that while do is obviously a special form the first var that snippet creates calls a symbol that's not interned until after it. do blocks are obviously meant to be sequentially executed, but that post on the "Gilardi Scenario" makes it seem something like that snippet should work regardless. I must not be understanding the issue.

sophiago18:06:19

Err, I guess the difference is the example in that post is still in the correct order inside the do block. I may still not be so lucid...

phil67216:06:20

I had a problem in JS-land last week where different settings of webpack caused the production build to fail (but not the development build) because an ES6 fn wasn't hoisted as normal. Moving the fn declarations around fixed it. Now I don't know what to think...

hiskennyness16:06:51

@phil672 Reminds me of when code that ran fine with the tooling that comes with lein mies then failed when I tried building with figwheel. Then someone spanked me for not figuring out why. Tough crowd. Where would I begin?

aaelony18:06:37

Hi everyone, there used to be a log of the slack channel (https://clojurians-log.clojureverse.org/clojure) but the last entry is ~ April 17. Is this a known issue or is there a reason it no longer captures recent dates?

spieden18:06:32

maybe someone forgot to pay =)

aaelony18:06:04

sounds plausible

spieden18:06:13

hmm yeah that’s odd. i imagine there’s a job supposed to be running somewhere that scrapes these

aaelony18:06:34

how are the bots going to learn clojure best practices if they don't have recent slack archives to read? 🙂

manutter5118:06:56

It is a known issue, IIRC someone is working on restoring/improving the data feed but I forget what the status is on that.

dpsutton18:06:30

i understand they have the logs but haven't fixed up the display of them yet. i could be mistaken

spieden18:06:41

hmm, clicking that link i get: > Relay is shutting down

lee.justin.m19:06:03

@spieden the relay thing was a bit of a diversion. the rest of the thread is the real context.

spieden19:06:18

got it, thanks

seancorfield19:06:06

Discussion of Slack/alternatives belongs in #community-development just FYI.

lee.justin.m19:06:57

sorry i meant to link to the thread rather than to that specific posting. the point wasn’t meant to be about slack alternatives, its about the work being done to fix the logging front end