Fork me on GitHub
#clojure
<
2020-01-05
>
tcrawley00:01:04

PSA: We just released some changes to Clojars. - rsyncing of the Clojars repo is now disabled (see https://github.com/clojars/clojars-web/issues/735) - GET or HEAD requests to the on-disk repo will now redirect to the CDN-fronted repo (see https://github.com/clojars/clojars-web/issues/734). This means that GET https://clojars.org/repo/whatever will return a 302 Found status code with a location header pointing to https://repo.clojars.org/whatever. We'll change that to a 301 Moved Permanently once this change has had time to settle and we don't have any issues reported that causes us to revert the change. Resolving dependencies should continue to work, since aether (the lib used by boot/lein/maven/tools.deps to resolve dependencies) should follow redirects correctly. Deploying to Clojars should be unaffected by this change - you can still deploy to https://clojars.org/repo/ or https://repo.clojars.org. If you (or someone you love) is negatively affected by either of these changes, please let us know by commenting on one of the above issues. See https://groups.google.com/d/msg/clojure/ZXBOQD5Ubbc/D4ktv8aFCQAJ for the rationale.

didibus08:01:44

What's the point of allowing to define an unbound non-dynamic var? Such as: (def a)

dpsutton08:01:09

(source declare) is an instance of binding to an unresolved binding

dpsutton08:01:17

And why it is necessary

dpsutton08:01:38

But nothing prevents repeatedly redefining the same var right? Namespaces are inherently mutable objects

didibus08:01:21

Ya, but I mean: (def a nil) is a thing too.

didibus08:01:03

Are you saying that (declare a) is just (def a) under the hood?

dpsutton08:01:27

That’s what (source declare) is telling me in replete

didibus08:01:58

Hum, it is haha

didibus08:01:00

`(do [email protected](map #(list 'def (vary-meta % assoc :declared true)) names))

didibus08:01:07

With a :declared true annotation

didibus08:01:03

I kind of would think it best if (def a) was a compile error. Since it feels like a mistake to me, or at least a pretty bogus thing.

dpsutton08:01:54

I’ll go grep the source in a minute, but I’m guessing there’s special casing for unbound vars in the compiler

dpsutton08:01:20

So you need to disambiguate nil (a legal value) from an unbound var

didibus08:01:58

Well, for the declare case I understand

didibus08:01:47

I guess I'm a bit surprised by it. I'd have thought that (def a) would be a compile error. Even more so that (def a) (a) would be one. But it isn't.

didibus08:01:03

So it seems you don't really need to declare, you can just define an Unbound var

dpsutton08:01:37

it looks like the only place that checks for the :declared meta is now no longer used? includesExplicitMetadata seems never called

p-himik09:01:58

I think it's used to check whether you're re-def'ing something and whether the signatures, if any, are different.

didibus08:01:54

Interesting, the meta makes it feel like maybe it wanted to not allow an unbound def unless it was specifically a declared one. But who knows

andy.fingerhut08:01:03

You do not get exception attempting to eval the second expression of (def a) (a) ?

dpsutton08:01:20

i think the thought was that (declare a) didn't do any more work than just (def a). was expecting a compiler error rather than a runtime error

dpsutton08:01:54

ie, expecting it to be the same as just (a) without the declaration

didibus08:01:24

Ya, but its run-time exception.

didibus08:01:45

I assumed since there is declare, that the reason was because an empty def was not allowed.

andy.fingerhut08:01:14

(declare a) followed by a call to a that is inside of another defn is not a compiler error by design, yes?

andy.fingerhut08:01:43

That's the primary use case for declare

didibus09:01:55

Ya, I understand for declare. What surprised me is that is is also true of (def a) (a)

andy.fingerhut09:01:05

Sorry, I may be catching up to what you meant earlier.

didibus09:01:08

And actually, declare under the hood just does a def

didibus09:01:21

no worries

dpsutton09:01:32

this is not true in clojurescript though. There the analyzer does use the metadata set from being declared rather than def'ed without an object

andy.fingerhut09:01:46

I've seen declare used in a few projects, but don't recall ever seeing anyone do def without a value.

didibus09:01:15

Sorry, wrong example above, I mean: (def a) (defn some-other-fn [] (a)) or some like that

andy.fingerhut09:01:11

Eh, it's at most a weird wrinkle that affects pretty much no one.

didibus09:01:15

Where I'd have expected (def a) to throw a compile error, and that you'd need to use declare instead.

dpsutton09:01:36

i'm not complaining. i'm just learning 🙂

andy.fingerhut09:01:48

There are much subtler bugs in code I've seen.

didibus09:01:03

ClojureDocs has a note about declare that says you can also use (def a) instead. I feel we should almost delete that note 🤔

andy.fingerhut09:01:05

that don't cause exceptions or obvious failures.

didibus09:01:24

Seems like it would be bad style

andy.fingerhut09:01:42

Like, see if you notice the bug here:

(deftest my-critical-test
  (= 5 (some-important-function {:a 1 :c 4})))

didibus09:01:04

Missing is ?

didibus09:01:35

And uneven map ?

didibus09:01:45

That's a compile error though I'm pretty sure

andy.fingerhut09:01:46

The man wins a prize! Not very obvious in a big file of tests, though, and I've seen dozens of projects with them. Granted, maybe not for "critical' tests, but they lie there doing nothing useful, catching no problems.

didibus09:01:29

Ya, I agree. I'm surprised deftest doesn't validate that. Would there ever be a reason to deftest without an is inside?

deep-symmetry21:01:50

Some tests care only that no exception was thrown, they don’t need is for that.

deep-symmetry21:01:24

Although, I suppose if you want to see the little green marker in Cursive or CIDER, you can throw an (is true) after the thing that might have blown up… 🙂

noisesmith19:01:58

also, you can invoke is inside a function inside a test, since is is dynamic, not lexical

andy.fingerhut09:01:37

missing is. No compiler warning or error. No run-time error.

didibus09:01:04

Sorry, I meant for the uneven map {:a 1 :4} but I guess that was just a typo 😛

andy.fingerhut09:01:22

Sorry, that was Slack doing something weird with a colon followed by a b

didibus09:01:05

I'll add an issue to clj-kondo about deftest without an is as well I guess 😛

andy.fingerhut09:01:07

I've edited it, which is unfair to your former answers. The missing is is the only mistake I intended to write.

andy.fingerhut09:01:22

One of the first things I implemented in Eastwood

didibus09:01:54

Ah that's good actually. Got eastwood running at work, glad it's got that check and I don't need to worry about our tests having that problem 😛

andy.fingerhut09:01:20

Along with finding this mistake:

(deftest my-good-tests
  (is (= 5 (something-important {:a 1 :c 5}))))

(deftest my-good-tests
  (is (= 7 (something-else-important 8))))

andy.fingerhut09:01:48

If you want a hint: very easy copy-and-paste mistake to make while maintaining a test file

didibus09:01:49

Oh, redefined the previous test ?

andy.fingerhut09:01:17

yes. The first ones never get run. No compiler error or warning. No run-time issues.

didibus09:01:00

Ya, that would be another good clj-kondo linter

andy.fingerhut09:01:24

And an existing Eastwood one for a few years.

didibus09:01:59

Actually seems clj-kondo catches that last one already

andy.fingerhut09:01:34

It's certainly nice if clj-kondo can do everything Eastwood does and more, given how much attention borkdude puts into it.

didibus09:01:40

I like Eastwood, but its pretty slow to run, which is why I have it only setup to run on build, and not like inside my editor as I type

didibus09:01:18

I feel clj-kondo is the best thing to happen to Clojure in the last year 😛 Sorry to any other amazing thing I'm forgetting. But for me, it has been a huge help to quickly correct some stupid mistakes like that. And its also great for beginners to help them out.

didibus09:01:34

There's a lot of things I feel I always wanted the Clojure compiler to catch and make as errors, which clj-kondo addresses as a linter.

didibus09:01:52

Hey, it appears clj-kondo already lints for missing test assertions in deftest!

andy.fingerhut09:01:22

Toss borkdude a hat tip of thanks when you see him next -- I don't know how much time he spends on that and other projects, but it seems huge.

didibus09:01:44

He's been on fire. clj-kondo, sci and babashka are probably the 3 Clojure projects I'm most excited about to see grow in the next few years.

didibus09:01:58

So ya, big shout out!

lmergen10:01:03

@borkdude ^

borkdude10:01:09

@didibus @andy.fingerhut there's also https://github.com/borkdude/missing.test.assertions for catching missing test assertions at runtime which is a pretty simple and small but effective lib

slipset16:01:13

Re #eastwood slowness. Most of it is caused by the analyzer. I’ve got a patch against tools.analyzer (https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FTANAL-130#issue/TANAL-130) which should speed things up a bit.

slipset16:01:11

Since tools.analyzer is bundled with Eastwood, I could of course monkey patch it.

slipset16:01:11

Since tools.analyzer is bundled with Eastwood, I could of course monkey patch it.

vemv16:01:58

I hit Eastwood on every (refresh) (asynchronously), so I'd be happy to check out the speedup! Perhaps the improvent can be offered in this fashion https://github.com/technomancy/leiningen/blob/4c601033354640aefa0bdf5c09bdd3646ff5e80c/sample.project.clj#L358

slipset17:01:10

Are you running eastwood from lein or from deps?

vemv17:01:33

I use Lein to manage my deps but I run Eastwood as a plan lib, integrated w/ refresh and reset

slipset17:01:28

Ok, was asking since if you used deps, you could have tried out the latest commits on master now.

vemv17:01:58

also can with lein checkouts is the feature ready? What speedup should I expect?

slipset18:01:56

This is what I get on our project:

== Linting done in 386103 ms ==
real	7m22.211s
user	8m40.180s
sys	0m9.512s

== Linting done in 253887 ms ==
real	5m4.637s
user	6m27.659s
sys	0m8.197s

vemv18:01:07

seems worthwhile! curious about its perf in smaller libs as well. can try myself

slipset18:01:44

Smaller libs should see smaller gains I guess.

erwinrooijakkers20:01:29

Hi I basically have a tree structure like this:

(def tree
  [:a :b :c
   [[:d
     [[:e]
      [:h :i]
      [:y :z]]]
    [:f :g]
    [:q :x]]])
Now I want to find all paths to the leaves in this tree:
[[:a :b :c :d :e]
 [:a :b :c :d :h :i]
 [:a :b :c :d :y :z]
 [:a :b :c :f :g]
 [:a :b :c :q :x]]
What is a good way to accomplish this?

Alper Cugun20:01:16

I struggled with this, here’s my answer to my own question and a bunch of other ways to work with trees in the other answers: https://stackoverflow.com/questions/59515105/how-to-calculate-the-sum-of-all-depths-in-a-tree-using-basic-recur/59519407#59519407

Alper Cugun21:01:05

You could modify mine to have not just the current branches under consideration in branches but the full path to get there from root instead.

erwinrooijakkers21:01:50

Haha I recognize some aoc there

erwinrooijakkers21:01:42

I don’t really see how to modify that code to get the paths instead of the depths

erwinrooijakkers21:01:07

Should keep the path so far instead of the counter or something

erwinrooijakkers21:01:29

Not sure if that’s possible. I’ll look again after sleep

Alper Cugun21:01:54

Drop both the counters, which are irrelevant to you.

Alper Cugun21:01:37

And branches will contain all the nodes at a certain level, say: ["B", "G"] and then go to the next level.

Alper Cugun21:01:01

Instead of just the nodes, you should modify recur so it has a list of nodes.

Alper Cugun21:01:08

(My tree may be off)

Alper Cugun21:01:29

But it should have something then such as: [["COM", "B"], ["COM", "G"]] and at the next level it replaces the first list with ["COM", "B", "C"]. Something like that.

Alper Cugun21:01:31

I could also be off.

didibus02:01:58

Isn't that tree a bit weid?

didibus02:01:11

What's :a :b :c ?

didibus02:01:22

Is that the value of the root?

didibus02:01:23

Anyways, you probably just want to keep track of the path as you traverse the tree until you reach a leaf

didibus04:01:49

(def tree
  [:a
   [[:b
     [[:c
       [[:d
         [:e
          [:h [:i]]
          [:y [:z]]]]
        [:f [:g]]
        [:q [:x]]]]]]]])

(defn find-paths
  ([node]
   (find-paths node [] []))
  ([node paths path]
   (if (vector? node)
     (into []
           (mapcat
            (fn [child-node]
              (find-paths child-node
                          paths
                          (conj path (first node)))))
           (second node))
     (->> (conj path node)
          (conj paths)))))

(find-paths tree)

erwinrooijakkers21:01:48

🙂 thanks a lot! Changing the data structure helps to make the recursion more understandable. :a :b :c is indeed the first nodes. 🙂

folcon22:01:12

Any reason transients don’t support first peek or empty?? The docs say:

Transients support the read-only interface of the source, i.e. you can call nth, get, count and fn-call a transient vector, just like a persistent vector.
Which implies they should…

andy.fingerhut22:01:20

It could be simply that implementing the necessary APIs for those operations on transient Java objects was overlooked. I looked through Clojure tickets in JIRA, and didn't notice any issues asking about that (but did find a dozen or so issues related to transients that have been fixed years ago).

dpsutton22:01:22

i thought i remember a ticket about empty? and that was going to be added?

ghadi22:01:54

There is a long historical debate about how many ops transients should support. They intentionally do not support all of them

folcon22:01:56

I’m getting the impression that the scope of using them is much smaller than I initially expected

andy.fingerhut22:01:06

https://clojure.atlassian.net/browse/CLJ-1872 is the ticket about implementing empty? on transients

folcon22:01:12

I should probably looking at transducers

andy.fingerhut22:01:14

Transients tend not to remain transients for long

folcon22:01:28

That’s fair

ghadi22:01:43

The goal of using transients should be to get to a persistent data structure, not to do a bunch of compute using transients

folcon22:01:40

It is in this context

folcon22:01:59

The problem is the persistent structure I’m trying to get is a map…

folcon22:01:08

All of this is done in a loop recur

folcon22:01:25

Which is what I thought the correct context was

ghadi22:01:28

Can you post an example?

folcon22:01:30

based on the docs

folcon22:01:26

I’m just trying to build up state, should I be breaking this into smaller steps?

ghadi22:01:15

Not sure - can you explain the code a bit?

ghadi22:01:18

Which part is the transient, etc

folcon23:01:58

Well how far I can go is highly dependant on how flexible transients are… Ultimately there are a lot of assoc’s happening from line 423 to 425.

folcon23:01:04

which get repeated a lot

folcon23:01:47

I originally was trying to use transient and persist at the level of the state on line 410.

andy.fingerhut23:01:03

If the lookups can be done in a persistent version that isn't "the most up to date", you can do that, of course, but you are probably only asking because you need the lookup results from the latest modified version at the time of the lookup.

folcon23:01:38

but it looks like I should perhaps split out :hull-tri and :hull-next in the loop recur

ghadi23:01:57

Remember transient only applies to one map, no nesting

folcon23:01:01

treat them as transients and then pull them back in after

folcon23:01:30

Right, that wasn’t super clear

folcon23:01:01

in fact I don’t think the word nest is even used in the docs

ghadi23:01:29

(BTW which docs were you excerpting from earlier?)

folcon23:01:45

Which I would expect to be canonical

folcon23:01:47

ok, so I should work out a good way of unpacking the nesting =)…

andy.fingerhut23:01:37

It is canonical. The official docs do contain edges/corner cases that could stand improvement in a few places.

folcon23:01:55

noted =)…

ghadi23:01:15

Perhaps explode the state into more loop locals

ghadi23:01:31

If there is no nesting in the data

ghadi23:01:44

(But that is a trade-off)

andy.fingerhut23:01:27

Depending upon how important time optimizations are for the code in question to you, remember that Java is Clojure's in-line assembler 🙂

ghadi23:01:08

Back to your original question, first will not work on transients because it needs a read only seq on the vec/map

ghadi23:01:23

There is an open bug for empty?

folcon23:01:28

I can’t quite do that @andy.fingerhut, I’m trying to keep the code cross platform. So any java I write, comes with writing an equivalent javascript…

andy.fingerhut23:01:15

Understood. Yeah, I forgot the .cljc suffix on your source file.

folcon23:01:29

It does seem like doing this sort of thing is still rather clunky… Really want to get to the write once run anywhere =)…

folcon23:01:42

Thanks @ghadi

andy.fingerhut23:01:16

You have the write once run anywhere, now you want to add "and even faster" on top

andy.fingerhut23:01:57

Those goals aren't always in tension/odds with each other, but certainly often are.

andy.fingerhut23:01:10

nth appears to work on transient vectors, so you can use index 0 and count-1 to emulate first and peek on transient vectors, at least.

andy.fingerhut23:01:28

I feel a table for operations that work vs. do not on transients in my future ...

Nate Sutton23:01:57

is there a way to get bulk archives of https://clojurians-log.clojureverse.org/ ?

andy.fingerhut00:01:59

Probably, but I don't know what it is. I did notice in their demo data that they say they do not offer the full logs publicly, so that they do not make it trivial for those with ill intent to mine the data.

seancorfield00:01:00

If you're after the messages via an API, see if Zulip would allow that since many channels here are mirrored to http://clojurians.zulipchat.com

Nate Sutton00:01:23

I'm just pulling the urls and mining it via nokogiri and it seems to be working well

Nate Sutton00:01:30

@andy.fingerhut ahhhh cool