Fork me on GitHub
#clojure
<
2020-09-10
>
coby02:09:04

is there a way to get the entire list (or tree) of namespaces that a given namespace depends on, given just the name (symbol) of the ns? e.g.

(ns-deps 'my.project.namespace)
;; ('my.other.namespace 'some.other.dep ...)

andy.fingerhut03:09:12

I am not sure if this is what you are after, but you can see a log of all namespaces that end up being require'd when you do a require with the :verbose option, e.g. like this (require 'my.project.namespace :verbose)

andy.fingerhut03:09:36

The output will be longer if none of the namespaces it requires have been loaded already. If any have, the things they require will not be shown in that log

noisesmith04:09:41

what about combining :reload-all and :verbose ?

andy.fingerhut05:09:12

Good idea. That should be more complete than leaving out :reload-all

noisesmith03:09:26

given that you can use a symbol by fully qualified name, that's not really possible - of course we can never use fully qualified names as a reasonable convention

noisesmith03:09:50

there are ns-aliases and ns-refers

noisesmith03:09:22

user=> (into #{} (map (comp namespace symbol val)) (ns-refers *ns*))
#{"clojure.pprint" "clojure.java.javadoc" "clojure.core" "clojure.repl"}
(ins)user=> (ns-aliases *ns*)
{}
(ins)user=> (require '[clojure.string :as string])
nil
(cmd)user=> (ns-aliases *ns*)
{string #object[clojure.lang.Namespace 0x73044cdf "clojure.string"]}

coby03:09:38

so by "never use fully qualified names" you mean always use :as? I think that can work for my use-case, thanks for the suggestion!

noisesmith04:09:43

I mean that you need either use or refer (which shows up in ns-refers ) or as (which shows up in ns-aliases, and avoid the situation where in one ns you use require [clojure.string :as string] and in another you use (clojure.string/join ...) - that's the situation where the relationship is invisible / unqueriable

3
didibus06:09:03

An adventure in OO (a short story) There's an API which creates something called a Registry. Registry is an object, currently it only contains an id. There's another API that deletes a Registry, it takes the same Registry object. Things changed, and when creating a Registry, people wanted it to also take a name, an owner, and some options. So they changed the Registry class with those additional fields. Now, when you want to delete a Registry, it turns out the delete API only really needs the id from it, but the Registry object now takes a bunch of stuff. The first issue is, people thought they'd be smart, and make that class so you're forced to provide all the mandatory parameters for creating a Registry, but that means when you delete you have to pass in data that's not needed and you might not have. So those validations were removed so everything except id is optional on this object now. But even then, the users of the API for deleting are really confused, why is there all these fields on the Registry? Which one need to be provided? Now, subtyping is the solution to everything (most of the time) in OO. So someone said, ok, we need to change the hierarchy of Registry. It turns out, we have a Registry that only has an id. And we also have a CreationRegistry that extend Registry with the additional fields like name, owner, and some options. Genius yes? Well, sure, except it is a breaking change, because you already have a bunch of clients calling that API and their code does: new Registry(). So if you make that change, you'd break them. The same thing happens if you try to make a DeleteRegistry object. So there's just no real solution to this, and everyone just deals with a stupid Registry object that depending where you use it, some fields are mandatory and in other place they are optional 😛 The End

flowthing06:09:24

Could that have been avoided if the method that deletes a Registry took only the ID instead of the entire Registry object?

flowthing06:09:12

I mean, that would also have been a breaking change, but since you have static types, it’s not really a problem, right? 😜

didibus06:09:03

Doing it now would be breaking, if people had had the foresight of this scenario, they could have made the delete take only the id

flowthing06:09:17

Exactly, yeah.

didibus06:09:34

But I've seen this play out a lot in OO, its because creating new class is cumbersome, people are lazy, so classes get shared as much as possible.

✔️ 3
didibus06:09:08

And people think of things as the class

didibus06:09:28

So an API called: deleteRegistry(...) ? what would it take as input if not a Registry ? 😛

didibus06:09:37

In fact, people would probably do: registry.delete()

didibus06:09:20

Also, for APIs, its kind of common I've seen for people to think... we might need to accept more things in the future, so if you have it take an object, it lets you add optional fields to it in the future. Where if you change the signature to have more argument, its breaking. Which is another reason people avoid doing deleteRegistry(id). But then, they don't think much further, and honestly, its normal, how can you anticipate everything, so I'm pretty sure people thought they were smart using Registry as input 😛

didibus06:09:40

Also, there's an integ test, and the test does:

registry = new RegistryBuilder().withId(123).withName("wtv").withEtc....;

client.createRegistry(registry);
// validate it was created
client.deleteRegistry(registry);
//validate it was deleted
Pretty sure people thought hey how convenient this API 😛 I can just use the same object back to back for both API.

flowthing06:09:44

Oh, absolutely.

ryan10:09:05

Why not add deleteRegistry(id) as an overload? Shouldn't break existing calls but allows people to use the more convenient method going forward

didibus04:09:54

That's not a bad idea. In my case, I don't think my client generator supports overload. But ya, I might look into that as an option.

vemv10:09:55

is there a j.u.concurrent mechanism (or clojure lib) that gives you a "grant" to use something (out of a pool of N items) for M seconds?

vemv10:09:39

of course clj being clj this is pretty easy to implement from scratch, I'm just curious. I think I had seen this mechanism somewhere

Alex Miller (Clojure team)13:09:17

Java has a Semaphore for granting but it doesn’t have the time aspect

Alex Miller (Clojure team)13:09:27

The time aspect seems nontrivial to me

vemv17:09:23

Thanks! Maybe a "time-expiring hashmap" could kill a big chunk of the problem. The clj/java alternatives seemed to be bit of a heavy dependency though

zilti15:09:43

Okay, creating uberjars using deps.edn and depstar... is that broken for anyone else? I get:

Execution error (FileAlreadyExistsException) at jdk.nio.zipfs.ZipFileSystem/createDirectory (ZipFileSystem.java:695).
/META-INF/versions/9

Full report at:
/tmp/clojure-13855814055111068112.edn

zilti15:09:26

Ah I guess uberdeps is the hot new thing now

seancorfield15:09:51

@zilti depstar is very actively maintained and I take bug reports very seriously. I've not seen that error before but I'm happy to help debug it (and fix it, if it's a bug in depstar).

zilti15:09:37

Oh, okay! Yes, I will file a bug report soon then. I guess it is fair to say it is in depstar, because uberdeps builds it fine

ghadi15:09:56

there is a #depstar room, maybe you should post your deps.edn in there

ghadi15:09:18

I don't think uberdeps handles MR jars, which is what /META-INF/versions/9 indicates you are using

☝️ 3
seancorfield15:09:25

Well, it may be that uberdeps simply ignores the problem and the resulting JAR just happens to work 🙂

6
ghadi15:09:26

MR == multi-release

seancorfield15:09:32

Yeah, what he said 🙂

zilti15:09:44

Yea, could also be

zilti15:09:00

Oh, I wasn't aware about there being a room 🙂

seancorfield15:09:36

So far, I've not hit problems with MR JARs -- they are supported by depstar -- but you may have encountered an edge case. The #depstar room is new so we don't flood #tools-deps with chatter about it 🙂

zilti16:09:39

I'll save it for now and will have a look

Yehonathan Sharvit17:09:07

What’s the idiomatic way to iterate over a sequence and execute a side effect functions for each element of the sequence given that we also want the returned values from the functions? For instance, iterating over do-stuff! that returns a meaninful value. One option is:

(mapv do-stuff! my-seq)

Yehonathan Sharvit17:09:24

Another option is: (doall (map do-stuff! my-seq))

Yehonathan Sharvit17:09:35

Or, I could write a loop/recur

ghadi17:09:03

run! doesn't return the results though

bronsa17:09:24

ah, I missed the last part of the question :)

Ben Sless17:09:11

mapv is good for this use case

Yehonathan Sharvit17:09:18

mapv doesn’t convey the fact that we are interested in the side effects (At least that’s how I perceive it)

Yehonathan Sharvit17:09:18

It seems that clojure.core is missing a function for this use case. Could it be that this use case is not interesting?

dpsutton17:09:00

can these fail? do you care about order? should you short circuit if they start failing? lots of different things change what this would look like in practice. can't imagine a single function in core really covering everything. do you need a thread pool to run these things? blocking, etc

💯 9
Yehonathan Sharvit18:09:10

What’s the difference between my use case and the use case covered by run! and doseq ?

dpsutton18:09:24

i have no idea what your use case is

dpsutton18:09:01

but run! will block and execute things in sequence. and no way to short circuit, all work done on current thread,

dpsutton18:09:24

if that fits your use case then bob's your uncle and your request for a function in core to handle your use case is satisfied

Yehonathan Sharvit18:09:36

Waht do you mean by “bob is my uncle”?

dpsutton18:09:59

its an idiom meaning "you're good to go"

dpsutton18:09:24

no idea where it comes from 🙂

Yehonathan Sharvit18:09:03

My use case is that I need something like run! with the slight difference that it returns a sequence of the values returned by the inner function

ghadi17:09:08

reduce/transduce

☝️ 3
borkdude17:09:07

Not sure if this is intentional:

user=> #_s/foo [1 2 3] ;; ok, although there is no alias s
[1 2 3]
user=> #_::s/foo [1 2 3] ;; not ok, reader fails!
Syntax error reading source at (REPL:5:10).
Invalid token: ::s/foo
[1 2 3]

ghadi17:09:28

we filed a similar bug around discarded tagged literals recently. Probably shouldn't try to autoresolve an alias for something that is discarded

borkdude17:09:52

should I make a JIRA about it?

hiredman17:09:46

I dunno, that seems pretty clear cut as the correct behavior for "read a valid form and discard it"

6
hiredman17:09:05

s/foo in the example reads correctly, even is s is not an alias in the current namespace

hiredman17:09:13

::s/foo doesn't

hiredman17:09:39

so ::s/foo is not a valid form if the alias s doesn't exist in *ns*

borkdude17:09:15

yeah, maybe you're right

ghadi17:09:25

it's definitely debatable

borkdude17:09:48

I just used #_ to comment out a form I just copy pasted from another namespace which then broke my CLJS compilation

ghadi17:09:54

I was taking valid form meaning not being able to discard 12ab34 because that's not readable at all

borkdude17:09:26

which is kind of what #_ is for right, commenting out forms temporarily

borkdude17:09:28

and it does accept s/foo if the alias doesn't exist, which gives kind of a precedent

bronsa17:09:52

but that's because 's/foo is valid even if s doesn't exist

bronsa17:09:56

while '::s/foo isn't

hiredman17:09:04

you'll get the same behavior from (comment ...) as well

borkdude17:09:20

that's only because comment is a macro. this is a reader implementation decision

hiredman17:09:33

user=> (comment s/foo)
nil
user=> (comment ::s/foo)
Syntax error reading source at (REPL:2:17).
Invalid token: ::s/foo
Syntax error reading source at (REPL:2:18).
Unmatched delimiter: )
user=>

hiredman17:09:05

it is because s/foo can be read and not evaled, while ::s/foo cannot be read

3
bronsa17:09:28

resolution of s/foo doesn't happen at read time

ghadi17:09:30

comment is different though

bronsa17:09:52

if we make #_::s/foo work then why not #_#{1 1} aswell

ghadi17:09:52

I'm considering it debatable because of the treatment of unregistered tags in CLJ-2577

ghadi17:09:51

(though edn has specific language around that, I think it could apply to alias resolution, too)

ghadi17:09:55

(it's a parallel problem)

lilactown17:09:10

one of the concrete ways this harms application development is if you have a require and an aliased keyword in a reader conditional. I ran into this last week, actually

hiredman17:09:25

alias resolution doesn't even exist in the edn spec

ghadi17:09:05

right, but tag handling and alias resolution are similar mechanisms (table lookup)

lilactown17:09:52

e.g. if you have some namespace:

(ns my-app.feature
   (:require
     #?(:clj [foo.bar :as fb])))

#?(:clj ::fb/baz)
if you try and use this namespace in a CLJS app, the reader will throw because fb doesn’t exist

hiredman17:09:14

which it should

lilactown18:09:04

I disagree; while it may match some definition of consistency, it harms CLJS apps. I don’t want to include foo.bar in my app, or literally can’t

lilactown18:09:47

it can work in the other direction too. I guess I should say it “harms cross-platform code bases”

Alex Miller (Clojure team)18:09:57

I personally do not see this as an error - the #_ is for reading, then skipping. ::s/foo where s is undefined is not readable.

borkdude18:09:40

it still seems inconsistent with #_s/foo to me, all technical details aside

borkdude18:09:02

I don't care, it's just an implementation decision

bronsa18:09:24

the difference is in the semantics, not an implementation detail

bronsa18:09:32

(read-string "s/foo") doesn't resolve s

Alex Miller (Clojure team)18:09:32

#_ means read, then skip

bronsa18:09:50

(read-string "::s/foo") does

borkdude18:09:25

so why are symbol namespaces not resolved in the reader again?

bronsa18:09:26

one could argue that one inconsistency is that

`s/foo
doesn't fail while ::s/foo does

bronsa18:09:18

@borkdude because how would you read (quote s/foo) if s was resolved by the reader?

andy.fingerhut18:09:22

(grabs :popcorn )

andy.fingerhut18:09:03

Any conversation with you, alexmiller, ghadi, and bronsa all submitting examples and points of view is guaranteed to exhibit something I've not thought about before.

andy.fingerhut18:09:01

Be very very careful with the #=(popcorn) though

😀 3
Alex Miller (Clojure team)18:09:25

the difference here is in :: which is inherently contextual

bronsa18:09:32

it'd be resolved-s/foo by the time quote sees it

ghadi18:09:30

calling a reader for tagged literal is similarly contextual, but the EDN spec says those should not be called when discarding

ghadi18:09:37

(I know clojure code != edn)

borkdude18:09:02

Meanwhile I'm littering my emacs buffer with ;; 🍿

andy.fingerhut18:09:05

I was going to jump in with ; as my go-to solution, but that was already known to all participants, of course.

borkdude18:09:36

And a nice recipe for messing up balanced parens

andy.fingerhut18:09:26

I have very practiced muscle memory of keeping parens balanced without using slurp/barf kinds of Emacs keybindings. Call me a luddite.

borkdude18:09:35

actually emacs clojure-mode does quite well with it, now that I try to mess it up

Alex Miller (Clojure team)18:09:06

yes, that is a clear violation of the intent of the spec

ghadi18:09:26

in XML there was a distinction between "well-formed" and "valid". I would like to be able to discard invalid things (tags that don't exist, aliases that don't exist) while maintaining the behavior of throwing when something isn't well-formed

6
dpsutton18:09:47

skip the following s-expression

borkdude18:09:55

unless it contains unbalanced s-expressions

try-not-to-cry 3
dpsutton18:09:19

is the obvious choice for the reader tag 🙂

lilactown18:09:19

this discussion hits on a couple of pain points that I have with using namespace aliases and keywords. I have two concrete use cases: 1. I want to require a namespace in a reader conditional and use that as an alias in a keyword for a reader conditional 2. I want to be able to create and use an alias without having a corresponding concrete namespace I feel like the solution to either of those two problems could be related.

lilactown18:09:54

specifically focusing on (1) for now, I think there should be some way to inform the reader, “do not progress further” and I would really appreciate if that was baked into reader conditionals

hiredman18:09:51

This kind of thing is why I wanted "reader-conditionals" to be a macro and not baked into the reader at all

hiredman18:09:40

I doubt it exists anymore but there used to be a confluence page with a fair bit if discussion about reader conditionals with lots of comments, etc

Alex Miller (Clojure team)18:09:08

I think that is open for discussion and may even already have a ticket

3
hiredman18:09:13

so it was very clear that whatever the code was for any branch it must be valid and readable on any other branch

Alex Miller (Clojure team)18:09:28

and 2 is something we definitely have a ticket for and are aware of as an issue that may see some attention in 1.11

6
lilactown18:09:04

I think that whether or not conditional branches were a macro or a reader form doesn’t really matter to me. I still have use cases that necessitate including code conditionally and using aliased keywords

hiredman18:09:27

and you insist on using ::

hiredman18:09:41

the very simple work around is just don't use ::

☝️ 6
hiredman18:09:32

which is mostly my position on :: regardless of comments and conditionals

fullNameHere18:09:10

Hey guys I know reading other people's code can feel worse than waking up after a night of drinking some really cheap vodka. But is anyone willing to do a little code review. Its about 130 loc.

dpsutton18:09:10

there's a #code-reviews channel. and also smaller snippets are always fair game in #clojure and #beginners and other topic related channels

fullNameHere18:09:34

Ill head there

hiredman18:09:31

the tying of namespaces for keywords to the namespaces for code is gross, and insisting that the language grow a feature to make generating empty code namespaces so that it is easier to use namespaced keywords is also gross

borkdude18:09:41

wasn't this inspired by spec initially?

hiredman18:09:03

what happened is the example docs for spec, for the sake of brevity use '::'

hiredman18:09:14

and people just copied that willy nilly

hiredman18:09:25

nothing requires the use of ::

lilactown18:09:42

> the very simple work around is just don’t use `::` yes that is a simple workaround. I disagree that it is a solution; otherwise, why have ::? I ran into this specifically when using a library that used namespaced keywords with long namespaces. :org.wscode.pathom.connect/input over and over is onerous to write and difficult to validate as a human reader (I have intentionally misspelled something here).

Alex Miller (Clojure team)18:09:49

autoresolved keywords have been part of Clojure since 1.0

hiredman18:09:52

(def long-key :org.wscode.pathom.connect/input)

3
hiredman18:09:50

I agree it is gross, I haven't had a problem with long keys

lilactown18:09:57

I also agree that generating empty code namespaces is gross. I wouldn’t suggest that as a solution

hiredman18:09:03

that is what you are pushing for

hiredman18:09:18

that is what all this easy alias stuff is

lilactown18:09:34

I have not suggested any solutions. I have only outlined my motivation for a solution to the use cases I have for my projects

Alex Miller (Clojure team)18:09:52

we are not expecting that to be the solution in Clojure

3
lilactown18:09:12

I spend 70% of my time in ClojureScript. generating an empty code namespace is not a workable solution at all in that context IMO

stephenmhopper19:09:12

I seem to recall watching a Clojure talk sometime ago that discussed the functional programming approach with Clojure. The basic steps were: 1. Define the problem 2. Define a data structure which solves the problem 3. Define functions which operate on that data structure I’m fuzzy on the exact details which is why I’m trying to find the original talk again. Does anyone happen to know which talk this might have been? It’s neither this https://www.youtube.com/watch?v=Tb823aqgX_0 nor this https://www.youtube.com/watch?v=vK1DazRK_a0 (although both address similar topics).

Alex Miller (Clojure team)19:09:36

there were a couple talks like this at very early conj's

Alex Miller (Clojure team)19:09:40

and probably others as well :)

dpsutton19:09:03

is there any issue with using async/onto-chan with large collections? 500k or so items? is this abusing the notion of channels or can they handle large work queues?

ghadi19:09:53

not that I know of, but make sure you use onto-chan! or onto-chan!!

dpsutton19:09:30

is that newer? not seeing it in our copy with (apropos "onto-chan")

Alex Miller (Clojure team)19:09:35

in the former case it will spin a go block to push to the channel, in the latter a thread

Alex Miller (Clojure team)19:09:41

these were added recently

dpsutton19:09:52

thanks for the pointers. i'll bump and check it out

dpsutton19:09:32

yeesh. goodbye "0.3.456" 🙂

Alex Miller (Clojure team)19:09:58

the difference there is not as large as you might presume :)

dpsutton19:09:15

yeah i figured. i'm bumping and expecting no work to do but use the new goodies

Alex Miller (Clojure team)19:09:18

actually, that is pretty old, nvm :)

Alex Miller (Clojure team)19:09:32

but shouldn't be anything breaking afaik

stephenmhopper20:09:39

So I have an ordered sequence of items . I need to map through this sequence with a function that accepts the previous element as well as all elements preceding it in my ordered sequence. Is there a good idiomatic way to do this in Clojure? Should I just map over the range of indices, take the current index + 1, and use butlast? Right now I have this:

(->> (range (count items))
  (map (fn [idx]
         (let [entries (take (inc idx) items)
               prev-items (butlast entries)
               item (last entries)]
              ;;do something interesting here
             ))))
Is there a more idiomatic way to do that?

ghadi20:09:02

butlast is O(N) complexity, so avoid that

ghadi20:09:40

clojure
Clojure 1.10.2-alpha1
user=> (doc partition)
-------------------------
clojure.core/partition
([n coll] [n step coll] [n step pad coll])
  Returns a lazy sequence of lists of n items each, at offsets step
  apart. If step is not supplied, defaults to n, i.e. the partitions
  do not overlap. If a pad collection is supplied, use its elements as
  necessary to complete last partition upto n items. In case there are
  not enough padding elements, return a partition with less than n items.

ghadi20:09:04

use (partition 2 1 coll) for overlapping pairs

stephenmhopper20:09:57

I don’t think partition will work because the partitions need to be of varying / increasing size

ghadi20:09:06

oh, I misread you

kenny20:09:45

Perhaps a loop ?

ghadi21:09:42

a loop if you're producing N values at the end, where N is the length of the list

ghadi21:09:55

otherwise a reduce is sufficient

dpsutton21:09:51

(reduce (fn [[previous acc] x]
          [(conj previous x) (apply + x previous)])
        [[] 0]
        (range 5))

stephenmhopper21:09:30

Yeah, I suppose either loop or reduce will work. I do need to produce N values at the end so I’ll probably go with loop

dpsutton21:09:47

you could also do some mapping over (reductions conj [] coll) but might run into memory issues?

stephenmhopper21:09:27

Oh, I like that even more, that should work for my use case.

apbleonard22:09:16

We use aliasing to shorten long keywords in natural language all the time. In the north of England "tea" means an evening meal, and "dinner" means lunch. A southern colleague was asked to deploy some code at dinnertime, and she stayed late! We could tell everyone to say ":northern/dinner" instead, and it might have helped her out, but they won't because most of the time in our context, there's no need. But you do have to clarify when you're crossing contexts 🙂 Similarly I can tell (and have told) developers their JSON keys in their APIs are too short and ambiguous, sufficing only for the narrow context they had at the time... And I long for namespaced clarity in the overloaded terms that are used multifariously in different contexts at my work. But I'm starting to appreciate that there's always shorthand language for local conversations, and more precise language for general audiences, and so aliasing is inevitable and to be supported.

❤️ 6
👍 3