Fork me on GitHub
#clojure
<
2018-01-29
>
rymndhng07:01:17

I was poking around tools.namespace and I found it interesting that the implementation of remove-lib is:

(defn remove-lib
  "Remove lib's namespace and remove lib from the set of loaded libs."
  [lib]
  (remove-ns lib)
  (dosync (alter @#'clojure.core/*loaded-libs* disj lib)))
I’m wondering if someone can enlighten me to explain why *loaded-libs* needs to exist, as opposed to checking clojure.lang.Namespace directly

seancorfield08:01:25

I suspect it's more about backward compatibility and legacy code -- but it's a good question and I'd love to hear the core team explain the background for it.

rymndhng21:01:15

@U04V70XH6 any suggestions on the best way of getting it on the view of the core team?

seancorfield21:01:23

You could open a JIRA issue about it...

seancorfield21:01:36

... But I'd expect it to be closed with no action taken. I can't imagine it'll be changed.

rymndhng23:01:27

yeah, creating an JIRA issue doesn't seem like the right place for a question like this, maybe clojure mailing list?

seancorfield00:01:57

Yup, that's a good place to ask about it's history and whether it's likely to ever go away...

rymndhng01:01:00

i can't tell if you mean *loaded-libs* or the mailing list itself 😄 the #clojurians slack definitely has higher engagement

seancorfield01:01:08

Hah, good point! Too many "that" and "it" 🙂 Yup, the mailing list is a good place to ask about the history of *loaded-libs* and whether *loaded-libs* is ever likely to go away.

andy.fingerhut08:01:49

I think it is because clojure.core/require adds namespaces to clojure.core/loaded-libs as they are loaded, and checks that list of namespaces before actually loading the code, to avoid loading namespaces multiple times unnecessarily.

andy.fingerhut08:01:16

Here is a link to functions load-one and load-all in clojure.core namespace, which are called to do actual namespace loading when you call require, use, etc. https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L5843-L5866

andy.fingerhut08:01:55

Oh, sorry, the question is why loaded-libs exists. Don't know, but it has been around since very early versions of Clojure.

seancorfield08:01:56

^ Yeah, I suspect "backward compatibility" is the real reason this still exists...

seancorfield08:01:43

...which is a perfectly good reason! It's why we can consistently take alpha and beta builds to production and why Clojure version upgrades are mostly so painless (unlike certain other languages!).

andy.fingerhut08:01:16

Looks like it was added Aug 2008, when ns macro was added.

seancorfield08:01:25

(and why are you awake and online at quarter past midnight?)

seancorfield08:01:11

Accretion is good. Not removing things is good.

seancorfield08:01:35

I was surprised they removed a predicate during the 1.9 alpha/beta process -- that broke our code!

seancorfield08:01:44

(an easy fix but...)

andy.fingerhut08:01:31

Because it was an exact replica of an existing function, but with a different name? Yeah, probably because Alex Miller knows how much each var causes in Clojure init time 🙂

seancorfield08:01:59

Yes, I didn't know about the function it replicated, my bad, so I just switched our code back to that (we had all of two instances of it, as I recall). I consider that the "bleeding edge tax" and Clojure is one of the only languages where I've felt that is worth paying!

seancorfield08:01:23

We've been using clojure.spec in production since the early alphas of 1.9 and it's been so totally worthwhile!!

andy.fingerhut08:01:25

And we thank you for paying that tax

seancorfield08:01:56

Alex was actually teasing me for not already being on 1.10.0-alpha2 🙂

seancorfield08:01:37

(we are multi-version testing against master-SNAPSHOT all the time tho'!)

rauh08:01:32

Next step is clearly to run with patches from Jira applied 😄

andy.fingerhut08:01:23

Huh. Total lines of text in all Clojure repo files (Java, Clojure, README, etc.): 301K. Total line of output in 'git log -p .' (full list of all changes made as diffs, with a couple of lines of context on each one, even: 586K

andy.fingerhut08:01:16

As soon as we find out what -secret project- is because it is in alpha4, I imagine you will move then

seancorfield08:01:20

Interesting stats. We're currently at around 75K lines of Clojure (about 55K production and 20K test).

seancorfield08:01:15

Yeah, Alex has been unusually teasy about that. It is somewhat annoying that these projects just "drop" into master but the care taken with them has usually been worth the "secrecy".

seancorfield08:01:21

I just spent most of yesterday and some of today bringing boot-tools-deps up-to-date against tools.deps.alpha (which is way more changeable than almost anything I've tracked in Clojure in seven years!). Nice to have Git and local deps projects fully supported.

seancorfield08:01:14

We'll probably jump to full-on tools.deps as a basis for our multi-subproject mono-repo soon but it's still a bit too "soft" even for me...

andy.fingerhut08:01:31

I have the perhaps bad habit of being on-line many hours a day, some for work, some for hobbies, anyway, and trying to get back to a more sane local time sleep schedule. In fact, heading AFK soon.

andy.fingerhut08:01:52

I was reminded how long it takes to transcribe 1 hour of video this weekend while transcribing Stuart Halloway's talk "REPL-Driven Development". Looking for a place to publish it where people can easily find it.

andy.fingerhut08:01:01

Guess I'll stick it in a public Github repo I have now, with a link to it in the comments of the video here: https://vimeo.com/223309989

buzzdan08:01:41

Security help anyone ?

seancorfield08:01:13

@andy.fingerhut That is a great talk that I keep watching over and over -- and referring people too, repeatedly!

seancorfield08:01:12

@buzzdan I doubt there's anything off-the-shelf for that. I think you'd be better off with a standard web application firewall in place instead of trying to reinvent that yourself.

buzzdan08:02:55

would firewall protect my api from incoming requests that contains params with malicious values? i'm talking about XSS or Injections. for example: POST https://www.my-app.com//api/user Body: { firstName: "aaa \d]<script>alert(document.cookie)</script>" lastName: "some name..." .... }

jumar11:01:15

What's the best way to turn existing map with unqualified keys to a namespaced map? Let's say I have a map {:id 1 :name "read-only"} and I want to turn it into #:role{:id 1 :name "read-only}

robert-stuttaford11:01:45

@jumar, you have to alter the keywords one by one. that it prints that way happens at the reader / pr-str level. so, (into {} (map (fn [[k v]] [(keyword "role" (name k)) v]) {:id 1 :name "read-only"}))

jumar11:01:03

I was afraid that I would need something like that 🙂. Thanks!

schmee14:01:25

with Specter:

user=> (def m {:id 1 :name "read-only"})
#'user/m
user=> (setval [MAP-KEYS NAMESPACE] "role" m)
{:role/id 1 :role/name "read-only"}

qqq13:01:52

besides writing

(fn self [...] ...)
is there a way to refer to "smallest containing function" ?

bronsa13:01:48

what does smallest containing function mean

qqq13:01:45

(fn [] .... <-- FOO
  ... ME1 ... 
   (fn [] ... <-- BAR
     .. ME2 ... )
  .. ME3 ...
)
at loc ME2, 'smallest containig function' is line BAR at ME1,ME3, 'smallest containing function' is line FOO

qqq13:01:56

what is the formal term I should be using ?

bronsa13:01:52

current enclosing function makes sense to me

bronsa13:01:58

and no, there’s no other way than to name it

qqq13:01:07

got it; thanks

matan14:01:21

I'd like to thank everyone who contributed so dearly to the discussion (yesterday) on how to start off the "data scientist, no/little OO exposure and little programming knowledge person type" on a productive clojure trajectory!

qqq16:01:36

what's wrong with:

(defn foo [{:keys [::depth ::inc-by]
                           :OR {::depth 0}
                           :as state}]
...)
?

qqq16:01:20

err,

(defn foo [{:keys [::depth ::inc-by]
                           :or {::depth 0}
                           :as state}]
not sure why the or became OR

bronsa16:01:02

should be :or {depth 0}

qqq16:01:21

ah, so it's 'name as symbol' and not 'name as keyword' okay; thanks

Alex Miller (Clojure team)17:01:29

the :or map always has the local symbols to bind as its keys

borkdude17:01:38

What do people use for searching XML with Xpath in Clojure? I’m using https://github.com/mudge/riveted but I’m wondering why it’s not more popular.

martinklepsch18:01:22

Is there any reason this doesn’t work in post conditions?:

{:post [nil?]}

rauh18:01:12

@martinklepsch You should use % as the result of the function. Run this:

(macroexpand
  '(fn [x]
     {:post [nil?]}
     (fn-body x)))

martinklepsch18:01:06

yeah, I realized that, just wondering why that is, seems… weird?

rauh18:01:42

I actually made the same mistake for months until I realized I was never actually checking anything since the val is just truthy

martinklepsch18:01:28

yeah, seems to me that unknowingly specifying post-conditions that will never fail is pretty easy

schmee18:01:58

what do the matchm, matchv and match-let macros in core.match do? they don’t seem to be documented at all

martinklepsch18:01:05

There was discussion about some improvements to :pre :post validation but never got merged https://dev.clojure.org/jira/browse/CLJ-1473

leontalbot18:01:44

hello! Do I need a doall when into is already there wrapping a pmap expression?

(spit "test.txt" (time (into [] (pmap inc (range 10000000)))))
(spit "test.txt" (time (into [] (doall (pmap inc (range 10000000))))))

sundarj18:01:28

into uses reduce, which is strict, so i don't believe you do

sundarj18:01:44

=> (type (take 100 (range)))
clojure.lang.LazySeq

=> (count (into [] (take 100 (range))))
100

sundarj18:01:35

any time 🙂

sundarj18:01:53

as an aside, a faster and simpler alternative to (into [] coll) is (vec coll)

sundarj18:01:56

=> (count (vec (take 100 (range))))
100

Victor Ferreira18:01:05

Hey guys I asked this in beginners chat but no one help me, maybe someone can do it here so: my question is that: how can I count key-value to a new object some like this: input -> {:type1 [{:type type1}] :type2 [{:type type2} {:type type2} {:type type2}]} output -> {:type1 1 :type2 3}

martinklepsch18:01:45

@victtorferreiralima

(->> {:type1 [{:type ’type1}] :type2 [{:type ’type2} {:type ’type2}  {:type ’type2}]}
     (map (fn [[k v]] [k (count v)]))
     (into {}))

noisesmith18:01:32

@victtorferreiralima depending on how you got that data, you might find it easier to use frequencies instead of group-by (that looks like the output of group-by)

noisesmith18:01:39

user=> (frequencies (map :type '[{:type type1} {:type type2} {:type type2}  {:type type2}]))
{type1 1, type2 3}

noisesmith18:01:32

oh, you need (comp keyword name) if it needs keywords as well

Victor Ferreira18:01:50

I tried use frequencies by I got diferent result I think that a don't use quote

noisesmith18:01:51

so it would become (map (comp keyword name :type) ...)

noisesmith18:01:34

@victtorferreiralima quote is an instruction to the reader telling it not to evaluate something, it isn't needed in output

noisesmith18:01:17

but if what you really want is keywords, the above will fix it (@martinklepsch’s example also gives you symbols as keys, but could use the same keyword / name trick in its mapping if needed)

martinklepsch18:01:33

I think mine should already return keywords

noisesmith18:01:20

oh right I misread

andy.fingerhut18:01:13

@rauh @martinklepsch I believe the Eastwood linter should warn about most such trivially-true pre and postconditions.

martinklepsch18:01:34

@andy.fingerhut I took this to #clojure-dev

andy.fingerhut18:01:59

And its docs for the warning give examples of the wrong and right way to do it. https://github.com/jonase/eastwood#wrong-pre-post

josh_tackett20:01:13

(r/fold
                   conj
                   (r/map (fn [base_item]
                            (r/fold
                              conj
                              (r/map (fn [target_item]
                                       (r/filter (match-percent-logic base_item
                                                                      target_item)
                                                 [(:din base_item)
                                                  (:din target_item)]))
                                     items)))
                          items))
I run this loop and get a bunch of objects: #object[clojure.core.reducers$folder$reify__6593 0x20038b5c "clojure.core.reducers$folder$reify__6593@20038b5c"] What is the best way to get the actual result?

hiredman20:01:52

r/fold is likely not going to be useful to you

hiredman20:01:20

r/fold needs to work over a full realized tree shaped datastructure to be useful

hiredman20:01:20

the reducer combinators return a thing that can be reduced

ghadi20:01:09

agree with hiredman about utility

hiredman20:01:35

conj is likely not going to work well for r/fold too

noisesmith20:01:36

oh - I had suggested reducers as a way to simplify parallelizing the combinatorial comparison

noisesmith20:01:55

but maybe something else is more appropriate - likely even

hiredman20:01:11

r/fold needs a reducing function and a combining function, some things like + can work for both, but conj isn't going to

josh_tackett20:01:09

(mapv (fn [base_item]
                         (mapv (fn [target_item]
                                 (when (match-percent-logic base_item
                                                            target_item)
                                   [(:din base_item)
                                    (:din target_item)]))
                               items))
                       items)

josh_tackett20:01:16

original loop we are trying to optimize ^^^

hiredman20:01:27

oh, actually, r/map and r/filter are folders, so r/fold will flow through

hiredman20:01:47

maybe try replacing (r/fold conj ...) with (r/fold cat conj ...)

hiredman20:01:26

I think your r/filter call inside the function passed to r/map isn't correct

hiredman20:01:05

which is are getting all those weird objects back, you are getting the reducible results of the filter

hiredman20:01:27

that should likely still be the (when ...) logic from the original version

josh_tackett20:01:07

I took out the r/filter and it worked well 🙂

hiredman21:01:01

how large is items?

arrdem22:01:06

Is there some way to bust open and introspect a reify produced object?

bronsa23:01:17

inspect how?

arrdem23:01:13

I’m building a thing that leverages spec and wanted to try and capture the closed-over multimethod on a multi-spec. Managed to get around it using s/describe*.

noisesmith22:01:31

is this a known eastwood issue?

== Linting penguin.babysit ==
Entering directory `/media/justin/806084F16084EEEA/clojure/penguin'
src/penguin/babysit.clj:60:21: wrong-arity: Function on var #'clojure.core/eduction called with 3 args, but it is only known to take one of the following args: [xform* coll]

noisesmith22:01:51

that * is supposed to represent a variable number of xform args

hiredman22:01:48

there was a whole mailing list thread a while back somewhat related to this. some tools attempt to use :arglists metadata to mecahnically check things, but some :arglists are written in such a way as to provide nicer documentation, not mechanical checking

hiredman22:01:00

(this was years ago)

noisesmith22:01:10

yeah, I recall similar conversations on IRC

noisesmith22:01:40

I'm just surprised something like that is not addressed - I guess not many folks are using eduction, and of those even fewer are using the variable number of xforms feature

hiredman22:01:56

it does seem weird that you would attach an arglist specically prettified for human consumption as a data structure though

noisesmith22:01:43

so there might be a config to explicitly tell it what to accept

andy.fingerhut22:01:10

@noisesmith If you mean not addressed specially in Eastwood, I don't recall any special checking for particular functions to change how the arg count warnings are done for particular ones. At least not right now.

andy.fingerhut22:01:55

For the reasons mentioned above, at least one approach would be to enumerate a list of known core functions that manually override :arglists for 'human readable purposes' vs. the purpose Eastwood more often uses them, and either do custom checks for those, or no checking at all.

noisesmith22:01:01

I'm mobile for the moment, I can make a small repro later- code that education is meant to handle and Eastwood rejects

noisesmith22:01:34

Thanks for the info, that's something to think on

andy.fingerhut22:01:28

Well, the beauty of Eastwood is that as long as you don't put it completely in the way of compiling and running your code, it doesn't actually reject anything 🙂 But yeah, false positives are very annoying, and Eastwood still doesn't have a general way to silence individual warnings on particular blocks of code.

andy.fingerhut22:01:57

e.g. specially formatted comments to suppress particular warnings, as many lint-type tools have

noisesmith22:01:12

Yeah, my install and deploy tasks just fail on Eastwood warnings, this is a good filter to keep junk artifacts out

noisesmith22:01:00

But ever so rarely it warns about something that I don't want to work around or fix

quoll23:01:29

I’m trying to update a project (not mine) to use clojure 1.9. It’s configured with Leiningen (and Monolith. This is a sub-project that’s failing). When I try to run lein test it fails during initialization when it tries to compile clojure/core_instance18.clj. Specifically, I get: Unbound: #'clojure.future/Inst is not a protocol This file extends the clojure.core namespace, but it looks like it’s decided that Inst is supposed to come from clojure.future, and I can’t figure out why. I’ve been at it a while, but I’m stuck. Does anyone have a suggestion where I could look please?

hiredman23:01:11

is it an aot project? have you deleted any class files laying around on disk?

quoll23:01:50

I don’t believe that it is. I’ll check to see if any classes exist

hiredman23:01:04

it looks like you are loading https://github.com/tonsky/clojure-future-spec and it isn't doing the right thing

quoll23:01:31

ah… if I am, then that’s wrong. I tried to remove it

quoll02:01:30

it was more complex than this, but you helped me narrow down the search a bit. @U8XJ15DTK too. Thank you.

greglook23:01:31

we ran into some clojure-future-spec vs 1.9 issues and eventually a coworker made a clojure-past-spec project to backfill the old namespaces until libs upgrade

quoll23:01:06

ah… it might be a dependency that’s using it

quoll23:01:07

thank you!

greglook23:01:05

lein monolith with-all deps :tree might help you track it down, otherwise you can look at the projects one by one