Fork me on GitHub
#clojure
<
2016-12-16
>
lvh04:12:08

Is there a way to automatically require a bunch of namespaces in a library without having to enumerate them? (I’m trying to reflectively enumerate all of the available nses.

arrdem04:12:55

Not really. Clojure doesn't really have a notion of the namespaces in a library, because it all gets mashed into the classpath.

arrdem04:12:11

tools.classpath does some stuff to find clj* files and parse a leading ns form

arrdem04:12:41

and you could potentially enumerate all files on the classpath and filter them by name to try and find the logical contents of some ns subtree

arrdem04:12:54

Maybe by looking at the classpath for the JARs on the path and then enumerating those by hand... that would probably be your best shot but assumes that you're working only in a packaged context.

lvh04:12:59

Gotcha. Thanks 🙂

lvh04:12:08

Yeah; that’d be a fair assumption for my use case

lvh04:12:49

I’m a little confused by clojure.tools.namespace; the docstring says it’s not about JARs, but there’s e.g. clojure.tools.namespace.find/sources-in-jar

seancorfield04:12:04

At work we have a Boot task that looks at all the source code in a bunch of folders and figures out, given a specified starting folder, which folders need to be on the classpath for that code, based on the network of namespace requires.

seancorfield04:12:50

@hiredman can tell you more about it -- he wrote it. I'd have to go digging through the build.boot file to give you more details.

lvh04:12:45

@seancorfield hm. I’m trying to load all the ns’es in amazonica, though

lvh04:12:08

So I might be able to enumerate from the classpath; but they’re not necessarily filesystem paths

seancorfield04:12:46

You can read entries from a JAR and locate namespaces that way easily enough. Hmm, let me link you some code.

hiredman04:12:56

Yeah, the class loader API is not good for enumeration

seancorfield04:12:17

https://github.com/framework-one/fw1-clj/blob/master/src/framework/one.clj#L415-L453 This finds namespaces based on a regex inside JAR files on the classpath

seancorfield04:12:38

(pay no attention to anything else in the file -- it's an interesting but ultimately failed thought experiment)

hiredman04:12:52

I might look at generating a manifest at build time using tools.namespace and then use that to load everything up in "production"

seancorfield04:12:55

The idea behind that code was that it could find files / namespaces on the classpath both in source file form and inside JARs

lvh04:12:10

@seancorfield FYI, I just did this:

(defn ^:private amazonica-ns?
  [ns-sym]
  (str/starts-with? (name ns-sym) "amazonica.aws."))

(->> (cp/classpath)
     (find-ns/find-namespaces)
     (filter amazonica-ns?))
Involves two new dependencies: clojure.{java.classpath,tools.namespace}

lvh04:12:17

and it did a thing 🙂

lvh04:12:02

Thanks for giving me a push in the right direction!

seancorfield06:12:49

This is one of the things that always amazes me about Clojure: some of the tools are so simple and yet so powerful!

pawandubey07:12:29

Hello, all. Has anybody had success with using the CIDER debug mode in Spacemacs? I can't seem to make it work.

rmuslimov07:12:38

@pawandubey please provide more details

rmuslimov07:12:50

and there is #cider channel for that

pawandubey07:12:43

@rmuslimov I am jacked in to the REPL, and if I do cider-debug-defun-at-point I expect CIDER to instrument the defun but instead it just evaluates it and I can't use the usual n, i etc key bindings.

pawandubey07:12:20

Nevermind, solved:D

roelofw07:12:06

may we know why so we can learn from it

pawandubey07:12:28

Oh, I was being a doofus and not executing the form after calling debug-defun-at-point.

luxbock09:12:18

I was reading the clojure.core.specs definition for defn and noticed that the multi-arity version also accepts an optional attribute map as the last argument (as well as the second/third alongside doc-string)

luxbock09:12:03

so in theory you could use two attribute maps in the definition

luxbock09:12:57

the last optinal attr-map seems totally unecessary since you can just have the same information in the earlier position

luxbock09:12:27

would anyone know what the reason for having it defined twice is?

karol.adamiec10:12:33

i need to prune nils and empty maps,vectors etc from a lot of stuff. as a helper i wrote a predicate like that:

(defn safe-empty? [a]
  (if (coll? a)
    (empty? a)
    (nil? a)))
is there maybe a nicer way of expressing that? more idiomatic or in core something i have missed?

rauh10:12:44

@karol.adamiec not-empty will nil empty collections as well as work with nil. So you could do (complement not-empty) or just (keep not-empty your-coll)

karol.adamiec10:12:59

will not empty work on a non sequable?

karol.adamiec10:12:29

that if coll check is for that, cause empty? throws on longs

karol.adamiec10:12:17

yep, throws as well

karol.adamiec10:12:41

but i might be able to sprinkle that in at the creation moment

karol.adamiec10:12:55

so should be fine 🙂, then i just prune nil?

dominicm12:12:10

Is there a version of juxt which executes immediately? I can't see anything, wanted something less ugly than ((juxt) x)

gfredericks12:12:21

don't think so

gfredericks12:12:47

[(f1 x) (f2 x) (f3 x)]

yenda12:12:15

In the previous snippet how come eval can use config in the repl but not when the code is compiled in a jar ?

gfredericks12:12:55

it's probably being evaluated in a different namespace

gfredericks12:12:30

(binding [*ns* (the-ns 'my-ns)] (eval ...)) would fix that

yenda12:12:17

thanks for the clarification !

Alex Miller (Clojure team)13:12:36

@luxbock: re your question above on the extra meta map in defn - I was not aware that existed till I wrote the spec and I asked Rich. The idea was that meta can get long and can obscure the meat of the function so this gives you the option of sticking it at the end.

gfredericks13:12:12

that's how I've used it

rastandy13:12:36

hello everyone, I’m using the component library from Stuart Sierra, but I’m not sure how to model a system with 2 instances of the same component, any idea?

donaldball13:12:59

There is a #component room, but briefly: your system can have multiple instances of the same component registered under different names.

rastandy14:12:16

@donaldball thanks, I’ll try

luxbock14:12:47

@alexmiller: thanks that's interesting

genec15:12:01

@jrheard thanks for the book suggesting

yenda15:12:59

anyone using the fs library chmod function ? it works in the repl but not in the jar for me and I wonder why

peterromfeld15:12:00

@karol.adamiec if you need something like empty-val? which als works on keywords and numbers i made this little helper for that

(defmulti empty-val? class)
(defmethod empty-val? :default [v] (empty? v))
(defmethod empty-val? Keyword [_] false)
(defmethod empty-val? Number [_] false)
(defmethod empty-val? Boolean [_] false)
(defmethod empty-val? Collection [c] (empty? c))
(defmethod empty-val? String [s] (empty? s))

kirill.salykin16:12:13

Hi all, is it possible/reasonable to use clojure.spec for webforms data validation/coercion?

pawandubey16:12:57

I need some help debugging some async code. I am working on an async web crawler (this is my first Clojure project) https://github.com/pawandubey/crawljer/blob/master/src/crawljer/core.clj

jr16:12:05

that map is lazy

jr16:12:27

so you're not actually adding the urls to chan until it is realized

jr16:12:52

use doseq for side effects

jr16:12:48

using an atom to track "seen urls" is a smell as well

jr16:12:04

if you're using core.async channels then I would recommend using a distinct transducer

pawandubey16:12:06

My Clojure knowledge is less than a week old. I am yet to explore transducers past their punch line.

pawandubey16:12:11

@jr wouldn't the desired effect also be achieved by wrapping the map with doall ?

jr16:12:55

that will force realization but generally you want to avoid map if you don't want to accumulate the results

pawandubey16:12:02

I saw that doseq returns a nil which is kind of not what I want /

jr16:12:17

mapv is also eager

pawandubey16:12:29

Cool. I will try that out and see how it goes.

jr16:12:17

I didn't realize you wanted those results

lmergen16:12:45

in that case, you can also use (into [] ...)

peterromfeld16:12:50

@pawandubey you can also do doall or dorun if you dont care about results (maybe other way around)

lmergen16:12:04

but mapv is probably more performant

pawandubey16:12:38

I actually decide whether to continue reading from the channel based on what the map returns in the function which wraps the go-loop.

peterromfeld16:12:54

not sure about performance wise doall vs mapv vs into

lmergen16:12:09

wouldn't that decision cause the seq to be evaluated then ?

pawandubey16:12:54

(defn read-urls
  []
  (async/go-loop []
    (when read-url
(recur))))

pawandubey16:12:45

I just check to see if read-url returns a truthy value to continue with the go-loop.

pawandubey16:12:20

read-url calls add-urls-to-chan as its last form which actually has the map call.

peterromfeld16:12:33

how about (doall (pmap ... (if you dont have anything else running on the server) 😄

peterromfeld16:12:50

i guess not really what you want for a lib

pawandubey16:12:00

I have to say I don't completely grok the effects of the lazy behavior of map when called in conjunction with other forms.

peterromfeld16:12:34

it doesnt work well with script (process that end when they think they are finished)

mping17:12:12

I’m having issue with this piece of code:

mping17:12:16

(->>  (Clojure.string/split-lines log)
      (map #(clojure.string/split % #","))
      (map reverse)
      (group-by first)
      (reduce-kv #(assoc %1 %2 %3)) {})

mping17:12:27

compiler says reduce-kv has wrong number of args

mping17:12:45

missed a {} but still it complains

mping17:12:15

nvmind I’m an idiot and can’t distinguish colors between parenthesis

mping17:12:18

the last {} should be moved to the inside of the parens

lvh17:12:35

Given a var, how can I extract its namespace?

lvh17:12:54

thanks! didn’t think to inspect the meta

lvh17:12:57

that makes sense

lvh17:12:24

I hope it will have that appropriate metadata since I’m generating it via classpath and find-namespaces

lvh17:12:02

(for future reference: yes; they had the appropriate meta)

viesti17:12:56

hmm, anyone thinking on writing a Clojure wrapper for http://mxnet.io ?

viesti17:12:52

since it is now a deep learning framework backed by AWS also

triss21:12:05

can any of you guys recommend a good free host to place a twitter bot on?

josh_tackett21:12:55

Anyone know a good way to turn a multilevel hashmap into a string or JSON and preserve the keywords when deserializing?

josh_tackett21:12:19

nvm got it: clojure.walk/keywordize-keys

hlship21:12:34

I’m having a tough time tying to create my own tagged value.

(ns com.walmartlabs.postmaster.address
  "Mailbox address data type."
  (:import ( Writer)))

(defrecord Address [path mailbox-id])

(defmethod print-method Address
  [addr ^Writer writer]
  (.write writer
          (str "#postmaster/address "
               (pr-str [(:path addr) (:mailbox-id addr)]))))

(defn parse-tagged
  "Invoked by EDN reader to convert tagged value into an Address.

  The value should be a vector of two: a string path, and a second
  value, often an UUID, to identify the specific mailbox at the path."
  [value]
  (apply ->Address value))
But the result in 1.9 and 1.8 is the same:
Clojure 1.9.0-alpha14
nREPL server started on port 59185 on host 127.0.0.1 - 
*data-readers*
=> #:postmaster{address #'com.walmartlabs.postmaster.address/parse-tagged}
(require '[clojure.edn :as edn])
=> nil
(edn/read-string "#postmaster/address [:path :id]")
RuntimeException No reader function for tag postmaster/address  clojure.lang.EdnReader$TaggedReader.readTagged (EdnReader.java:784)

hlship22:12:51

It seems to have read data_readers.clj properly.

hiredman22:12:18

it isn't a keyword

hlship22:12:30

Debugging gets us to EdnReader line 778, and the valyue is not present in RT.DEFAULT_DATA_READERS.

hlship22:12:38

It’s not a keyword; that’s 1.9 namespaced map syntax.

hiredman22:12:38

the key isn't supposed to be a keyword

hlship22:12:58

data_readers.clj:

{postmaster/address com.walmartlabs.postmaster.address/parse-tagged}

hiredman22:12:14

edn/read-string doesn't use data-readers if I recall, you ahve to manually pass them in

hiredman22:12:03

default-data-readers is a different var

hlship22:12:16

Then what’s the point of data_readers.clj ; I though that was to seed RT.DEFAULT_DATA_READERS.

hlship22:12:29

From *data-readers* doc:

Reader tags without namespace qualifiers are reserved for
Clojure. Default reader tags are defined in
clojure.core/default-data-readers but may be overridden in
data_readers.clj or by rebinding this Var.

hiredman22:12:06

data_readers.clj does work with the regular reader by default, it is just the edn reader you have to pass them in to

bfabry22:12:11

it's for the core reader, core/read-string, which is used by the compiler

hiredman22:12:00

default-data-readers is the built in list of readers clojure ships with (inst and uuid), data-readers is user added readers

hlship22:12:00

Ah! I was out-clevering myself by using edn/read-string!

hiredman22:12:35

edn/read-string doesn't use user defined readers unless you explicitly pass them in

hlship22:12:21

Ok then:

(read-string "#postmaster/address [:a :b]")
java.lang.IllegalStateException: Attempting to call unbound fn: #'com.walmartlabs.postmaster.address/parse-tagged
Loading src/com/walmartlabs/postmaster/address.clj... done
(read-string "#postmaster/address [:a :b]")
=> #postmaster/address[:a :b]
… you just have to load the namespace explicitly before attempting to read it.

hiredman22:12:06

yeah, it creates the namespace and the var, but doesn't load code from disk 🙂

justinr23:12:03

Does anyone have a moment for a beginner question?

gfredericks23:12:48

usually better to just ask

gfredericks23:12:23

there's also a dedicated room for that I believe

justinr23:12:54

Ah, thanks. I am trying to use the test lib, but am apparently missing something. Here's the code:

justinr23:12:34

(ns test) evaluates to nil

gfredericks23:12:12

I think that's normal

justinr23:12:24

The is statement results in:

clojure.lang.Compiler$CompilerException: java.langRuntimeException: Unable to resulve symbol: is in this context, compiling ...

gfredericks23:12:41

(ns test (:require [clojure.test :refer [is]]))

gfredericks23:12:45

should fix that

justinr23:12:02

Awesome, will give that a try.

justinr23:12:32

Does that mean that I want to import test, so long as the 'is' keyword is there?

justinr23:12:46

I'm trying to break that statement down 🙂

gfredericks23:12:49

it means that's the only thing you're "importing"

gfredericks23:12:08

a more general thing is (:require [clojure.test :as t])

gfredericks23:12:22

and then you can use t/is and t/deftest and everything else in clojure.test that way

gfredericks23:12:37

you can mix the two together too, to taste

justinr23:12:49

Great, that makes sense. And yes, that does work. Thank you!!

chromaticgliss23:12:45

justinr, here's a pretty thorough rundown on importing/namespaces http://www.braveclojure.com/organization/