Fork me on GitHub
#clojure
<
2016-05-31
>
patrkris08:05:43

Hi people. Is there an easy way to start a lein repl and include a specific jar in the classpath for just that session?

plexus09:05:01

@patrkris: lein update-in :dependencies conj \[ring-middleware-format\ \"0.7.0\"\] -- repl

patrkris09:05:16

@plexus: thanks, will give it a try

aspra10:05:08

hi all! Anyone knows if there is a nice way to show some info about what is been tested at that time using clojure.test?

aspra10:05:39

I know there is “testing” however it seems that it only outputs when the test failed

danielcompton10:05:17

@aspra do you mean logging what is being tested?

danielcompton10:05:48

I don’t see that often in Clojure, but you could add your own logging calls, or wrap it around testing, or similar

lambeta11:05:01

Hi, do you guys know how to get those profiles with specific task? for example: when I run lein test, I wanna know how many profiles being merged.

aspra11:05:22

@danielcompton I mean more that when a test is running it outputs a piece of text, a description, so it is easy to see which test is in progress.Like what midje does with the text input in "fact"

danlebrero11:05:59

I am having some trouble with a “clojure.lang.Var$Unbound.throwArity” exception

danlebrero11:05:38

which I assume is due to the way functions are serialized/deserialized

octahedrion12:05:49

wouldn't it be nice if Leiningen coordinates given on githubs were text, not an image link to Clojars

octahedrion12:05:16

(so one could copypaste immediately rather than having to go to Clojars and come back again)

chrisbetz12:05:53

@danlebrero: sounds like your not compiling stuff properly. Check your uberjar for the classes and/or your project.clj. Otherwise send me more info...

danlebrero14:05:07

@chrisbetz: The issue is when I try to use the spark streaming “foreachRRD” function. The code looks like “(.foreachRDD dstream (sparkling.function/function2 f))

danlebrero14:05:43

and to use it (mysparkling/foreach-rdd dstream (fn [rdd _] (sparkling/foreach-partition ….)))

danlebrero14:05:40

@chrisbetz: the fn that I pass to “foreach-partition” is the one that throws the unbound exception

danlebrero14:05:48

@chrisbetz: I found that doing a (Class/forName “com.ig.somenamspace__init”) is a workaround

danlebrero14:05:27

@chrisbetz: but (import ‘com.ig.somenamespace__init) or (require ‘com.ig.somenamespace) does not work

danlebrero14:05:53

@chrisbetz: I do that from within the fn passed to “foreach-partition"

jstokes14:05:23

@danlebrero not sure about the foreachRDD with the arity 2 function, but i’ve been successful with this -

(defn foreach-rdd [f dstream]
  (.foreachRDD dstream (function f)))

danlebrero14:05:19

@jstokes: do you do a "sparkling.core/foreach-partition” inside the “f”?

jstokes14:05:12

(stream/foreach-rdd (fn [rdd] (spark/foreach println rdd)) totals)

danlebrero14:05:26

If I understand Spark correctly, “println” will be run on the driver, right?

chrisbetz14:05:04

@danlebrero: is your fn a thing from defn?

danlebrero14:05:50

@chrisbetz: no, it is an anon fn, I just changed it to a defn and trying to test it

jstokes14:05:56

i believe on the worker node @danlebrero

chrisbetz14:05:52

Anon fn should be ok... Just not f from protocols

chrisbetz14:05:36

Did you make sure to require it in the executor-vm? One way is to require it from serializer code? Because requiring should fix it

danlebrero14:05:46

@chrisbetz: the “unbounded” fn is in the same namespace

danlebrero14:05:56

moving all fn to defn did not work

bronsa15:05:03

@nathanmarz: in your static-field example -- I can't see why you can't already do that just by closing over a volatile reference

nathanmarz15:05:47

It has to be declared inside the function

nathanmarz15:05:13

(fn [...] (select [:a :b :c] ...))

nathanmarz15:05:27

My select macro is what's responsible for declaring that it needs to use a static field

nathanmarz15:05:17

Right now it generates a UUID string at macro-time for doing gets/puts into a global map

danlebrero15:05:37

@jstokes @chrisbetz found the root cause of the issue. sparkling.core/foreach-partition uses (comp f iterator-seq), so Clojure creates a fn in clojure.core$fnxxx for it, which means that my namespace is never hits the sparkling.serialization.Utils/requireNamespace method to load the namespace, so it is never initialized

danlebrero15:05:25

@jstokes @chrisbetz workaround is to use a define a foreach-partition without the comp, or to make sure that other fn in the same namespace is used before and will trigger the ns load

chrisbetz15:05:01

Thanks for investigating. I'll have alook at that

josh_tackett16:05:44

Anyone know how to add a + to a regex?

plexus16:05:59

@josh_tackett: you want to match a literal +? Try escaping it. #"\+"

josh_tackett16:05:32

it was in JSON so it was actually #”\\+"

josh_tackett16:05:37

That’s why I was getting tripped up

zane16:05:15

What's the right place to look for the full 1.8 ↔️ 1.9 changelog?

gastove16:05:33

I didn’t think there was a full changelog yet — I doooon’t think 1.9 is out? You could check out the in-progress release notes on the Clojure JIRA tho http://dev.clojure.org/jira/secure/ReleaseNote.jspa?projectId=10010&amp;version=10750

wei16:05:34

how can I tell if a data structure supports empty?

plexus16:05:01

@wei empty? internally calls (seq coll) and quite a lot of things can be used with seq

plexus16:05:00

anything that implements ISeq, Seqable, java Iterables, Arrays, CharSequences, Maps

zane17:05:48

Huh. boolean? isn't a thing?

wei17:05:06

@plexus is there something like seqable?

wei17:05:52

basically, I want to test for emptiness via empty? but need to make sure the target thing can support empty?

noonian17:05:22

@wei: can do (and (seq? coll) (empty? coll))

plexus17:05:58

Seqable is an interface, you can test that with instance?. I don't think clojure.core has seqable?, but have a look on crossclj I'm sure you'll find half a dozen implementations

plexus17:05:07

(on my phone on the run)

noonian17:05:31

Could also just call empty? in a try catch and return false or nil for an exception

lopalghost17:05:14

Does anyone do a lot of interop with Clojure and MS Excel? I don't suppose there's a library that skips Apache POI and parses Excel xml directly into Clojure?

pesterhazy21:05:38

there was a tool or library using Clojure/.net interfacing with excel, if I remember correctly

noonian21:05:20

@lopalghost: might checkout https://github.com/tomfaulhaber/excel-templates and the Clojure West talk linked from the readme. May or may not be the type of thing you are looking for.

loganmac21:05:37

I'm not sure if this is the right channel, but is the ClojureCLR project dead? it seems kind of dead, it's way behind in terms of updates

loganmac21:05:14

I had dreams of combining it with Xamarin to make mobile games/apps now that they killed RoboVM

tomfaulhaber21:05:57

@lopalghost: excel-templates won't help you if you're trying to read Excel files (it's really a templating library for Excel). But docjure does that. There's also https://github.com/outpace/clj-excel

lopalghost21:05:40

Excel-templates actually looks pretty nice--I'm looking to read and write

tomfaulhaber21:05:40

I think excel-templates is the best on the output side, but I would. I wrote it 🙂.

lopalghost21:05:40

But the problem with all these libraries is that none of them do exactly what I want, so I end up using POI directly

lopalghost21:05:21

its mutable objects give me a headache 😞

tomfaulhaber21:05:17

POI is really the most complete in terms of covering all the functionality. If you were going to wrap it, you'd be wrapping the mutability which is pointless

tomfaulhaber21:05:48

I've been noodling dropping POI in excel-templates and simply doing immutable ops on the Excel zip/xml structure. I'm already doing that in some places.

lopalghost21:05:55

Yeah, that's why I'd like a library that translates the XML directly to clojure data and vise versa

lopalghost22:05:20

That's cool, I imagine its a lot of work

tomfaulhaber22:05:38

@lopalghost: we'll that wouldn't be too hard to do yourself

tomfaulhaber22:05:12

but then you need to understand all the semantics of Excel

lopalghost22:05:36

Its mostly that I'm not very familiar with ooxml--any good resources?

tomfaulhaber22:05:25

the spec is the best resource. Look here for a link and some of my thoughts on the subject: https://github.com/tomfaulhaber/excel-templates/issues/28

lopalghost22:05:13

@tomfaulhaber: thanks! I will look into it

lopalghost22:05:46

excel-templates should cover a lot of what I need to do

kenny22:05:46

Are qualified keywords allowed to start with a number?

kenny22:05:57

The compiler doesn't seem to care if you create a qualified keyword that starts with a number, however, if you alias the ns in a different ns and try to use the qualified keyword that started with a number the compiler barfs.

kenny22:05:56

Example:

(ns foo)
(s/def ::1d-vector (s/and coll?
                          #(or (empty? %)
                               (number? (first %)))))

(ns bar
  (:require [foo :as f]))

::f/1d-vector
Compiler barfs on the ::f/1d-vector line. Using 1.9.0-alpha3

hiredman23:05:46

in general, keywords of any kind should not begin with a number

hiredman23:05:06

the reader and the keyword function have at various times been more accepting

kenny23:05:38

Just strange that the compiler doesn't care as long as the qualified keyword that begins with a number is used in the ns it was created in.

kenny23:05:05

Seems like the compiler should barf on any keyword starting with a number.

kenny23:05:22

Or at least some consistent behaviour.

hiredman23:05:26

it isn't the compiler, it is the reader

hiredman23:05:52

well, I am mean, I am not sure, I haven't tried it, but I am pretty sure it would be the reader

kenny23:05:10

A CompilerException is thrown

kenny23:05:51

clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Invalid token: ::ms/1d-vector, compiling:

hiredman23:05:10

in a bare clojure repl it is a clojure.lang.LispReader$ReaderException

hiredman23:05:44

the compilerexception wrapper is a result of whatever tooling you are using

kenny23:05:06

Okay. Anyways, not relevant to the discussion. It seems like there should be consistent behaviour unless there is another reason for an error not to be thrown.

hiredman23:05:08

in 1.8, if I recall, there was a commit to change the reader to be more inline with the documentation about the reader, which suggests that keywords shouldn't start with numbers

hiredman23:05:26

but that was reverted because of people

hiredman23:05:07

but it wouldn't surprise me if the reader was using slightly different regexes for parsing slightly different keyword syntaxes

kenny23:05:54

What does reverted because of people mean?

hiredman23:05:19

because, even though the docs on the reader say symbols and keywords must start with non-numeric characters, because the reader allowed it, they started using it, and then got ticked off when someone actually tried to make it consistent with the docs

hiredman23:05:35

which is, you know, the kind of thing people do all the time

hiredman23:05:04

the other thing is, people really want the set of keywords you can create via the keyword function to match exactly with the set of keywords you can create using the reader

kenny23:05:54

Interesting. Though I think you can make the behaviour more consistent by disallowing qualified keywords to start with a number as you cannot use them outside of the ns that they were created so it should not affect anyone.