Fork me on GitHub
#beginners
<
2016-03-24
>
bendy00:03:41

Hey all, I have a really general question. What is the/your preferred clojure terminal setup? Like, do you start a clojure server, print everything out to there, and have a repl in another terminal? Or do you log out to the same terminal you have your repl in? Or do you log into log files and tail them when you need to?

bendy00:03:25

I guess a lot of it comes down to personal preference, but I’m just wondering if there’s any standardization around that / a ‘clojure way’

bwstearns03:03:26

@bendy If it's just a repl you're after then I recommend lein repl

bwstearns03:03:27

it has tab completion, scroll up for previous inputs, all the nice repl-y things.

seancorfield05:03:42

@bendy: I think a lot of people use a REPL in their editor / IDE for live development. For me that REPL is mostly standalone but I also run REPL servers inside most of my long-running processes and can connect my editor to that for live development (I use Emacs / CIDER / nrepl).

seancorfield05:03:40

As for logging, it depends whether I want to just "see information" (in which case I log to stdout / console), or whether I might want to post-process the log data, in which case it either goes to a file (for ingestion to Red Shift etc) or a database (for more real time queries).

seancorfield05:03:48

So "it depends".

urbanslug05:03:52

Getting the following error while attempting to run lein bikeshed on lein bikeshed

(defproject lein-bikeshed "0.3.1-SNAPSHOT"
  :description (str "A Leiningen plugin designed to tell you your code is bad, "
                    "and that you should feel bad")
  :url ""
  :license {:name "Eclipse Public License"
            :url ""}
  :eval-in :leiningen
  :dependencies [[org.clojure/tools.cli       "0.3.1"]
                 [org.clojure/tools.namespace "0.2.6"]]
  :plugins [[lein-bikeshed "0.3.0"]])
The error:
$ lein bikeshed
Retrieving lein-bikeshed/lein-bikeshed/0.3.0/lein-bikeshed-0.3.0.pom from clojars
Retrieving lein-bikeshed/lein-bikeshed/0.3.0/lein-bikeshed-0.3.0.jar from clojars
Could not find artifact lein-bikeshed:lein-bikeshed:jar:0.3.1-SNAPSHOT in clojars ()
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

urbanslug05:03:58

What am I missing?

rauh06:03:55

@urbanslug: I'd run lein deps :tree first with such a problem

urbanslug06:03:08

@rauh:

[cider/cider-nrepl "0.9.1"]
   [org.tcrawley/dynapath "0.2.3" :exclusions [[org.clojure/clojure]]]
 [clojure-complete "0.2.4" :exclusions [[org.clojure/clojure]]]
 [org.clojure/tools.cli "0.3.1"]
   [org.clojure/clojure "1.4.0"]
 [org.clojure/tools.namespace "0.2.6"]
 [org.clojure/tools.nrepl "0.2.10"]
 [refactor-nrepl "1.1.0"]

rauh06:03:32

@urbanslug: Remove the "eval-in-leiningen". Always causes problems

rauh06:03:38

@urbanslug: Alternatively, you can just run a "lein install" first then you'll have it installed locally

urbanslug06:03:12

rauh: hmmm I think I’ll try that, fighting the dependency tree isn’t my definition of programming or problem solving.

urbanslug07:03:16

removing :eval-in … fixed it :0

gowder18:03:15

Can I ask an annoying noob set of questions about pmap? I'm trying to learn clojure by building a library to ultra-efficiently process large amounts of text data, and trying to wrap my head around how all the sexy efficiency things work. 1. Does pmap scale to hardware capacities seamlessly? Like, if I run the same algorithm with pmap on a dual-core machine, a quad-core machine, and an Amazon EC2 instance with like 8 virtual cores, do I just get a number of parallel processes appropriate for the number of cores, without having to do anything special? (I assume this is handled at the JVM level, like the JVM allocates threads to the cores available?) 2. Do I need to do anything special, function composition-wise, to take the best advantage of pmap combined with laziness? I have this vision of a number of processing a large number of documents and offering an API to users that allows them to compose a variety of utility functions to transform them (probably involving a ton of macros, whew), and then apply some standard transformational, all in one lovely step that just applied all at once. But I have the sneaking suspicion that there are probably parallelism gotchas here that I don't even know about... :-) Thanks!

bwstearns18:03:01

@gowder So I don't fully understand it, but from a quick read here (http://stackoverflow.com/questions/5021788/how-many-threads-does-clojures-pmap-function-spawn-for-url-fetching-operations) it seems that it launches 2+n threads where n is the number of processors available?

bwstearns18:03:59

I did see one note on another resource that said that since all I/O goes through one thread then that quickly becomes the bottleneck if there's enough I/O

bwstearns18:03:01

Also, I didn't know this until yesterday but (source f) is awesome. I had been relying on (doc f) until then.

gowder18:03:40

Thanks @bwstearns --- I'll start there!

bwstearns18:03:26

As far as part 2 I think the big thing to worry about is that the order of execution is not guaranteed, so in terms of side effects ideally you won't have any, but if they do have side effects then make sure the order they occur in doesn't matter.

gowder18:03:15

The I/O on a single thread thing sounds like a serious time/space tradeoff with large collections of documents: either load the whole mess into memory at least once so that costly transformations can be applied in parallel, or lazily apply the transformations on the file level, including I/O, then run into that bottleneck. Hmm... wonder if there's some workaround there. Is that a JVM limitation?

bwstearns18:03:23

And the other warning I have seen come standard with pmap is that you want the individual operation being mapped to be more expensive than the cost of thread creation. The more expensive the operation, the more you gain in parallelism.

bwstearns18:03:14

@gowder I am certain on the more expensive operations save more by being pmaped though.

alexmiller18:03:53

all of those comments on pmap seem reasonably accurate to me

alexmiller18:03:12

pmap is generally better when the units of work are "chunky" so that the cost of handing off the work and getting it back is not the dominant factor

alexmiller18:03:02

other options for this kind of thing are Java's java.util.concurrent.Executors (queue/worker pattern) or a Clojure wrapper with more control like Claypoole

alexmiller18:03:32

pmap returns results in the order of inputs - if that's not a constraint for you, then you might ultimately get better results from something else

adamkowalski19:03:29

if I am in a repl session and I have multiple namespaces across multiple files, is there a way I can go to my root namespace which imports everything else and have it find and pull in all the files it needs before loading itself?

adamkowalski19:03:30

Right now I am actually going into all of the namespaces manually, loading them into the repl then going back to my main file and loading it in

adamkowalski19:03:33

and any time i make a change in a file I need to load it back it, then go to the other files and reload those so they get the latest changes as well

donaldball19:03:35

(require ‘main.ns :require-all) is one way

adamkowalski19:03:35

awesome thank you! so tools.namespace essentially builds a dependency graph of all your namespaces so you can reload code into your repl right?

gowder19:03:28

Thanks @alexmiller ---I should check out claypoole. So a quick Google does suggest that people have achieved parallel I/O in Java, so the JVM should be able to handle something. Hmm...

adamkowalski19:03:01

woah tools.namespace + component seems like a really good way to get a nice workflow going

adamkowalski21:03:17

is there a way to treat a clojure string as a seq so that something like (reverse “hello”) will return “olleh” instead of a list of characters

adamkowalski21:03:58

that way I could make general functions that work on all sorts of lists and not have to treat string separately with (apply str (reverse “hello”)

seancorfield21:03:41

Well, reverse calls reduce which calls seq.

seancorfield21:03:05

(seq "String") produces a sequence of characters.

seancorfield21:03:10

clojure.string has several of the functions you want tho’, including reverse: (clojure.string/reverse "hello") => "olleh"

adamkowalski21:03:54

thanks, thats what I figured was happening

adamkowalski21:03:49

why doesn’t clojure check the type of the argument and dispatch the correct function like it does with first for example?

adamkowalski21:03:10

so if it is a string call clojure.string/reverse otherwise do the regular reverse?

seancorfield22:03:59

And first calls seq on its argument too.