Fork me on GitHub
#beginners
<
2017-07-07
>
ghadi02:07:06

One thing I treat differently with clojure libraries than I do with other lang libraries is that I don't mind projects with old commit dates.

ghadi02:07:48

My experience is that there is minimal bitrot

ghadi02:07:16

Most projects don't just continually accrue features and changes and weight. They tend to pick one thing (like hiccup chose only element rendering) get it right, then stop.

ghadi02:07:16

It sounds like the original poster wanted to treat html as html, and thus prefer enlive over hiccup

seancorfield02:07:53

@ballpointcarrot We (World Singles) use Selmer extensively in production for situations where we're interacting with designers on the HTML (most web pages, and HTML emails). For situations where we're fully in control of the HTML and the template would be very complex (lots of conditions and/or lots of nested loops), then we use Hiccup.

donyorm03:07:45

@billbarnhill Selmer (as is Hiccup) is bundled with luminus, which gives it some measure of popularity, at least with newer folks.

moogey03:07:14

I’m coming from python. And the whole CIDER - Figwheel - nREPL - Lein relationships are tripping me up really hard. Is it one of those things that just takes use until it clicks? Is there something that nicely explains it? http://www.sherv.net/cm/emoticons/fighting/bash-head.gif

noisesmith04:07:06

leiningen is a dependency manager, nrepl connects a clojure process to another process via the network, cider uses nrepl to integrate with emacs / elisp for code eval and reloading and debugging, figwheel uses nrepl to connect to a browser for the same stuff

noisesmith04:07:37

I actually don't like that beginner tutorials use all of those - you can learn most of the important stuff with java and clojure.jar

noisesmith04:07:59

the other things are nice, but when you get them all at once, there's lots of weird subtle points of potential failure or confusion

moogey04:07:56

I walked through the om beginner tutorial, and when I’m doing just clojure stuff, I get that I need to start CIDER and it looks for a project.clj. But I’m not certain how to keep figwheel stuff in emacs.

moogey04:07:07

Also, thank you @noisesmith

bnvinay9204:07:27

Can I use re-natal to integrate react-native into an existing native android app? I would like to have just a couple of screens be written in react-native so I can leverage code-push.

seancorfield05:07:31

@moogey What editor are you using for learning Clojure? What editor did you use for Python?

seancorfield05:07:52

Trying to learn Clojure by learning ClojureScript seems like a very fraught endeavor to me -- getting a ClojureScript dev env up and running is very fiddly, and the toolset keeps changing and evolving...

moogey15:07:06

For learning python I used Sublime Text, for work I use PyCharm. I did the Land of Lisp book and Advent of Code in Common Lisp in Emacs, so I’ve been using Emacs, every once in a while I’ll jump out to another clojure editor, then head back to Emacs a couple 45 minutes later. I mostly do webdev stuff, so sometimes I struggle to think of projects that aren’t in the browser. But I agree, learning clojurescript at the same time seems like a confusing road.

shaun-mahood15:07:09

I learned through ClojureScript and it was pretty reasonable. It's easy to get into the weeds but if you focus on one area (in my case, building a simple SPA with re-frame) then it can be a really good way to learn. I suspect that the emacs and cider learning curve is a big part of the issue - if you start out with Cursive and get a figwheel environment set up (not totally straightforward but not too bad), then I think that learning with ClojureScript will be pretty much equivalent to learning with Clojure.

shaun-mahood15:07:46

Happy to help you go through the setup as well, and I've been happy developing both Clojure and CLJS on Linux, Mac and Windows with the same setup. Pretty awesome how well it all works now once you get through the initial hump.

moogey15:07:35

That would be awesome. My brain keeps trying trying/wanting to treat things like python. I know its a bit different, but I haven’t gotten enough questions answered for the separation to be made. 😐

shaun-mahood15:07:12

@moogey: Let me know what you need help with - I'm not too python savvy but have done a little bit years ago. There's a lot to take in when you're getting started and I found it really helpful to take it one step at a time and try to understand the Clojure/CLJS version of something I knew well in another language. There are also lots of good books and resources to tackle specific areas too.

seancorfield16:07:55

Happy to help with the Clojure side too. I used Emacs for years, but switched to Atom and #protorepl at the end of last year and I love that. Not sure how easy it is to use for cljs but I saw some folks talking about Figwheel setup just the other day in that channel.

moogey16:07:18

Going to keep plugging away at things. Not sure how much of an ask this is, would someone be willing to do a hangout or something going through .. something. Where I could interrupt and ask questions?

seancorfield16:07:30

I'd offer but I'm working on a bunch of proprietary code...

seancorfield16:07:06

(we have a code base of around 60,000 lines at work)

shaun-mahood16:07:41

@moogey: Same here except my code base is much smaller and way less polished 🙂 I could probably spend some time helping you get setup or find materials if there's a particular project you want to tackle.

moogey16:07:07

I’ve had something in mind that uses twilio or something. ¯\(ツ)

moogey17:07:26

btw, thanks both of you for replying 🙂 I appreciate it

shaun-mahood17:07:35

@moogey: No problem!

victora07:07:26

Can somebody help me with post requests using ring-mock? The documentation is awful...

jumar08:07:14

@victora look at https://github.com/ring-clojure/ring-mock/blob/master/src/ring/mock/request.clj#L83 Something like this should work:

(mock/request :post "/api/resource" {"param1" "value1"})

tsulej11:07:00

@swizzard look at my approach made in clojure2d https://github.com/Clojure2D/clojure2d/blob/master/examples/ex34_noc22_forces_many.clj check-edges is just 5 lines. I think it can be done similarly in quil.

shan15:07:55

What’s the clojure way of comparing lists/seqs when you don’t care about the order of the items? I’ve got a list of maps, currently I’m just converting the expected and the actual into sets and then comparing with =

donaldball15:07:00

shan: That’s fine if you don’t care about duplicates. If you do, you could apply a sort to each seq before comparing, or compare frequencies

noisesmith15:07:35

oh that’s smart, (apply = (map frequencies l1 l2 ...))

slester17:07:13

Is there a more idiomatic way of getting a hashmap from a list of hashmaps besides (first (filter #(= "target" (:name %)) hm-list))?

noisesmith17:07:04

that’s pretty much it, but you can get fancy

=> ((comp first filter) (comp #{"target"} :name) [{:name "foo"} {:name "bar" :id 2} {:name "target" :id 3}])
{:name "target", :id 3}

noisesmith17:07:00

the #{x} thing is most useful when you have a few matches that count

noisesmith17:07:18

(and none are a false or nil)

moogey18:07:06

not sure if its more clojure-y but.

(first ((clojure.set/index hm-list [:name]) {:name "target"}))

ghadi18:07:48

I find the (( double-banana pattern (invoking an invocation) really hard to read, however composing a data accessor and a set is really nice and common (comp #{thing} :field)

bschrag19:07:32

Getting this error on Windows, with jdk1.8.0_131 installed.

foo.core>  (sh "dir")
IOException CreateProcess error=2, The system cannot find the file specified  java.lang.ProcessImpl.create (ProcessImpl.java:-2)
My core.clj includes...
(ns foo.core
  (:require [clojure.java.shell :only [sh]])
  (:gen-class))
Ideas?

dpsutton19:07:40

look at the comments of the top answer on https://stackoverflow.com/questions/6734908/how-to-execute-system-commands. it looks like you need "cmd" and "\c" in there

bschrag19:07:34

dpsutton: Good point. However, I get the same error with "notepad", which that page suggests should work.

bschrag02:07:52

Hmm, this error has stopped happening, perhaps because I'm handling ns better now...

goomba19:07:03

anyone want to play "why is my code so slow" with me? 😛

goomba19:07:30

trying to do some data processing for some crypto research

goomba19:07:35

(defn -main [& args]
  (let [value (slurp (nth files 1020))
        ws (words value)]
    (as-> {} x
        (time (assoc x :bigrams (bigrams ws)))
        (time (assoc x :trigrams (trigrams ws)))
        (time (assoc x :quadgrams (quadgrams ws))))))

"Elapsed time: 1570.11396 msecs"
"Elapsed time: 1957.212454 msecs"
"Elapsed time: 2403.794646 msecs"

goomba19:07:03

those times seem unacceptable to me... I've got 40,000 books to process!!!

goomba19:07:50

(defn ngrams [n contents]
  (let [ngs (partition n 1 contents)]
    (frequencies ngs)))

(defn bigrams [contents]
  (ngrams 2 contents))

(defn trigrams [contents]
  (ngrams 3 contents))

(defn quadgrams [contents]
  (ngrams 4 contents))

goomba19:07:34

I'm hoping there's a "hey idiot, you should be doing this" answer 😅

donaldball19:07:24

Prolly nothing earth-shattering here but I mean you are looping over your input collection thrice, and you could do it in a single loop, though you’d have to maintain your own n-gram state and frequencies values. Assuming you did that, you’d probably want to use a transient collection

goomba19:07:34

yeah but even for an individual time those are rough

goomba19:07:24

I like to be the guy that says to everyone in the office "all the Clojure" but I ended up having to resort to Python because there a 3 fold order of magnitude speed difference

goomba19:07:16

collections.Counter(partition(2, 1, contents))) runs really fast

goomba19:07:11

which makes zero sense to me because assoc operations in Clojure are several orders of magnitude faster than in Python

goomba20:07:19

haha damn! how appropriate!!

alwyn21:07:08

I saw partition and I wondered if both approaches are single threaded or not?

goomba00:07:10

yes... my approach was single threaded 😅

moogey01:07:14

@goomba what did you end up going with?

goomba02:07:22

Honestly I just had to get this thing running so I wrote a Python-clojure interop 😅

goomba02:07:53

Interop is a strong word... More like a hack

goomba22:07:36

holy smokes I got a protip that sped up my code by a factor of 10

goomba22:07:36

(defn slow-ngrams [n contents]
  (let [ngs (partition n 1 contents)]
    (frequencies ngs)))

(defn fast-ngrams [n contents]
  (->> contents
       (subs-windows-educt n)
       (frequencies)))

goomba22:07:49

(defn slow-book []
  (let [content (slurp (nth files 1020))
        book (extract-book content)
        ws (words book)]
    (time {:bigrams   (bigrams ws)
           :trigrams  (trigrams ws)
           :quadgrams (quadgrams ws)})
    ))

(defn fast-book []
  (let [content (slurp (nth files 1020))
        book (extract-book content)
        ws (words book)]
    (time (into {}
            (pmap (fn [[k v]] [k (v ws)])
                  [[:bigrams bigrams]
                   [:trigrams trigrams]
                   [:quadgrams quadgrams]])
            ))))

goomba22:07:03

it's now substantially faster than my Python code!!!! yayyyyyy!!!

goomba22:07:28

;; Python code
"Elapsed time: 1536.049366 msecs"

;; old-time (Clojure)
"Elapsed time: 6634.489171 msecs"

;; new-time (Clojure)
"Elapsed time: 1084.17433 msecs"

goomba22:07:39

that's without even using xforms yet!

moogey15:07:53

Having not done java outside of a small android tutorial in 2012, this stuff is going a bit over my head, heh. I’m glad you found a fast solution though. And thank you for sharing 😄 I enjoy looking

kkruit21:07:26

why doesn't the following run the map? (do (map prn ["a" "b" "c"]) {:success true})

kkruit21:07:41

Because lazyness, use dorun?

jeremys21:07:49

@goomba I don’t that the assoc is the problem, try to time the ngrams function, I think this is where the most time might be spent. If that is the case you might want to look if you can do the processing with transducers to partition and get frequencies in one traversal of the contents. You might even be able to do the bi-tri-quad in only one travesal of the content using transducers from Christophe Grand’s xform library.

goomba21:07:30

It's absoultely the ngrams function... was using the frequencies and partition functions (now that you mention it I'm not actually sure which one of those is the bottleneck so that bears some looking into)

goomba21:07:16

it's frequencies for sure...

(let [ps (time (partition 2 1 ws))]
  (time (frequencies ps)))

"Elapsed time: 0.035318 msecs"
"Elapsed time: 1689.346757 msecs"

jeremys21:07:52

I don’t know if there is a frequency transducer, it might be more of a reducing function. Take a look at this library https://github.com/cgrand/xforms

goomba21:07:15

oops just realized partition is lazy, so that does contribute some time

goomba21:07:21

I'll take a look at that... thank you

jeremys21:07:54

I have just taken a look at frequencies implementation, it isn’t a transducer nor a reducing function. It actually uses reduce internally.

jeremys21:07:43

you might want to code your own version using transduce instead of reduce and use the partition transducer. The library I linked might be able to let you do the 3 processing in one iterating over the contents once using transjuxt but I am not sure there.

cgrand12:07:32

jeremys: @goomba

(defn ngrams [n s]
  (x/into {}
    (comp (x/partition n 1)
      (x/by-key identity x/count))
    s))

;; or if you know that your input is made of chars, you can treat partitions as strings which will be lighter than a collection
(defn ngrams-str [n s]
  (x/into {}
    (comp (x/partition n 1 x/str)
      (x/by-key identity x/count))
    s))

cgrand12:07:03

On a random string of 1M chars (chosen between A and Z), I get:

user=> (count (time (ngrams-naive 2 data)))
"Elapsed time: 1249.028 msecs"
676
user=> (count (time (ngrams-str 2 data)))
"Elapsed time: 434.007 msecs"
676
where ngrams-naive is frequencies + core/partition

jeremys13:07:06

Thanks for the answer, I hope this helps @goomba. Your xform library is really interesting although I haven’t taken the time to study it properly and I quite frankly find it really hard to grasp. I just knew he might find the tools he needed there.

cgrand13:07:36

I should definitely write an example-driven guide to xforms.

jeremys13:07:55

You might though I understand it is no small task. The by-key transducer can do so much and it is only one from the many you provide.

sneakypeet22:07:45

I have the below code. I instantiate an instance of LogPubSub type in a namespace other than any of these below and pass it into the get-command-handler function as the pubsub param. When I ignore the line (enqueue pubsub events) everything works fine. the :pre conditions pass and I get the type I expect. however when adding the (enqueue pubsub events) line I get the following compiler exception

java.lang.RuntimeException: Unable to resolve symbol: enqueue in this context, compiling:(tenandsix/domain/commands.clj:18:7)
How do I tell (enqueue pubsub events) that it should call the enqueue method from the protocol?
(ns tenandsix.domain.core)

(defprotocol PubSub
  (enqueue [this events]))

;;----
(ns dev-integration
  (:require [tenandsix.domain.core :refer [PubSub]]))

(deftype LogPusSub []
    PubSub
  (enqueue [this events]
     ...))

;; ---

(ns tenandsix.domain.commands
  (:require [tenandsix.domain.core :refer [Database PubSub] :as d]
            [clojure.spec.alpha :as s]
            [slingshot.slingshot :as slingshot]
            [tenandsix.common :as c]))

(defmulti handle-command (fn [type data db] type))

(defn get-command-handler [db pubsub]
  {:pre [(satisfies? Database db) (satisfies? PubSub pubsub)]}
  (fn [type data]
    (let [events (handle-command type data db)
          events-valid? (s/valid? :d/events events)]
      (when-not events-valid?
        (slingshot/throw+ (c/->app-error "Command Handler Needs to Return a Vector Of Events" events)))
      (enqueue pubsub events)))) 

tsulej22:07:24

(:require [dev-integration :as di])

tsulej22:07:34

(di/enqueue ...)

jeremys22:07:44

@sneakypeet your require is bit wrong you can’t refer a protocol, but you should be able to refer functions that are part of the protocol.

tsulej22:07:40

yes, enqueue is a function and is defined in dev-integration namespace, refer to it like to other functions defined in any namespace

sneakypeet22:07:42

so I do not want to (:require [dev-integration :as di]) as I do not want the code to know where the type was defined. but would

(:require [tenandsix.domain.core :as d])
d/enqueue
not work?

sneakypeet22:07:04

sweet d/enqueue works

tsulej22:07:03

oh, didn't know that you can refer to protocol fn, nice

sneakypeet22:07:21

so defining here is what happens

sneakypeet22:07:42

(defprotocol P
  (foo [x]

sneakypeet22:07:56

foo gets defined as a fn in the given namespace

sneakypeet22:07:19

(defprotocol P1
  (foo [x]))

(defprotocol P2
  (foo [x]))

sneakypeet22:07:34

will give a warning that foo is being redefined

jeremys22:07:23

yep you need P1 & P2 to be defined in different namespaces so the foo fns don’t clash

sneakypeet22:07:30

I had all the info, just could not put the dots together 😄

jeremys22:07:46

namespaces can be tricky, especially when protocols and records come into play