Fork me on GitHub
#beginners
<
2018-06-26
>
chris56800:06:19

are there any static analysis tools for clojure that might give me some insight into the complexity of a given function?

chris56800:06:32

i’m running into issues were all the logic is correct and the data is clean but I’m getting null pointer or out of bounds errors leading me to believe some process is ballooning in size

val_waeselynck05:06:40

Such errors usually indicate a problem in the logic, not the algorithmic complexity though

bjr12:06:56

everything can be verified - so i’d isolate and make observations

bjr12:06:27

if you dislike unit tests throw some paranoid assertions throughout your code, you can always turn them off for production builds

chris56813:06:25

@ The logic is correct for any given small subset of the data though, the errors only occur when scaling up the amount of data processed which points to a resource limitation. If I can identify the expensive processes then I can develop a solution.

sova02:06:37

are there files or array indexes involved?

cristibalan13:06:38

Hi. I'm making a hash out of a vector of hashes based on a key and I have two working variants for now. Is there a more idiomatic way do do it?

(def items [{:id 1 :val "a"} {:id 2 :val "b"} {:id 3 :val "c"}])

(reduce #(assoc %1 (:id %2) %2) {} items)
;; {1 {:id 1, :val "a"}, 3 {:id 3, :val "c"}, 2 {:id 2, :val "b"}}

(->> items
     (map (juxt :id identity))
     (map (partial apply hash-map))
     (apply merge-with concat))

sundarj13:06:53

@cristibalan

=> (def items [{:id 1 :val "a"} {:id 2 :val "b"} {:id 3 :val "c"}])
#'boot.user/items
=> (group-by :id items)
{1 [{:id 1, :val "a"}], 2 [{:id 2, :val "b"}], 3 [{:id 3, :val "c"}]}
perhaps?

dpsutton13:06:22

or (into {} (map (juxt :id identity)) items) ?

chris56813:06:28

@sova there is one csv read at initialization, i’m not sure if there are array indices involved, I am using nth on sequences though, do these get converted into arrays under the hood?

cristibalan13:06:30

Nice. Thanks, guys!

alexandre.almosni13:06:24

Complete noob question please - I have a Clojure app running on a server. I'd like to write a web app (likely in Python, but would consider Clojurescript or Clojure), open with the world, on the same server, that communicates with the Clojure app typically querying for data. What's the easiest way to communicate? So far I've found exposing Clojure through a REST api (Liberator?), or some socket connection such as zeroMQ. Is there anything else easier / obvious? Thank you!

ordnungswidrig13:06:54

By Web app do you mean a JS app (SPA) or an Http service with an HTML frontend talking to your (other) service?

alexandre.almosni14:06:23

sadly I'm not sure about the difference but I think the latter - users would log in from any PC and be able to run reports, with the data prepared by the Clojure service

ordnungswidrig14:06:53

I’d create a http service with liberator that talks to your existing app and have a clojurescript frontend talk to that.

porkostomus14:06:21

How do I learn how to write tests? It seems like every resource assumes that you already know how to do it.

pez14:06:31

@porkostomus: do you mean technically or how to do it well?

porkostomus14:06:16

Technically, as in how to do it at all. It seems that there are countless tutorials on practically everything, taking you by the hand step by step, but I've never written tests and have absolutely no idea what to do.

pez14:06:00

I have my own little test driven process. When I write a function I add a :test function to the attributes map of the function and figure out how I want my function to behave by writing a call to the function and testing what it returns. Then I write the function. When I run the tests (either my editor doing it or vi lein test) the test runner (in clojure.test) will find these embedded tests and run them.

pez14:06:02

I wrote this earlier today:

(defn- split
  "Splits text at idx"
  {:test (fn []
           (is= ["(foo\n " "\n bar)"]
                (split "(foo\n  \n bar)" 6))
           (is= [" " " "]
                (split "  " 1)))}
  [text idx]
  [(subs text 0 idx) (subs text idx)])

pez14:06:42

The is= function is from clojure.test.

vale14:06:43

that's a very interesting way to go. i learned something today again

pez14:06:32

For me this replaces writing “exploration” code in (comment) blocks in the file. I can place the cursor before the (is= (or before (split in this case), and run the code in the REPL.

pez14:06:18

No idea if this answers your question, @porkostomus, but it is a way to get started.

porkostomus14:06:06

Thanks... that seems to make sense. But how did you learn how to do that? Is there some intro doc that I'm missing? Somehow I've managed to write many programs... I hack on it until it works, and if unexpected things happen I fix it. Testing? What? Everybody talks about how important it is, but there seems to be a void of info on how to get started with it...

pez14:06:45

I have been mostly a test-first coder for many years now. Picked it up from collagues and books and vids and shit. 😃 But I am new to Clojure and this paricular way embedding the tests with the function-under-test I picked up at a Clojure crash course seminar.

pez14:06:09

The more common way to unit test is to organize the code in a src and a test directory and mirror the src files with test files. I do this too most often, but generally like how the tests become documentation when they are embedded in the functions they are testing.

pez14:06:52

The test runner in clojure.test will find tests embedded with the code like I do above, tests with namespaces ending in -test (associating them with the matching namespace, w/o that suffix). There is also some (with-tests macro that I haven’t explored.

pez14:06:32

Now lets hope someone links to a good Clojure testing primer article.

porkostomus14:06:35

Yeah that's what I'm after. Clearly I'm missing a huge part of my programming experience

pez14:06:00

I can recommend doing it TDD style with the embedded test technique. It encourages you to write pure functions and functions that have a specific task and since you design the function’s interface by “using it” before you implement it quite often you will get nice interfaces. 😃 I can also recommend just going ahead doing it. You can add that missing part of your programming experience easily, just by moving some of those REPL explorations inside the :test functions.

porkostomus14:06:36

(I just need to learn how to write tests in order to try it!)

pez14:06:42

Me too. ClojureScript testing is rather much harder than Clojure testing so far.

pez14:06:40

(What I do for that is that I write code in CLJC files and test them using Clojure. But it is not always feasible.)

pez14:06:26

Today I found out that clojure.string/split-lines trims of beginning and ending newlines, so that for instance (clojure.string/split-lines "\n\n\n") => []. (This is because of how regexp splitting is behaving in Java.) Through the Java docs I also found that to stop this mad behaviour I could use a third parameter to clojure.string/split, giving it -1. So (clojure.string/split "\n\n\n" #"\r?\n" -1) => ["" "" "" ""]. Just leaving that here, in case some other beginner can avoid this trap.

mfikes14:06:54

Heh. That also works in ClojureScript. 🙂

pez15:06:36

Yeah, I tested it in both, since I need it in a CLJC file. 😃

josh_tackett15:06:15

Hey anyone know what (conf! ...) is?

bherrmann15:06:16

in what context? conf! looks like a function… the ! usually means that it modifies something….

porkostomus19:06:47

Is anyone else having trouble getting started with testing? I've managed to write programs without them so far, but if I ever want to work with people who do TDD that's not gonna fly. So I'm gonna do a deep dive and put together a tutorial or something. Any recommended resources are welcome.

bjr19:06:14

How do you know your programs work?

bjr19:06:19

I learned to use tests to write better software by 1) trying to answer that question, 2) recognizing that any manually performed verification should be covered by an automated test, 3) any test that is hard to write indicates poorly structured source code, 4) test code is source code, 5) take the lessons described in Stuart’s talk Debugging with the Scientific Method and apply to testing (tests are experiments informing your theory of what the code does)

bjr19:06:36

not sure I found any particular resource about testing particularly helpful

bjr19:06:11

failing to maintain brittle tests and finding bugs in “well-tested” code has been very informative

seancorfield19:06:12

My experience is that TDD is somewhat different in Clojure than other languages because you tend to work very heavily in/with the REPL while developing code (or you should be!).

seancorfield19:06:35

Consequently, where I would write tests for TDD and then write the code to make them pass in other languages, with Clojure I tend to write expressions and evaluate them in the REPL, building up to the code I need, rather than writing distinct tests first.

seancorfield19:06:10

I write the expressions typically inside a (comment ...) form in a source file and evaluate each expression as I go. Once I have "working" code, I can lift that code out into a function -- and might also lift out a result into a test (calling the newly-lifted function).

seancorfield19:06:44

Or I might just leave the (comment ...) in place for future developers to run the expressions inline.

seancorfield19:06:50

The ideal would probably be to create specs and/or property-based tests from that REPL session into code... but I haven't quite integrated that into my regular workflow.

seancorfield19:06:15

The only places where I tend to do "pure TDD" that would be recognized by developers in other languages is when I'm developing a very specific API and I have requirements in front of me that I can convert directly into tests (which fail), and then I'll switch to a more REPL-oriented workflow to design and build the code behind the API to make those tests pass.

seancorfield19:06:43

I highly recommend Stu's talk on "RDD": https://vimeo.com/223309989

bjr19:06:24

@ do you ever copy the expressions inside a (comment ...) into a test namespace?

porkostomus19:06:06

Thanks a lot!

seancorfield19:06:10

@ it's more likely that I'll copy the REPL result into a test namespace with a call to the function I just created. But then I'll often create a few more example-based tests from that (different arguments, different expected results).

seancorfield19:06:54

It would be better to create property-based tests tho' I think.

seancorfield19:06:39

I certainly don't find I need as many tests with Clojure as with some other languages.

bjr19:06:13

nice, I do the same. yeah…I have not integrated spec & property-based tests into my process yet.

alexmiller19:06:23

Clojure Applied has a chapter that goes through some basics on clojure.test, expectations, test.check, etc

pez19:06:24

I do something similar, but instead of (comment …) blocks I use the :test entry in the functions attributes map and then I leave some of the experiments there as tests (and as examples for future devs).

pez21:06:10

Thanks for the RDD link. Fantastic!

porkostomus21:06:33

yeah, I remember looking for that last year and couldn't find it... didn't realize it wasn't on youtube.

seancorfield21:06:51

@ Ah, that's an interesting approach... I tend to forget that tests are just metadata on functions... I might try that as a new workflow!

pez21:06:59

It seems that cljs.test does not discover these tests, though. So not always the way to go. But as long as it is clojure.test running the tests, it works. And, even so, using it for exploration and documentation still works.