Fork me on GitHub
#beginners
<
2020-09-02
>
joel38001:09:51

in maven the tests are kept in separate structure (same package), usually Java the classes end with *Test. With clojure, does it make more sense to keep the same namespace the same?

seancorfield01:09:53

@joel380 Most tools assume -test at the end of the ns.

seancorfield02:09:49

so foo/bar.clj would have tests in foo/bar_test.clj -- and we also usually have a src tree and a separate test tree because that makes it easier to package source code into a JAR while omitting test code.

seancorfield02:09:24

(and just to be clear, those would have namespaces foo.bar and foo.bar-test respectively)

seancorfield02:09:25

Also, since both src and test will be on your classpath during testing, the namespaces must have different names otherwise they will conflict.

joel38002:09:23

thanks, that's the way i started, and then questioned it.

joel38002:09:14

I thought I'd be able to (print/dump) my nested map structure, cut/paste into unit test, and just have it work, but that's not quite the story... have to quote the empty lists... doesn't seem practical now.

dpsutton02:09:23

I thought most tools used the metadata from deftest and not the namespace name.

seancorfield03:09:53

The test runners expect to find -test namespaces. Certainly Cognitect's does. I think CIDER assumes some conventions as well? That my.namespace will have my.namespace-test as the place for the tests?

dpsutton02:09:53

Empty lists evaluate to themselves. You shouldn’t need to quote them

joel38002:09:26

must be doing something wrong then.

joel38002:09:56

i have like .... :keys ("x","y") .... and I have to quote the list, as well as the empty ones.

joel38002:09:48

just deep quote the whole thing it looks like does it.

dpsutton02:09:41

That’s not an empty list

dpsutton02:09:24

user=> ()
()

dpsutton02:09:00

but if you want to just read a list you'll need to quote it.

nfedyashev07:09:37

Is there any way to count lines in un-gzipped file without slurp/reading it as a whole?

(ns gzip-playground
  (:require [ :as io]
            [clojure.string :as str])
  (:import
   ( StringReader BufferedReader ByteArrayOutputStream ByteArrayInputStream PrintWriter)
   (java.util.zip GZIPOutputStream)
   ))

  (->> (io/input-stream "")
       (java.util.zip.GZIPInputStream.)
       (slurp)
       (str/split-lines)
       count
       )

nfedyashev07:09:16

not sure how to get line-seq after this java.util.zip.GZIPInputStream. step

nfedyashev07:09:00

sorry, just found a way that seems to work

(->> (io/input-stream "")
       (java.util.zip.GZIPInputStream.)
       io/reader
       line-seq
       count
       )

jsyrjala08:09:09

Is that any better than one with slurp? I’d guess both read the whole thing to memory and keep it in memory while counting happens.

jason35808:09:12

@jsyrjala it's not, AFAICS, because the last variant is lazy

alexmiller12:09:59

If you don’t hold the head then the seq can be gc’ed behind you as you count. Slurp makes a string in memory of the whole contents.

borkdude13:09:29

it's recommended to close/clean resources at the end btw. I've had issues with jar files still being opened (which are similar to zip files) in clj-kondo, which sometimes blew up the entire process when you linted a huge classpath (https://github.com/borkdude/clj-kondo/issues/542#issuecomment-549054475)

joel38016:09:27

are private functions considered best practice?

joel38016:09:47

so used to doing it java it's a habit.

seancorfield16:09:58

I've gone back and forth on them. I started out using them a lot and then often found I wanted to test and/or reuse private functions so I made them private. Then I got into the habit of making every function public except for very specific implementation details that fell out of refactoring (breaking up a larger public function). Now I've started to lean more toward private functions by default because I'm used to clj-kondo telling me a private Var isn't used (and can be deleted) when I'm refactoring code.

seancorfield17:09:57

If I change the API of a public function, I have to search for all call sites -- including some legacy non-Clojure code that calls into our Clojure code which uses munged names (i.e., _ instead of -, _QMARK_ instead of ? etc). If a function is private, I know I only have to check for uses within the current namespace. Well, unless there's some rogue #'my.ns/private-fn references to it somewhere 🙂

joel38017:09:51

yeah, i think i'm going to use them, and make the test ns match.

seancorfield17:09:52

@joel380 I'm not sure what you mean by "make the test ns match"?

joel38017:09:54

use the same namespace for the test file. so that i can invoke the private fns.

hiredman17:09:02

don't do that

joel38017:09:31

what issue will that run into?

joel38017:09:12

feels right, similar to java/smalltalk approach.

hiredman17:09:15

a lot of things assume a 1:1 file to namespace mapping

hiredman17:09:47

or at least that there is a single resource on the classpath with a given name that when loaded will create a given namespace

joel38018:09:50

so should i keep tests in same file as code?

hiredman18:09:08

no, make a test namespace, but the tests in there

joel38018:09:12

how to invoke private functions though from that namespace?

jr0cket18:09:37

Test private functions through testing the public functions. I avoid testing private functions as it leads to lots of low value unit tests, slowing down the speed of running tests

joel38018:09:25

i guess answer is "don't"?

joel38018:09:19

i saw there was concept to have a file with a "master namespace"... that doesn't apply here?

alexmiller18:09:19

use the var quote

joel38018:09:43

what is var quote?

alexmiller18:09:56

#'my.ns/private-fn

joel38018:09:10

invoke method using a quoted fn name?

alexmiller18:09:43

it's ugly, but I think that's actually a good reminder that you're hitting something private in your test

alexmiller18:09:54

which is imo perfectly fine

joel38018:09:53

also, another nit related to maven... maven will automatically run the tests (good). But i can't run the tests when developing unless I add the (run-tests 'ns) line to the test file. If I leave that in then the tests run twice during build... I'm thinking there's a better expected way to do this?

alexmiller18:09:45

I usually put something like that in a (comment... ) block at the bottom so I can eval it to the repl from my editor

alexmiller18:09:09

or some editors have built-in support to "run tests in this ns"

joel38018:09:58

yeah, probably the latter is better... i just know those comment blocks will be forgotten.

joel38018:09:32

btw, thanks for the java-to-clojure examples @alexmiller

alexmiller18:09:27

forgotten to use? or forgotten in the code?

alexmiller18:09:15

if the former, just a matter of habits (I have them in a lot of files but I always put them at the end) if the latter, who cares?

seancorfield18:09:29

Likewise. We have a lot of those Rich Comment Forms all over our code for REPL convenience when developing (Stu Halloway's name for them, I gather, because Rich Hickey uses them a lot 🙂 ).

joel38018:09:21

so you uncomment to use them right? then end up committing them by accident, or i'm missing something?

alexmiller18:09:33

no, they are always in the comments

seancorfield18:09:39

No, eval code via the editor from within the comment form.

seancorfield18:09:19

In Atom, that's ctl-; b for evaluate block, when the cursor is at the edge of each form you want to evaluate.

seancorfield18:09:34

(well, with the most common keymap setup)

seancorfield18:09:39

All editors with Clojure integration should have hotkeys for evaluating a form, evaluating a top-level form, evaluating selected/highlighted code (although that's rarely as useful), and somehow loading or evaluating a complete file.

seancorfield18:09:02

(I never type into a REPL, I only ever eval code from my editor into my REPL)

seancorfield18:09:18

I highly recommend https://purelyfunctional.tv/courses/repl-driven-development-in-clojure/ if you don't mind paying $49 for a month of access to all Eric's courses. That one RDD course is worth the price on its own.

seancorfield18:09:33

Monthly/yearly subscriptions are available via this page https://purelyfunctional.tv/register/

seancorfield18:09:48

Developing a really tight REPL-based development workflow pays off over and over, when doing Clojure.

joel38018:09:46

i did think his free web content was really good.

seancorfield18:09:16

Re: test namespaces -- the standard practice is to have your source files and your test files in separate trees that match in terms of directories, but the test files end in _test.clj (and their namespaces end in -test). Several test runners assume that -- and it's what you'll see if you use any of the "new project" generators (`lein new ...`, clj-new, etc).

seancorfield18:09:44

@joel380 Like this:

> clj -A:new app joel/example
Generating a project called example based on the 'app' template.
> tree example
example
|____.gitignore
|____.hgignore
|____CHANGELOG.md
|____deps.edn
|____doc
| |____intro.md
|____LICENSE
|____pom.xml
|____README.md
|____resources
| |____.keep
|____src
| |____joel
| | |____example.clj
|____test
| |____joel
| | |____example_test.clj

seancorfield18:09:23

The namespaces in this new project are joel.example in the source tree and joel.example-test in the test tree.

seancorfield18:09:00

lein new app joel/example would create almost exactly the same structure, except with project.clj for Leiningen instead of deps.edn for CLI.

joel38018:09:24

yeah, i noticed that the hard way... build kept complaining about unfound tests, as intellij (w/o cursive anyway) renamed the namespace like the file name.

joel38018:09:43

so now i have the _ file and the - test ns.

jr0cket18:09:59

@joel380 https://practicalli.github.io/clojure/testing/unit-testing/ covers the basics for simple unit tests. I also suggest using generative tests to minimise test code whilst increasing testing.

rickheere19:09:48

I have a question about clojure.spec. I'm writing a spec for a function and the first parameter is a string out of a set of possible strings. But to get all the options I have to query an API. The question is. In spec is it a normal thing to query for data from an external data source or should that be avoided?

alexmiller19:09:33

generally, I would avoid it and spec it as a string

alexmiller19:09:51

unless you dynamically generate the static spec (which I have seen some people do)

rickheere19:09:59

Thanks for the advice Alex