Fork me on GitHub
#beginners
<
2020-03-29
>
glfinn8308:03:35

I can find a lot of material on how spec works, but I'm not finding a lot of examples of how it's used for functions. I want to make sure that the map I've received from Mongo has certain fields, and if not then log the explain. Anyone have a good example of this?

glfinn8308:03:35

Or is maybe not a usecase for spec? I'm also wanting to validate a map going intoba mongo update function

aviv08:03:21

In clojure-style-guide it's recommended to use Nil Punning, e.g: prefer (when (seq s)) over (when-not (empty? s)) why is this preferable? I find the latter more readable and concise. plus overhead of moving a map/vector to sequence

andy.fingerhut08:03:34

I think part of the reason is that it is idiomatic for experienced Clojure developers -- you should understand this when you see it, because it is used frequently.

andy.fingerhut08:03:24

Another reason is a little bit of efficiency. The definition of empty? is basically (not (seq Coll)) , so using not around empty? is sometimes double-negating the return value of seq

andy.fingerhut09:03:53

In terms of the part of the definition of the word "concise" that means "brief", (when (seq s)) is of course also shorter.

andy.fingerhut09:03:24

I do not understand what you mean by "overhead of moving a map/vector to a sequence", but happy to address that concern if you can explain what you mean.

andy.fingerhut09:03:16

For example, if you are thinking that calling seq on a vector immediately does an O(n) time traversal of all n elements of the vector, that is not the case. seq returns in O(1) time, creating and returning an object that you can call first and rest on it, each of which returns a value in O(1) time (well, being nit-picky they are not always O(1), but the ones that aren't are typically something like O(log n) where the base of the logarithm is 32)

andy.fingerhut09:03:35

Using seq and rest to traverse all elements of a vector/map/set will take O(n) time and allocate O(n) new objects. That is the nature of things when dealing with immutable values. Clojure reduce can avoid that O(n) memory allocation for built-in collections, and for custom collections that implement the necessary support for it to avoid the O(n) memory allocation.

aviv09:03:18

Thanks! also for the o(n), didn't know that :)

stebokas11:03:30

is there a variant of this -> that only passes non nil values to the next function?

quentin.leguennec111:03:44

@stebokas some-> ?

glfinn8312:03:13

Anyone here use Leiningen and have a nice way of separating integration tests from unit tests when running tests? Something similar to maven's Test and Integration-Test phases and the surefire and failsafe plugins?

glfinn8312:03:38

actually, lein help test gave me all the info needed 🙂

stebokas14:03:01

I am trying (spit "/tmp/ok" (make-array Byte/TYPE 12)) but it outputs something like [[email protected] how to spit actual bytes?

dpsutton14:03:22

are you ok with not using spit?

dpsutton14:03:24

(with-open [fos (FileOutputStream. "bytes.out")]
  (.write fos (into-array Byte/TYPE (.getBytes "bob"))))

stebokas15:03:07

thanks 😉

mfikes16:03:58

There are some nice facilities in as well.

(require '[ :as io])

(with-open [os (io/output-stream "bytes.out")]
  (io/copy (into-array Byte/TYPE (.getBytes "bob")) os))

mfikes16:03:39

Or even:

(io/copy (into-array Byte/TYPE (.getBytes "bob")) (io/file "bytes.out"))

quentin.leguennec114:03:46

@stebokas I think you would have to format it yourself, possibly using format

quentin.leguennec114:03:06

That is, converting it into string first

frederikdieleman19:03:28

Hi! After watching this presentation on data-driven development: https://www.youtube.com/watch?v=Tb823aqgX_0&amp;t=941s, I'm trying to incorporate data-driven development in my code, basically passing around a nested hashmap and applying things on it. Unfortunately I'm getting burnt a bit too often by NullPointerExceptions a bit too often, for example when changing a key name and forgetting to change it everywhere, or when changing a sub hashmap into a sequence of hashmaps. (both cases, a get call will return nil, creating the exception somewhere down the line). Very frustrating, especially because my calva lein repl often doesn't give any stacktrace (is this normal, or is there something wrong with my repl?). So I decided I should start using spec to explicitly check that my data structure has the keys/values/structures a function expects, but I'm doubting what the best approach to do this is. In a package I use (mxnet) they seem to validate the incoming parameters of functions by explicitly validating the parameters inside the function, e.g. with this utility function:

(defn validate! [spec value error-msg]
  (when-not (s/valid? spec value)
    (s/explain spec value)
    (throw (ex-info error-msg
                    (s/explain-data spec value)))))
Another option seems to be using s/fdef and then using stest/instrument to apply upon function call. What's the preferred approach for data validation during function calls?

pez19:03:21

> calva lein repl often doesn't give any stacktrace Can you describe a scenario when this happens?

frederikdieleman19:03:31

I'll post it here the next time it happens 🙂 Typically a nil value that shouldn't be there, e.g. it should have been a function, but I messed up, now clojure tries to call (nil & args), throws a NPE, but without stack trace, only 2-3 lines that point to read evaluation of the REPL, nothing about the location of where it happens inside my code.

frederikdieleman19:03:47

As a clojure beginner, rather challenging to debug 🙂

pez19:03:54

I think you run into a limitation of Calva there. Run the same code in the Calva REPL window and it should produce a stack trace for you. (You'll need to expand it, using the All, Clojure, etc links).

frederikdieleman19:03:58

I run it in the calva repl window inside vs code, or is that not what you mean?

frederikdieleman19:03:22

(thanks for the help btw! you're the creator of Calva right? In that case, just wanted to say that calva + vs code really made clojure a lot more approachable! So thanks for that!)

pez19:03:13

You're welcome!

pez19:03:43

I usually do not evaluate things in the REPL window, but when I am curious about a stack trace I do. So what does it look like when you do not get a stack trace? Can you paste a screenshot here?

frederikdieleman19:03:18

Sometimes I get this, sometimes it's without the interruptible_eval part, just 1-2 lines pointing to read-eval-print. This is in my repl window in vs code

frederikdieleman19:03:26

In this case, the NPE comes from calling nil as a function.

hindol.adhya19:03:41

JVM has a parameter. @frederikdieleman Are you using Lein?

pez19:03:29

That looks like it is a bit outside what Calva can influence, unfortunately.

frederikdieleman20:03:54

I am @, should I use some parameters in my project.clj file?

frederikdieleman20:03:43

@ no problem, writing specs will hopefully minimize the amount of times this creates frustrations 🙂

andy.fingerhut20:03:12

Using the option -XX:-OmitStackTraceInFastThrow is often a good idea (maybe always?) when starting a JVM, and I believe is not the default behavior.

andy.fingerhut20:03:33

I do not know if that will make a difference in your case, but good general advice for using Clojure/Java.

hindol.adhya20:03:56

Yeah, I was thinking about this option. But I read somewhere it is on by default in Lein.

andy.fingerhut21:03:49

One could use a command like ps in macOS or Linux to see all command line options to the java command after starting lein to confirm or refute that.

andy.fingerhut21:03:40

I just started lein repl outside of any Lein project directory, and it started a REPL with a java command line that does not use that option.

andy.fingerhut21:03:45

When I created a brand new lein project directory, changed into it, then ran lein repl , it did use that command line option.

andy.fingerhut21:03:09

I don't know the rule for how lein operates here. If I wanted it to be in there reliably, I would personally add it to my project.clj file.

frederikdieleman16:03:22

I also read that lein uses it as a default, but I'll double check 🙂

sfyire18:04:11

if your nested thing gets too big it might be worth investigating a database like Datahike, Datascript, Datomic etc that will free your data from being coupled to a particular tree, with one graph you can create as many context dependent trees as you like using datalog and pull

sfyire18:04:50

then when you have that ability you can make changes to one tree without affecting the others, at the moment I imagine your app is all coupled together because everywhere expects one shared structure (or one shared tree to put it another way)

seancorfield19:03:19

@frederikdieleman instrument is intended to be dev/test only. For production it's appropriate to use s/valid?

frederikdieleman19:03:06

So I should put explicit s/valid? calls inside most of my functions?

seancorfield19:03:30

Functions at the boundary, not "most".

seancorfield19:03:33

Also, perhaps, taking more time and care to design the "data API" so you aren't "changing a key name".

frederikdieleman19:03:39

Ok, thanks 🙂 So considering I'm not exactly creating production code, just some code to train a ML model, and still changing functions all the time, it's an ok approach to use instrument? Once the code is stable and using it as an "api", I should get rid of those and validate at the boundaries?

seancorfield19:03:12

Yeah, absolutely use instrument while you're developing -- that can be very helpful.

seancorfield19:03:59

I recently had to work on some legacy Clojure code that I'd never worked with before and it was heavily data-centric but I didn't have a sense of the different shapes of data through the pipeline I found it very helpful to write a lot of specs, instrument a lot of functions, and explore it in the REPL. Took me about four or five days to build up a full model in Spec so I could safely begin changing the code 🙂

frederikdieleman20:03:00

Thanks for the advice! Sometimes hard to know which tools to use when while being new to a language

seancorfield20:03:17

Yeah, and it can be particularly difficult for beginners with Clojure, given its focus on small, composable pieces -- and composable ideas -- so there is so much "assembly required" when putting together your dev env and your workflow.