Fork me on GitHub
#beginners
<
2021-07-12
>
seancorfield00:07:36

It's an interesting idea to use (defn ^:private foo [] ...) given that we don't have def- so a private var has to use (def ^:private thing ...) -- there's a consistency to that I guess. I would probably still lean toward defn- since that's what I "grew up with" in Clojure and it's definitely more common.

sova-soars-the-sora01:07:38

@corasaurus-hex you are 100% correct about it being shorthand

👍 2
Rob Haisfield02:07:53

Is it possible to run bash from within Clojure code?

sova-soars-the-sora02:07:31

Uh… @borkdude probably knows xD

phronmophobic02:07:25

there are a number of ways to run bash depending on the use case. As an example:

test> (use '[clojure.java.shell :only [sh]])
nil
test> (sh "/bin/bash"  "-c" "pwd")
{:exit 0, :out "/var/tmp\n", :err ""}

noisesmith15:07:17

there's also the more flexible ProceessBuilder / Process

user=> (-> (ProcessBuilder. ["/bin/bash" "-c" "pwd"]) (.inheritIO) (.start) (.waitFor))
/home/justin/clojure-experiments
0

noisesmith16:07:17

better match for the functionality though

(let [builder (ProcessBuilder. ["/bin/bash" "-c" "pwd"])
      process (.start builder)
      out (.getInputStream process)
      err (.getErrorStream process)]
  {:result (.waitFor process)
   :out (slurp out)
   :err (slurp err)})
but that's just feature-for-feature match of shell/sh, you can also use the streams interactively / async while keeping the process alive

popeye07:07:07

(#(map * %&) [1 2 3] [4 5 6] [7 8 9]) gives an error where as (#(apply map * %&) [1 2 3] [4 5 6] [7 8 9]) gives result, What does apply do here

jsn08:07:17

what you're trying to do is probably (map * [1 2 3] [4 5 6] [7 8 9])

jsn08:07:10

your #(... %&) wraps the list of vectors in a list (a sequence, really), and your apply then unwraps it, doing effectively the same

João Galrito15:07:53

hello, is there a way to filter a coll based on some state computed from the previous items? I know I could use reduce for this but I'm trying to avoid build the whole coll

João Galrito15:07:00

maybe this doesn't make sense

noisesmith15:07:54

you can use reduced to terminate reduce early, otherwise your other option is loop which is more complex than using reduce

João Galrito15:07:56

yea I think using loop will work

noisesmith15:07:37

the thing is, with loop you need to traverse the list "by hand" and reduce already does that for you, loop makes the code more complex if you are walking a list in order

João Galrito15:07:12

yea nvm, the thing I need to do can be accomplished with reduce as well

popeye16:07:55

how much time memorize will keep the values in cache ?

popeye16:07:54

i am calling same database twice in my two different function which gives same result! can I make use of memorize instead ?

dpsutton16:07:59

the clojure.core function memoize (note no r) has no eviction policy for cached results, and is suitable only for functions with a small set of inputs or in non-production environments. There's a library clojure.core/memoize that exposes more advanced strategies for managing the cache of results, like bounded queues and time based cache evictions

dpsutton16:07:15

is the library. and the always fantastic http://grep.app can help find usage in the wild

popeye16:07:03

how can avoid calling database twice from different function twice ? Can I assign those values in def and make use of that ?

sova-soars-the-sora17:07:03

Yes, rather than memo-ize, you can have a map called cache-db-results or something and check it for entries first. If empty, query the db and store the result

noisesmith17:07:23

def is for globals (and shouldn't be re-defined at runtime from regular code), you can use core.cache for result caching

noisesmith17:07:37

or just a hash-map inside an atom

noisesmith17:07:53

@popeyepwr since this is #beginners I feel like I should warn that caching is a harder problem than it looks like at first, and you gain a lot by using a dedicate pre-existing cache library

noisesmith17:07:47

memoization is typically meant for pure computations, while caching is meant for arbitrary data that might be expensive to acquire

Cora (she/her)17:07:51

the TTL in there is probably what you want unless you want to do cache eviction (which is really really really really hard to do right)

popeye17:07:09

my database query returns only 5 result everytime, So i thought of using memoize , But till when the values are cached if we use it?

noisesmith17:07:41

the memoize function returns a new function that stores results until the function goes out of scope

popeye17:07:43

just wanted to improve the code, Do not want to use new dependency

noisesmith17:07:19

memoize will not work here because it never lets go of any of the data

noisesmith17:07:45

@popeyepwr as I mentioned before this is a much harder problem than it looks like at first, and rolling this yourself is a lot more work (and almost guaranteed to be buggy) compared to adding a dep

Cora (she/her)17:07:45

memoize uses the library I linked I believe

noisesmith17:07:52

no it does not

noisesmith17:07:21

org.noisesmith.expecting=> (source memoize)
(defn memoize
  "Returns a memoized version of a referentially transparent function. The
  memoized version of the function keeps a cache of the mapping from arguments
  to results and, when calls with the same arguments are repeated often, has
  higher performance at the expense of higher memory use."
  {:added "1.0"
   :static true}
  [f]
  (let [mem (atom {})]
    (fn [& args]
      (if-let [e (find @mem args)]
        (val e)
        (let [ret (apply f args)]
          (swap! mem assoc args ret)
          ret)))))
nil

mario-star 2
noisesmith17:07:41

it creates an atom and puts every arg list and result in the atom

noisesmith17:07:04

it's not appropriate for caching an endpoint at all

Cora (she/her)17:07:45

I was thinking of that, sorry 😬

noisesmith17:07:05

right, clojure's memoize doesn't use either of those - it's very naiive, see the snippet above

Cora (she/her)17:07:08

which is a different thing than the core memoize

Cora (she/her)17:07:59

depending on what you're doing, one good alternative is to look up all the records ahead of time and pass them in to the function calls. saves on repeated lookups in tight loops and often makes functions more pure since you can put all the i/o in a single location instead of nested wherever

noisesmith17:07:41

you could even reasonably call memoize inside a smaller scope that doesn't live as long as the entire program, if you know a given arg list should always map to the same result and don't mind the occasional double-call when there's a data race

2
Noah Moss17:07:07

Is there a good way to write a test that asserts that two functions, say f and g, have approximately the same runtime? I.e. it doesn’t matter whether they’re fast or slow, we just want to make sure that the average runtime of f and g doesn’t differ by more than 10% (or whatever threshold). time doesn’t seem quite accurate enough, and I was getting weird results when running it in a loop (maybe because of laziness?). But a benchmarking lib like Criterium seems way too heavy and not fit for a testing use case

pithyless17:07:04

I think performance regression testing is a good case for criterium; I would just tag the tests as benchmark tags that don't get run all the time (e.g. only in CI or between releases, etc)

noisesmith17:07:13

instead of running time in a loop, run a loop inside time

noisesmith17:07:23

(sounds like a doctor who quote doesn't it...)

Cora (she/her)17:07:45

that would be a neat library

noisesmith17:07:49

(time (dotimes [_ 10000] (your code goes here)))

Noah Moss17:07:49

that makes sense, I feel dumb for not thinking of that haha. thanks!

noisesmith17:07:28

criterium does this (both figuring out how many times to run, and showing stats on variation)

Cora (she/her)17:07:21

showing the variation in runtime is so nice

noisesmith17:07:35

@noah688 I removed the doall / dorun above because they only work if your code returns a sequence, I think it's up to you to figure out how to ensure the full result of your code is realized inside the dotimes body

👍 2
noisesmith17:07:53

(eg. what if you returned a lazy-seq of lazy-seqs etc.)

noisesmith17:07:04

I guess (dorun (tree-seq coll? seq x)) is generally safe but it's probably more than you actually need and probably colors the timing result

teodorlu17:07:55

(require '[hiccup.core :as h])

(->> (edn-paths)
     (map slurp)
     (map edn/read-string)
     (map h/html))

;; fails with:
;; Can't take value of a macro: #'hiccup.core/html
Is there an alternative to h/html that I can use as a function?

noisesmith17:07:17

you could look at what h/html expands to, or use #(h/html %)

teodorlu17:07:51

Wrapping in a function didn't look like it helped too much:

(->> (edn-paths)
     (map slurp)
     (map edn/read-string)
     (map #(hiccup.core/html %)))
;; 1. Caused by java.lang.IllegalArgumentException

ghadi17:07:37

always paste the full stack trace. (in a snippet if possible CMD+Shift+Enter)

teodorlu17:07:05

Thanks for the shortcut - I suspect that's going to save me some time.

ghadi17:07:05

eliding it throws away the ability for others to help you, and perhaps for you to help yourself

teodorlu17:07:18

above full stacktrace ☝️

ghadi17:07:59

seems like some element vector without an element name?

👀 2
noisesmith17:07:09

" is not a valid element name." - looks like bad input

👀 2
ghadi17:07:20

inspect your input without the terminal call to hiccup/html

dpsutton17:07:25

is not a valid element name. yeah. somehow you ended up with [] or [nil]

👀 2
teodorlu17:07:44

Some invalid conclusions on my side. I'll test a bit before asking further 🙂

ghadi17:07:04

Debugging is science™

👍 2
2
teodorlu17:07:49

I found a [] in my edn file. Removed that, and now it produces valid HTML. So you guys were right ☝️ Thanks!

sova-soars-the-sora17:07:34

Genius! 😄

✔️ 2
Matheus Silva20:07:47

I'm separating items by dates and wanted to get his index from the list he belongs to, but I don't know how to get the index from a lazy-seq

(defn part-by [item items]
  (partition-by #(cond
                   (= (:date %) (:date item)) "value"
                   :else "string")
                items))

noisesmith20:07:35

lazy-seq is not indexed, you'll need to count it yourself

👍 2
noisesmith20:07:43

(or add indexes to the input items)

👍 2
ChillPillzKillzBillz20:07:16

How does one go about writing tests for their clojure code...? Is there a guide which explains?

seancorfield20:07:34

@abhishek.mazy It's probably worth looking at some OSS projects on GitHub that have good test suites. Most folks use clojure.test because it's "built-in". I don't know if there are any good tutorials on writing tests in Clojure tho'...

ChillPillzKillzBillz20:07:13

Hi Sean, Thanks for that. I was asking about clojure.test itself. I am developing in clojurescript for now... but I don't understand how or even if clojure.test links with that... if yes how? The documentation on this is thin from my searches...

seancorfield20:07:11

ClojureScript has the clojure.test namespace too, yes. and Olical's cljs-test-runner project lets you run tests for ClojureScript.

seancorfield20:07:56

(the whole project is .cljc files so it can be run as both Clojure and ClojureScript)

seancorfield20:07:07

The syntax for importing the test ns is slightly different, but the tests are then nearly all identical for both languages: https://github.com/seancorfield/honeysql/blob/develop/test/honey/sql_test.cljc#L6-L7

seancorfield20:07:06

Ah, I use cljs.test but I'm pretty sure clojure.test exists too? Anyways, there's an example of how to write cljs tests and run them...

ChillPillzKillzBillz07:07:15

thanks for all the info Sean!! cljs.test or clojure.test - I am not informed enough to know the difference or point out the distinction!! I'll look through the links!! Many thanks again!

lread14:07:29

Ya, you can now get away with only requiring clojure.test in cljc files. ClojureScript will automatically convert to cljs.test. For example: https://github.com/clj-commons/rewrite-clj/blob/52f429e09429a6e942837fabb481045698cba54f/test/rewrite_clj/zip_test.cljc#L4

seancorfield17:07:15

@UE21H2HHD Oh, you don't even need the conditional and the :refer-macros stuff now? (I don't track the cljs world so I haven't seen if/how it has advanced in terms of language compatibility lately)

lread17:07:30

That’s right, no need anymore!

seancorfield17:07:29

Good to know -- I'll create an issue to clean that up in HoneySQL. Thanks.

ChillPillzKillzBillz20:07:13

Hi Sean, Thanks for that. I was asking about clojure.test itself. I am developing in clojurescript for now... but I don't understand how or even if clojure.test links with that... if yes how? The documentation on this is thin from my searches...

gabor20:07:26

hi, in lein project.clj I have

(defproject foo "0.1.0-SNAPSHOT"
      :dependencies [ [clojure-csv/clojure-csv "2.0.1"]
which I can see in lein deps :tree but in the lein repl if I try (use '[clojure-csv [parse-csv :as pp]]) it errors
Could not locate clojure_csv/parse_csv__init.class, clojure_csv/parse_csv.clj or clojure_csv/parse_csv.cljc on classpath.
I does seem to be there under ~/.m2/repository/clojure-csv/clojure-csv/2.0.1/, anyone can give my noob self a hint please?

seancorfield20:07:09

It should be either (use 'clojure-csv.core) which will refer in all symbols from that namespace or something like (require '[clojure-csv.core :as csv :refer [parse-csv]]) which will refer in just parse-csv but will also make everything else available via the alias csv.

seancorfield21:07:29

The namespace in a project very often does not match the library name. The README for the library has a Use section that says "The clojure-csv.core namespace exposes two functions to the user:" so that's where you'd figure out what namespace to require @gabor.jasko

gabor21:07:29

> The namespace in a project very often does not > match the library name These simple things! thank you @U04V70XH6

Jakub Šťastný21:07:12

Probably really silly question, but (let [filtered-names (filter (fn [name] (match-name name search-key-chars)) names)] (println filtered-names)) I don't think this is how things are done? I mean assigning like that result of a whole function in the body of the let block? If not, than what? Always a separate (probably private?) function?

Jakub Šťastný21:07:05

By one doesn't assign variables using def within a function, correct?

seancorfield21:07:15

Correct. def is always global.

seancorfield21:07:28

let is how you declare local bindings.

seancorfield21:07:15

Whether you bother to bind a local symbol, depends on whether you're going to use it multiple times.

seancorfield21:07:56

If all you want to do is print the filtered list, I'd just do

(println (filter #(match-name % search-key-chars) names))

👍 2
Jakub Šťastný21:07:00

OK, fair enough, just wanted a sanity check.

seancorfield21:07:53

A lot of the time, it's purely about readability. Some people might prefer:

(let [match-name #(match-name % search-key-chars)]
  (println (filter match-name names)))
(I'm deliberately shadowing match-name there to get a similar name in a more limited context -- but I'd also say that match-name is perhaps not a very descriptive name in the first place)

seancorfield21:07:54

(depending on what your match-name function actually does, I might just inline it -- I don't find much value in wrapping core functions, if it's just doing a re-find for example)

Jakub Šťastný21:07:27

OK, that's what I wanted to know. Yeah it can get ((((((very))((nesty)))(((very))((fast)).

lsenjov00:07:17

If it's getting that way quickly, would recommend having a look at https://clojuredocs.org/clojure.core/-%3E https://clojuredocs.org/clojure.core/-%3E%3E

👍 3
lsenjov00:07:08

When things are very nesty, this (and the others in the -> family) tend to help

👍 3
seancorfield22:07:58

With rainbow parens in your editor, you eventually just stop seeing the parens 🙂

popeye11:07:04

Same experience when I started though! But loving it now!