Fork me on GitHub
#beginners
<
2020-11-24
>
hiredman00:11:23

Patterns of what?

hiredman00:11:05

Spec has regex ops for recognizing patterns of data structures, the implementation of those most closely matches taking the derivative of the regular language, which is a nice simple method for implementing regular expressions bit doesn't get used much

hiredman00:11:45

I believe https://github.com/ztellman/automat uses the traditional NFA transformation to recognize sequences of values

👍 3
hiredman00:11:30

There is another library whose name escapes me that is more deliberately regexes on top of clojure sequences, I don't know what the internals of that are like

hiredman00:11:09

If you just need to implement regexes, the derivative route is way easier

noisesmith01:11:03

if you have a java lib you can just use it

zach01:11:52

Hi! I am working to understand clojure.spec on a project and hoping to find a bit of illumination on the exercise-fn and the test/check functions. Just as a toy example, I wrote a function that takes an email and returns a string.

(defn toy-fn [email] (str "the email is " email))

zach01:11:45

I then wrote a spec for email as

(s/def ::email (s/and string? #(re-matches email-regex %))

zach01:11:28

and I wrote a function spec as

(s/fdef toy-fn
:args (s/cat :email ::email)
:ret string?)

zach01:11:35

When I do either a (test/check toy-fn) or (s/exercise-fn toy-fn) I get an error of: Couldn’t satisfy such-that predicate after 100 tries

seancorfield01:11:52

You'll need a custom generator on your ::email Spec.

zach01:11:04

I read in this spec faq (https://blog.taylorwood.io/2018/10/15/clojure-spec-faq.html) that this is because it’s trying to match just for string

seancorfield01:11:30

The problem is that string? generates a very broad range of random strings and the vast majority will not satisfy your regex predicate.

zach01:11:15

right right! I was wondering what would be the idiomatic way of doing this, or a suggested way of doing this. Is it more common to write the generator as part o the spec, or as something I feed into a test function?

seancorfield01:11:55

Gary Fredericks has a library called test.chuck that includes a regex generator which you could use to generator strings that would conform to your Spec. Or you could just generate some "canned" emails for testing (e.g., using a set of email addresses).

zach01:11:00

I have a few specs like this, and some that are maps of these things (e.g. a map of a username and an email, with both fitting a regex)…and I was excited about writing less code with clojure.spec , but it seemed like i’d be writing custom generators for much of my specs…and wanted to check there wasn’t a better way.

seancorfield01:11:53

If you write the generator as part of the Spec, then aggregates of those Specs should generate automatically.

seancorfield01:11:50

Here's an example from our codebase at work:

(s/def ::email (s/with-gen (s/and (bounded-string 5 128)
                                  wstr/valid-email?)
                 (wgen/fn-string-from-regex wstr/email-regex)))

seancorfield01:11:41

(defn fn-string-from-regex
  "Return a function that produces a generator for the given
  regular expression string."
  [regex]
  (fn []
    (let [string-from-regex (requiring-resolve 'com.gfredericks.test.chuck.generators/string-from-regex)]
      (string-from-regex regex))))
That's just done so we only depend on test.chuck at runtime, during testing.

seancorfield01:11:17

and here's our email regex:

(def email-regex
  "Sophisticated regex for validating an email address."
  (re-pattern
   (str "(([^<>()\\[\\]\\\\.,;:\\s@\"]+(\\.[^<>()\\[\\]\\\\.,;:\\s@\"]+)*)|"
        "(\".+\"))@((\\[[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\])|"
        "(([a-zA-Z\\-0-9]+\\.)+[a-zA-Z]{2,}))")))

zach01:11:31

oh my gosh this is handy.

zach01:11:46

And because yr generator is defined as part of the spec, if you had (`test/check email)` and (`test/check coll-of-emails)`, you don’t have to feed sample emails into both tests.

seancorfield01:11:11

It generates gnarly stuff:

user=> (s/exercise ::email 10)
(["\"𭏩\"@[991.8.76.94]" "\"𭏩\"@[991.8.76.94]"]
      ["\"񜑣𶤍\"@-a.JX" "\"񜑣𶤍\"@-a.JX"]
      ["𛴹.񓬝󾆜򆘔.񎔞@[0.2.306.04]" "𛴹.񓬝󾆜򆘔.񎔞@[0.2.306.04]"]
      ["󝖣󄂪@[5.293.2.240]" "󝖣󄂪@[5.293.2.240]"]
      ["򪤷.𵲥藮@[96.9.946.5]" "򪤷.𵲥藮@[96.9.946.5]"]
      ["𰔀򟡙񛼏�.󶰿񛧂򥙭󅟜.򲰴􌭨𭄁􏔛󗤠񡄃@E.NN6.erjjWW" "𰔀򟡙񛼏�.󶰿񛧂򥙭󅟜.򲰴􌭨𭄁􏔛󗤠񡄃@E.NN6.erjjWW"]
      ["\"􆓑𬸸򯐜򿳪񜺈讴񃪕\"@J.w0P.21x.hRxb" "\"􆓑𬸸򯐜򿳪񜺈讴񃪕\"@J.w0P.21x.hRxb"]
      ["񯸯.𓱨񷤖񕂉𥺼򇔵򡜕򞝹𸏙.󂶹.󿎛򨐪愸񱞶.󺖆򬃞󼾽󍎲.򴹧볿𲄆.󋲽𵛍򩠐󌡇򣛳󥊰򢍕󰅚.񊍁𲒣񊽫򘱌𨇪@[2.393.765.1]" "񯸯.𓱨񷤖񕂉𥺼򇔵򡜕򞝹𸏙.󂶹.󿎛򨐪愸񱞶.󺖆򬃞󼾽󍎲.򴹧볿𲄆.󋲽𵛍򩠐󌡇򣛳󥊰򢍕󰅚.񊍁𲒣񊽫򘱌𨇪@[2.393.765.1]"]
      ["󲶍򂇇򤍩󐞨󂁚񵚌.ⵜ񫱽𬀄򘖲󺫟򆳫鶆񗈡玉.񣰮󆪕�􋆉򦻼@l0u.C6d4qPVy.R3s.F3P2kkwd.br62Nkb4.aE-JqLl.bO.IKffxYXLW.gqyIcL" "󲶍򂇇򤍩󐞨󂁚񵚌.ⵜ񫱽𬀄򘖲󺫟򆳫鶆񗈡玉.񣰮󆪕�􋆉򦻼@l0u.C6d4qPVy.R3s.F3P2kkwd.br62Nkb4.aE-JqLl.bO.IKffxYXLW.gqyIcL"]
      ["\"񅸦򪅢댴򨱹𒊱򰍉\"@lzFjoe.aWJ9k.-9CxZ1b.TdUWCK.Nn.urDeD.FDpBg" "\"񅸦򪅢댴򨱹𒊱򰍉\"@lzFjoe.aWJ9k.-9CxZ1b.TdUWCK.Nn.urDeD.FDpBg"

seancorfield01:11:58

Right, so

user=>  (s/def ::emails (s/coll-of ::email :kind vector?))
user=>  (s/exercise ::emails 3)
([["򸢧@[6.84.4.329]" "󐿎@[8.4.444.65]" "\"񳾈\"@[59.775.8.4]" "\"򒙊\"@5.Ub" "񂚝@L.Sq" "򒏙@[3.87.8.441]" "\"􆄅\"@u.fn" "󾟅@[618.644.46.084]" "\"񓥹\"@S.HM" "򎹉@[8.56.028.56]" "\"󖧩\"@[4.30.361.922]" "\"󕪤\"@w.WG" "\"𒨴\"@[307.508.705.640]" "\"󰡻\"@s.tq" "󙴏@M.pw" "�@I.tb" "\"񿴛\"@[04.9.3.2]" "\"񁽰\"@n.Eg" "\"􄚯\"@A.vm" "\"󆆻\"@[2.4.95.3]"]
       ["򸢧@[6.84.4.329]" "󐿎@[8.4.444.65]" "\"񳾈\"@[59.775.8.4]" "\"򒙊\"@5.Ub" "񂚝@L.Sq" "򒏙@[3.87.8.441]" "\"􆄅\"@u.fn" "󾟅@[618.644.46.084]" "\"񓥹\"@S.HM" "򎹉@[8.56.028.56]" "\"󖧩\"@[4.30.361.922]" "\"󕪤\"@w.WG" "\"𒨴\"@[307.508.705.640]" "\"󰡻\"@s.tq" "󙴏@M.pw" "�@I.tb" "\"񿴛\"@[04.9.3.2]" "\"񁽰\"@n.Eg" "\"􄚯\"@A.vm" "\"󆆻\"@[2.4.95.3]"]]
      [["󘖺.�񟑀@DC.fsX" "\"񐦲\"@[27.378.381.633]" "\"񪵐񺛕\"@Y.Sv.Hsf" "\"󱽠򚌳\"@[02.73.791.23]" "\"󤗭󱥀\"@[24.9.7.46]" "񅏪󆯊@[093.6.1.4]" "\"𹁮\"@[8.31.839.0]" "\"󺭵𵫎\"@E.dIk" "\"򚎾򆳈\"@[3.751.87.647]" "\"򝁾\"@5.wdM"]
       ["󘖺.�񟑀@DC.fsX" "\"񐦲\"@[27.378.381.633]" "\"񪵐񺛕\"@Y.Sv.Hsf" "\"󱽠򚌳\"@[02.73.791.23]" "\"󤗭󱥀\"@[24.9.7.46]" "񅏪󆯊@[093.6.1.4]" "\"𹁮\"@[8.31.839.0]" "\"󺭵𵫎\"@E.dIk" "\"򚎾򆳈\"@[3.751.87.647]" "\"򝁾\"@5.wdM"]]
      [["\"󜟞򅿅󓄹\"@o-.QQfc" "\"񟮻쇪򾆯\"@aUe.I.Db.FmjY" "񐭿.򻺩򰩠@[37.4.6.976]" "񸯛񷊾󌹃@uT.6aZ.fsX" "\"󣅼\"@YJ0.uh" "\"񼈙򬎈\"@z.j.DNle" "\"򓢅򷳮\"@J.RsE" "\"񴌜\"@ZUW.owBf" "\"񜞯񿴨򁯵\"@u.U.USjW" "򊂮.𐊹𠱣𡷗.񏂶@lrw.zqn.gN" "𫸽񽅆򱕞@[31.9.15.13]" "𱌼󋥌.򏙘�񌺊@[00.441.127.31]" "󪫮.񭞪򪟝@m.E5U.Ab.bfy"]
       ["\"󜟞򅿅󓄹\"@o-.QQfc" "\"񟮻쇪򾆯\"@aUe.I.Db.FmjY" "񐭿.򻺩򰩠@[37.4.6.976]" "񸯛񷊾󌹃@uT.6aZ.fsX"...

seancorfield01:11:42

Sometimes you may want to override a generator for a specific test case -- you might want readable test data, for example.

zach01:11:47

That makes sense. I read a mention of overriding in the blog I linked above, but I couldn’t find an example of this in the spec guide.

zach01:11:43

it said to feed a map to test/check…would it be something like (test/check email-fn {:gen [my list of emails]})

seancorfield01:11:46

:gen takes a hash map from symbols to generators so it's a bit more involved than that

seancorfield01:11:11

(assuming you mean clojure.spec.test.alpha/check there)

👍 3
seancorfield01:11:39

:gen map from spec names to generator overrides

zach01:11:53

gotcha. Thank you!

juniusfree09:11:14

👋 I'm learning and building in public. Anyone else here who's learning in public? https://twitter.com/juniusfree/status/1330368422727667713

👍 6
practicalli-johnny15:11:20

I've been learning Clojure in public for the last two years (and 3 months). It has been very rewarding. https://practicalli.github.io/

👍 9
juniusfree22:11:52

@U05254DQM 😲 I saw this before when I'm searching for repl. Do you have a twitter account?

dpsutton21:12:57

there's probably no need to keep sending these to the channel. if you have specific questions or are particularly proud you should share them but this is starting to become just a blog feed

dpsutton22:12:41

oh no worries. love the enthusiasm. but its just a string of tweets is all

👍 3
Emil12:11:52

I'm trying to figure out how one would go about executing cljs builds using clojure tools/deps.edn without shadow, lein etc. Something like this (I'm aware of the fact that you can run cljs.main with :main-opts set)

{:paths ["src"]
 
 :deps 
 {org.clojure/clojurescript {:mvn/version "1.10.758"}}


 :aliases
 {:build/release 
  {:exec-fn cljs.build.api/build
   :exec-args {:src "src" :opts {:optimizations :advanced} } }}}

noisesmith16:11:26

I'd start with figuring out the java command that invokes the cljs compiler in the way you want, then translate to deps.edn config. Also there are features that shadow provides beyond compiling your clojurescript, and it can be used with deps.edn.

Jonas Claesson13:11:36

I'm trying to write a generative test, but get an exception "Wrong number of args (10) passed to: jobtech-taxonomy-api.test.generative-test/send-concept-request" when running (spec/exercise-fn `send-concept-request) in the repl. I want to debug why this has happend, but I can't find a way to inspect the exception stack. I can see the stack with *e, but not the values passed between each frame. In C I would have written 'bt full' in gdb to get the full stack. But how to you do that in Clojure? I want to see the 10 arguments, and how they were generated and passed down.

noisesmith16:11:55

The jvm stack trace object doesn't carry that data, sadly. The jdb command has many of the features of gdb, but will also require being familiar with the constructs that clojure bytecode creates. Alternatively you can easily wrap send-concept-request so that it captures its args for debugging.

Jonas Claesson16:11:25

Thanks for the answer. Too bad this info is not available, I have needed it a few times. But I'll try the wrapping instead.

noisesmith16:11:20

also if you use the cursive plugin with intellij, you'll have more native java integration features (I find the IDE paradigm obnoxious so I don't use it, but ymmv)

noisesmith17:11:06

you might want to check out this API - I haven't tried it myself but it seems to provide what you are looking for https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jdi/com/sun/jdi/StackFrame.html

st3fan14:11:46

Hm fun one .. I start a message queue worker in a (def) and I think that is why lein uberjar is hanging

noisesmith16:11:33

right, any side effects in namespaces are invoked when you aot compile

noisesmith16:11:49

this is a very common trap for beginners to fall into

st3fan19:11:34

a beginner i am

Gabriel Augusto Reis da Silva14:11:35

I'm trying to make tests in my API for study, using nubank matcher combinators. But, in my "post" route test the output show this error:

java.lang.Exception: JSON error (unexpected character): N
Could anyone help me please? 😉

Darin Douglass15:11:48

first thought is that N is the beginning of Not found or something that's not json

3
Gabriel Augusto Reis da Silva16:11:41

Thanks, I thought it was just that

jstaab15:11:24

Check your payload, that's an error you'll get from json/read-str

3
Gabriel Augusto Reis da Silva16:11:15

Thanks, you helped me a lot

jstaab15:11:35

(require 'clojure.data.json)
nil
user=> (clojure.data.json/read-str "N")
Execution error at clojure.data.json/-read (json.clj:230).
JSON error (unexpected character): N

MatthewLisp16:11:54

Hello Clojurists 😄 What's the easiest way to achieve simple parallelism with clojure? By that i mean, running two functions together at the same time. Someone recommended me to use promise but i can't see how this is achieved through promises, because when i try to deref the first, it will block until it's computation is done, and only then proceed to the next one.

noisesmith16:11:35

promise doesn't do parallelism

MatthewLisp16:11:51

oh, right, reading the docs, makes sense

noisesmith16:11:28

perhaps they were thinking of js, which does have promises but no parallelism

MatthewLisp16:11:15

i have a very newbie question, because i don't have much knowledge about parallelism and so on. But for example:

(ns user)

(require '[throttler.core :refer [throttle-chan throttle-fn]])

(def +# (throttle-fn + 100 :second))

(let [p1 (promise)
      p2 (promise)
      p3 (promise)]
  (deliver p1 (time (dotimes [_ 300] (+# 1 1))))
  (deliver p2 (time (dotimes [_ 300] (+# 1 1))))
  (deliver p3 (time (dotimes [_ 300] (+# 1 1)))))

;; "Elapsed time: 3044.38462 msecs"
;; "Elapsed time: 3079.034443 msecs"
;; "Elapsed time: 3075.497436 msecs"

(let [p1 (future (time (dotimes [_ 300] (+# 1 1))))
      p2 (future (time (dotimes [_ 300] (+# 1 1))))
      p3 (future (time (dotimes [_ 300] (+# 1 1))))])

;; "Elapsed time: 9145.961132 msecs"
;; "Elapsed time: 9186.730517 msecs"
;; "Elapsed time: 9196.688568 msecs"
I'm using a library for throttling functions (changing throughput rate) just to analyse the time results. On the first approach, where i use promises (that DOESN't achieve parallelism), i can see that the total time of execution is roughly 9 seconds. On the second approach i use futures (that DOES achieve parallelism) and the total execution time is also roughly 9 seconds. What have i really achieved in terms of parallelism if the total time is the same as when doing it synchronously?

noisesmith17:11:13

the timing is happening in parallel though

noisesmith17:11:35

sorry, I misread for a moment

noisesmith17:11:43

this is a classic problem with parallelism

noisesmith17:11:57

so that throttling - is it being applied per invocation, or globally?

MatthewLisp17:11:36

i think it's per invocation, if per invocation means that it's applied when you call the function

noisesmith17:11:09

I mean does it limit "maximum throughput for this function, across all call sites" or "maximum throughput at this call"

noisesmith17:11:19

because those numbers make it look like the former

noisesmith17:11:50

@matthewlisp yeah, looking at the project, it looks like what this lib does is slow down your function globally, making parallelism pointless https://github.com/brunoV/throttler

noisesmith17:11:21

it creates a single "spigot" controlling throughput of your function across all calls

noisesmith17:11:28

try just using (defn slow-plus [& args] (Thread/sleep 1000) (apply + args))

noisesmith17:11:54

that sleeps 1000 ms on each invocation, but the slowdown is per call, not globally, so parallelism will actually do something

noisesmith17:11:37

@matthewlisp

(cmd)user=> (time (let [a (promise) b (promise) c (promise)] (doseq [p [a b c]] (deliver p (slow-plus 1 1)))))
"Elapsed time: 3000.869278 msecs"
nil
(cmd)user=> (time (let [a (future (slow-plus 1 1)) b (future (slow-plus 1 1)) c (future (slow-plus 1 1))] (doseq [f [a b c]] (deref f))))
"Elapsed time: 1000.468 msecs"
nil

noisesmith17:11:36

that throttler lib does expose an important point though: if your bottle neck is a global resource, rather than CPU, parallelism won't help

noisesmith17:11:49

also, this usage of promises is weird - they are quite useful if you need to coordinate between threads and don't need the full complexity of core.async, but here they are doing nothing

MatthewLisp17:11:18

ohhh right, i got it

MatthewLisp17:11:27

thanks for the help

clojuregeek17:11:02

I'm using deps.edn, and clojure reports 1.10.1 when i start the repl. Yet on my mac it accepts -m to run my tasks... and linux (also reporting same clj version) it has to use -a ... why does it report the same version but yet have these minor differences?

Alex Miller (Clojure team)17:11:01

1.10.1 is the Clojure (language) version. if you do clj -Sdescribe you can see the version of your clojure tools

Alex Miller (Clojure team)17:11:48

any version of the Clojure tools can use any version of Clojure (forward and backward) but they share a number because that's the default you'll get if you don't specify (and the version the tools themselves are using)

Alex Miller (Clojure team)17:11:19

I assume your linux version is older

clojuregeek17:11:37

ok thanks, i'll check the tool version

seancorfield18:11:42

@clojuregeek The change in behavior (around -M / -A -- uppercase) happened in 1.10.1.697 in the stable releases of the tools per: https://clojure.org/releases/tools (there were a whole bunch of prerelease versions between 1.10.1.561 and that).

seancorfield18:11:12

I ended up using linuxbrew for installing the Clojure CLI on Linux so that i can brew upgrade clojure/tools/clojure on both macOS and Linux to keep up with the latest versions.

seancorfield18:11:02

(well, if I was on stable, that's what I'd do -- I actually use brew install to install specific versions since I'm always running the prerelease builds)

clojuregeek18:11:33

Thanks, i'll look at Linxu brew. I was installing clojure via instructions on Eric Normands site https://purelyfunctional.tv/guide/how-to-install-clojure/#linux ... but Linux Brew might be better solution here 🙂

seancorfield18:11:22

I switch between macOS and Ubuntu (WSL2 on Windows) all the time so I just find it so much easier to be able to use brew everywhere and not have to think about which platform I'm one! 🙂

👍 6
Stuart23:11:37

HI, I'm trying to solve this problem on codewars in Clojure https://www.codewars.com/kata/52a78825cdfc2cfc87000005/train/clojure I have the tests almost passing but I'm failing on things like

Test Failed
expected: (= (sol/calc t2) (calc t2))
  actual: (not (= 1.1818181 1.1818181818181819))
When I parse out the numbers I'm using (Float/parseFloat). Does that test make it look like I should be using a different number type?

noisesmith23:11:38

1.0 is a double literal in clojure, not a float literal, I'd use Double/parseDouble

Stuart00:11:22

thanks, another question if I may. Why does (float) reduce precision?

(let [op1 (Float/parseFloat "4.42")
      op2 (Float/parseFloat "5.55")]
  (+ op1 op2))

=> 9.970000267028809

(let [op1 (Float/parseFloat "4.42")
      op2 (Float/parseFloat "5.55")]
  (float (+ op1 op2)))
=> 9.97
??

noisesmith00:11:48

because float is a concrete type with less precision, you might want the double function instead

noisesmith00:11:30

in the jvm, Float / float are IEEE floats with 32 bits, Double/ double are 64 bits

noisesmith00:11:55

see also "int", which is 32 bits, and "long", which is an integer value with 64 bits

Stuart00:11:27

ah ok, that makes sense. Thank you for help.

dgb2323:11:48

There are several specs and spec arguments (primitives ?) that take concrete values, such as s/int-in and :min-count etc. Is it weird/wrong to provide these values via a binding and create specs dynamically for validation or parsing (with conform). For example it seems useful to read them from a database and then register a spec based on that. A slightly more involved example would be to create a spec for a map or tuple where one value is dependant on the other.

seancorfield23:11:01

@denis.baudinot I wouldn't say it is weird or wrong, but it is hard with Spec 1. It's much easier with Spec 2 (but that's not ready for production use yet).

dgb2300:11:20

why would you say it is hard? wouldn’t it simply mean to lets say create a function that binds the values and registers a spec? I tried that in a REPL and it seemed to work. Am I missing something?

dgb2300:11:32

Also thank you for the response

seancorfield00:11:52

Spec 1 is nearly all macros and isn't designed for parameterized specs. You can't just write top-level Specs like that. You've found a workaround, but you've also now got to make sure that your function is called early enough in your app setup process to happen before anything that needs the spec -- and any Specs that depend on those specs also have to be registered dynamically, and that's going to start to affect your ability to do instrument in tests and so on...

👍 3
seancorfield00:11:20

Spec 2 draws a clearer line between spec forms and spec objects and provides facilities for constructing specs dynamically directly. You'll still have some of the ordering issues to deal with, but it's much easier to build and use dynamic specs.

👍 3
seancorfield00:11:30

We maintained a Spec 2 compatible branch of our codebase at work for several months, but it's very buggy and it's still changing a lot -- Rich is apparently going to redesign function specs fairly substantially at some point -- so we gave up after a while.

dgb2300:11:18

Thank you for the clarifications. I didn’t think about the incidental complexity my workaround implies!

seancorfield00:11:56

Mostly we've just avoided using Spec in those (few) cases where we would otherwise need to build them dynamically after app startup.

dgb2300:11:15

My instinct is that using something like integrant would maybe help to structure this. But I’m working on a small one man thing so I can afford it for now.