Fork me on GitHub
#beginners
<
2021-05-26
>
seancorfield00:05:12

@francesco.losciale It’s so that you don’t leak connections if you consume the result lazily.

seancorfield00:05:18

That was the bug it was fixing. Previously, if you lazily consumed just the first portion of the result (from either find-maps or find-seq), then the connection was never explicitly freed up (because you didn’t reach the end of the sequence).

seancorfield00:05:39

(it’s why laziness + side-effects = trouble in most cases)

gibi07:05:39

interesting, thanks @seancorfield

Joni Hiltunen04:05:04

can someone help me understand what's wrong with my spec for ::main-script because it accepts invalid scripts... see gist for example, L33 shows the definition and L38-39 how I'm testing it https://gist.github.com/Sose/97dac8f1785d87d282d20e97fe1d41eb#file-spec-cljs-L32

seancorfield04:05:35

@djonih [start & rest] is going to bind start to [:start 1 2 3] and that's valid for ::start right?

Joni Hiltunen04:05:30

@seancorfield but shouldn't rest bind to ([:call :circle "invalid"]) and that shouldn't be a valid ::script and i'm using (s/and)?

phronmophobic04:05:58

what does s/conform report?

Joni Hiltunen04:05:52

kilppari.spec> test-sc
;; => [[:start 1 2 3] [:call :circle "invalid"]]
kilppari.spec> (s/conform ::script (rest test-sc))
;; => :cljs.spec.alpha/invalid
kilppari.spec> (s/conform ::main-script test-sc)
;; => [[:start 1 2 3] [:call :circle "invalid"]]

seancorfield05:05:16

So the first one is invalid and the second one is valid.

phronmophobic05:05:02

I forgot that s/conform isn’t super helpful in this scenario.

seancorfield05:05:21

s/conform is telling the truth though 🙂

phronmophobic05:05:14

Indeed! s/conform can sometimes be useful for telling you how a spec is conformed. But not in this case.

lassemaatta04:05:12

should that s/and in the predicate be just a plain and?

Joni Hiltunen04:05:51

umm right I think so... I guess I don't really understand the difference between those two

Joni Hiltunen04:05:24

changing it to a normal andseems to fix my problem.. Now I only have to figure out what s/and actually does 😄 thank you

lassemaatta04:05:35

I'm no expert on the matter, but my understanding is that s/and is one way to compose specs, where the given value must match all predicates. Typical example is something like (s/and int? #(< 10 %)) , which accepts integers over 10. It has a bunch of details related to how the values propagate from one predicate to the next, and how it works when generating values, which I won't go into. But I think the relevant part here is that s/and returns a new spec, not a truthy/falsy value like and.

Joni Hiltunen04:05:10

ahh.. I think I understand

lassemaatta04:05:49

but as Adrian suggested, you might want to checkout s/cat for speccing a sequence. One day you'll want to generate sample data from the specs and find out that spec can't generate values that match your custom predicate.

Joni Hiltunen05:05:17

thanks for the suggestion, I guess using s/cat that would be

(s/def ::main-script (s/cat :start ::start
                            :script (s/* ::instruction)))

seancorfield05:05:13

s/and “flows” the result of the first predicate into the second predicate. and just checks both predicates are true.

seancorfield05:05:58

Since you don’t have an actual predicate being applied, you don’t get what you expect (you have expressions that just return Boolean).

seancorfield05:05:35

(and, yes, s/cat is what you want to specify and sequence of Specs over a sequence of values)

lassemaatta05:05:53

and I imagine that because s/and returns a spec it's value is truthy -> (s/def ::foo (fn [v] (s/and .. .. ..))) is essentially (s/def ::foo (constantly true)) ?

Joni Hiltunen05:05:13

I got interested in trying to generate sample data for fun but it's failing on stuff like (s/def ::start (s/tuple #(= :start %) number? number? number?))because of the anonymous function... I tried to look but I don't see if there's actually a spec way for saying "something is equal to this"?

seancorfield05:05:01

You can use a set as an equality predicate: #{:start}

seancorfield05:05:05

That should generate.

seancorfield05:05:13

(! 532)-> clj -A:test
Downloading: org/clojure/test.check/maven-metadata.xml from central
Downloading: org/clojure/test.check/maven-metadata.xml from sonatype
Clojure 1.10.3
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::start (s/tuple #{:start} number? number? number?))
:user/start
user=> (s/exercise ::start)
([[:start 1.0 0 -1] [:start 1.0 0 -1]] [[:start 1.0 -1 0] [:start 1.0 -1 0]] [[:start 0 -1 0] [:start 0 -1 0]] [[:start -2.25 -2 -3] [:start -2.25 -2 -3]] [[:start -0.5 -1 -0.5] [:start -0.5 -1 -0.5]] [[:start 0.5625 14 -15] [:start 0.5625 14 -15]] [[:start 1 1.453125 2] [:start 1 1.453125 2]] [[:start -1 1 60] [:start -1 1 60]] [[:start 7 0.34375 2] [:start 7 0.34375 2]] [[:start -6 -1 2] [:start -6 -1 2]])

seancorfield05:05:28

(the :test alias brings in test.check)

Joni Hiltunen05:05:46

yep that works, fun.. doesn't work for ::instruction though, and not sure I understand the error

kilppari.spec> (gen/generate (s/gen ::instruction))

Execution error (Error) at (<cljs repl>:1).
Vector's key for assoc must be a number.
;; => :repl/exception!

seancorfield05:05:34

I suspect (since I’m not terribly familiar with multi-spec) that you need to define ::instruction as (s/and (s/cat keyword? ...) (s/multi-spec ...)) although I’d have to think about what that first ... was

seancorfield05:05:58

I don’t know that multi-spec is going to generate on its own.

Joni Hiltunen05:05:22

Ah. So I'd have to look into writing a generator for it I guess?

seancorfield05:05:59

Depends. If your Spec for what an instruction sequence could be was tighter, it might generate out of the box.

seancorfield05:05:47

Maybe (s/cat ::command ...) if you made ::command a literal set of possible keywords (since a set will always generate).

seancorfield05:05:32

Your ::command can only be a limited number of things (keywords).

seancorfield05:05:26

But the general idea is that you define an s/and Spec so that the first predicate will generate and the remaining predicates will filter (and hopefully succeed).

Joni Hiltunen05:05:34

Hm.. right, I'll see if I can make it work

seancorfield05:05:13

(but it’s worth pointing out that Spec isn’t designed to be a parser)

Joni Hiltunen07:05:54

I got it to generate something after bunch of messing around and trying to google things... The namespaces seem to matter too and I don't undestand them too well I guess... I'm not even sure what actually fixed it in the end 😄 <https://gist.github.com/Sose/32aa813eea3004ecf7c3176b8b7c5a68>

Joni Hiltunen08:05:56

(number? ##NaN) => true , this definitely surprised me 😄 "is 'not a number' a number?" "yes"

Joni Hiltunen10:05:34

I wonder if there's a better way of doing this :thinking_face:

(def mymap {:a 1 :b 2 :c 3 :d 4})
(def mykeys [:a :c])
(def myvals (map mymap mykeys)) ; => (1 3)

(def myanswer (map vector mykeys myvals)) ;=> ([:a 1] [:c 3])

noisesmith16:05:22

another option: (map find (repeat mymap) mykeys)

noisesmith16:05:02

that gives literally the same result you had (select keys-gives back a map with the same keys, which will have the right contents if you call seq on it, but maybe not in the expected order)

Joni Hiltunen11:05:28

I wonder if there's a good pattern or method for testing things that are very "iterative" (?) by nature... Like stepping an interpreter and checking some values about the environment after each step? This is pretty painful to write imo... I mean it's obviously possible to write something but I wonder if there's a great builtin solution for these kinds of cases https://gist.github.com/Sose/f4fb8ffe55e54d9856e1e96bcd3eb51b

lassemaatta12:05:05

I recently did something similar (in Java), where I needed to submit events to a state machine and check the state after each event: create a vector of pairs, where each pair consists of a) an input value (in your case scripts) and b) a predicate function, which checks the state machine. Then you can reduce over this vector by submitting a script to the turtle thing and verify its state using the predicate.

Joni Hiltunen12:05:14

Yeah I ended up doing something like that.. Though I guess it could be made a little more general still.. Current 4th version https://gist.github.com/Sose/f4fb8ffe55e54d9856e1e96bcd3eb51b#file-test-cljs-L53

Joni Hiltunen12:05:20

not sure that even works 😄

Joni Hiltunen12:05:44

ahh I need to (dorun ...) the result of the final map for the tests to actually run

lassemaatta12:05:27

perhaps something not quite unlike this

Joni Hiltunen12:05:40

Hmm, thank you

Joni Hiltunen13:05:14

I think I actually like this version the most because if you pass a predicate function, you don't seem to get the "expected" and "actual" reports when tests fail. It just says "(pred xxx)" failed. Unless I'm doing something wrong 😄 But maybe the most general version isn't the best in this case https://gist.github.com/Sose/5c4440f85b317e9f322c8b4fdca7d28d

Joni Hiltunen13:05:42

maybe it would be better to pass in kv pairs as maps and compare all of those so it could test multiple things at once

Joni Hiltunen13:05:43

easy enough with clojure.data/diff it seems

afry15:05:01

I'm super stuck on a dependency issue, I'm hoping I can describe it a bit and maybe get some suggestions for where to go looking: I've installed Datahike in my app via tools.deps (i.e., using a deps.edn file as opposed to Boot or Lein). The app works perfectly when I'm running it with the clj utility, so no problems there. However, after I bundle the app into an uberjar using Depstar, all of a sudden I'm getting errors that Datahike can't be found on the classpath. It's a fairly simple setup, I've got src and resources listed as my paths in deps.edn, so I'm scratching my head as to why it works with clj but not when running the jar with java. I'm also unsure whether this is specifically a Datahike problem, but it seems unlikely.

dpsutton15:05:35

can you share your deps edn file?

afry16:05:08

Sure thing!

afry16:05:22

I'm building the jar with this command: clojure -A:depstar -m hf.depstar.uberjar mailfile.jar And running it like this: java -cp mailfile.jar clojure.main -m backend.core

afry16:05:29

Welp ... turns out Datahike was the culprit after all. Version 0.3.6 gives me that classpath problem, but 0.3.2 doesn't. Thanks for taking a look anyway @U11BV7MTK, appreciate the offer 🙂

dpsutton16:05:11

i can make a simple uberjar with 0.3.6 and run it. i'm not able to reproduce your issue

afry16:05:07

Huh, interesting. Downgrading fixed it for me, but if it hasn't affected you then there's something else going on

afry16:05:13

Looks like you're using AOT and a few other things which aren't in my aliases, I'll give that a shot and see if it makes a difference

dpsutton16:05:23

i can remove them. i think the warning even says i didn't mark any namespaces so its ignoring it

afry16:05:17

Nah, it's fine. I actually just started getting that same classpath error I was seeing with Datahike with a different library, so that shows me that the issue isn't with my libs

afry16:05:56

Those alternative Depstar args are looking like they could put me on the right path

afry16:05:57

and you're using a much later version of Depstar too come to think of it, like 2 major versions later than what I was using, that could very well be it

afry16:05:55

🙌 bingo

afry16:05:34

really appreciate it, thanks @U11BV7MTK 🙂

Joni Hiltunen17:05:28

ehmm I'm stuck on how to "get rid of" one layer of structure 😄 flatten isn't really what I want

kilppari.turtle> (def v [1 [[:a 3] [:b 5]]])
;; => #'kilppari.turtle/v
kilppari.turtle> (rest v)
;; => ([[:a 3] [:b 5]])
kilppari.turtle> (drop 1 v)
;; => ([[:a 3] [:b 5]])

Joni Hiltunen17:05:27

hm I guess destructuring but are there other options?

phronmophobic17:05:38

what output are you looking for?

Joni Hiltunen17:05:21

without the outer list layer.. [[:a 3] [:b 5]]

phronmophobic17:05:49

I would probably do something like (-> v rest first)

dpsutton17:05:58

(second [1 [[:a 3] [:b 5]]])

dpsutton17:05:17

there are two elements in that vector, 1 and the second one is the one you want

👍 1
Joni Hiltunen17:05:28

ohh you're right 😄 thanks a lot

thumbnail18:05:22

I have a script which interactively walks through items received from a backend. Currently I (ab)use lazy-cat to achieve that. I shouldn't rely on this because it's side-effects in lazy sequences. which is not cool. What would be recommended in this case? I put a simplified repl-ready example in the 🧵

thumbnail18:05:35

(doall (get-items {})) would take roughly 1second, while (take 1 (get-items {})) will take 200ms. (the desired behaviour).

phronmophobic18:05:32

It really depends on how you want to handle errors.

phronmophobic18:05:11

If it's for a dev script that rarely fails and rerunning the script is acceptable, then the lazy version is ok.

phronmophobic18:05:00

If it's for a website where random, unhandled errors cost money, then you probably want a different strategy

thumbnail18:05:33

> If it's for a dev script that rarely fails and rerunning the script is acceptable, This is indeed the case. It's part of a script the team uses to query deployable artifacts from a registry. I was just wondering if there's something I should be careful about; not too familiar with lazy sequences.

phronmophobic18:05:09

Some common gotchas when mixing lazy sequences and I/O: • you can't control chunk sizes. Even doing (take n my-items) may realize more than n and there's not a good, reliable way to control how many items are realized with lazy sequences • Since you can't control chunk sizes, you also can't control when or which thread is used to realize items. • Exceptions may be thrown wherever items may be realized • Since you can't control when items are realized, if you try to stream items, then resources like network connections may close or time out depending on when items are consumed. It looks like your example is grabbing chunks all at once, so this is less of a problem • Since you can't control when items are realized, you can't rely on dynamic vars for configuration (not usually a problem, but it can come up).

1
phronmophobic18:05:29

Ymmv, but it seems like it might work reasonably well for a dev script. It seems like the type of thing where someone finds a TODO 6 years later that says "TODO: replace lazy-seq with a better alternative", but it's been working fine the whole time

😅 1
thumbnail19:05:47

Thanks for the thourough explanation! I'll make sure to include that Todo so the prophesy may come true ;)

😆 1
Joni Hiltunen20:05:19

Does anyone know if CIDER in Emacs can do something like "auto reload on save"? I got so used to that in ClojureScript that if I'm editing Clojure code, I sometimes forget to manually re-evaluate things and get confused why my changes are doing nothing

dpsutton20:05:43

emacs has a general after-save-hook. You could make a function that checks if the buffer 's mode is clojure-mode, and cider is connected then call cider-ns-reload

Rob Haisfield20:05:40

Has anyone here been using http://fig.io? If so, I would love some help building a Leiningen autocomplete spec

noisesmith20:05:19

of course you can always replace (fn [a b] (* a b)) with *

noisesmith20:05:33

for factorial, you might want *' instead

noisesmith20:05:58

(ins)user=> (reduce *' (range 1 101))
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000N
(cmd)user=> (reduce * (range 1 101))
Execution error (ArithmeticException) at user/eval314 (REPL:1).
integer overflow

noisesmith20:05:27

a nice thing about clojure is that most of the time, with good code most of the code disappears

Rob Haisfield20:05:33

Huh I’ve never seen the ' after a multiplier. Is that a general syntax thing? What’s going on there?

noisesmith18:06:43

@U02108ERRU5 it's an old lisp idiom to name something foo' if it's an extended or enhanced version of foo, and to name it foo* if it's a less featureful intermediate form of foo in clojure it seems to only get used for operations that upgrade to bignum:

)user=> (->> (all-ns) (mapcat (comp keys ns-publics)) (map name) (filter #(re-find #"'$" %)))
("+'" "dec'" "inc'" "-'" "*'")

noisesmith18:06:29

but I've definitely used x' in let blocks to mean "next value of x"