Fork me on GitHub
#beginners
<
2021-05-12
>
Franco Gasperino03:05:54

How do i model a spec def to validate the value of an optional key in a map? Assuming the :c keyword and associated value are intended to be optional:

(spec/def ::acceptable-value? 
    (spec/and int? pos-int?))
  
  (spec/def ::a? ::acceptable-value?)
  (spec/def ::b? ::acceptable-value?)
  (spec/def ::c? ::acceptable-value?)

  (spec/def ::a ::a?)
  (spec/def ::b ::b?)
  (spec/def ::c ::c?)

  (spec/def ::acceptable-map? 
    (spec/and 
     (spec/map-of keyword? ::acceptable-value?)
     (spec/keys :req-un [::a ::b ::c])))

  (def my-map {:a 1 :b 2})
  (spec/valid? ::acceptable-map? my-map) => false
  (spec/explain ::acceptable-map? my-map) => {:a 1, :b 2} - failed: (contains? % :c) spec: ...

dpsutton03:05:43

(require '[clojure.spec.alpha :as s])
(s/def ::c pos-int?)
(s/valid? (s/keys :opt-un [::c]) {}) ;; true
(s/valid? (s/keys :opt-un [::c]) {:c 3}) ;; true
(s/valid? (s/keys :opt-un [::c]) {:c "3"})  ;; false

dpsutton03:05:04

the secret sauce you were missing was :opt-un

dpsutton03:05:14

the docstring of s/keys didn't have quite the information i was expecting. > Creates and returns a map validating spec. :req and :opt are both > vectors of namespaced-qualified keywords. The validator will ensure > the :req keys are present. The :opt keys serve as documentation and > may be used by the generator.

Franco Gasperino03:05:44

that's the secret sauce. works as expected.

David Pham06:05:21

I am sorry if I should ask this on ask.Clojure or not, but what would be the potential disadvantage of using clojure.core.match, except for the potential slower speed?

dpsutton06:05:12

two things i know of. 1) i believe it uses exception throwing as part of its backtracking. i thought i remembered reading somewhere that creating lots of exceptions can have some cost to it. 2) it's susceptible to creating code that is too large for a single method on jvms. The bad part of this is that it rears its head when you've gone down that path and add a few more cases and can be tricky because there's not much you can do

Stuart11:05:39

I have a vector of tuples, like so:

[["foo" 1] ["bar" 1] ["quax" 1] ["king" 2] ["queen" 3] ["pawn..." 6]]
Ordererd by the second item. I want to pull out all the items with the smallest second item. So in this case
[["foo" 1] ["bar" 1] ["quax" 1]]
Right now I'm doing something like
(let [smallest (second (first results))]
  (filter #(= (second %) smallest) results))
Is their a nicer way?

Stuart11:05:53

Oh, this seems better

(first (partition-by second [["foo" 1] ["bar" 1] ["quax" 1] ["king" 2] ["queen" 3] ["pawn..." 6]]))
=> (["foo" 1] ["bar" 1] ["quax" 1])
The list is always going to small, only couple of dozen items.

Alex Miller (Clojure team)12:05:28

sort-by, then partition-by ?

👍 2
Ivan Koz14:05:06

In a recent article by Rich about history of clojure, there was two examples of lazy-seq provided, the last one line-seq doesn't wrap the first result "line" into LazySeq. Is that a design choice or just a matter of style? https://gyazo.com/09011955983e53357d340672b590aa01.png

Alex Miller (Clojure team)14:05:50

I don't understand the question

Alex Miller (Clojure team)14:05:20

(that is the actual implementation of line-seq btw)

Ivan Koz14:05:06

interleave-2 wraps first result in LazySeq, line-seq doesn't, the question is why

andy.fingerhut14:05:57

As in, could interleave-2 also work if its call to lazy-seq were just before the recursive call to itself?

andy.fingerhut14:05:21

that appears to be one way one could create a modified version of interleave-2 that would be implemented more like line-seq is.

andy.fingerhut14:05:54

My first impression is that for some reason that I don't yet understand, interleave-2 is "one step lazier" than line-seq.

Alex Miller (Clojure team)14:05:42

(note that the self call is not recursive btw - macro expansion is happening here in a loop, no stack frames are harmed)

andy.fingerhut14:05:51

Not sure what you want to use to name those calls -- what you would normally name a recursive call -- a call to the function currently being defined?

Ivan Koz14:05:41

yes why not do (cons (first s1) (cons (first s2) (lazy-seq (interleave-2 (rest....))))

Alex Miller (Clojure team)14:05:01

I think it's to make the result more lazy

Alex Miller (Clojure team)14:05:39

I think in the line-seq case, you have to readLine so you've effectively already forced the head, so might as well reify

andy.fingerhut14:05:30

@UH16CGZC2 I have not tested this, but you could probably get a function that behaves nearly identically to line-seq if you modified it by moving the lazy-seq call to wrap the entire body. That function would be "one step more lazy" than current line-seq, and more like interleave-2. If it works correctly, then it seems like a minor difference perhaps for no deep reason.

Ivan Koz14:05:34

"you have to readLine" am i missing something? One could delay reading the first line and wrap read in LazySeq as in the interleave-2 case?

Ivan Koz14:05:37

@U0CMVHBL2 yes, that was my question, kinda. Looks like there is no strict reason why line-seq couldn't be more lazy.

andy.fingerhut14:05:49

"no deep reason" approximately meaning "if Rich found a performance issue with his use of line-seq where it was doing the first readLine method call every time you called it, and he wanted to optimize for a case where he called line-seq on many files and never realized their first lines, he would probably change line-seq"

Alex Miller (Clojure team)14:05:51

I don't know, maybe it would work - the things I'd want to look at are termination, empty file case, and where you discover errors

andy.fingerhut14:05:21

"no strict reason" 🙂

Alex Miller (Clojure team)14:05:46

(like what's the behavior if the file isn't readable)

Alex Miller (Clojure team)14:05:38

it's probably preferable to have that exception throw on the call to line-seq rather than on the realization of the seq resulting from it

andy.fingerhut14:05:40

Note: My comments are ignoring backwards compatibility issues on cases like Alex is mentioning, which is also important now that line-seq has been implemented in the current way for N years.

Ivan Koz14:05:50

right, failing fast sounds reasonable

Ivan Koz14:05:49

thank you guys

andy.fingerhut14:05:24

no problem. "Why" is one of the most important and interesting questions to ask, but in many cases also the most difficult to answer accurately.

Ivan Koz14:05:34

bad, bad docs, undocumented semantics =)

andy.fingerhut14:05:00

Sometimes I think the view of the Clojure maintainers is "says no more than it needs to say, and every word is significant. Don't tie one's hands unnecessarily by the documentation in ways that prevents implementation changes in the future." I know that this is at odds with the kinds of documentation that many developers want from a library.

andy.fingerhut14:05:15

You can either accept that as a decision of the Clojure maintainers that seems unlikely to ever change, or you can go to alternate resources like http://ClojureDocs.org that say more, but are not "official" but community-contributed, or a book like https://www.manning.com/books/clojure-the-essential-reference that is very deep in details of the behavior of the implementation, but again written by someone who is not a Clojure maintainer, but has spent large chunks of time analyzing the Clojure implementation.

Ivan Koz14:05:28

right, most of the times reading clojure sources is enough for me, yet sometimes the thing is not obvious

didibus02:05:36

I see this a lot with lazy-seq implementation, some people will lazy-seq the rest, and some will lazy-seq the whole thing. I don't think there's always a reason why people chose one over the other, as more they just did.

didibus02:05:03

In practice it means you shouldn't treat things as fully lazy, but mostly lazy, there's a few functions here and there that end up not being fully lazy, like line-seq, so if you want to be fully lazy, you should design things that way yourself.

Ivan Koz02:05:29

most of the times documentation says "returns a lazy-sequence" however if first element was already realized the returned sequence is not that lazy

didibus02:05:39

Ya, that's what I mean, you should most of the time think of something that says lazy or returns a lazy-seq as "mostly lazy" and not "strictly lazy". Because there's a lot of edge cases where you'll have more elements evaluated then you'd expect. So none of the lazy sequence machinery guarantees full 1 element at a time lazy was including the first element.

👍 3
didibus02:05:03

Yes, this is kind of disappointing when you were actually wanting to be fully 1 element at a time lazy, and hoping you could just use Clojure's lazy sequence functions to do so.

Ivan Koz02:05:39

Not so to say disappointing, just unexpected, not documented properly.

👍 3
didibus02:05:44

That's one of the reasons you'll hear advice that says, don't mix side-effects and sequence functions together. Cause if say you send an email on realizing an element, there's be a lot of occasion you might be surprised like oh.nonit sent way too many emails.

didibus03:05:40

Ya, I agree with you there. More consistent terminology in the documentation would be good.

didibus03:05:48

The whole lazy sequence in Clojure is best thought of as an optimization for performance and memory use. And not really as a mechanism to implement lazy evaluation for your own use cases.

Ivan Koz03:05:25

yep, for that we have promises, futures

didibus03:05:47

Ya, promise, delay, or using transducers. You can also do it with lazy-seq, but you need to be careful, cause some of the sequence functions will realize more than you might want. If you keep to your own functions and lazy-seq it can work as well.

didibus03:05:26

Now why is line-seq like this is hard to say and probably speculation. I wonder if the type hint makes it it needs to be this way :thinking_face: It's possible if wrapped around lazy-seq the hint is lost? Or you know could just have been an afterthought, like no one thinking it mattered if the first line was read immediately or not.

Ivan Koz03:05:49

or as we concluded to fail-fast if buffered reader\\file misbehaves

didibus03:05:32

Ya possible

didibus03:05:49

I've come to realize that most details in Clojure often have good reasons and were thought through with a trade off chosen, but that still, with all that thoughtful process, there are details that slip through and were "accidental" in a sense, or overlooked. And sometimes it's hard to say if you've got a case of something that's overlooked, or of something that was a conscious choice but wasn't documented.

didibus03:05:42

Whenever you learn about some details like this, please go to http://ClojureDocs.org and add a notes or example for it. That stuff can really help others, even future you.

Stuart21:05:10

I have this code:

(def rules {"mov" [has-two-arguments? first-argument-is-a-register?]
            ,,, other keys and rules elided.})

(defn validate-instruction [instr args]
  (keep (fn [f]
          (let [error (f args)]
            (when error
              (str "Invalid `" instr "` call, " error)))) (rules instr)))
All my rules take one argument, so this is working fine. However, I'd like to extend it so that I can have rules with 1 argument and rules with 2 arguments... The obvious solution (that I don't like), is just to make all the rules 2 arguments and discard one on most of them... The other option is to remove all this and just hard code a function for each key in rules. BUt I don't like that either. Can anyone think of a better way?

phronmophobic21:05:50

Have you considered using spec? This type of validation seems like something spec would both support and be a good fit for.

Stuart21:05:44

I know nothing about spec, I thought it was a thing used for validating inputs to functions etc. Can it be used as a general purpose thing for doing validation?

Stuart21:05:59

I mean, I can just valid any old string ?

phronmophobic21:05:11

yes and yes. Not only that, it can generate data matching your spec

phronmophobic21:05:34

is the instruction a string?

phronmophobic21:05:06

Generally, I would usually separate parsing into its own step where you parse a string into data.

phronmophobic21:05:19

and then the instructions would just be data

Stuart21:05:04

yes, so I have a little toy language I've written. With very basic assembly language style instructions. e.g. "mov :a 5" I want to parse this as things like has 2 instructions, first instruction is a register (i.e. it starts with a : ) But I could also have "mov %1 %2" if this mov was declared in a macro (I support macro expansion). So when validating "mov %1 %2", the rules are sorta different, since its in a macro I want to check fi the first argument starts with % as I know this will be expanded out to a register at macro-expansion stage. SO the function needs 2 inputs, the args and a bool for whether its in a macro or not.

Stuart21:05:13

hmmm, maybe the solution is just to write a proper EBNF or something

Stuart21:05:07

I'll look into spec though, I didn't realise you could use it like that

phronmophobic21:05:26

yea, although if you're ok with a simple grammar at first. parsing can be simple as line-seq and (clojure.string/split line #" ")

phronmophobic21:05:58

learning spec and something like instaparse may be a little too much yak shaving depending on your goals. Another option that might be interesting is just 1. parse using line-seq and clojure.string/split 2. create multimethods for validating and processing instructions that look like ["mov" arg1 arg2] that dispatch on first

phronmophobic21:05:25

You can also create a simple parser with java.util.Scanner .

Stuart21:05:05

Think I might try with spec, as the whole point of this project is just to help me learn clojure and I wanted to look into spec at some point

👍 2
clojure-spin 2
Stuart21:05:55

thanks for your advice

phronmophobic21:05:02

I haven't seen spec used for just validating strings. spec will be much more effective if it's validating a string that's already been parsed into data, but there might be a good way to use it on validating strings directly

Stuart21:05:05

I can have it parsed into [:mov :a 5] or [:add :a :b], [:call "foo"] etc at this point. That part already works great. WOuld spec be better used on these structures?

Stuart21:05:27

and i validate on these?

Mike Martin22:05:00

Currently when I find a clojure library I want to try out that doesn’t specify it’s full maven name and version I go to https://mvnrepository.com/ and look up the artifact to find that information so I can add it as a dependency in deps.edn. Is there a more idiomatic way to do this?

Noah Bogart23:05:43

Clojars? Most libraries link directly to their clojars page. You can also use the git repo in deps.edn

Mike Martin23:05:47

totally forgot that deps.edn has git support, that probably would suffice for my cases. Thank you!

👍 3
Eric Scott22:05:51

I'm using a piece of java-based proprietary software that allows you to extend certain of its classes. This basicially wraps a single function in a bunch of Java ceremony that allows it to fit into the rest of the system.

(ns my.ns.Add1
 (:require ...)
 (:import ...)
 (:gen-class
 :extends their.AbstractClass
 :yadda
 :yadda
 :yadda
)

(defn -doit [x] (pack (inc (unpack x))))
So Add1 works, but I'm hoping I can write a macro that works something like this:
(defmacro def-doer [func arg-description])

(def-doer Add1 inc [long long]) => class my.ns.Add1
(def-doer IsPrime prime? [boolean long]) => class my.ns.IsPrime
;; etc
So here's where my mental model breaks down. When I spell out the gen-class explictly, lein compile has a named class it can generate, but invoking the equivalent macro seems to introduce some added level of indirection, and no class gets defined. Could I trouble someone to clue me in here? Thanks,

didibus02:05:16

Did you try using proxy instead? It can extend a Java class and override its methods as well, and is more convenient when it is possible to use it.

didibus02:05:51

Otherwise you might need to show your macro implementation if you want help

didibus02:05:30

Or do you need help with writing such a macro?

Eric Scott03:05:49

I think I need a specifically named Class, unfortunately. Unfortunately the actual macro is wrapped up in proprietary code. Let me try to put together a toy example and I'll come back.

Eric Scott03:05:37

I don't think it's so much writing the macro as making the resulting class visible to aot at compile time.

didibus03:05:02

Well there's a :name property you can pass to gen-class to give it whatever name you want (it's the fully qualified package name + class name)

didibus03:05:20

If you run macroexpand-1 on your macro, do you get the correct gen-class call?

didibus03:05:59

Also, does your macro generate a (ns) or a (gen-class) ? I'd suggest the latter

didibus03:05:17

And make sure your macro calls are top-level, otherwise they won't run at compile time

Eric Scott04:05:18

Well I get something that compiles and looks OK to me. When I put the name of the class in the :aot clause of my project file, the compiler claims not to see it. Maybe I'm not specifying it quite right. I'm away from my desk now, but I'll concentrate on that when I revisit the problem.

Eric Scott04:05:40

The macro generates gen-class statements.

Eric Scott04:05:46

I'm defining the macro in one module and calling it in my 'core' module at top-level.

Eric Scott04:05:14

Thanks for your help!

didibus04:05:48

Oh, so don't put the class name in :aot

didibus04:05:17

Put the namespace name of the clojure file that contains the calls to your macro

Eric Scott04:05:12

Ah. So if the full path is my.ns.MyClass, the aot should be 'my.ns'?

didibus04:05:14

No, it should be like:

(ns foo.bar)
 
(gen-class
  :name my.ns.Class
  ...)
And in :aot in project.clj you want to put foo.bar

Eric Scott04:05:07

Ah! The ns containing the the macro call?

didibus04:05:09

Which I mean, yes if you class is called foo.bar.MyClass (assuming my example), then you just want to put foo.bar

didibus04:05:17

Yes the macro call

Eric Scott04:05:49

OK. Yeah. Sure, that makes perfect sense now.

didibus04:05:22

Basically you don't compile the gen-class, you compile a Clojure source file which will contain calls to (gen-class ...) inside it at the top level, and when compiling that Clojure source file, the compiler will also emit Class files for each of the gen-class directives it encounters.

Eric Scott04:05:36

Yes, yes, yes. That was the missing piece of my mental model. Thanks @U0K064KQV! I have to step away now, but I thank you for clearing that up for me!

didibus04:05:07

No problem, hopefully that'll solve your issue.

Rob Haisfield23:05:49

Any idea why I’m seeing this error? Everything else executes well

dpsutton23:05:11

I can recreate this error with mal-formed edn:

search=> (clojure.edn/read-string "{:a}")
Execution error at metabase.api.search/eval125364 (REPL:2235).
Map literal must contain an even number of forms

Rob Haisfield23:05:32

So the problem is with the EDN?

dpsutton23:05:38

if you read the stacktrace by evalling *e you would know for sure. But seems like a good guess. You're reading edn data structures and getting an error that there's a map literal with an odd number of forms. the form you are evaluating has only a single well-formed map in it

Rob Haisfield23:05:13

How would I read this to know for sure?

dpsutton23:05:35

the top of that stacktrace is the function which through the error

dpsutton23:05:29

Util isn't super helpful, but the next line is EdnReader MapReader, which is pretty descriptive. The function that reads edn has a function that reads maps and it threw an error that maps must contain an even number of forms

Rob Haisfield23:05:12

Is there a way I could find where the map has an uneven number of forms?

dpsutton23:05:17

and a few lines below that is read_eval_print, which is run in a loop and is your REPL

dpsutton23:05:23

it's not in your app

dpsutton23:05:54

its in the file "/Users'roberthaisfield/Dropbox/Roam-ups/..."

Rob Haisfield23:05:42

So how would I identify the error point in that file?

hiredman23:05:07

there aren't distinct syntax classes for keys and values, so the reader can't tell if one is missing, it can only tell that it got to the end of the map and read an uneven number of things

hiredman23:05:44

if you know, for example, that all the keys should be keywords and none of the values are, you can create an input stream from the file and read one character (skipping past the initial {) and then read forms from the input stream checking that the first one, and then everyone after that is a keyword

hiredman23:05:31

that of course assumes the outermost form is a map and it is that map that has the issue

hiredman23:05:52

my guess is you have a '...' somewhere in the map, because you printed at the repl with *print-level* and/or *print-length* bound

hiredman23:05:35

user=> (binding [clojure.core/*print-length* 3] (prn {:a 1 :b 2 :c 3 :d 4}))
{:a 1, :b 2, :c 3, ...}
nil
user=>
something like that

hiredman23:05:08

where likely you didn't even know that *print-length* was set, because your tooling sets it for you

hiredman23:05:24

and you just spit that printed invalid/truncated map out to a file

dpsutton23:05:42

yeah. you can visually inspect the file. you're in something that has structural movements i'd start trying to identify sub parts that were valid edn to narrow down the tree

Rob Haisfield23:05:43

Okay, I think I might know what the problem was. Opening up the EDN directly in a text editor a while ago reformatted it in some way (VS Code asked to fix something) This time I just redownloaded an EDN export and didn’t open it directly (except through code) and that worked

clojure-spin 3
Rob Haisfield23:05:05

Thank you for helping. This is helping me understand how to find the issue better

dpsutton23:05:15

weird. glad its sorted