you can't pass a value. you have to use a literal vector ther

same here. think of macros as syntax. there's no values involved

user=> (def x ['a 1])
#'user/x
user=> (let x (+ a 1))
Syntax error macroexpanding clojure.core/let at (REPL:1:1).
x - failed: vector? at: [:bindings] spec: :clojure.core.specs.alpha/bindings

The syntax is that its a literal association of symbol to value

Sorry , what is the meaning of using literal vector?

@U11BV7MTK How can we sdo that ?>

the same as the let example above. You can't use a value input which is a def for a vector, you have to use the vector itself

(s/def ::test (s/keys :req [::id ::name ::address]))

hmmm.. but my requirement is req keys are cretaed dynamically and it will be stored in a collection! So any alternative ?

@U01J3DB39R6 What we do at work is treat the specs as the "system of record" and then use s/form to pull the specs apart and get the list of keys. Spec 2 will allow dynamic sets of keys because it supports programmatic specs. However, it's not being developed right now (other priorities and still some design issues need hammock time). So I get I'll ask, what problem are you trying to solve in the large? If you have specs, they are checked for all keys in a map, regardless of what's in :req...

But you can use regular Clojure to check all keys are present -- and know that all specs are checked for keys in a map.

@U04V70XH6, I am here trying to write a general spec, I am generating required keys dynamically! So for different input I get different type of req-keys , So I was planning to generate those keys and pass it to :req dynamically ,

Not possible with Spec 1.

Like I say, you can check "requiredness" with regular Clojure and you can check all qualified keys just by having the Specs for them.

If you had (s/def ::test (s/keys [])) and you validate {::id 42} against that, it will still check the value of ::id because you have a spec for it - even tho' it is not listed in ::test

(a lot of people don't realize that all keys get checked even when they're not listed in s/keys)

@U04V70XH6, That is strange right? If we pass some invalid keys and it is returning true values,

Not sure what you mean.

Spec is designed to validate by key.

user=> (s/def :acct/person (s/keys :req [:acct/first-name]
                            :opt [:acct/phone]))
Syntax error compiling at (REPL:1:1).
No such namespace: s
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def :acct/person (s/keys :req [:acct/first-name]
                            :opt [:acct/phone]))
:acct/person
user=> (s/explain :acct/person {:acct/first-name  "Bugs"
                                          :acct/address  "address"})
Success!
nil
user=> 

it suppose to say :acct/address is not present right ?

Huh? Why would it say that?

Maps are open by design.

You only said :acct/first-name was required.

user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::id int?)
:user/id
user=> (s/def ::name string?)
:user/name
user=> (s/def ::test (s/keys :req []))
:user/test
user=> (s/valid? ::test {::id "foo" ::name 42})
false
user=> (s/explain ::test {::id "foo" ::name 42})
"foo" - failed: int? in: [:user/id] at: [:user/id] spec: :user/id
42 - failed: string? in: [:user/name] at: [:user/name] spec: :user/name
nil
user=>

☝️:skin-tone-2: It will validate all keys that have specs.

Requiredness is a separate concern. Spec 2 addresses this with schema and select.

I agree to your point, But if I pass just a invalid keys in a spec , I thought I would expect that , there is no such key defined in a spec

If there's a spec defined, the key will be checked. If there's no spec defined, it's "ok" by definition.

That's pretty much what "open by design" means. Not sure why you'd expect different?

Ahh I see, that's how the design is ! Thanks for your time and detailed explanation 🙂

Thanks to @U11BV7MTK also for your inputs

And for requiredness: (every? #(contains? data %) [::id ::name ::address]) will check that all expected keys are present -- so it can be dynamic.

(and, like I say, Spec 2 addresses this shortcoming of Spec 1)

is spec2 released ?

No, it's experimental (and, as I noted above, no longer being worked on due to other priorities).

There's a repo for it, but no releases.

At some point, Rich will figure out the remaining parts of the design, and it'll get "finished" and integrated into the core of Clojure.

🙌

For a while we had a branch of our code at work that was tracking the Spec 2 repo -- and it has some really neat stuff in it and some really interesting ideas -- but it never got to "production" quality and there are some big unanswered questions still to be resolved.

thank you @U04V70XH6

make the functions that are different params to parse-param function

What @U0LAJQLQ1 said ☝️

(defn parse-params
  ([params]
   (parse-params params util/bytes->str)) ; <-- default
  ([{:keys [response relay-state] :as params} decode-fn]
   (cond-> params
     response (update :response (comp decode-fn encoding/unb64))
     relay-state (update :relay-state (comp util/bytes->data encoding/unb64)))))

sorry, i was really tired when i wrote that. depending on if you want to have the outside world change the behaviour of your function, you have some options, but they require more code than just having a function as a param. multimethods and protocols can give you polymophism too. if you get to the point where the function as a param becomes a problem you can look into those ways of solving your problem.

1. I think #C053AK3F9 is a great fit for this question. 2. I think instaparse should work. Not sure about the ambiguity, but if we're lucky, you can put the aa rule before the a rule and hope that leads to aa matching working as expected. 3. I don't know of any other alternatives. :)

@UC1DTFY1G: have you explored https://web.stanford.edu/~laurik/fsmbook/home.html, or more generally the use of https://www.oreilly.com/content/how-to-get-superior-text-processing-in-python-with-pynini/ for defining complex string-to-string relations? There’s a lot of interesting prior art there for the kind of linguistic string transformation you’d be doing here.

@U02STT6CVK5 No, this is the first time I'm hearing about this. I will check it out. Thanks very much

I hope you find it useful! It’s very cool stuff, and I wish it were more widely known (and that there were good libraries for working with it in more languages). By the way, if you end up continuing to do language-processing work on Sinhala using functional programming, you might also be interested in Grammatical Framework: https://www.grammaticalframework.org/

i've done some instaparse work. ambiguous grammars is a pretty big problem, and a real langauge (not a typical constructed language that gammars are mainly written for) will produce a lot of problems generating consistent parsing. if you just want a mapping, even a bit of a complicated mapping that involves a processing step to make the map, then instaparse is probably going to hurt much more than it helps. if you could show some examples of the non-trivial translations that would help a lot.

a map in clojure can represent very complicated translations compared to other languages. you can have arrays as keys, you can access nested maps using arrays.

{"a"{nil "tr1" "comb1" tr2}}

i just looked at the examples again, it wasn't obvious to me that "aa" is a combination, same with "kaa".

dhyaa -> ධ්‍යා
madhyama -> මධ්‍යම

(defn parses 
  "Use parser to parse the text.  Returns lazy seq of all parse trees
   that completely parse the text.  If no parse tree is possible, returns
   () with a Failure object attached as metadata.

i have found https://github.com/clojure/core.match to be very helpful in taking the nodes of a parse tree and translating them to something else. if the tree is very nested then clojure.walk works well with match to do translations, though debugging that stuff can sometimes be very tricky.

@U0LAJQLQ1 Thanks very much for the suggestions. I had written a trivial grammar for a subset of the language yesterday and I did run into the problem of incorrect parse trees. I used the parses function and I see that the correct parse is also in there though not as the first element. I will grapple with it a bit more and come back for any help.

you have to change the grammar to fix that type of bug. if (parses produces more than 1 tree than you will need to make a more complicated translation function (maybe that is the easier path to go down...?). i spent a lot of time fixing my grammar to not produce ambiguous parsing. it's difficult because very small changes in the grammar can have very large effects.

one issue i can see is in aa and kaa and dhyaa you will have to work a bit to make a grammar that respects the longer parses over the shorter ones. dhyaa is likely to be parsed as [:node "dhy"][:node "aa"]

@U0LAJQLQ1 On the other hand, multiple parses might be useful in a real transliteration context. They can provide the user with multiple matches for their input (like a drop down - This is what Google input tools does). We can apply some rules over the produced parse trees (For example in Sinhalese vowels can appear only at the beginning of a word) to produce multiple valid outputs? Just thinking out loud.

@U0LAJQLQ1 Yes. I’ve tried to put the most specific rules (dhyaa) at the bottom of the grammar and build up from there.

you likely don't want anything like your parse output to hit the user, and you probably want to have a good idea of what your parse output will be (reduce ambiguous parsing). you are going to translate your parse tree to something else that maybe your user could see. for my parsing i had a few translations before my final output, but i was doing programming language translations (doing paradigm translations too).

instaparse lets you order your parsing using a special character (i felt like it was like cheating a bit, but i had to use it for some of my issues)

https://github.com/Engelberg/instaparse#peg-extensions

i'm not sure order of your grammar matters at all. PEG ordered choice does that job

Great. I will check this out. I’m pretty sure there’s a long way to go to get stuff working properly.

Actually, I could get a decent parser working. A simple grammar:

S = word (whitespace word)*
word = (any | vowel | pureconsonant | consonant)+
vowel = e | E | a | U | O | i | u | A | I | o | aa | ii | ei | AA | ou | au | sru | srU
pureconsonant = T | d | n | K | w | s | f | L | p | j | G | J | v | B | P | t | k | b | r | y | g | l | N | h | m | D | dh | ch | Dh | xd | xj | Sh | xb | sh | xg | Ch | Th | th | thh | xmb | chh | xdh | xgdh | xkdh
consonant = pureconsonant vowel
any = #".*"
whitespace = #"(\s|\n|\t)+"

looking good

Can you share the contents of core.clj? In general the way to troubleshoot this is to comment out everything in the namespace, confirm it works and then start adding it back one by one (first the requires/imports, then the code) until you find out what's causing it.

some tool, some nrepl middleware, you are using is throwing an exception as you code to evaluate is passing through all the layers of middleware before being evaluated

the root exception is: "Caused by: java.util.zip.ZipException: zip file is empty" from cider.nrepl.middleware.track_state$fn__3437.invokeStatic(track_state.clj:155)

not sure if whatever jar file that is should be not empty or that middleware should be better able to handle that, but that seems like the thing to look at

@U0NCTKEV8 hmmmm, would that something be in the ~/.m2/repository/ directory or in the <project>/.cpcache?

wouldn't be in .cpcache

might be in m2 repo, but I don't know what that does

I would move to #C0617A8PQ, bring it up there, they might want to open a bug for it something. basically the classpath lein built for the java process it launched, that is running your clojure repl, includes an empty file

same exception as

user=> (java.util.jar.JarFile. "/dev/zero")
Execution error (ZipException) at java.util.zip.ZipFile$Source/zerror (ZipFile.java:1598).
zip file is empty
user=>

and cider includes some nrepl middleware that attempts to open all the jar files and look for clojure source in there

it is possible for maven downloads to fail, leaving bad stuff (but could also be an actually empty jar for some reason)

i will try #C0617A8PQ , interestingly, a new directory containing an empty core.clj & a stripped down deps.edn works just fine.

well, lein and deps.edn are different, so that is different

depending on what kind of repl you are creating with deps.edn, nrepl wouldn't be involved at all, so a problem with nrepl middleware just won't exist

failing project is deps.edn as well

oh, sorry, I just saw all the nrepl preamble print out and assumed lein

no worries

but a stripped down deps.edn makes sense to, because it is likely one of the deps in there that is the empty jar file

find ~/.m2/ -type f -empty might tell you which jar is empty, and if it isn't empty on purpose, if you delete it it might download again correctly

good find!

(base) ➜  repository find . -type f -empty
./clj-commons-exec/clj-commons-exec/1.0.7/clj-commons-exec-1.0.7.jar
(base) ➜  repository ls -l ./clj-commons-exec/clj-commons-exec/1.0.7/clj-commons-exec-1.0.7.jar
-rw-r--r-- 1 ebodine staff 0 Oct  6 10:53 ./clj-commons-exec/clj-commons-exec/1.0.7/clj-commons-exec-1.0.7.jar

@U0NCTKEV8 that is an awesome thought to find empty jars. I’ve had that a few times

yeah, the download is bad. keeps pulling down an empty jar file

that is definitely the problem - thanks again @U0NCTKEV8

and @U064X3EF3 for bad maven download idea

i think its a bad artifact in clojars

yeah, I was wondering that too :)