Fork me on GitHub
#malli
<
2023-02-06
>
renewdoit10:02:03

I am trying to transform schemas, but due to the need for documentation, I have to read the source code; one potential problem is there is no API namespace to mark which function/protocol is considered internal and subject to change.

ikitommi12:02:47

Good point. All protocols are part of the extender API. See https://github.com/metosin/malli#alpha, could add that to docs. Not planning on breaking those, will bump a minor if those change. Add a proper description what changed

ikitommi12:02:38

the section also defines the public / extender API difference.

Ben Sless13:02:30

Playing a bit with persisting schemas in datascript, going by the AST I find its structure is not very regular. A value can be a schema reference or a concrete value. This makes writing a decent schema for when things are components, references and unique a headache. How would you approach this?

refset14:02:40

> This makes writing a decent schema (I assume you mean DataScript schema here) I don't have an answer for you...but I guess it may help to clarify: are you specifically hoping to use Datalog queries to perform analysis on the Malli schema(s)?

Ben Sless15:02:28

yes, specifically, I'm trying to "unify" different schema registries, assuming they specify an ad-hoc key as communication channel, to check for rules and gain insights

👍 2
Ben Sless15:02:14

for example, you send a message over a queue and say it's a string which starts with "abc". I read that message and say it's a string. We can unify these constraints

Ben Sless15:02:35

but to do that with complex schemas I need a way to store them reasonably in a datalog database

refset16:02:42

Cool, that sounds pretty interesting! If you don't have a massive graph, I wonder whether you may have an easier time with using (e.g.) Meander directly on the Malli schemas ...although I've not actually used Meander before, I'm just speculating 🙂

Ben Sless16:02:57

Recursive unification with meander is hard

til 2
Ben Sless16:02:11

Speaking from experience

😅 2
Ben Sless17:02:25

I do think this is a datalog shaped problem

sun-one18:02:44

First off amazing library, it's been a pleasure to use. My question is related to massive schemas. How do you store and use them? Currently I'm generating malli schemas off of a standard called FHIR which has massive amounts of data describing the datas schemas here's a sample of the size https://gitlab.com/genfhi/genfhi/-/blob/malli_changes/cljc/generator/resources/structure_definitions/structure_definitions/hl7/profiles-resources.json 30+mb json file that defines schema for various data types. I generated a massive clj file https://gitlab.com/genfhi/genfhi/-/blob/schema_alts/cljc/generator/src/gen_fhi/generator/r4/schemas.cljc but unsurprisingly it gets killed on AOT compilation with method size too large (note this works just fails during AOT compilation). Should I store these massive schemas on edn file and read-string them in (related to that I have another issue that comes up in that I have functions that would need to be evaled so I need to read the edn than eval it). Is this the right approach to be generating a giant map of schemas (all these data types ref one another).

sun-one18:02:12

I could split these into chunks of data that get merged at the end too. But just curious to hear if this is the right approach for schemas of this size.

ikitommi19:02:38

thanks! I get "...structure_definitions/hl7/profiles-resources.json" did not exist on "master"

ikitommi19:02:38

If you could share the files, happy to take a look what can be done

sun-one19:02:11

Oh sorry I moved the branch let me update those links

sun-one19:02:47

@U055NJ5CC links are updated here's the json file representing the source of schemas and my generated malli schema map. https://gitlab.com/genfhi/genfhi/-/blob/malli_changes/cljc/generator/resources/structure_definitions/structure_definitions/hl7/profiles-resources.json https://gitlab.com/genfhi/genfhi/-/blob/schema_alts/cljc/generator/src/gen_fhi/generator/r4/schemas.cljc I suspect I should be reading it from edn and doing the neccessary wiring with sci (I do use dynamic bindings which I'm assuming wouldn't be an issue). Validation was working for me when I run it on dev it's really just the AOT compilation that was causing me issues (I could maybe hack around this and do a trick to avoid AOT comp for this massive map).

escherize22:02:14

Do time schemas have generators? I couldn’t find anything on that

escherize23:02:05

My colleague found:

'[malli.experimental.time :as mtime]
 '[malli.experimental.time.generator :as timegen]

🙂 2
Ben Sless05:02:18

Is it missing from the documentation?

escherize16:02:39

Probably not

escherize23:02:05

My colleague found:

'[malli.experimental.time :as mtime]
 '[malli.experimental.time.generator :as timegen]

🙂 2