This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-02-06
Channels
- # announcements (58)
- # babashka (43)
- # babashka-sci-dev (22)
- # beginners (8)
- # biff (8)
- # calva (62)
- # circleci (3)
- # clerk (6)
- # clj-kondo (27)
- # cljsrn (9)
- # clojure (61)
- # clojure-austin (4)
- # clojure-conj (3)
- # clojure-europe (11)
- # clojure-losangeles (2)
- # clojure-nl (2)
- # clojure-norway (4)
- # clojure-spain (5)
- # clojure-uk (2)
- # clojurescript (51)
- # data-science (1)
- # datascript (4)
- # emacs (33)
- # events (14)
- # funcool (14)
- # gratitude (13)
- # introduce-yourself (1)
- # jobs (9)
- # lsp (58)
- # malli (23)
- # missionary (31)
- # nextjournal (9)
- # off-topic (35)
- # proletarian (2)
- # re-frame (5)
- # remote-jobs (7)
- # shadow-cljs (2)
- # spacemacs (7)
- # sql (26)
- # testing (12)
- # vim (1)
- # web-security (3)
- # xtdb (2)
I am trying to transform schemas, but due to the need for documentation, I have to read the source code; one potential problem is there is no API namespace to mark which function/protocol is considered internal and subject to change.
Good point. All protocols are part of the extender API. See https://github.com/metosin/malli#alpha, could add that to docs. Not planning on breaking those, will bump a minor if those change. Add a proper description what changed
Playing a bit with persisting schemas in datascript, going by the AST I find its structure is not very regular. A value can be a schema reference or a concrete value. This makes writing a decent schema for when things are components, references and unique a headache. How would you approach this?
> This makes writing a decent schema (I assume you mean DataScript schema here) I don't have an answer for you...but I guess it may help to clarify: are you specifically hoping to use Datalog queries to perform analysis on the Malli schema(s)?
yes, specifically, I'm trying to "unify" different schema registries, assuming they specify an ad-hoc key as communication channel, to check for rules and gain insights
for example, you send a message over a queue and say it's a string which starts with "abc". I read that message and say it's a string. We can unify these constraints
but to do that with complex schemas I need a way to store them reasonably in a datalog database
Cool, that sounds pretty interesting! If you don't have a massive graph, I wonder whether you may have an easier time with using (e.g.) Meander directly on the Malli schemas ...although I've not actually used Meander before, I'm just speculating 🙂
First off amazing library, it's been a pleasure to use. My question is related to massive schemas. How do you store and use them? Currently I'm generating malli schemas off of a standard called FHIR which has massive amounts of data describing the datas schemas here's a sample of the size https://gitlab.com/genfhi/genfhi/-/blob/malli_changes/cljc/generator/resources/structure_definitions/structure_definitions/hl7/profiles-resources.json 30+mb json file that defines schema for various data types. I generated a massive clj file https://gitlab.com/genfhi/genfhi/-/blob/schema_alts/cljc/generator/src/gen_fhi/generator/r4/schemas.cljc but unsurprisingly it gets killed on AOT compilation with method size too large (note this works just fails during AOT compilation). Should I store these massive schemas on edn file and read-string them in (related to that I have another issue that comes up in that I have functions that would need to be evaled so I need to read the edn than eval it). Is this the right approach to be generating a giant map of schemas (all these data types ref one another).
I could split these into chunks of data that get merged at the end too. But just curious to hear if this is the right approach for schemas of this size.
thanks! I get "...structure_definitions/hl7/profiles-resources.json" did not exist on "master"
@U055NJ5CC links are updated here's the json file representing the source of schemas and my generated malli schema map. https://gitlab.com/genfhi/genfhi/-/blob/malli_changes/cljc/generator/resources/structure_definitions/structure_definitions/hl7/profiles-resources.json https://gitlab.com/genfhi/genfhi/-/blob/schema_alts/cljc/generator/src/gen_fhi/generator/r4/schemas.cljc I suspect I should be reading it from edn and doing the neccessary wiring with sci (I do use dynamic bindings which I'm assuming wouldn't be an issue). Validation was working for me when I run it on dev it's really just the AOT compilation that was causing me issues (I could maybe hack around this and do a trick to avoid AOT comp for this massive map).
My colleague found:
'[malli.experimental.time :as mtime]
'[malli.experimental.time.generator :as timegen]
In fact: https://github.com/metosin/malli#generators---malliexperimentaltimegenerator it’s right here
My colleague found:
'[malli.experimental.time :as mtime]
'[malli.experimental.time.generator :as timegen]