Fork me on GitHub
#meander
<
2020-05-12
>
jeroenvandijk10:05:14

As follow up of yesterday, I’ve been able to create a pretty good mapping with your hints (`m/some` and m/app). Now I’m running into the point that it is almost perfect, but not completely and I cannot easily find out. Mismatches are silently ignore. Not sure how to fix this. Here is my current mapping https://gist.github.com/jeroenvandijk/62e3a52550f3aef5b69e4f0e91b73d18#file-meander_import-clj-L19-L82 The original data is also linked in the gist if you are interested

jeroenvandijk10:05:28

Maybe I need to apply more strict mapping with m/some for all the fields that I know will be there for certain?

Jimmy Miller15:05:43

m*/attempt is what is silently ignoring things. That is the point of attempt.

Jimmy Miller15:05:31

I know you are doing that be because you are doing bottom-up which will match on things smaller than just maps.

Jimmy Miller15:05:35

You could add a clause that was something like to the very end of your match:

{& ?data} [:this-fell-through ?data]
To find out what maps are being ignored.

jeroenvandijk16:05:49

I’m not really consciously using attempt 🙈, but now you are mentioning it. It makes sense that it is silently ignoring things. Without attempt it cannot work i’m affraid, at least nog in the recursive manner?

jeroenvandijk16:05:19

Your trick doesn’t catch the missing items. I’m guessing the data is being eaten at other levels. Ok I’ll need to study this better I’m affraid

jeroenvandijk17:05:49

But the trick does get me further when I disable other rules

jeroenvandijk17:05:59

Thanks for all the help!

Jimmy Miller18:05:09

Glad you got things working!

noprompt22:05:32

I rarely use the strategy namespace for stuff like this; I’d recommend using rewrite and m/cata if possible because it allows for much more control and will probably result in a snappier transform. I took the gist you shared — which was awesome, thanks — and https://gist.github.com/jeroenvandijk/62e3a52550f3aef5b69e4f0e91b73d18#gistcomment-3301768 for you which seems to work out pretty well. Let me know if it does the trick. I noticed that you were using m/some on keys, use m/some on the values. Also, for the "children" key, use m/seqable and m/cata to recursively transform the values, this handles nil automatically and avoids the need for separate rules. It will also make your output consistent i.e. you will always have :node/children that at least will be an empty vector.

noprompt22:05:09

If you want to have more control over how children are rewritten, I can show you another technique.

jeroenvandijk07:05:14

Great thank you! I managed to get it working (probably with the worst hacks, given your feedback), but I’ll incorporate your changes

jeroenvandijk07:05:57

The next step i’m considering is flattening the datastructure. Eventually this data needs to be transacted in Datascript/Datomic db. I’m not sure if this is feasible, but given your Hiccup example I feel this might be possible. I’ll play around with this idea first

noprompt15:05:19

It’s easy to do something like that with search 👍

noprompt22:05:38

@U0FT7SRLP how’d things end up?