I like the idea of ancillary functions taking a large map of all the data, then only operating on the subsets the function needs. It's very "inversion of control" style.

I think you could improve your usage of the style by having your ancillary functions return the whole map, after performing some update-in calls to update the keys that pertain to that function.

Then where you are calling the ancillary functions you can use a threading macro:

(->> the-data
     do-thing-1
     do-thing-2
     do-thing-3)

Ah, I see it's basically what you're doing but instead of calling update-in and letting the other values come along for the ride by default, you are manually reconstituting the map including all the keys that don't pertain to the function.

Here are a couple examples; https://gist.github.com/bgrabow/0306387873ec31e053091eff73c50626

Yeah, that's a whole ton more readable. The way I was doing it I was losing some keys when I tried that. Do you not have to enumerate ll the keys, or just the ones you use?

All the data will be preserved if you bind the whole map in your arg list (the :as the-data part).

And then instead of returning a map from scratch, you return the result of calling (update the-data k f)

(let [m {:a 1 :b 2 :c 3}]
  (update m :b inc)) ; => {:a 1, :b 3, :c 3}

@U09NVRNG2 I think the second example in my gist was a little off. If you have a brand new value for a key, you should use assoc. If you want to apply a function on the old value to get the new value, you should use update. Hope that helps.

I have finally gotten the update assoc distinction, so at least I'm not learning that! 🙂

You might even say the business requirements aren't clear

mm, I'd say they're off by a small amount

I’ve actually never seen/used volatile! before. Looks like a new 1.10 thing? Time to learn something new 🙂

they've been there since 1.7 https://github.com/clojure/clojure/blob/master/changes.md#21-transducers

not sure if they're documented except for the (tautological?) docstrings

I found this, https://stackoverflow.com/questions/31288608/what-is-clojure-volatile#comment50623688_31288608 - looks like a good summary

I wonder is there a rule of thumb when volatiles are safe to use

and when atoms are required instead

My rule of thumb is to use volatile unless I have concurrency. There is also transient for IEditableCollection. https://clojure.org/reference/transients

basically keep them contained in a single fn right?

There are two aspects: - for correctness: ensure the volatiles/transients are not accessed by multiple threads. They don't ensure atomicity. - for code organization: If I use state I prefer to keep it as local as possible so in this case inside the function

good points both

It's pretty clear in the text to ignore the 0, and use the 1's as zero. Your lucky with using clojure since sometimes the index is higher then the number of children. With clojure they must are ignored, which not always is a good thing.

I'm talking about the "meta-data`, which (in my test data) includes no 0s

I agree that it's clear from the text (I was hasty once more)

As a beginner I’d like to ask why you prefer loop over reduce here?

essentially because the 2nd argument in the reduce-fn is ignored

I just saw you mentioned it in the main thread. Thanks tons!

=== Running clojure test aoc.y2018.d07.clashthebunny

Running tests in #{"src"}

Testing user

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.

=== Running cljs test aoc.y2018.d07.clashthebunny
#error {
 :cause Library name must be specified as a symbol in :require / :require-macros; offending spec: [] at line 1 /Users/ranmason/code/advent-of-cljc/cljs-test-runner-out/gen/cljs_test_runner/gen.cljs
 :data {:file #object[java.io.File 0x70c69586 /Users/ranmason/code/advent-of-cljc/cljs-test-runner-out/gen/cljs_test_runner/gen.cljs], :line 1, :column 1, :tag :cljs/analysis-error}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message Library name must be specified as a symbol in :require / :require-macros; offending spec: [] at line 1 /Users/ranmason/code/advent-of-cljc/cljs-test-runner-out/gen/cljs_test_runner/gen.cljs
   :data {:file #object[java.io.File 0x70c69586 /Users/ranmason/code/advent-of-cljc/cljs-test-runner-out/gen/cljs_test_runner/gen.cljs], :line 1, :column 1, :tag :cljs/analysis-error}
   :at [cljs.analyzer$error invokeStatic analyzer.cljc 718]}]

That file is generated with the empty brakets at the top:

cat /Users/ranmason/code/advent-of-cljc/cljs-test-runner-out/gen/cljs_test_runner/gen.cljs
(ns cljs-test-runner.gen
       (:require [doo.runner :refer-macros [doo-tests]] []))

I've done a git clean -fdx, so I am pretty sure I'm not being bitten by cache.

@U09NVRNG2 What is that empty vector doing at the end?

(Of your ns form.)

It's an autogenerated file from cljs_test_runner.

So I assume I made either made an egregious error in the clojure side of things or I have a bug in the versions of things I use...

That empty vector would normally have the test namespaces listed.

Mine looks like:

(ns cljs-test-runner.gen
       (:require [doo.runner :refer-macros [doo-tests]] [aoc.y2018.d08.mfikes]))

Perhaps you deleted the deftest?

No... that doesn't provoke that kind of failure.

Damn! I figured it out.

It's the whole uppercase wasn't allowed in the username and then I ran it again with lowercase.

Ahh. Damn!

Thanks for the help!

So script/test-one aoc.y2018.d07.ClashTheBunny works for both cljs and clj, but script/test-one aoc.y2018.d07.clashthebunny only works for clj.

Is this something that would be expected? Should cljs downcase the namespace?

Or should clj not be downcasing it?

No... this isn't really a case-specific problem. It happens if you ask it to run tests for which there is no namespace.

Or maybe you are onto something... hrm.

I don't know where the problem lies, either Michiel's script, or upstream in one of the test runners. Perhaps file a ticket against advent-of-cljc.

I had the same. The test input isn’t stacked deeply. Do you need another test input? 2 1 3 3 0 1 2 0 3 7 8 9 1 1 0 2 6 12 1 4 1 2 1 3 0 1 17 1 4 4 2

That one broke my first implementation as did the real data.

thanks, I’ll try

If you need the numbers for part 1 or 2, just tell me.

(They’re comfortable enough to do by hand)

can you tell me the number for the test input you just gave me?

Part 1: 80

Do you want Part 2 as well?

I get 80 as well

sweet

but the real file still doesn’t work? 😮

yeah

I’m sorry that the additional test input didn’t help 😞

no problem

Have you checked your iterations?

Hang on.

Have you considered that there is a newline at the end of the test file?

(That one kicked my bucket a few times already and I guess that’s not it. But as a tester I have to ask 😉 )

got it now

\o/

what was it?

Yeah, it's basically just turning the recursion stack into an explicit one. But it would certainly make the parts that consume the stream easier to handle, I suspect, by breaking apart the functions that handle different read states (reading header, reading body, reading metadata)

It's possible because I'm doing it, but it's much easier with mutable data, but you can have that as well in clojure.

I still enjoy the lisp-y way of pushing environments onto an explicit stack. That said, I'm probably making life hard for myself 🙂

I can see how to do that with part 1 but with part 2 I don’t think I stand a chance to to something without a second pass. Unless I don’t understand what you mean with “second pass” :thinking_face:

Accumulating your total with a single walk through the tree's data, beginning to end, no back-tracking

Urgh. Nope. I couldn’t do that. But I will now attempt to use tree-seq for walking the tree after I built it… I also want to change how I built the tree because it’s s l o w.

I’m almost certain that this will turn into manually managed frames.

Thanks. And OMG I am not last

Uhm… I just learned a new word. Reify. I am not sure yet if I understand it completely though I consulted a lexicon

In this case: There’s an implied data structure in the sequence of numbers for Day 8. Reifying it means to make it into clojure maps, lists, and vectors instead of requiring pre-understood knowledge.

Structuring it? Like building a tree?

Text is hard 😄

Yepp. Especially when you’re not a native speaker 😄

It is a tree, but it’s represented as a list of numbers. Better is to make the relationships explicit.

So instead of

[2 0 0 3 10 11 12]

{:metadata []
 :children [{:metadata [10 11 12]}]}

How do you walk the first? Well… You the developer must be told what each number means.

How do you walk the second? The relationships are explicit in the structure. It’s immediately obvious.

I’m far from it, too 😄

But I am learning tons by AoC. First I try to solve it and now I am trying to optimize and to try things out.

@U04V437EA Do you have an example?

pick any from the gist I’ve posted, i.e. day1 pt 2

the problem is there are plenty of built-in functions whose complexity is unknown

I have four Revisions in this gist and especially the difference between Rev 3 and Rev 4 are mind blowing https://gist.github.com/MeikeMertsch/f1f3a1bfc157c3ed3a6cfcb1dec0ff57

@U04V437EA I’m not sure what you mean by that statement. The complexity is pretty well stated for most operations on most structures.

But it is sometimes polymorphic. So you must be aware of the type of the underlying structure.

@U0A7TVBLN Is there a particular change you’re interested in hearing about. Instead of “this whole diff,” it’s easier to talk about, “This change from concat to conj” or whatever.

(map-invert
          (frequencies
            (flatten
              (vals
                ids-by-minutes))))

O(n) (3*n I think)

sry 2*n

but O(n) anyways

flatten is lazy O(n), frequencies is eager O(n), map-invert is eager O(n). The lazy will get rolled into the first eager O(n) so it doesn’t contribute.

Language is hard… what’s “O(n)“?

oh. Just check the diff from rev 3 to 4. it’s just two lines. And really just a conversion

But it’s obviously not polynomial or more. It’s going over some data a few times.

yeah, I meant the C in front of O(n) is difficult to estimate

@U0A7TVBLN The last revision changes behavior. It’s not an isolated optimization.

knowing something is O(n) helps, for sure

Right, gotcha. So is it the laziness that’s difficult?

I noticed. 39 seconds worth of a behavioral change

y

Yeah, that takes a while to get used too, I agree.

@U0A7TVBLN One of those should return an incorrect result, I would presume.

So it might be faster because you’ve dropped some data. idk the context.

no. Both rune fine. Before trying that out I changed all peeks, pops and conjs, so they would work the same for vecs and lists

that’s why the vector version is so slow already. With peeks, pops, and conjs I gain almost half a second speed (see version 2)

ah I’m sorry, I misread this

Obviously I access data WAY more often from the end than from the front. And lists aren’t built for that, right??

Okay, so you’re running into the same lazy evaluation issues @U04V437EA was talking about

calling vec goes over all the data immediately

so you’re potentially adding a lot of extra processing

depending on the length of the list

using vec made me 39seconds (!) faster (:shocked_face_with_exploding_head: )

vec vs list is like array with direct access vs linked list, get element from vec is O(1), from list it’s O(n)

So I think I’m misunderstanding the question 🙂 It looks like you took out calls to vec (I presume to make it faster)

But, in general, there’s a lot going on in the solution, so it’s difficult to characterize where it’s spending its time.

haha. Sorry for the confusion. My tests ran 500msec and I wanted them faster so I checked my collection operations and spotted that I did a whole lot of conversions to vectors. So I refactored them out. With the result that everything was s l o w e r

😄 😄 😄

so let’s talk about butlast

the docstring on it directly says it’s a linear scan

pop is constant time

I’m not certain that’s your issue, but that’s a common mistake

I am quite impressed with what you can read out of those docs. Most of the internals are just “blahb” to me

I can read the words but I don’t understand what they mean.

I'm not even sure if is correctly placed

I don’t think it’s doing anything, because parse-data never returns a function

I saw that. And I appreciated it while it lasted 😛

you saw what?

your original posting re: haskell

just before you deleted it

oh yeah 😉