https://github.com/wotbrew/idx

Obviously don't use this, still needs a lot of work before its done. I mean I need to do my full property test suite...

• Would people use it? 'is it worth the effort getting it lib quality' • I am worried about misuse of auto-idx • I am worried people might not understand what the hell this is even about.

As this is quite general purpose I'd hope it'd be useful quite broadly on most kinds of non-trivial application.

This sort of library I feel needs a massively high quality bar so I will probably spend a lot of time on it before I release it, hence my request for a cold shower.

Don't want to do a load of work that produces something nobody wants to use.

I am quite a Clojure noob, so I am not quite sure what I’d use that for. Maybe it would be possible to add some examples?

I don't know if I'd recommend this for beginners, but I think a section on what your options are and where this fits would be good.

(having only looked at the README) i would like clojurescript support - we manually construct secondary indexes on a few collections in our re-frame app-db map

Yes this would be very suitable for re-frame

as your collections are not opaque, they look and smell exactly like normal collections all the subscription invalidation would work on these whilst still providing some nifty query support

I would be looking to do a clojurescript implementation at some point, but my dream would be somebody would help me with that.

I'd probably polish the clj implementation first to test the waters and see if there is demand

I am fine with it not being suitable for beginners, but I’d least like to understand where I could (maybe) apply it. So one day I could go “ah, maybe lib x would be good here”

@UM8P1G72Q I agree, I'm gonna work on something in the readme 🙂

readme is normally more work than the actual lib tbh

@UM8P1G72Q see the Why section I just added? Does that clarify at all?

Yeah that’s quite useful 🙂 Two comments. “… introducing incidental complexity to their signatures, which I find very, very smelly.” • I only know inherent and accidental complexity, I assume you mean the later? • “which I find very, very smelly” To me that is not helpful. Why is it smelly? Is it because it creates unnecessary coupling or any other “metric” that is more objective?

Interesting lib… I think I see the usecase, as I’ve built secondary indexes too… particularly in reagent/re-frame app dbs. However I tend not to do so much of this now, as we use an indexed in memory graph database to do this sort of thing. i.e if you’re maintaining various secondary indexes you’re probably also approaching wanting a query language too. That said I can imagine there are still uses. Anyway in terms of what I think would benefit this: 1. better more explicit examples that show the data and results too. Eliding them means readers are just guessing at what things do. 2. Benchmarks 3. Comparison or suggestions for alternative approaches, e.g. in memory databases, perhaps also memoize or core.cache is better for some smaller use cases, also perhaps specter?

Also to be clear I think cljc support rather than just cljs support would be useful (i.e. working isomorphically in both environments)

yea, I would attempt a .cljc first. I believe though if your .jar contains both a idx.cljs and idx.clj file the runtime will pick the right one? I am worried the branches will noise up the code, I mean there is a lot of java-specific stuff in there.

yes you can organise it like that — or with reader conditionals.

Regarding the database point, I seem to recall cgrand making a good point on this… which is that using complex clojure datastrutures as databases is similar to using a hierarchical database from the 1960s. A model that was essentially destroyed by the relational model — because they’re painful to query, update, index etc. I find that argument quite compelling, though I guess I didn’t have idx at the time… Curious what your thoughts are about this and that spectrum of options though

I live and breath relational data, and long for the day that programming with logic, rules and relations is the norm (no pun intended). This lib is not really trying to compete with databases though. If you can pour all of your data into datascript and use that great.

A lot of use cases for idx will be internally in functions where you whack on an auto-idx do some work and then unwrap and return. Much like you would use a transient.

In the large use relational databases, in the small use databases when it makes sense. Use idx for ad-hoc collection indexing as you would group-by.

Love this library. Been watching it since you created it.

I haven't figured out yet if it would help me reorder elements in an array efficiently, but that would be great.

@U09LZR36F I'm not exactly sure how this would help? Do you have an example of the problem at hand?

@danstone: yeah that makes a lot of sense, I’ve just seen the changes you made to the why section in the README. I could certainly see myself using something like this.

@danstone if you have [1 2 3 4 5] and you want to place 5 between 2 and 3 that's annoyingly difficult to do efficiently.

The classic solution involves subvec and other magic :)

5 is not the best example there. Maybe placing 4 between 2 and 3. The instructions are arbitrary.

Can idx could help with that? I might well be missing something.

I mean you could use the sorted index to write a function that does that in a linear pass. If you want to do multiple reinsertions you might be able to use replace-by.

(let [coll (auto-idx [1 2 3 4 5])
      target 5
      left (ascending coll identity <= 2)
      right (remove #{target} (ascending coll identity >= 3))]
  (concat left [target] right))

idx really doesn't offer you anything you couldn't do yourself manually of course. (e.g I could create a sorted map there and do the same thing)

Can I just say how amazing I think it is that you are trying to be inclusive to people with photo-sensitivity. :fist: