When doing a large import, is it recommended to leave off the indexes and then add them later?

@casperc: recommended is maybe strongly stated, but it’s definitely something that makes sense to do if indexing is a bottleneck for the import. If you haven’t might be helpful to review the [import docs](http://docs.datomic.com/capacity.html#data-imports) for other ideas.

I did check them out, but I can’t e.g. do batching, since I am adding data to the transaction.

@bkamphaus: How can I tell if the indexing is the bottleneck? I only have the logs to go by for now

Is the AlarmBackPressure metric logged as well?

@casperc: yes, any AlarmBackPressure in the logs will indicate transactions are pushed back waiting on indexing.

I have a question about a way I’m trying to write a datalog query where I’m using a collection binding and negation. Long story short, I want the entire list of attributes that are components except for a select few. My query looks like this:

[:find [?name ...]
         :in $ ?e [?skip ...]
         :where
         [?e ?a ?v]
         [?a :db/ident ?name]
         [?a :db/isComponent true]
         (not [?a :db/ident ?skip])]

So if I pass in my entity-id and a list that contains only one attribute that I want left out, it works. Any other variation on the list and it doesn’t work… so an empty list (saying that I want no attributes ignored) or a list with 2 or more (saying I want all of these attributes ignored).

Does anyone know why that’s happening? I can’t explain the empty list scenario at all but the 2+ scenario seems to be doing an ‘and' instead of an ‘or'.

@biggert: trying to parse through the problem. My quick reading of this query is that with 2+ it will union several subsets of relations with one thing excluded, which would be the same as the query with no exclusions (without the not clause). I.e. one thing gets excluded at a time and the results are unioned.

Gotcha. From the queries we’ve played with and parsing the different results, I can see that being the case. I’m having difficulty coming up with a different approach in datalog to avoid that.

pull ‘[*] and dissoc resulting map maybe?

Yeah... We have another approach where instead of the ?skip we map over the query and build it dynamically adding nots for every individual attribute and it works but we'd like to do that work in datalog and avoid a dynamic query if possible.

[:find ?e
 :in $ ?e ?atts-to-skip
 :where
 [?e ?a ?v]
 [?a :db/isComponent true]
 [?a :db/ident ?i]
 (not [(?atts-to-skip ?i)])]

This also seems to work if you pass in the atts-to-skip as a set

Sorry about the typo in the find clause. Obviously not looking for ?E

yes, you can pass a collection and invoke Clojure collection manipulation in the body of Datalog (i.e. in this case testing set membership), Not usually a solution I tend to reach for, though it may be a fit for this case. Still thinking on it a bit.