cpmcdaniel: n00b here, but could you use the dosync macro on multiple cell swaps?

does dosync work on cells?

Yeah, I just read that on the Javelin site and saw it in a talk video: https://github.com/hoplon/javelin#transactions

up to this point, I have been treating my world state as one big cell, with the view using cell= formulas for each “block” on the 2d svg screen

I then have a main loop that responds to keypress. Then a chain of systems update the world state.

So (swap! world #(process-command (input/do-input % (.-key event))) does all the work

and process command is doing things like moving the player, etc.

I see, perhaps another way to get at two parts of the world state (in a single cell) is via Lenses?

so I don’t think I need dosync. My main concern is that each subsystem (movement, combat, magic, etc) will have to iterate over lots of entities

so I was pondering making the world state mutable JS arrays & objects, and having a system that syncs that to the view state in one swap! call

this is fortunately turn-based, so rendering can happen last

looking at lenses

JS array iteration is supposed to be fast, iirc

oh, can you pass JS arrays straight to (map) and others that take seqs?

I’m not totally sure if this helps you, but this is what I was referring to (I think, it’s been a while): http://swannodette.github.io/2013/06/10/porting-notchs-minecraft-demo-to-clojurescript

Sorry, I’m not totally clear on what you really need, and don’t want to send you on a wild goose chase.

When you mentioned “iterating” through a bunch of things, I remembered this tidbit.

@U0564EGNY is the problem just that you want control over when/how calculations are run?

I have control, its just that there are a lot of entities to iterate through, and some systems will need to iterate through them multiple times

per turn

So here is a bit of what my world structure looks like, if you can imagine…

An entity is just an ID, so lets say the player is entity 0

world has an :etoc key that is a vector of maps. Each map is a mapping of component name to component data for the entity at that index. Example:

(get-in world [:etoc 0 :position]) => {:x 1 :y 2}

things that move have a :velocity component

so to do movement, you have to find all things with :velocity component and in order to move them, you then have to iterate over everything with a :position component to avoid collisions

else you would walk right through a wall

now, if you have N monsters and a map that is 198 x 66, you have a lot to iterate through

now, I also store :ctoe in world, which is a mapping of component name to a set of entity ids

so that helps

but still, for collision detection, it’s a lot of unnecessary iteration

so I could store a :positions mapping in the world and keep it in sync. Otherwise I will be iterating over pretty much every entity in the dungeon when there are at most 8 I really need to look at

N x 198 x 66 scares me

but, it’s how a lot of games are designed

lots of looping

so the trade-off is more memory with a hash-map index

or drop to JS

what if you added the entity id, and velocity to the position map. (making it a “state” map) for an entity

Then you could filter a set or vector of entities by presence of attributes (such as velocity), and it would take a single pass

actually, the position map would actually be a 2D vector (rows and colums)

so you would index into by (x, y) coordinates.

and the values would be a set of entity IDs

because some entities stack

for example: floor <- scroll <- monster

all can occupy the same spot on the grid

actually, might use a vector of entity IDs to keep them in the right stacking order

om uses that denormalized data thing, with the parser

maybe use something like that with an index?

although, I would probably prefer putting entity objects in a set or vector and filtering them, or directly looking them up via a map

I wonder if I am going to get into other situations like this (wanting to optimize) when I get into other systems like ranged combat

there you will want to probably start with all entities that have the :health component

(get-in world [:ctoe :health]) => #{0 4 5 12}

but then you have to do line of sight checks

the position optimization is probably a good trade-off though

because rendering requires almost every entity

also, so many things besides rendering require knowledge of position - combat, range combat, line of sight, monster AI

I should also state that this is not graphically complex

@U0564EGNY is this logic something that could be handled with datascript queries?

not that it magically improves the speed of a loop, but it has some things like internal indexes that could help

sounds like a good idea to have a bit of a "cache" of entities that are actually close enough to matter

yeah, would make for less code, certainly

but I’d be worried about performance

the very first thing you could do would be run filter over your entities, with a distance calculation function

so it's one big, but relatively fast, iteration up front each turn, but everything else is quick

I guess there are lots of ways to slice this. I’ve been taking my inspiration from other rogue-likes and comonent-entity-systems in general from game design

the ones in C actually use a bitmask array of length = max_entities

the bitmask tells you what components the entity has

i don't know about performance

but datascript seems like a pretty good fit for keeping track of attributes of game entities

then for each component you have an array (also max_entities), and the data for each entity at the appropriate index

I might play with it a bit

it's a datomic clone in cljs

yeah, that much I get

if you use namespaced attributes like :position/x and :monster/hp i imagine it should be relatively easy to keep track of what's what

so those component arrays end up being sparse, but array iteration in C is so fast

hmm, I didn’t think to use the namespace of the keyword as the component indicator

well then it fits with spec

well, ant that is how datomic treats things: entity-attribute-value

i just checked, datascript implements 3 of the datomic indexes

EAVT, AEVT and AVET indexes

so you don't get VAET

but i think you want AVET anyway

datomic at least says "The AVET index also supports the indexRange API, which returns all attribute values in a particular range."

yeah, give me all things that have a position component, which I could use either :position/x or y for

you could just create ranges for x and y, basically a "bounding box"

I wonder if transact! works on javelin cells

it does

@U0564EGNY https://github.com/hoplon/hoplon/wiki/Hoplon-with-Datascript

just need to add a little bit of meta data to the cell

well, even more than just "give me everything with a position"

you could say "give me everything with a position within this area"

thanks, I’m going to play with this

… and just when I had basic movement working!

hah, i spend way more time reworking than working

well, half the purpose of this was to learn.

@U0D4G0Q4U ok, so how do you create a javelin formula cell that queries the db cell?

@U0564EGNY you don't, you query db values, not cells

here's a random example

(defn entity->list-title
 [e]
 (let [list-id (-> e :item/list-id :db/id)
       db (d/entity-db e)]
  (when list-id
   (first
    (d/q '[ :find [?t]
            :in $ ?id
            :where  [?e :item/list-id ?id]
                    [?e :item/list-title ?t]]
          db
          list-id)))))

ah soz, doesn't actually have a formula cell in that one 😛

(let [ids-titles (j/cell=
                   (into (sorted-set)
                    (d/q
                     '[:find ?lid ?t
                       :where [?e :item/list-id ?lid]
                              [?e :item/list-title ?t]]
                     conn)))]

there you go

(def test-var (cell= (into [] (d/datoms (d/db dw) :avet :position/y 3))))

dw is my conn cell

you don't need the d/db

formula cells deref other cells, and dw is a cell

(d/datoms dw :avet :position/y e) should be fine

#object[Error Error: Assert failed: (db/db? db)]

really?

I got a regular query to work

dw should be a db already because of the formula cell wrapping it

(def test-var (cell= (into [] (d/datoms dw :avet :position/y 3))))

works for me...

(h/if-tpl (j/cell= (d/datoms conn :eavt))
     (el.project-list.dom/project-list :conn conn)
     (h/div :class "welcome"
      (h/span :class "empty-text"
       "You don't have any projects yet! "
       (h/br)
       (h/a
        :click #(project.wire/+! conn)
        "Create a new project")
       " to get started."))))))

this is how i build my conn cells

(defn conn-cell
  "Builds a fresh conn cell wrapping an empty db"
  ([]
   (conn-cell {}))
  ([schema]
   {:pre  [(map? schema)]
    :post [(d/conn? %) (j/cell? %) (= {} (-> % meta :listeners deref))]}
   (conn-cell-from-db (d/empty-db schema))))

do you have a schema that supports the :avet indexing?

(def dw (conn-cell {:position/x {:db/index true}
                    :position/y {:db/index true}}))

also get the error with :eavt

ok cool, so you have db/index set

hmmm

what if you just prn dw?

what does (d/conn? dw) and (d/db? <@U04VCMKQN>) return?

(def test-var (j/cell= (d/q '[:find ?e :where [?e :position/x _]] dw)))
@test-var

works

both true

some sort of bug?

#object [javelin.core.Cell <#C07V8N22C|datascript>/DB {:schema #:position{:x #:db{:index true}, :y #:db{:index true}}, :datoms [[1 :appearance/symbol "@" 536870913] [1 :health/current 75 536870913] [1 :health/max 100 536870913] [1 :player/name "Frodo" 536870913] [1 :position/x 1 536870913] [1 :position/y 2 536870913] [1 :position/z 100 536870913] [1 :velocity/x 0 536870913] [1 :velocity/y 0 536870913] [2 :appearance/symbol "." 536870913] [2 :position/x 1 536870913] [2 :position/y 3 536870913] [2 :position/z 0 536870913]]}]

(into [] (d/datoms <@U04VCMKQN> :avet :position/x 1))

so something with the way d/datoms and cell= are interacting

hmmm

what versions of datascript, javelin and cljs are you using?

i don't understand how (d/db? <@U04VCMKQN>) could be true, but then (j/cell= (d/db? dw)) would be false

that might be out of my ability to debug remotely >.<

let me just try that much

ok, that works

😕

but that's all d/datoms is doing internally

https://github.com/tonsky/datascript/blob/130e06f87aff3269b25a43c458709903cc39858b/src/datascript/core.cljc#L82

ok, it’s working now

must have had things in a funky state

yay

also, if you just want positions, you can look at making a filtered db and running your queries against that

I think > 90% of entities will have position

So, the one main concern I have with using datascript for this is that, previously, my whole turn-loop was one big atomic cell swap operation. And that works really well because it is turn-based. With datascript, I don’t see how I could avoid doing lots of little transactions within a single turn, causing lots of cascading “change events” to dependent cells

I can’t simply queue up transactions for the end of turn. Consider movement. What if 2 entities want to move into the same space?

I suppose I could sort them by priority and use a db function to avoid collisions, but then we’re getting rather complicated

perhaps I could just use the datalog parts of datascript to query my own custom state cell (the one I have now)

…or… I could copy the db at the start of the turn and swap it after (sort of like react with the vdom)

A lot of what you’re describing sounds like “game logic.” In which case perhaps you don’t want it in Datascript queries? Since it sounds like you’re already building that into a library of Javelin cells. Just a thought.

yeah, Datascript is very similar to what I want in terms of how data is represented (entity/attr/val)

but there’s some other things that make it not quite a great fit

@U0564EGNY you can stop the propagation

propagation only happens when vals change

so put an "end of turn" cell in between your db and the downstream calculations

one sec, i'll put together an example

(defn turn-cell
 "Returns a lens that always updates to a new random-uuid when swapped/reset"
 []
 (let [t (j/cell (random-uuid))]
  (j/cell= t #(swap! t random-uuid))))

(defn end-of-turn-cell
 "Given a cell and a turn cell, returns a cell that syncs with the given cell only when the turn updates"
 [c turn]
 {:pre [(j/cell? c) (j/cell? turn)]}
 (j/with-let [c' (j/cell nil)]
  (h/do-watch turn #(reset! c' @c))))

; Example usage
(let [conn (conn-cell)
      turn (turn-cell)
      ; the lens means when we update turn to anything it just result in a random-uuid anyway.
      end-turn! #(reset! turn true)
      turn-conn (end-of-turn-cell conn turn)]
 (transact! conn [{:monster/id "turtle" :position/x 1 :position/y 5}]) ; conn propagates
 (transact! conn [{:monster/id "fish" :position/x 3 :position/y 10 :position/z -1}]) ; conn propagates
 (end-turn!)) ; turn-conn propagates

@U0564EGNY ^^

hmmm….still planning to test 3 methods: custom world cell, datascript, and pure js world object

in the latter case, I would update a cell at the end of each turn from the js object

@U0564EGNY this technique isn't specific to datascript

the point is that your concept of a "turn" is now explicit in data somewhere - it has a uuid

yeah, but also I’m not sure I want every tile being a formula cell either

so you can watch and respond to that directly

I’m thinking I keep a 2D vector of input cells and only update the ones that need to change

why does every cell have to be a formula cell?

every tile

soz, yes tiles

why does moving the "turn" into a uuid have anything to do with that?

at the end of turn the view must be updated

my point was just that if you try to co-ordinate "end of turn" with callbacks or queues or whatever else

it becomes a structural thing

but if "current turn" is data, then sure, you can build callbacks and whatever on top of that

yeah, I think a queue of which tiles to update might be most efficient

but you have an explicit thing that can be referenced, rather than an implicit emergent behaviour that needs to be managed

efficient to what?

execute? think about? write? refactor later?

also, i don't see how having a queue is mutually exclusive to tracking the current turn?

wouldn't you just put a flush-queue! function on a do-watch for the current turn?

and also, if you have a queue anyway, why can't the queue be a vector to transact into datascript in bulk?

@U0564EGNY soz, it's a little tricky because i can't see your code >.<

i'm asking like a million questions 😛

yeah, I’ll put it up on github at some point

but right now I’m throwing lots of darts

yeah, i know that feeling 🙂

my preliminary tests (no datascript yet) indicate low-level js performance is about 100x faster than cljs for creating world state, and about 20x faster searching for entities by position.

I’ll post my code to github this weekend and we can tear it apart

I need to comment it a bit so intent is clear

@U0564EGNY i'm not surprised really, lower level tools are often faster

@U0564EGNY the question is always "fast enough?" not "fastest?"

for a turn based game, i'd imagine (without knowing much of the details) that you could get away with a few hundred ms, or even a few seconds of "thinking" per turn

which is a lot of wiggle room