 |
Flecs v3.2
A fast entity component system (ECS) for C & C++
|
|
Flecs v3.2
A fast entity component system (ECS) for C & C++
|
At the core of an Entity Component System are queries, which make it possible to find entities matching a list of conditions in realtime, for example:
This query returns all entities that at least have the Position and Velocity components. Queries provide direct access to cache efficient storages of matched components, which gives applications the ability to process large numbers (think millions) of entities each frame.
Here are some of the highlights of Flecs queries:
and, or, not and optional operators.Position component from an entity and its parent.| Name | Description |
|---|---|
| Id | An id that can be matched, added and removed |
| Component | Id with a single element (same as an entity id) |
| Pair | Id with two elements |
| Tag | Component or pair not associated with data |
| Relationship | Used to refer to first element of pair |
| Target | Used to refer to second element of pair |
| Source | Entity on which a term is matched |
| Iterator | Object used to iterate a query |
| Field | A single value or array of values made available by an iterator. An iterator usually provides a field per query term. |
Make sure to check out the code examples in the repository:
https://github.com/SanderMertens/flecs/tree/master/examples
Flecs has different query types, which are optimized for different kinds of use cases. This section provides a brief overview of each kind:
Filters are cheap to create, low overhead, reasonably efficient to iterate. They are good for ad-hoc queries with runtime-defined conditions. An example:
Cached queries cache the output of a filter. They are more expensive to create and have higher overhead, but are the fastest to iterate. Cached queries are the default for systems. An example:
Rules are a constraint-based query engine capable of traversing graphs. They are more expensive to create than filters, have low overhead, and their iteration performance depends on query complexity. An example:
For more information on how each implementation performs, see Performance.
This section explains how to create queries in the different language bindings and the flecs query DSL.
Query descriptors are the C API for creating queries. The API uses a type called ecs_filter_desc_t, to describe the structure of a query. This type is used to create all query kinds (ecs_filter_t, ecs_query_t, ecs_rule_t). An example:
The example shows the short notation, which looks like this when expanded:
Query descriptors can also be used by the C++ API. However because C++ does not support taking the address of a temporary, and not all language revisions support designated initializers, query descriptors in C++ should be used like this:
The following table provides an overview of the query types with the init/fini functions:
| Kind | Type | Init | Fini | Descriptor type |
|---|---|---|---|---|
| Filter | ecs_filter_t | ecs_filter_init | ecs_filter_fini | ecs_filter_desc_t |
| Query | ecs_query_t | ecs_query_init | ecs_query_fini | ecs_query_desc_t |
| Rule | ecs_rule_t | ecs_rule_init | ecs_rule_fini | ecs_filter_desc_t |
Additionally the descriptor types for systems (ecs_system_desc_t) and observers (ecs_observer_desc_t) embed the ecs_filter_desc_t descriptor type.
Query builders are the C++ API for creating queries. The builder API is built on top of the descriptor API, and adds a layer of convenience and type safety that matches modern idiomatic C++. The builder API is implemented for all query kinds (filters, cached queries, rules). An example of a simple query:
Queries created with template arguments provide a type safe way to iterate components:
The builder API allows for incrementally constructing queries:
The following table provides an overview of the query types with the factory functions:
| Kind | Type | Factory |
|---|---|---|
| Filter | flecs::filter | world::filter_builder |
| Query | flecs::query | world::query_builder |
| Rule | flecs::rule | world::rule_builder |
Additional helper methods have been added to the C++ API to replace combinations of the term method with other methods. They are the following:
| Term | Equivalent |
|---|---|
.with<Component>() | .term<Component>() |
.without<Component>() | .term<Component>().not_() |
.read<Component>() | .term<Component>().read() |
.write<Component>() | .term<Component>().write() |
The query DSL (domain specific language) is a string format that can represent a query. The query DSL is used by convenience macros in the C API like ECS_SYSTEM and ECS_OBSERVER, and makes it easier to create queries at runtime for tools like https://www.flecs.dev/explorer/. An example of a simple query in the DSL:
An example of how the DSL is used with the ECS_SYSTEM convenience macro:
Queries can be created from expressions with both the descriptor and builder APIs:
The query DSL requires the FLECS_PARSER addon to be included in a build.
This section describes the different ways queries can be iterated. The code examples use filters, but also apply to cached queries and rules.
In the C API an iterator object of type ecs_iter_t can be created for each of the query kinds, using the ecs_filter_iter, ecs_query_iter and ecs_rule_iter functions. This iterator can then be iterated with the respective next functions: ecs_filter_next, ecs_query_next and ecs_rule_next.
An iterator can also be iterated with the ecs_iter_next function which is slightly slower, but does not require knowledge about the source the iterator was created for.
An example:
Iteration is split up into two loops: the outer loop which iterates tables, and the inner loop which iterates the entities in that table. This approach provides direct access to component arrays, which allows compilers to do performance optimizations like auto-vectorization.
The indices provided to the ecs_field function must correspond with the order in which terms have been specified in the query. This index starts counting from 1, with index 0 reserved for the array containing entity ids.
The each function is the default and often fastest approach for iterating a query in C++. each can be called directly on a flecs::filter, flecs::query and flecs::rule. An example:
A flecs::entity can be added as first argument:
A flecs::iter and size_t argument can be added as first arguments. This variant of each provides access to the flecs::iter object, which contains more information about the object being iterated. The size_t argument contains the index of the entity being iterated, which can be used to obtain entity-specific data from the flecs::iter object. An example:
When a query contains a template argument that is an empty type (a struct without any members), it should be passed by value instead of by reference:
Alternatively an empty type can be specified outside of the query type, which removes it from the signature of each:
The iter function has an outer and inner loop (similar to C iterators) which provides more control over how to iterate entities in a table than each. The iter function makes it possible, for example, to run code only once per table, or iterate entities multiple times.
An example:
Instead of using flecs::iter as iterator directly, an application can also use the flecs::iter::count method which provides more flexibility:
The component arguments may be omitted, and can be obtained from the iterator object:
This can be combined with an untyped variant of the field method to access component data without having to know component types at compile time. This can be useful for generic code, like serializers:
Entities can be moved between tables when components are added or removed. This can cause unwanted side effects while iterating a table, like iterating an entity twice, or missing an entity. To prevent this from happening, a table is locked by the C++ each and iter functions, meaning no entities can be moved from or to it.
When an application attempts to add or remove components to an entity in a table being iterated over, this can throw a runtime assert. An example:
This can be addressed by deferring operations while the query is being iterated:
An application can also use the defer_begin and defer_end functions which achieve the same goal:
Code ran by a system is deferred by default.
This section goes over the different features of queries and how they can be expressed by the query descriptor API, query builder API and in the query DSL.
Supported by: filters, cached queries, rules
A component is any single id that can be added to an entity. This includes tags and regular entities, which are ids that are not associated with a datatype.
To match a query, an entity must have all the requested components. An example:
Only entities e2 and e3 match the query Position, Velocity.
The following sections describe how to create queries for components in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To query for a component in C, the id field of a term can be set:
The id field is guaranteed to be the first member of a term, which allows the previous code to be rewritten in this shorter form:
The ecs_id macro converts the component typename into the variable name that holds the component identifier. This macro is required for components created with ECS_COMPONENT, but not when querying for regular tags/entities:
Components can also be queried for by name by setting the .first.name member in a term:
An easy way to query for components in C++ is to pass them as template arguments to the query factory function:
This changes the returned query type, which determines the type of the function used to iterate the query:
The builder API makes it possible to add components to a query without modifying the query type:
When template arguments are mixed with the builder API, the components added by the term function will be placed after the components provided as template arguments.
The builder API makes it possible to query for regular entity ids created at runtime:
Components can also be queried for by name. To query for component types by name, they have to be used or registered first.
To query for a components in the query DSL they can be specified in a comma separated list of identifiers. The rules for resolving identifiers are the same as the ecs_lookup_fullpath / world.lookup functions. An example:
Any named entity can be specified this way. Consider:
The entity e from this example will be matched by this query:
When an identifier in the query DSL consists purely out of numeric characters it is converted to an entity id. If in the previous example Npc has id 100 and Platoon_01 has id 101, the following query string would be equivalent:
The , symbol in the query DSL is referred to as the and operator, as an entity must have all comma-separated components in order to match the query.
Supported by: filters, cached queries, rules
Wildcards allow a single query term to match with more than one (component) ids. Flecs supports two kinds of wildcards:
| Name | DSL Symbol | C identifier | C++ identifier | Description |
|---|---|---|---|---|
| Wildcard | * | EcsWildcard | flecs::Wildcard | Match all |
| Any | _ | EcsAny | flecs::Any | Match at most one |
The Wildcard wildcard returns an individual result for anything that it matches. The query in the following example will return twice for entity e, once for component Position and once for component Velocity:
The Any wildcard returns a single result for the first component that it matches. The query in the following example will return once for entity e:
When using the Any wildcard it is undefined which component will be matched, as this can be influenced by other parts of the query. It is guaranteed that iterating the same query twice on the same dataset will produce the same result.
Wildcards are particularly useful when used in combination with pairs (next section).
Supported by: filters, cached queries, rules
A pair is an id that encodes two elements. Pairs, like components, can be added to entities and are the foundation for Relationships.
The elements of a pair are allowed to be wildcards. When a query pair contains the Wildcard wildcard, a query returns a result for each matching pair on an entity. When a query pair returns an Any wildcard, the query returns at most a single matching pair on an entity.
The following sections describe how to create queries for pairs in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To query for a pair in C, the id field of a term can be set to a pair using the ecs_pair macro:
The id field is guaranteed to be the first member of a term, which allows the previous code to be rewritten in this shorter form:
When an element of the pair is a component type, use the ecs_id macro to obtain the identifier to the id variable of the component type:
The ecs_isa, ecs_childof and ecs_dependson convenience macros can be used to create pairs for builtin relationships. The two queries in the next example are equivalent:
Pair queries can be created by setting their individual elements in the first.id and second.id members of a term:
Alternatively, one or both elements of a pair can be resolved by name. The two queries in the next example are equivalent:
When a query pair contains a wildcard, the ecs_field_id function can be used to determine the id of the pair element that matched the query:
When both parts of a pair are types, a flecs::pair template can be used. Pair templates can be made part of the query type, which makes them part of the argument list of the iterator functions. An example:
When using the iter function to iterate a query with a pair template, the argument type assumes the type of the pair. This is required as the component array being passed directly to the iter function. An example:
Pairs can also be added to queries using the builder API. This allows for the pair to be composed out of both types and regular entities. The three queries in the following example are equivalent:
Individual elements of a pair can be specified with the first and second methods. The methods apply to the last added term. An example:
Individual elements of a pair can be resolved by name by using the first and second methods:
When a query pair contains a wildcard, the flecs::iter::pair method can be used to determine the id of the pair element that matched the query:
To query for a pair in the query DSL, the elements of a pair are a comma separated list surrounded by parentheses. An example:
A query may contain multiple pairs:
Queries for pairs that contain wildcards should use the symbols for either the Wildcard or Any wildcards:
A pair may contain two wildcards:
Supported by: filters, cached queries, rules
Access modifiers specify which components of a query can be read and/or written. The different access modifiers are:
| Name | DSL identifier | C identifier | C++ identifier | Description |
|---|---|---|---|---|
| In | in | EcsIn | flecs::In | Component is readonly |
| Out | out | EcsOut | flecs::Out | Component is write only |
| InOut | inout | EcsInOut | flecs::InOut | Component can be read/written |
| None | none | EcsInOutNone | flecs::InOutNone | Component is neither read nor written |
| Default | n/a | EcsInOutDefault | flecs::InOutDefault | Default modifier is selected for term |
Access modifiers can be used by API functions to ensure a component cannot be written, for example by requiring a component to be accessed with a const modifier. APIs may also infer access modifiers where possible, for example by using the In modifier for a query term with a type that has a const modifier.
When using pipelines, the scheduler may use access modifiers to determine where sync points are inserted. This typically happens when a system access modifier indicates a system writing to a component not matched by the query (for example, by using set), and is followed by a system that reads that component.
Access modifiers may also be used by serializers that serialize the output of an iterator (for example: ecs_iter_to_json). A serializer may for example decide to not serialize component values that have the Out or None modifiers.
When no access modifier is specified, Default is assumed. This selects InOut for components owned by the matched entity, and In for components that are from entities other than the one matched by the query.
When a query term can either match a component from the matched entity or another entity (for example: when a component is inherited from a prefab) the Default access modifier only provides write access for the results where the component is owned by the matched entity. This prevents accidentally writing to a shared component. This behavior can be overridden by explicitly specifying an access modifier.
When a query term matches a tag (a component not associated with data) with a Default modifier, the None modifier is selected.
The following sections show how to use access modifiers in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
Access modifiers can be set using the inout member:
Access modifiers can be set using the inout method:
When the const modifier is added to a type, the flecs::In modifier is automatically set:
This also applies to types added with term:
When a component is added by the term method and retrieved from a flecs::iter object during iteration, it must meet the constraints of the access modifiers. If the constraints are not met, a runtime assert may be thrown:
The builder API has in(), inout(), out() and inout_none() convenience methods:
Access modifiers in the query DSL can be specified inside of angular brackets before the component identifier:
Supported by: filters, cached queries, rules
The following operators are supported by queries:
| Name | DSL operator | C identifier | C++ identifier | Description |
|---|---|---|---|---|
| And | , | EcsAnd | flecs::And | Match at least once with term |
| Or | \|\| | EcsOr | flecs::Or | Match at least once with one of the OR terms |
| Not | ! | EcsNot | flecs::Not | Must not match with term |
| Optional | ? | EcsOptional | flecs::Optional | May match with term |
| Equal | == | EcsPredEq | flecs::PredEq | Equals entity/entity name |
| Not equal | != | EcsPredNeq | flecs::PredNeq | Not equals entity/entity name |
| Match | ~= | EcsPredMatch | flecs::PredMatch | Match entity name with substring |
| AndFrom | AND \| | EcsAndFrom | flecs::AndFrom | Match all components from id at least once |
| OrFrom | OR \| | EcsOrFrom | flecs::OrFrom | Match at least one component from id at least once |
| NotFrom | NOT \| | EcsNotFrom | flecs::NotFrom | Don't match any components from id |
Supported by: filters, cached queries, rules
The And operator is used when no other operators are specified. The following sections show how to use the And operator in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
When no operator is specified, And is assumed. The following two queries are equivalent:
When no operator is specified, And is assumed. The following two queries are equivalent:
The builder API has a and_ convenience method:
Query expressions with comma separated lists use the And operator:
Supported by: filters, cached queries, rules
The Or operator allows for matching a single component from a list. Using the Or operator means that a single term can return results of multiple types. When the value of a component is used while iterating the results of an Or operator, an application has to make sure that it is working with the expected type.
When using the Or operator, the terms participating in the Or expression are made available as a single field. Field indices obtained from an iterator need to account for this. Consider the following query:
This query has 4 terms, while an iterator for the query returns results with 3 fields. This is important to consider when retrieving the field for a term, as its index has to be adjusted. In this example, Position has index 1, Velocity || Speed has index 2, and Mass has index 3.
The following sections show how to use the Or operator in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To create a query with Or terms, set oper to EcsOr:
To create a query with Or terms, use the oper method with flecs::Or:
The builder API has a or_ convenience method:
To create a query with Or terms, use the || symbol:
Supported by: filters, cached queries, rules
The Not operator makes it possible to exclude entities with a specified component. Fields for terms that uses the Not operator will never provide data.
A note on performance: Not terms are efficient to evaluate when combined with other terms, but queries that only have Not terms (or `Optional`) can be expensive. This is because the storage only maintains indices for tables that have a component, not for tables that do not have a component.
The following sections show how to use the Not operator in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To create a query with Not terms, set oper to EcsNot:
To create a query with Not terms, use the oper method with flecs::Not:
The builder API has a not_ convenience method:
To create a query with Not terms, use the ! symbol:
Supported by: filters, cached queries, rules
The Optional operator optionally matches with a component. While this operator does not affect the entities that are matched by a query, it can provide more efficient access to a component when compared to conditionally getting the component in user code. Before accessing the value provided by an optional term, code must first check if the term was set.
A note on performance: just like the Not operator Optional terms are efficient to evaluate when combined with other terms, but queries that only have Optional terms can be expensive. Because the Optional operator does not restrict query results, a query that only has Optional terms will match all entities.
When an optional operator is used in a rule, and a variable written by the optional term is read by a subsequent term, the subsequent term becomes a dependent term. This means that if the optional term does not match, the dependent term will be ignored. For example:
Because the second term is optional, the variable $planet may or may not be set depending on whether the term was matched. As a result the third term becomes dependent: if $planet was not set, the term will be ignored.
The following sections show how to use the Optional operator in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To create a query with Optional terms, set oper to EcsOptional:
To create a query with Optional terms, call the oper method with flecs::Optional:
The builder API has an optional convenience method:
To create a query with Optional terms, use the ? symbol:
Supported by: rules
Equality operators (==, !=, ~=) allow a query to ensure that a variable equals a specific entity, that the entity it stores has a specific name, or that the entity name partially matches a substring.
The left hand side of an equality operator must be a variable. The right hand side of an operator can be an entity identifier or a string for the == and != operators, and must be a string in case of the ~= operator. For example:
Test if variable $this equals Foo (Foo must exist at query creation time):
Test if variable $this equals entity with name Foo (Foo does not need to exist at query creation time):
Test if variable $this stores an entity with a name that has substring Fo:
When the equals operator (==) is used with a variable that has not yet been initialized, the right-hand side of the operator will be assigned to the variable.
Other than regular operators, equality operators are set as first, with the left hand being src and the right hand being second. Equality operators can be combined with And, Not and Or terms.
Terms with equality operators return no data.
Supported by: filters, cached queries
The AndFrom, OrFrom and NotFrom operators make it possible to match a list of components that is defined outside of the query. Instead of matching the id provided in the term, the operators match the list of components of the provided id as if they were provided as a list of terms with And, Or or Not operators. For example, if entity e has components Position, Velocity and is combined in a query with the AndFrom operator, entities matching the query must have both Position and Velocity.
The AndFrom, OrFrom and NotFrom operators are especially useful when combined with prefab entities, which by default are not matched with queries themselves. Components that have the DontInherit property are ignored while matching the operators, which means that using a prefab in combination with AndFrom, OrFrom and NotFrom will not cause components like Prefab or ChildOf to be considered.
Component lists can be organized recursively by adding an id to an entity with the AND and OR id flags.
Fields for terms that use the AndFrom, OrFrom or NotFrom operators never provide data. Access modifiers for these operators default to InOutNone. When a the AndFrom, OrFrom or NotFrom operator is combined with an access modifier other than InOutDefault or InOutNone query creation will fail.
The following sections show how to use the operators in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To use the AndFrom, OrFrom and NotFrom operators, set oper to EcsAndFrom, EcsOrFrom or EcsNotFrom
To use the AndFrom, OrFrom and NotFrom operators, call the oper method with flecs::AndFrom, flecs::OrFrom or flecs::NotFrom.
The builder API has the and_from, or_from and not_from convenience methods:
To create a query with the AndFrom, OrFrom and NotFrom operators in the C API, use AND, OR and NOT in combination with the bitwise OR operator (|):
Supported by: rules
Query scopes are a mechanism that allows for treating the output of a number of terms as a single condition. For example, the following query has two terms with an Or operator that are negated by a Not operator:
A query scope can contain any number of terms and operators. The following query has a scope with mixed operators:
Query scopes allow for the creation of complex queries when combined with variables and relationships. The following query finds all entities that have no children with Position:
Note how this is different from this query, which finds all children that don't have Position:
Whereas the first query only returns parents without children with Position, the second query returns parents that have at least one child that doesn't have Position.
When a scope is evaluated, the entire result set of the scope is treated as a single term. This has as side effect that any variables first declared inside the scope are not available outside of the scope. For example, in the following query the value for variable $child is undefined, as it is first used inside a scope:
Scopes currently have the following limitations:
!{ ... }). Future versions of flecs will add support for combining scopes with Or operators (e.g. { ... } || { ... }).The following examples show how to use scopes in the different language bindings:
Supported by: filters, cached queries, rules
Source is a property of a term that specifies the entity on which the term should be matched. Queries support two kinds of sources: static and variable. A static source is known when the query is created (for example: match SimTime on entity Game), whereas a variable source is resolved while the query is evaluated. When no explicit source is specified, a default variable source called $This is used (see Variables).
When a query only has terms with fixed sources, iterating the query will return a result at least once when it matches, and at most once if the query terms do not match wildcards. If a query has one or more terms with a fixed source that do not match the entity, the query will return no results. A source does not need to match the query when the query is created.
When a term has a fixed source and the access modifiers are not explicitly set, the access modifier defaults to In, instead of InOut. The rationale behind this is that it encourages code to only makes local changes (changes to components owned by the matched entity) which is easier to maintain and multithread. This default can be overridden by explicitly setting access modifiers.
The following sections show how to use variable and fixed sources with the different language bindings. The code examples use filter queries, but also apply to queries and rules.
To specify a fixed source, set the src.id member to the entity to match. The following example shows how to set a source, and how to access the value provided by a term with a fixed source:
Note how in this example all components can be accessed as arrays. When a query has mixed fields (fields with both arrays and single values), behavior defaults to entity-based iteration where entities are returned one at a time. As a result, i in the previous example will never be larger than 0, which is why this code works even though there is only a single instance of the SimTime component.
Returning entities one at a time can negatively affect performance, especially for large tables. To learn more about why this behavior exists and how to ensure that mixed results use table-based iteration, see Instancing.
A source may also be specified by name by setting the src.name member:
This examples shows how to access the entities matched by the default $This source and a fixed source:
The entities and count member are solely populated by the number of entities matched by the default $This source. If a query only contains fixed sources, count will be set to 0. This is important to keep in mind, as the inner for loop from the last example would never be iterated for a query that only has fixed sources.
To specify a fixed source, call the src method to the entity to match. The following example shows how to set a source, and how to access the value provided by a term with a fixed source:
Note how in this example all components can be accessed as arrays. When a query has mixed fields (fields with both arrays and single values), behavior defaults to entity-based iteration where entities are returned one at a time. As a result, i in the previous example will never be larger than 0, which is why this code works even though there is only a single instance of the SimTime component.
Returning entities one at a time can negatively affect performance, especially for large tables. To learn more about why this behavior exists and how to ensure that mixed results use table-based iteration, see Instancing.
The next example shows how queries with mixed $This and fixed sources can be iterated with each. The each function does not have the performance drawback of the last iter example, as it uses instancing by default.
When a query has no terms for the $This source, it must be iterated with the iter function or with a variant of each that does not have a signature with flecs::entity as first argument:
A source may also be specified by name:
To specify a source in the DSL, use parenthesis after the component identifier. The following example uses the default $This source for Position and Velocity, and Game as source for SimTime.
In the previous example the source for Position and Velocity is implicit. The following example shows the same query with explicit sources for all terms:
To specify a source for a pair, the second element of the pair is placed inside the parenthesis after the source. The following query uses the default $This source for the (Color, Diffuse) pair, and Game as source for the (Color, Sky) pair.
In the previous example the source for (Color, Diffuse) is implicit. The following example shows the same query with explicit sources for all terms:
Supported by: filters, cached queries, rules
Singletons are components that are added to themselves, which can be matched by providing the component id as source.
The following sections show how to use singletons in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
A singleton query is created by specifying the same id as component and source:
The singleton component data is accessed in the same way a component from a static source is accessed.
A singleton query can be created by specifying the same id as component and source:
The builder API provides a singleton convenience function:
The singleton component data is accessed in the same way a component from a static source is accessed.
A singleton query can be created by specifying the same id as component and source:
For convenience the $ character may be used as source, which resolves to the component id:
Supported by: filters, cached queries, rules(!)
Relationship traversal enables a query to search for a component by traversing a relationship. One of the most common examples of where this is useful is a Transform system, which matches Position on an entity and the entity's parent. To find the Position component on a parent entity, a query traverses the ChildOf relationship upwards:
The arrows in this diagram indicate the direction in which the query is traversing the ChildOf relationship to find the component. A query will continue traversing until it has found an entity with the component, or until a root (an entity without the relationship) has been found. The traversal is depth-first. If an entity has multiple instances of a relationship a query will first traverse the first instance until its root entity before continuing with the second instance.
Using the relationship traversal feature will in most cases provide better performance than doing the traversal in user code. This is especially true for cached queries, where the results of traversal are cached. Relationship traversal can in some edge cases cause performance degradation, especially in applications with large numbers of cached queries and deep hierarchies. See the Performance section for more details.
Any relationship used for traversal must have the Traversable property. Attempting to create a query that traverses a relationship that does not have the Traversable property will cause query creation to fail. This safeguards against creating queries that could end up in an infinite traversal loop when a cyclic relationship is encountered.
Components that have the DontInherit property cannot be matched through traversal. Examples of builtin components that have the DontInherit property are Prefab (instances of prefabs should not be considered prefabs) and ChildOf (a child of a parent is not a child of the parent's parent).
Relationship traversal works for both variable and fixed sources.
Traversal behavior can be customized with the following bitflags, in addition to the relationship being traversed:
| Name | DSL identifier | C identifier | C++ identifier | Description |
|---|---|---|---|---|
| Self | self | EcsSelf | flecs::Self | Match self |
| Up | up | EcsUp | flecs::Up | Match by traversing upwards |
| Down | down | EcsDown | flecs::Down | Match by traversing downwards (derived, cannot be set) |
| Parent | parent | EcsParent | flecs::Parent | Short for up(ChildOf) |
| Cascade | cascade | EcsCascade | flecs::Cascade | Same as Up, but iterate in breadth-first order |
If just Self is set a query will only match components on the matched entity (no traversal). If just Up is set, a query will only match components that can be reached by following the relationship and ignore components from the matched entity. If both Self and Up are set, the query will first look on the matched entity, and if it does not have the component the query will continue searching by traverse the relationship.
Query terms default to Self|Up for the IsA relationship. This means that components inherited from prefabs will be matched automatically by queries, unless specified otherwise. When a relationship that is not IsA is traversed, the entities visited while traversing will still be tested for inherited components. This means that an entity with a parent that inherits the Mass component from a prefab will match a query that traverses the ChildOf relationship to match the Mass component. A code example:
When a component is matched through traversal and its access modifier is not explicitly set, it defaults to flecs::In. This behavior is consistent with terms that have a fixed source.
When a component is matched through traversal, the behavior is the same as though the component was matched through a fixed source: iteration will switch from table-based to entity-based. This happens on a per-result basis: if all terms are matched on the matched entity the entire table will be returned by the iterator. If one of the terms was matched through traversal, entities are returned one by one.
While returning entities one by one is measurably slower than iterating entire tables, this default behavior enables matching inherited components by default without requiring the user code to be explicitly aware of the difference between a regular component and an inherited component. An advantage of this approach is that applications that use inherited components can interoperate with third party systems that do not explicitly handle them.
To ensure fast table-based iteration an application can enable instancing. Instanced iteration is as fast as, and often faster than regular iteration. Using inherited components that are shared across many entities can improve cache efficiency as less data needs to be loaded from main RAM, and values are more likely to already be stored in the cache.
Note: the C++
eachAPI always uses instancing, which guarantees fast table-based iteration.
This list is an overview of current relationship traversal limitations:
Down flag is currently reserved for internal purposes and should not be set when creating a query.Cascade flag only works for cached queries.The following sections show how to use traversal in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
By default queries traverse the IsA relationship if a component cannot be found on the matched entity. In the following example, both base and inst match the query:
Implicit traversal can be disabled by setting the flags member to EcsSelf. The following example only matches base:
To use a different relationship for traversal, use the trav member in combination with the EcsUp flag. The following example only matches child:
The EcsParent flag can be used which is shorthand for EcsUp with EcsChildOf. The query in the following example is equivalent to the one in the previous example:
If a query needs to match a component from both child and parent, but must also include the root of the tree, the term that traverses the relationship can be made optional. The following example matches both parent and child. The second term is not set for the result that contains parent.
By adding the EcsCascade flag, a query will iterate the hierarchy top-down. This is only supported for cached queries:
Relationship traversal can be combined with fixed source terms. The following query matches if the my_widget entity has a parent with the Window component:
The two queries in the following example are equivalent, and show how the implicit traversal of the IsA relationship is implemented:
By default queries traverse the IsA relationship if a component cannot be found on the matched entity. In the following example, both base and inst match the query:
Implicit traversal can be disabled by calling the self method for the term. The following example only matches base:
To use a different relationship for traversal, use the up method with the relationship as argument. The following example only matches child:
The parent method can be used which is shorthand for up(flecs::ChildOf). The query in the following example is equivalent to the one in the previous example:
If a query needs to match a component from both child and parent, but must also include the root of the tree, the term that traverses the relationship can be made optional. The following example matches both parent and child. The second term is not set for the result that contains parent.
By calling the cascade method, a query will iterate the hierarchy top-down. Note that the example could also have called both parent() and cascade(). The cascade feature is only supported for cached queries:
Relationship traversal can be combined with fixed source terms. The following query matches if the my_widget entity has a parent with the Window component:
The two queries in the following example are equivalent, and show how the implicit traversal of the IsA relationship is implemented:
By default queries traverse the IsA relationship if a component cannot be found on the matched entity. The following query matches Position on entities that either have the component or inherit it:
Implicit traversal can be disabled by adding self enclosed by parentheses after the component identifier:
To use a different relationship for traversal, specify up with the relationship as argument:
The parent keyword can be used which is shorthand for up(ChildOf). The query in the following example is equivalent to the one in the previous example:
If a query needs to match a component from both child and parent, but must also include the root of the tree, the term that traverses the relationship can be made optional:
By adding the cascade keyword, a query will iterate the ChildOf hierarchy top-down. The cascade feature is only supported by cached queries:
Relationship traversal can be combined with fixed source terms, by using a colon (:) to separate the traversal flags and the source identifier. The following query matches if the my_widget entity has a parent with the Window component:
The two queries in the following example are equivalent, and show how the implicit traversal of the IsA relationship is implemented:
Supported by: filters, cached queries, rules
Note: this section is useful when optimizing code, but is not required knowledge for using queries.
Instancing is the ability to return results with fields that have different numbers of elements. An example of where this is relevant is a query with terms that have both variable and fixed sources:
This query may return a result with a table that has many entities, therefore resulting in many component instances being available for Position and Velocity, while only having a single instance for the SimTime component.
Another much more common cause of mixed results is if a matched component is inherited from a prefab. Consider this example:
The filter in this example will match both inst_1, inst_2 because they inherit Mass, and ent_1 and ent_2 because they own Mass. The following example shows an example of code that iterates the filter:
With iteration being table-based the expectation is that the result looks something like this, where all entities in the same table are iterated in the same result:
Instead, when ran this code produces the following output:
The prefab instances are returned one at a time, even though they are stored in the same table. The rationale behind this behavior is that it prevents a common error that was present in the previous code example, specifically where the Mass component was read:
If inst_1 and inst_2 would have been returned in the same result it would have caused the inner for loop to loop twice. This means i would have become 1 and m[1].value would have caused an out of bounds violation, likely crashing the code. Why does this happen?
The reason is that both entities inherit the same instance of the Mass component. The Mass pointer provided by the iterator is not an array to multiple values, in this case it is a pointer to a single value. To fix this error, the component would have to be accessed as:
To prevent subtle errors like this from happening, iterators will switch to entity-based iteration when a result has fields of mixed lengths. This returns entities one by one, and guarantees that the loop variable i always has value 0. The expression m[0].value is equivalent to m->value, which is the right way to access the component.
Note: As long as all fields in a result are of equal length, the iterator switches to table-based iteration. This is shown in the previous example: both
ent_1andent_2are returned in the same result.
While this provides a safe default to iterate results with shared components, it does come at a performance cost. Iterating entities one at a time can be much slower than iterating an entire table, especially if tables are large. When this cost is too great, iteration can be instanced, which prevents switching from table-based to entity-based iteration.
Note: The implementation of the C++
eachfunction always uses instancing.
The following diagram shows the difference in how results are returned between instanced iteration and non-instanced iteration. Each result block corresponds with a call to the iter method in the above example:
Note that when iteration is not instanced, inst_1 and inst_2 are returned in separate results, whereas with instanced iteration they are both combined in the same result. Iterating the right result is faster for a few reasons:
However, what the diagram also shows is that code for instanced iterators must handle results where Mass is either an array or a single value. This is the tradeoff of instancing: faster iteration at the cost of more complex iteration code.
The following sections show how to use instancing in the different language bindings. The code examples use filter queries, but also apply to queries and rules.
Queries can be instanced by setting the instanced member to true:
Note how the ecs_field_is_self test is moved outside of the for loops. This keeps conditional statements outside of the core loop, which enables optimizations like auto-vectorization.
Queries can be instanced by calling the instanced method:
Note how the it.is_self() test is moved outside of the for loops. This keeps conditional statements outside of the core loop, which enables optimizations like auto-vectorization.
Partial support: filters, cached queries. Full support: rules
Query variables represent the state of a query while it is being evaluated. The most common form of state is "the entity (or table) against which the query is evaluated". While a query is evaluating an entity or table, it has to store it somewhere. In flecs, that "somewhere" is a query variable.
Consider this query example, written down with explicit term sources:
The first term to encounter a variable is usually the one to populate it with all candidates that could match that term. Subsequent terms then use the already populated variable to test if it matches. If the condition matches, the query moves on to the next term. If the condition fails, the query moves back to the previous term and, if necessary, populates the variable with the next candidate. These kinds of conditions are usually referred to as predicates, and this evaluation process is called backtracking.
This process effectively constrains the possible results that a term could yield. By itself, the Velocity term would return all entities with the Velocity component, but because $This has been assigned already with entities that have Position, the term only feeds forward entities that have both Position and Velocity.
While using variables as source is the most common application for variables, variables can be used in any part of the term. Consider constructing a query for all spaceships that are docked to a planet. A first attempt could look like this:
When rewritten with explicit sources, the query looks like this:
This returns all spaceships that are docked to anything, instead of docked to planets. To constrain the result of this query, the wildcard used as target for the DockedTo relationship can be replaced with a variable. An example:
When the second term is evaluated for the first time, $Location will not yet be populated. This causes the term to do two things:
$This has (DockedTo, *)$Location with the id matched by *.After evaluating the second term, the $Location variable is populated with the location the spaceship is docked to. We can now use this variable in a new term, that constrains the location to only entities that have Planet:
This query returns the desired result ("return all spaceships docked to a planet").
Variables can also be used to constrain matched components. Consider the following example query:
This query returns serializable components for all entities that have at least one.
By default variables are assigned while the query is being iterated, but variables can be set before query iteration to constrain the results of a query. Consider the previous example:
An application can set the $This variable or $Location variables, or both, before starting iteration to constrain the results returned by the query. This makes it possible to reuse a single query for multiple purposes, which provides better performance when compared to creating multiple queries.
The following sections show how to use queries in the different language bindings. The code examples use rules queries, which currently are the only queries that support using variables other than $This.
Query variables can be specified by setting the name member in combination with setting the EcsIsVariable bit in the flags member:
An application can constrain the results of the query by setting the variable before starting iteration:
Query variables can be specified by specifying a name with a $ prefix:
Alternatively, variables can also be specified using the var method:
An application can constrain the results of the query by setting the variable before starting iteration:
Alternatively the variable name can be provided to set_var directly:
Supported by: cached queries
Change detection makes it possible for applications to know whether data matching a query has changed. Changes are tracked at the table level, for each component in the table. While this is less granular than per entity tracking, the mechanism has minimal overhead, and can be used to skip entities in bulk.
Change detection works by storing a list of counters on tracked tables, where each counter tracks changes for a component in the table. When a component in the table changes, the corresponding counter is increased. An additional counter is stored for changes that add or remove entities to the table. Queries with change detection store a copy of the list of counters for each table in the cache, and compare counters to detect changes. To reduce overhead, counters are only tracked for tables matched with queries that use change detection.
The change detection feature cannot detect all changes. The following scenarios are detected by change detection:
setmodified for an entity/componentinout or out termsThe following scenarios are not detected by change detection:
get_mut without calling modifiedecs_ref_t or flecs::ref) without calling modifiedA query with change detection enabled will only report a change for the components it matched with, or when an entity got added/removed to a matched table. A change to a component in a matched table that is not matched by the query will not be reported by the query.
By default queries do not use change detection. Change detection is automatically enabled when a function that requires change detection is called on the query, for example if an application calls changed() on the query. Once change detection is enabled it will stay enabled for both the query and the tables the query is matched with.
When a change occurred in a table matching a query, the query state for that table will remain changed until the table is iterated by the query.
When a query iterates a table for which changes are tracked and the query has inout or out terms that are matched with components of that table, the counter for those components will be increased by default. An application can indicate that no components were modified by skipping the table (see code examples).
When a query uses change detection and has
outorinoutterms, its state will always be changed as iterating the query increases the table counters. It is recommended to only use terms with theinaccess modifier in combination with change detection.
The following sections show how to use change detection in the different language bindings. The code examples use cached queries, which is the only kind of query for which change detection is supported.
The following example shows how the change detection API is used in C:
The following example shows how the change detection API is used in C++:
Supported by: cached queries
Sorted queries allow an application to specify a component that entities should be sorted on. Sorting is enabled by setting the order_by function in combination with the component to order on. Sorted queries sort the tables they match with when necessary. To determine whether a table needs to be sorted, sorted queries use change detection. A query determines whether a sort operation is needed when an iterator is created for it.
Because sorting relies on change detection, it has the same limitations with respect to detecting changes. When using sorted queries, make sure a query is able to detect the changes necessary for knowing when to (re)sort.
Query sorting works best for data that does not change often, as the sorting process can be expensive. This is especially true for queries that match with many tables, as one step of the sorting algorithm scans all matched tables repeatedly to find an ordered set of slices.
An application should also prevent having sorted queries with conflicting sorting requirements. This can cause scenarios in which both queries are invalidating each others ordering, which can result in a resort each time an iterator is created for one of the conflicting queries.
Sorted queries are encouraged to mark the component used for sorting as In. If a sorted query has write access to the sorted component, iterating the query will invalidate its own order which can lead to continuous resorting.
Components matched through traversal can be used to sort entities. This often results in more efficient sorting as component values can be used to sort entire tables, and as a result tables themselves do not have to be sorted.
Sorted queries use a two-step process to return entities in a sorted order. The sort algorithm used in both steps is quicksort. The first step sorts contents of all tables matched by the query. The second step is to find a list of ordered slices across the tables matched by the query. This second step is necessary to support datasets where ordered results have entities interleaved from multiple tables. An example data set:
| Entity | Components (table) | Value used for sorting |
|---|---|---|
| E1 | Position | 1 |
| E2 | Position | 3 |
| E3 | Position | 4 |
| E4 | Position, Velocity | 5 |
| E5 | Position, Velocity | 7 |
| E6 | Position, Mass | 8 |
| E7 | Position | 10 |
| E8 | Position | 11 |
To make sure a query iterates the entities in the right order, it will iterate entities in the ordered tables to determine the largest slice of ordered entities in each table, which the query will iterate in order. Slices are computed during the sorting step. For the above set of entities slices would look like this:
| Table | Slice |
|---|---|
| Position | 0..2 |
| Position, Velocity | 3..4 |
| Position, Mass | 5 |
| Position | 6..7 |
To minimize time spent on sorting, the results of a sort are cached. The performance overhead of iterating an already sorted query is comparable to iterating a regular query, though for degenerate scenarios where a sort produces many slices for comparatively few tables the performance overhead can be significant.
The following sections show how to use sorting in the different language bindings. The code examples use cached queries, which is the only kind of query for which change detection is supported.
The following example shows how to use sorted queries in C:
The function signature of the order_by function should look like the following example:
A query may only use entity identifiers to sort by not setting the order_by_component member:
The following example shows a function that sorts by entity id:
The following example shows how to use sorted queries in C++:
Queries may specify a component id if the component is not known at compile time:
Queries may specify zero for component id to sort on entity ids:
Supported by: cached queries
Grouping is the ability of queries to assign an id ("group id") to a set of tables. Grouped tables are iterated together, as they are stored together in the query cache. Additionally, groups in the query cache are sorted by group id, which guarantees that tables with a lower group id are iterated after tables with a higher group id. Grouping is only supported for cached queries.
Group ids are local to a query, and as a result queries with grouping do not modify the tables they match with.
Grouping is the least granular, most efficient mechanism to order results of a query. Grouping does not rely on sorting individual tables or entities. Instead, sorting only happens when a new group id is introduced to a query, which is rare in many scenarios. Queries maintain an index that maps group ids to the first and last elements in the cache belonging to that group. This allows for tables to be inserted into the cache in a fast constant time operation, while also providing ordered access to tables.
The grouping mechanism is used internally by the cascade feature. Queries that use cascade use a computed hierarchy depth as group id. Because lower group ids are iterated before higher group ids, this provides in a bread-first topological sort of tables and entities that is almost free to maintain.
Queries may be grouped and sorted at the same time. When combined, grouping takes precedence over sorting. Tables are first assigned to their groups in the query cache, after which each group is sorted individually.
An example of an internal query that uses cascade grouping in combination with sorting is the builtin pipeline query. The pipeline query first groups systems by their depth in the DependsOn relationship tree. Within the depth-based groups systems are ordered based on their entity id, which ensures systems are iterated in order of declaration.
A group iterator iterates over a single group of a grouped query. This can be useful when an application may need to match different entities based on the context of the game, such as editor mode, day/night, inside/outside or location in the world.
One example is that of an open game which is divided up into world cells. Even though a world may contain many entities, only the entities in cells close to the player need to be processed. Instead of creating a cached query per world cell, apps can create a single query grouped by world cell, and use group iterators to only iterate the necessary cells.
Grouped iterators, when used in combination with a good group_by function are one of the fastest available mechanisms for finding entities in Flecs. The feature provides the iteration performance of having a cached query per group, but without the overhead of having to maintain multiple caches. Whether a group has ten or ten thousand tables does not matter, which makes the feature an enabler for games with large numbers of entities.
The following sections show how to use sorting in the different language bindings. The code examples use cached queries, which is the only kind of query for which change detection is supported.
The following example shows how grouping can be used to group entities that are in the same game region.
The following example shows what the implementation of group_by_target could look like:
The following example shows how grouping can be used to group entities that are in the same game region.
Supported by: rules
Component inheritance allows for a query to match entities with a component and all subsets of that component, as defined by the IsA relationship. Component inheritance is enabled for all queries by default, for components where it applies.
It is possible to prevent inheriting from a component from by adding the Final property. Queries for components with the Final property will not attempt to resolve component inheritance.
Inheritance relationships can, but are not required to mirror inheritance of the types used as long as it does not impact the layout of the type. Component inheritance is most often used with tags.
The following sections show how to use component inheritance in the different language bindings. The code examples use rules, which is the only kind of query for which component inheritance is currently supported.
The following example shows a rule that uses component inheritance to match entities:
The following example shows a rule that uses component inheritance to match entities:
Supported by: rules
When a transitive relationship is used by a query, a query will automatically look up or test against pairs that satisfy the transitive property. Transitivity is usually defined as:
If R(X, Y) and R(Y, Z) then R(X, Z)
In this example, R is the transitive relationship and X, Y and Z are entities that are used both as source and second element. A typical example of a transitive relationship is LocatedIn. If Bob (X) is located in Manhattan (Y) and Manhattan (Y) is located in New York (Z), then Bob (X) is also located in New York (Z). Therefore "located in" is transitive.
A relationship can be made transitive by adding the transitive property. This would ensure that both (LocatedIn, ManHattan) and (LocatedIn, NewYork) queries would match Bob. When the location is replaced with a variable, the variable will assume all values encountered while traversing the transitive relationship. For example, (LocatedIn, $Place) would return results with Place = [ManHattan, NewYork] for Bob, and a result with Place = [NewYork] for ManHattan.
An example of a builtin relationship that has the Transitive property is the IsA relationship.
The following sections show how to use transitive relationships in the different language bindings. The code examples use rules, which is the only kind of query for which transitive relationships are currently supported.
The following example shows a rule that uses transitivity to match entities that are located in New York:
Queries for transitive relationships can be compared with variables. This query returns all locations an entity is in:
Variables can be used to constrain the results of a transitive query. The following query returns locations an entity is in that are a city:
The following example shows a rule that uses transitivity to match entities that are located in New York:
Queries for transitive relationships can be compared with variables. This query returns all locations an entity is in:
Variables can be used to constrain the results of a transitive query. The following query returns locations an entity is in that are a city:
Transitivity in a query is enabled by adding the Transitive property to a relationship. As a result, a query for a transitive relationship looks the same as a query for a non-transitive relationship. The following examples show the queries used in the C/C++ examples:
Match all entities located in New York:
Return all locations an entity is in:
Return the city entities are in:
Supported by: rules
When a query matches a reflexive relationship, a query term will evaluate to true if the source and target are equal. Reflexivity can be defined as:
R(X, X)
For example, if relationship IsA (R) is reflexive, then a Tree (X) is a Tree (X). An example of a builtin relationship that has the Reflexive property is the IsA relationship.
The following sections show how to use transitive relationships in the different language bindings. The code examples use rules, which is the only kind of query for which transitive relationships are currently supported.
The following example shows a rule that uses the IsA reflexive relationship:
The following example shows a rule that uses the IsA reflexive relationship:
Reflexivity in a query is enabled by adding the Reflexive property to a relationship. As a result, a query for a reflexive relationship looks the same as a query for a non-reflexive relationship. The following example shows the query used in the C/C++ examples:
This section describes performance characteristics for each query type.
A filter is a data type that describes the structure of a query in Flecs. Filters are the cheapest to create, as creating one just requires initializing the value of the filter object.
Filters can serve different purposes, like iterating all matching entities, or testing if a specific entity matches with a filter. Filters can also be used to filter the output of other queries, including other filters.
While the exact way a filter is evaluated depends on what kind of filter it is, almost all filter evaluation follows steps similar to the ones in this diagram:
Each node in the diagram represents a function that can return either true or false. When a node returns true, evaluation moves on to the next node. When it returns false, evaluation goes back one node. These kinds of functions are called predicates, and this evaluation process is called backtracking.
Whether a node returns true or false depends on what its function does:
select(Position). This function finds all tables with Position, and will forward these tables to the next node. As long as select is able to find more tables with Position, the node will keep returning true. When no more tables can be found the node returns false.with(Velocity). This function takes the table found by the previous node and returns true if the table has Velocity, and false if it does not.Both functions are O(1) operations.
When a table reaches the yield node it means that it matched all parts of the filter, and it will be returned by the iterator doing the evaluation.
A table groups all entities that have exactly the same components. Thus if one entity in a table matches a node, all entities in the table match the node. This is one of the main reasons queries are fast: instead of checking each individual entity for components, we can eliminate a table with thousands of entities in a single operation.
Because filters are fast to create, have low overhead, and are reasonably efficient to iterate, they are the goto solution for when an application cannot know in advance what it needs to query for, like finding all children for a specific entity:
While filter evaluation for a single component is very fast, each additional component adds some overhead. This is not just because of the time it takes to do an additional check, but because more tables will get discarded during evaluation. The cost of discarded tables adds up as they pass through one or more nodes before getting rejected, consuming resources while not contributing to the query result.
This cost can be particularly significant for filters that match a large (5-10) number of components in applications with many (>1000s) tables. While Flecs implements several strategies to limit this overhead, like storing empty tables separately from non-empty tables, this is something to keep in mind when using filters.
A cached query is an object that caches the output of a filter. Cached queries are significantly faster to iterate than a filter, as an iterator just needs to walk the list of cache entries and return each one to the application.
This also makes the performance of cached queries more predictable. The overhead per returned table for a filter depends on how many components it matches with, and how many tables it had to discard. For a cached query the cost per table is much lower, and is constant time.
A benchmark showing the difference between cached queries and filters.
This predictable behavior is why cached queries are the default for Flecs systems. Once tables are created and caches are initialized, the (often time critical) main loop of an application does not have to spend time matching entities with systems, which frees up resources for application logic.
Another benefit of cached queries is that they are compatible with good code practices. Cached queries add minimal overhead to systems, which means applications do not have to compromise on design where the cost of evaluating a query has to be weighed against the benefit of having many small decoupled systems.
The following example shows how cached queries are used:
The scenarios described so far are best case scenarios for cached queries, where their performance is amongst the fastest of any ECS implementation. To build games that perform well however, it also helps to know when cached queries perform badly:
Position on a parent entity, where the component is removed from the parent. This triggers cache revalidation, where a query reevaluates its filter to correct invalid entries. When this happens for a large number of queries and tables, this can be time consuming.