Copyright 2023 Travis J. West, https://traviswest.ca, Input Devices and Music Interaction Laboratory (IDMIL), Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), McGill University, Montréal, Canada, and Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
SPDX-License-Identifier: MIT
A component has a name and at least one of the following subroutines or endpoints structures: an initialization subroutine, a main subroutine, an external sources or destinations subroutine, inputs, or outputs. An assembly is defined recursively as an aggregate struct that contains only components or assemblies.
This document provides a means of identifying components and assemblies, and accessing their functionality.
// @='Component Concepts'
@{SimpleAggregate}
@{has_main_subroutine}
@{inputs and outputs}
@{throughpoints and plugins}
template<typename T>
concept Component
= has_name<T>
&& ( has_init_subroutine<T>
|| has_external_sources_subroutine<T>
|| has_main_subroutine<T>
|| has_external_destinations_subroutine<T>
|| has_inputs<T>
|| has_outputs<T>
)
;
@{assembly_predicate}
template<typename T>
concept Assembly = detail::assembly_predicate<T>::value && not Component<T> && not Tuple<T>;
Assemblies, as well as the input and output structures of components, are required to be simple aggregates. Our notion of a simple aggregate comes from boost::pfr, which we rely on for the ability to iterate over lists of endpoints and components in the form of structures, an essential functionality for our purposes. We would ideally defer to boost::pfr for our test of simple aggregate requirements, but a straightforward concept–
–does not provide the required results. It appears as though the compiler considers this to be a valid expression regardless of the result, i.e. even if boost::pfr itself would raise several (overly verbose and difficult to debug) static assertion failures with the given T.
Attempts to defer to std::is_aggregate_v<T> in a similar fashion were also unsatisfactory, with various test structures passing the test even through boost::pfr rejects them at compile time through static assertions. It doesn't help that boost::pfr's implementation and documentation don't always agree on what the requirements are, as of 2023:
// @+'tests'
// boost::pfr docs say: aggregates may not have base classes
static_assert(6 == boost::pfr::tuple_size_v<not_simple_aggregate4>); // works fine, even though the docs say it's not allowed, although arguably this should return 2?
// docs say: aggregates may not have constructors
struct not_simple_aggregate5 { float f; not_simple_aggregate5(float a, float b) : f{a + b} {} };
static_assert(not std::is_aggregate_v<not_simple_aggregate5>); // a class with a constructor is not aggregate
// auto failure = boost::pfr::structure_to_tuple(nope); // static assertion failure
// @/
In most cases, boost::pfr seems to work when std::is_aggregate_v is true, and to fail when it is not. The only exceptions we noted in our tests are when T is scalar, such as a single float, when T is a union, and when T has any base class. We can easily detect when T is scalar or union using std::is_scalar and std::is_union respectively. However, it is challenging to detect when T has a base class. For now, we defer this issue to boost::pfr, which will in any case raise a static assertion failure if we try to pass it a type with a base class, giving us the following incomplete-but-likely-sufficient implementation of our SimpleAggregate concept:
The main limitation we encounter due to our incomplete test of simple-aggregate-ness is that we cannot admit nested aggregates of endpoints in case any of our endpoints have inherited a class type; such non-base endpoints pass our SimpleAggregate concept, but trigger static assertion failures from boost::pfr. For this reason, we currently assume that endpoint containers are not nested.
Component Basics
Using our small function reflection library we can tell whether a component has a main subroutine by checking whether the return type of the expected methods (T::main or T::operator()) have the expected type (void). This concept will not be satisfied if T::main is a variable, since function reflection is impossible in this case and checking the return type is thus an error.
As well as detecting the existence of a component's main subroutine, it may also be necessary to access its type for function reflection. We provide simple reflection template structs inheriting most of their functionality from the function_reflection template developed in the function concepts document.
Along with the has_name concept defined in the metadata concept header, this completes the specification of basic component requirements.
Inputs and Outputs
To detect whether a component inputs or outputs, we employ the same structure parameterized on the expected name of the nested simple aggregate structure. We use a macro to avoid repetition. A similar pattern is used to define has_name.
An assembly is a simple aggregate struct that contains only components. To validate the second requirement, first we insist that the type must be a simple aggregate. Then we generate a type list of its members using boost::pfr. Using boost::mp11, we then check if all of these members are components or assemblies. If so, we have an assembly.
using tup = decltype(boost::pfr::structure_to_tuple(std::declval<T&>()));
using valid_components = boost::mp11::mp_transform<assembly_predicate, tup>;
using all_valid = boost::mp11::mp_apply<boost::mp11::mp_all, valid_components>;
staticconstexprbool value = all_valid::value;
};
}
// @/
This is somewhat complicated by the fact that the check is necessarily recursive. The predicate needs to be true for both components and assemblies, and the predicate needs to be used in its own definition. But we don't want our Assembly concept to return true for components (which are not assemblies). We therefore need a seperate more generate predicate that does the recursive check, allowing the final actual concept to weed out components.
The container can be seen as the root of a tree-like structure.
( container // a component container
, ( c1 // a component
, ( inputs // an input endpoint container
, ( in1 // an input endpoint
, in2 // another input endpoint
)
)
, ( outputs // an output endpoint container
, ( out // an output endpoint
)
)
)
, ( c2 // a component
, /* etc, as above */
)
)
Each node in this tree structure has a certain type, such as described in comments above. A set of tag classes and type traits are provided to distinguish between these types at compile time.
Generalising boost::pfr's facility of taking a structure and returning a tuple of references to or copies of its elements, we aim in this section to enable tuple-like semantics for components and assemblies.
In a first attempt, I used boost::mp11 to first generate a type list from a general component. However, this turns out to provide very little use when it comes time to access the nodes of the component-tree. Instead, the first step is to generate a runtime tuple of references. This runtime structure can be manipulated more fluently using a combination of runtime programming and template metaprogramming, and if compile-time execution is required, the transformations can generally be constexpr or constinit and thus executed at compile-time, or easily converted into pure type-generating metaprograms using e.g. std::declval and decltype.
Non-standard Tuple Support
Component Tree
For starters, we define a function component_to_tree that generates a tuple of nested tuples that preserves the tree-like structure of an assembly, augmenting our references with a tag type that annotates what kind of object a node represents (e.g. component or endpoint), as in the following example.
Attention
Each element of the tree is a tuple whose first element is a node and subsequent elements are that node's subtrees as tuples of the same form. Leaf nodes are thus represented as tuples containing a single element.
Helper struct for defining recognized tags. This is immediately undefed, so don't try to use it!
Definition sygah-endpoints.hpp:238
First consider the case where we have been passed an assembly T& component. component_to_tree should return something of the form: <head, <subtree>, <subtree>, ...> where the head of the tuple is simply the component tagged node::assembly, and each subtree in the tail of the tuple is the tree for one component or subassembly. To make the tail, we can get a tuple of references to the subcomponents using structure_tie. Then using tuplet::tuple::map we can turn each subcomponent reference into its corresponding subtree, completing the tail. We then must use tuple_cat to unpack the tail, which is one tuple containing all of the sublists, so that we will have < head, <subtree>, <subtree> ... > where the tail is zero or more subtree tuples, instead of < head, < <subtree>, <subtree>, ...> > where the tail is a single tuple containing the subtree tuples.
// @='component container tree case'
auto subcomponents = sygaldry::pfr::structure_tie(component);
auto head = tpl::make_tuple(tagged<node::assembly, T>{component});
auto tail = subcomponents.map([](auto& subcomponent)
{
return component_to_tree(subcomponent);
});
return tpl::tuple_cat( head, tail);
// @/
We could access the head or tail with the following methods:
// @+'tests'
TEST_CASE("sygaldry tuple head and tail")
{
struct {
int a;
int b;
int c;
} x = {0,0,0};
auto tup = sygaldry::pfr::structure_tie(x);
auto head = tuple_head(tup);
static_assert(std::same_as<int, decltype(head)>);
auto tail = tuple_head(tup);
auto empty_tuple = tpl::tuple<>{};
auto empty = tuple_head(empty_tuple);
}
// @/
Now consider the case of a regular component. The subtree has the form: < head, <maybe inputs>, <maybe outputs>>. We have to work around the possibility that inputs and outputs may not exist. Our strategy is to always tuple_cat several tuples, where the inputs and outputs tuples may be empty tuples, which will vanish during the tuple_cat operation. One awkward consequence of this strategy is that the head of the tree has to be a tpl::tuple before being passed to tuple_cat.
The tree structure is useful for certain applications, but most of the time it is more convenient to work with a flat tuple. The following function flattens a given node tree, returning such a flat tuple.
Flattening the tree is a simple matter of recursively peeling apart the input. The function aims to return a single flat tuple. This is achieved using tuple_cat to join a flat tuple made from the head of the input with a flat tuple made from the end. In case the head is an element, and not a nested tuple, a flat tuple is made by simply wrapping the element in a tuple. Otherwise, the function recurses, so that the nested head tuple will be peeled apart in successive calls until an element is found and recursion can stop. Similarly, the tail of the input is always passed down recursively, so that it can get the same treatment for its new head element. Thus the recursive calls to component_tree_to_node_list(head) can be seen as doing the actual flattening, while the calls to component_tree_to_node_list(tail) merely continue along the tree.
Note that, although labelled constexpr, this operation may not be very efficient. It hasn't been verified whether or to what extent the compiler is able to optimize this operation into oblivion. In the test case below, compiled without optimizations enabled, where the flattened tuple is declared there appear about 18 pairs of lea, mov instructions (one pair for each element of the tuple?), suggesting that the compiler was able to compute the flattened tuple at compile time, and all it does at runtime is move the computed data structure into local memory; this is certainly encouraging, but likely relies on the declaration of the flattened tuple as constexpr.
Also note that this implementation is only possible based on the assumption that the type of every element in the tree will be unique (from the perspective of the type system), so that the return type of any two invocations of component_tree_to_node_list are guaranteed to be different, meaning there's never an issue with inconsistent deduced return types. If this assumption ever becomes an issue, it may be possible to enforce it by passing arbitrary unique template parameters, such as integer constants, to each recursive call to component_tree_to_node_list.
Notice also that, although the flattened tuple is declared constexpr, we are still able to extract a mutable reference to our component data from it. Neat!
A flat tuple is easy to filter. We can pass a predicate metafunction to the filter, and then use tup.map from tuplet to transform the flat tuple into one of wrapped values or empty tuples based on the predicate, then tpl::apply the result to tuple_cat to get a filtered tuple out.
A search can be construed as a filtering operation. Here we leverage node_list_filter to find elements with a particular type within our flattened tuple of tagged component tree nodes. Based on the assumption that every node in the tree has a unique type, this is a way of finding a particular node in the tree. A shortcut is provided for working directly from a component.
In the following test, only_in1 is fully inlined by the compiler. No runtime computation is required; the address of the variable is simply loaded from memory.
We can filter for a particular type of node (e.g. input endpoints or parts containers or components) in a similar fashion. With the definition of a for_each_node_in_list function that runs a callback for each node in a component node list, this provides a succinct method to visit each node that matches one of the node types in a variadic list of node tag template type parameters, allowing the user to select which types of nodes they wish to visit by first filtering the list.
// @+'tuples of nodes'
template<typename ... RequestedNodes>
struct _search_by_tags
{
template<typename Tag> using fn = boost::mp11::mp_contains<tpl::tuple<RequestedNodes...>, typename Tag::tag>;
};
template<typename ... RequestedNodes>
constexprauto node_list_filter_by_tag(Tuple auto tup)
We model the path of a named node as a tuple whose first element is the root of the component tree in which the node is found, whose last element is the named node, and whose intervening elements are the named parents of the node in order.
// @+'tests'
auto in11_path = path_of<in11>(component_to_tree(accessor_test_container));
Generating a path, given a node's (untagged) type and a component tree, is similar to flattening the tree while searching for a particular node.
The input to the search is a tuple representing a part of the overall tree of components. If the tuple is empty, this is the tail of a leaf node of the tree. We return the empty tuple unchanged.
Otherwise, we split the tuple into its head and its tail. If the head is a node that matches the node that we are looking for, we return a tuple containing it. Otherwise, if the head is a different node, we search for the sought node in the tail of the tuple. If it is found, this search will return a non-empty tuple. If the current node is named, we prepend the current node, which is a named parent of the sought one, to this tuple, and return it, or else simply return the found sub-path. Otherwise we return an empty tuple, since this node is not part of the path.
Finally, if the head is a tuple, then we return the tuple_cat of the recursive search over the head subtree and tail subtree(s). If any of these find the sought node, the result of the concatenation will be a non-empty tuple. Otherwise, it will be an empty tuple.
// @+'tuples of nodes'
template<typename T, Tuple Tup>
constexprauto path_of(const Tup tree)
{
ifconstexpr (std::tuple_size_v<Tup> == 0) // tail of a leaf node
elsereturn tpl::tuple<>{}; // node is in another subtree
}
template<typename T, typename C>
requires Component<C> || Assembly<C>
constexprauto path_of(C& component)
{
return path_of<T>(component_to_tree(component));
}
// @/
Untagged List
// @+'tests'
auto outputs = remove_node_tags(node_list_filter_by_tag<node::output_endpoint>(component_tree_to_node_list(component_to_tree(accessor_test_container))));
Often it's not necessary to access a value instance of a node list, but rather it's tpl::tuple type, i.e. a type list. The following template type aliases are provided to facilitate direct access to useful type lists.
As an alternative to the node-list based approach above, the following for_each implementation directly iterates over a component tree with compile time branching. This approach provides nearly identical performance in informal benchmarks, compared to for_each_node_in_list, provided that the list in the latter case is generated at compile time, i.e. declared constexpr.
for_each_node runs the user-provided callback function for each of these nodes. To help the callback distinguish between a component container, component, input or output endpoint aggregate, parts aggregate, or endpoint, an empty tag class is passed as the second argument to the callback. Some helpers are provided to facilitate compile-time branching depending on the type of node.
The function itself takes a variadic list of these tags, allowing the user to select which types of nodes they wish to visit. The default, in case no tags are provided, is to visit every node.
// @+'for each'
template<typename T, typename ... RequestedNodes>
constexprauto for_each_node(T& component, auto callback)
{
using boost::mp11::mp_list;
using boost::mp11::mp_contains;
using boost::mp11::mp_empty;
using nodes = mp_list<RequestedNodes...>;
@{for each component container case}
@{for each component case}
}
// @/
In case the input is a component container, we optionally call back for said container, and then recurse for each contained component.
The pattern is almost identical for each of the optional aggregates of a component. For endpoint containers, we can skip the iteration over the aggregate if the user has not requested the corresponding type of endpoints be visited. However, we always recurse over parts in a parts container since the user must wish to visit some of its nested nodes, if any exist.
Many components have endpoints with message semantics, e.g. by its value having semantics similar to a pointer or std::optional. These values are expected to be cleared so that they have a false interpretation unless the value has been set just before or during the current call to the component's main subroutine. To facilitate bindings implementing this behavior in a consistent way, the following subroutine is implemented, that reflects over a component's endpoints and clears all of its endpoints with this semantics.
A test is not provided for this function here, since its use in the CLI binding is well tested.
The initialization and main subroutines of components can be generically triggered using the following functions. Overloads are also provided that will initialize or activate all components in a component container, recursively. This currently includes parts in subassemblies, but TODO it probably shouldn't.
1.9.8