Finished section 8.1

bib/main.bib

@@ -598,6 +598,16 @@ @InProceedings{hajij2022high
   year      = {2022},
 }
 
+@Article{hamilton2017representation,
+  author  = {William L. Hamilton and Rex Ying and Jure Leskovec},
+  journal = {IEEE Data Engineering Bulletin},
+  title   = {Representation Learning on Graphs: Methods and Applications},
+  year    = {2017},
+  number  = {3},
+  pages   = {52--74},
+  volume  = {40},
+}
+
 @Article{hanocka2019meshcnn,
   author    = {Hanocka, Rana and Hertz, Amir and Fish, Noa and Giryes, Raja and Fleishman, Shachar and Cohen-Or, Daniel},
   journal   = {ACM Transactions on Graphics},

rmd/07-push-forward-and-pooling.rmd

@@ -50,7 +50,7 @@ knitr::include_graphics('figures/image_pooling.png', dpi=NA)
 
 ```{proposition, image-pool, name="Realization of image ppooling"}
 An image pooling operator can be realized in terms of a push-forward operator from the underlying image domain to a 2-dimensional CC obtained by augmenting the image by appropriate 2-cells where image pooling computations occur.
-\end{proposition}
+```
 
 ```{proof}
 The proof is a straightforward conclusion from the definition of image pooling.

rmd/08-hasse-graph-interpretation.rmd

@@ -98,3 +98,30 @@ knitr::include_graphics('figures/hasse_graph_pooling_scaled.png', dpi=NA)
 ### Augmented Hasse diagrams, message passing and merge nodes
 
 The difference between constructing a CCNN using the higher-order message passing paradigm given in Section \@ref(definition-of-higher-order-message-passing) versus using the three elementary tensor operations given in Section \@ref(the-main-three-tensor-operations) has been demonstrated in Section \@ref(merge-nodes-and-higher-order-message-passing-a-qualitative-comparison). In particular, Section \@ref(merge-nodes-and-higher-order-message-passing-a-qualitative-comparison) mentions that merge nodes naturally allow for a more flexible computational framework in comparison to the higher-order message-passing paradigm. This flexibility manifests in terms of the underlying tensor diagram as well as the input for the network under consideration. The difference between tensor operations and higher-order message passing can also be highlighted with augmented Hasse graphs, as demonstrated in Figure \@ref(fig:hasse-diagram-examples). Figure \@ref(fig:hasse-diagram-examples)(a) shows a tensor diagram obtained from a higher-order message-passing scheme on a CCNN. We observe two key properties of this CCNN: the initial input cochains are supported on all cells of all dimensions of the domain, and the CCNN updates all cochains supported on all cells of all dimensions of the domain at every iteration given a predetermined set of neighborhood functions. As a consequence, the corresponding augmented Hasse graph exhibits a uniform topological structure. In contrast, Figure \@ref(fig:hasse-diagram-examples)(b) shows a tensor diagram constructed using the three elementary tensor operations. As the higher-order message-passing rules do not impose constraints, the resulting augmented Hasse graph exhibits a more flexible structure.
+
+### Higher-order representation learning
+
+The relation between augmented Hasse graphs and CCs given by Theorem \@ref(thm:hasse-theorem) suggests that many graph-based deep learning constructions have analogous constructions for CCs. In this section, we demonstrate how *higher-order representation learning* can be reduced to graph representation learning [@hamilton2017representation], as an application of certain CC computations as augmented Hasse graph computations.
+
+The goal of graph representation is to learn a mapping that embeds the vertices, edges or subgraphs of a graph into a Euclidean space, so that the resulting embedding captures useful information about the graph. Similarly, higher-order representation learning [@hajijcell] involves learning an embedding of various cells in a given topological domain into a Euclidean space, preserving the main structural properties of the topological domain. More precisely, given a complex $\mathcal{X}$, higher-order representation learning refers to learning a pair $(enc, dec)$ of functions, consisting of the *encoder map* $enc \colon \mathcal{X}^k \to \mathbb{R}^d $ and the *decoder map* $dec \colon \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$. The encoder function associates to every $k$-cell $x^k$ in $\mathcal{X}$ a feature vector $enc(x^k)$, which encodes the structure of $x^k$ with respect to the structures of other cells in $\mathcal{X}$. On the other hand, the decoder function associates to every pair of cell embeddings a measure of similarity, which quantifies some notion of relation between the corresponding cells. We optimize the trainable functions $(enc, dec)$ using a context-specific *similarity measure* $sim \colon \mathcal{X}^k \times \mathcal{X}^k \to \mathbb{R}$ and an objective function
+\begin{equation}
+\mathcal{L}_k=\sum_{ x^k \in \mathcal{X}^k     } l(  dec(  enc(x^{k}), enc(y^{k})),sim(x^{k},y^k)),
+(\#eq:loss)
+\end{equation}
+where $l \colon \mathbb{R} \times \mathbb{R} \to \mathbb{R}$ is a loss function. The precise relation between higher-order and graph representation learning is given by Proposition \@ref(prp:convert-graphtocc).
+
+```{proposition, convert-graphtocc, name="Higher-order representation learning as graph representation learning"}
+Higher-order representation learning can be reduced to graph representation learning.
+```
+
+```{proof}
+Let $sim\colon \mathcal{X}^k \times \mathcal{X}^k \to \mathbb{R}$ be a similarity measure. The graph $\mathcal{G}_{\mathcal{X}^k}$ is defined as the graph whose vertex set corresponds to cells in $\mathcal{X}^k$ and whose edges correspond to cell pairs in $\mathcal{X}^k \times \mathcal{X}^k$ mapped to non-zero values by the function $sim$. Thus, the pair $(enc, dec)$ corresponds to a pair $(enc_{\mathcal{G}}, dec_{\mathcal{G}})$ of the form $enc_{\mathcal{G}}\colon \mathcal{G}_{\mathcal{X}^k} \to \mathbb{R}$ and $dec_{\mathcal{G}}\colon \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$. Thereby, learning the pair $(enc, dec)$ is reduced to learning the pair $(enc_{\mathcal{G}}, dec_{\mathcal{G}})$.
+```
+
+[TopoEmbedX](https://github.com/pyt-team/TopoEmbedX), one of our three contributed software packages, supports higher-order representation learning on cell complexes, simplicial complexes, and CCs. The main computational principle underlying TopoEmbedX is Proposition \@ref(prp:convert-graphtocc). Specifically, TopoEmbedX converts a given higher-order domain into a subgraph of the corresponding augmented Hasse graph, and then utilizes existing graph representation learning algorithms to compute the embedding of elements of this subgraph. Given the correspondence between the elements of the augmented Hasse graph and the original higher-order domain, this results in obtaining embeddings for the higher-order domain.
+
+```{remark}
+Following our discussion on Hasse graphs, and particularly the ability to transform computations on a CCNN to computations on a (Hasse) graph, one may argue that GNNs are sufficient and that there is no need for CCNNs. However, this is a misleading clue, in the sense that any computation can be represented by a computational graph. Applying a standard GNN over the augmented Hasse graph of a CC is not equivalent to applying a CCNN. This point will become clearer in Section \@ref(on-the-equivariance-of-ccnns), where we introduce CCNN *equivariances*.
+```
+
+## On the equivariance of CCNNs