Random initialization in pre-trained GNN
Published:
Summary: The random initialized embeddings is equivalent to multiple a random matrix to the original feature matrix. The value of each dimension in generated embeddings are the weighted summation of values of each dimension in original features. In the random initialization, the embeddings are still distinguishable since the ‘weights’ are randomly generated and not identical. In ogb datasets with enough training data, the downstream classifier can still fit it.