> LLMs reduce a very high-dimensional vector space into a very low-dimensional vector space.
What do you mean? There is an embedding size that is maintained constant from the first layer to the last. Embedding lookup, N x transformer layers, softmax - all three of them have the same dimension.
Maybe you mean LoRA is "reducing a high-dimensional vector space into a lower-dimensional vector space"
What do you mean? There is an embedding size that is maintained constant from the first layer to the last. Embedding lookup, N x transformer layers, softmax - all three of them have the same dimension.
Maybe you mean LoRA is "reducing a high-dimensional vector space into a lower-dimensional vector space"