Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

step-wise patch embedding的实现 #11

Open
LindgeW opened this issue Jun 17, 2023 · 2 comments
Open

step-wise patch embedding的实现 #11

LindgeW opened this issue Jun 17, 2023 · 2 comments

Comments

@LindgeW
Copy link

LindgeW commented Jun 17, 2023 •
edited
Loading

你好，请问论文中所说的step-wise patch embedding的实现具体体现在哪里呢？是通过不同stage设置不同patch size的patch embedding来体现的吗？
@danczs
Copy link
Owner

danczs commented Jun 18, 2023

你好，是这样的，step-wise patch embedding就是每一步做embedding时只做尺度为2x2大小的patch embedding，具体就是每次使用步长为2，卷积核大小为2的卷积，类似于CNN中的下采样层，同时通道数加倍。结果就是原来8x8的patch embedding被多个2x2的patch embedding替代。

@LindgeW
Copy link
Author

LindgeW commented Jun 22, 2023 •
edited
Loading

多谢回复！

请问一般overlapping的cnn stem更好，还是(你们的)non-overlapping的更好呢？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@danczs @LindgeW