Visual Studio Code – vscode can't open certain folder with remote ssh

i am able to remote-ssh to my vm with vscode, however there are some folder I used to be able to open but now i get ssh timeout, however some other folder works fine.

any idea?

e.g
~/code/folder_a —-ok
~/code/folder_b —-ssh timeout

if I rename folder_b to something else, I then can open it, if I rename it back to folder_b, still get timeout.