I believe I read somewhere that you need to upgrade Dutch's tent first in order to get the Fast Travel to the camp.... but I am not that far into it to confirm.
You need to upgrade Dutch's tent to fast travel at all. I did the upgrade, but I haven't used it yet. But, my understanding is that you can use it to fast travel from the camp to certain locations that you have been to, but you can't fast travel back.