Will asteroid bases appear as quantum markers always, or is exploration mission, or is exploration stuff necessary to travel with them?

No, they will be always there, always present for you. As I said, you can also go to them if there's no mission running even the mission sandbox you want to encounter asteroid bases you can travel there without requiring a mission. There will be AI population regardless. The mission stuff will add on top of that. But all of them you can see from the get-go. There might be plans for the future.