When do I get my first mount?

Short Answer

You need to have done the level 20 MSQ quest "The Company you keep", to unlock the quest "My Little Chocobo", that will give you access to the chocobo mount and any other mounts you may already have.

To complete "My Little Chocobo" you will need 200 Seals to buy the Chocobo Issuance from the Grand Company you selected. To get these 200 seals you need to finish the MSQ quest that follows the one you just did.

Until you have done this quest, you will not be able to use mounts, even the ones bought from the Online Store.