When do I need to start leveling Crafters or Gatherers?

Short Answer

Whenever you want to! There is no pressure to level crafters, as its NOT necessary to be relevant in end-game content.