You use the little rubber stick to clean clogged belts of wood, leather, plastic. Belts don't clog much from steel.
You use a wheel dressing stone (I use a diamond dresser or rake it with the edge of lathe bit) to break up dulled grits on a dull belt to expose fresh edges. Usually, once a belt is dull, it is also spent. But, if you've been using it at high speed and light pressure it is possible it is only glazed and has more life left in it if you dress it. This normally only applies to lower grit belts and structured abrasive belts.
If the belt is designed to run in either direction, sometimes you can just take it off and reverse it to find sharp grit.
Around here we're taught to treat belts like they're free and discard them once they start to dull. This is because pressing a dull belt is where mistakes occur. Sometimes these belts have a little more life left in them if you dress them, depending on how hard the abrasive is and how you have been using it.