Walking away is the best tactic, good job! You never know what that person is carrying or who is his "backup".
A couple of things to keep in mind for NYC, pepper spray while it can be legal for self defense is very clearly regulated in size, strength and where it was purchased.
When it comes to the infamous "gravity knife" I have spoken to police officers who say the rules allow them to flick open even a leatherman wave when holding the blade. It doesn't matter if they are wrong, YOU are the one inconvenienced by dealing with the judge and cost of court. This is not done maliciously, a lot of police officers do not know the ins and outs of every law on the books.