Taking a quick photo or sly little video of the band is fine...filming the entire set is not. You are there to watch the band not watch the band on your tiny screen. Photos are there to capture a moment in your life and if that moment in your life is pointing a camera at a band then maybe its time to put the camera down and actually experience the band yourself.
Again, there is nothing wrong with using your camera, but if you're holding your hand up throughout an entire gig and obstructing other peoples view you need to stop!!