It could be a sign the diaphragm is going out, but without disassembling it and inspecting it there is really no way to tell other than the output. I can't say about that particular air pump, but all the one's I've had any experience with disassemble very easily. I'm not sure if it's because they are generally very simple designs or if they were designed that way. If you aren't comfortable taking it apart, you can do some basic troubleshooting without doing it. Remove it from the air line and plug it in. Listen to the tone of the humming running with no obstructions. Once you get an "ear" for the sound or tone of it running, plug it with your finger. You should immediately hear it start to bog down as it's loaded. If it doesn't bog down much or at all, you are in need of either the diaphragm or the valve, possibly both.
I'm looking at youtube to see if I can find a video for you for a basic diaphragm swap...
I can't say for sure, but this looks to be closest to the type of pump you have: