VOIPo does not typically proxy its own audio. This means that the RTP stream will usually come from a different IP address than the SIP proxy which was used to set up the call. Some routers, particularly those with symmetric NAT, will drop packets which are sent to the correct address and port, but from a different IP address. That's at least one explanation. I'm sure there are others as well.