It is important to clarify a few things:
VoIP calls take about 100Kbps each way. In reality, a little less, and you can play some games with codecs and compression - but to be on the safe side, assume 100kbps.

QoS is not helping if you have insufficient download bandwidth (and with 1Mbps watching HD movies and downloading large files from fast servers might do it for you). If incoming traffic saturates your modem, nothing that you do on your LAN (and QoS is set up on your router - that is, on your LAN) will help. Therefore, QoS will not slow down your Netflix, but you will still have to run around your house asking who is downloading large file. It might have worked before, but all those HD movies are more likely to impact your download capabilities today, and even more tomorrow).

Insufficient upstream bandwidth can be fixed by QoS. It doesn't matter whether you have modem == ATA with router == router or modem == router == ATA with router sequence.