Some guesses;
Considering that PD is rather like us in that it's cross-platform and for realtime stuff I'd predict that PD also uses RTAudio so it might not be so surprising that it behaves the same way. It might be a logical next step to try something realtime that uses a different library.
Another possibility: if any one random program is taking a lot of CPU then there will be audio hickups but that needn't mean the audio system itself has a problem. Did you check how CPU usage is distributed?
The world's history is full of incidences of blaming something (or preferably somebody) else first before doing a lot of work yourself so let's try if we can do this as well :¬).
Kas.