which program are you using? Im am using ogle right now. I did have some problems with DMA but thanks to lovechild overcoming my ignorance, it works now. Something else you might want to check, dont know if it will be applicable to you, but try setting the xvattributes. There is some buffer in there that does something...wait one.
There seems to be some problems with Xv and nVidia cards (GeForce...).
Setting the attribute XV_DOUBLE_BUFFER to 0 should make the video smooth. You can use 'xvattr' to do this.