mmvideo: Make sure the rle does not write over the frame boundaries