apedec: check output buffer size after calculating actual output size