Fix h5repacktst (test file generating binary) to address HDFFV-8016 h5repacktst failed in AIX 32bit mode
Description:
Segmentation fault occurred when h5repacktst try to generate a dataset when
system memory is limited to 256MB.
Add error handling to display apporiate message to exit and adjust the test
dataset according to the original size which didn't causing out of memory.
Tested: test was done by limit memory in csh.
jam (linux32-LE), ostrich (linuxppc64-BE)
c11b378c89