Ticket #218 (closed defect: fixed)

Opened 13 years ago

Last modified 13 years ago

Fix memory issue with OLE-based formats

Reported by: melissa Owned by: melissa
Priority: critical Milestone:
Component: bio-formats Severity: major
Keywords: Cc:
Blocked By: Blocking:

Description

Zeiss ZVI, FV1000 OIB, ImagePro IPW and SimplePCI all have significant memory issues due to POI. Right now, POI is structured such that the *entire* file is read into memory - this is less than optimal under any circumstances, but becomes really problematic for files > 200 MB.

As far as I can tell, POI keeps a list of all of the data blocks. We should probably replace this with a list of offsets to the data blocks, and read the actual data on demand. This might be a bit slower, but should save lots of memory.

Change History

comment:1 Changed 13 years ago by melissa

  • Status changed from new to closed
  • Resolution set to fixed

Fixed in r3372. Used the solution described above, which changed POI's API slightly.

For comparison, the overhead for reading an OLE file was (file size) + 50 MB; it's now ~12 MB. Also, there doesn't appear to be a significant speed decrease, which is nice.

Note: See TracTickets for help on using tickets.