With the fix for including associated streams, this test now finds more text on the first page. I've verified using Aspose.PDF and by viewing the ErcotFacts.pdf file being tested that yes, it was indeed missing part of the text before.
* read last line of ignore file
- do not cancel other matrix jobs if one test fails
- read all lines of the ignore list even if it doesn't end with a newline
- add ignore list for 0008 and 0009
* support missing object numbers when brute-forcing
the file 10404 (ironically) contains not found references with number 43 0
for its info dictionary. changes brute-force code so that objects can be
entirely missing
* fix test since document is now opened successfully but mediabox is broken
* move file parsing to single-pass static methods
for the file 0002973.pdf in the test corpus we need to completely overhaul
how initial xref parsing is done since we need to locate the xref stream by
brute-force and this is currently broken. i wanted to take this opportunity to
change the logic to be more imperative and less like the pdfbox methods with
instance data and classes.
currently the logic is split between the xref offset validator and parser methods
and we call the validator logic twice, followed by brute-force searching again
in the actual parser. we're going to move to a single method that performs
the following steps:
1. find the first (from the end) occurrence of "startxref" and pull out the location
in bytes. this will also support "startref" since some files in the wild have that
2. go to that offset if found and parse the chain of tables or streams by /prev
reference
3. if any element in step 2 fails then we perform a single brute-force over the
entire file and like pdfbox treat later in file-length xrefs as the ultimate arbiter
of the object positions. while we do this we potentially can capture the actual
object offsets since the xref positions are probably incorrect too.
the aim with this is to avoid as much seeking and re-reading of bytes as
possible. while this won't technically be single-pass it gets us much closer. it
also removes the more strict logic requiring a "startxref" token to exist and be
valid, since we can repair this by brute-force anyway.
we will surface as much information as possible from the static method so that
we could in future support an object explorer ui for pdfs.
this will also be more resilient to invalid xref formats with e.g. comment tokens
or missing newlines.
* move more parsing to the static classes
* plumb through the new parsing results
* plug in new parser and remove old classes, port tests to new classes
* update tests to reflect logic changes
* apply correction when file header has offset
* ignore console runner launch settings
* skip offsets outside of file bounds
* fix parsing tables missing a line break
* use brute forced locations if they're already present
* only treat line breaks and spaces as whitespace for stream content
* address review comments
---------
Co-authored-by: BobLd <38405645+BobLd@users.noreply.github.com>
