in #1082 and other issues relating to annotations we're running into
constraints of the current model of building a pdf document. currently
we skip all link type annotations, i think we can support copying of links
where the link destination is outside the current document. however the
more i look at this code the more i think we need a radical redesign of
how document building is done because it has been pushed far beyond
its current capabilities, i'll detail my thinking in the related pr in more
detail
in order to remove reflection from the core content stream operators
construction we ensure all types covered by the operations dictionary
have corresponding switch statement support. this moves the remaining
2 operators to the switch statement. fix for #1062
we encountered a fence constructed in the middle of a field for an unknown
reason so we demolished it. i think this was intended to catch flaws in the
parser logic but the reality is in a pdf anything can happen so we no longer
want to catch these issues and this restores a green test run in debug mode.
fix for #915
when adding a page to a builder from an existing document using either
addpage or copyfrom methods the added page's content stream can contain
a global transform matrix change that will subsequently change all the locations
of any modifications made by the user. here whenever using an existing stream
we apply the inverse of any active transformation matrix
there could be a bug here where if you use 'copy from' with a global transform
active, we then apply the inverse, and you use 'copy from' again to the same
destination page our inverse transform is now active and could potentially
affect the second stream, but I don't think it will
* fix colorspace error when form xobject contains a transparency group
when a form xobject contains a reference to a group xobject this can only
be used to change attributes of the transparency imaging model. the old
code was setting the main colorspaces incorrectly causing errors when the
transparency component had a different number of channels. this was
causing #1071 in addition to the failure in file 0000355.pdf of the test corpus
* add master integration tests for corpus group 0000
* tidy up actions
* remove invalid reference in echo
* move new action to different branch
investigating the corpus at
https://digitalcorpora.s3.amazonaws.com/s3_browser.html#corpora/files/CC-MAIN-2021-31-PDF-UNTRUNCATED/zipfiles/0000-0999/
the input file 0000000.pdf contained a utf-16 surrogate pair in an input
defined as usc2. the approach of various parsers varies here, adobe
acrobat seems to hard crash, pdf js returns the same text we now
parse, chrome parses the intended text (2 invalid characters and
"ib exam"). we don't care too much about matching chrome exactly
so doing the same as firefox is fine here
if parsing in lenient mode and encountering a malformed base name
(in this case 'helveticai') we fallback to times-roman as the adobe font
metrics file for a standard 14 font. this aligns with the behavior of pdfbox.
we also log a more informative error in non-lenient modes
this fixes document 0000086.pdf from the corpus
* fix off-by-one and optimize brute force xref search #1078
when performing a brute force xref search we were ending up
off-by-one, update the search to use a ring buffer to reduce
seeking and fix xref detection
* make method testable and add test coverage
* normalize test input on other platforms
* seal circular buffer class
* back-calculate first char if last char and widths present
when a truetype font has a last char and widths array in its font
dictionary the first char can be calculated #644
* fix off by 1 in last char calculation
* use correct bounding boxes for standard 14 glyphs #850
previously every bounding box for type 1 standard 14 fonts was assumed
to start at 0,0 and ignored the bounding box data in the font metrics file.
now we correctly read the glyph bounding box while preserving the
existing advance width values for advancing the renderer position
* update test case for new logic
when copying from a ancestor node of a page's resource dictionary
we were incorrectly writing nested nodes of e.g. /fonts to the root
of the target dictionary, here we write to the intended target node
correctly
align with the behavior of pdfbox and c implementations where
single character final blocks are ignored rather than being written.
also makes the error more informative in case it is ever encountered
again.
add more test cases.
it is possible this is hiding the problem and will move the error elsewhere
but this matches the implementation behavior of the 2 reference
implementations. one other potential source for the error is if pdf supports
'<~' as a start of data marker which i can't find in the spec but wikipedia
says might be possible? without documents to trigger the error i think
this is the best fix for now
when parsing a stream object with multiple endstream tokens
the last parsed token was selected instead of the actual stream
token so instead we just skip all following tokens if the first
is a stream and the following tokens are `endstream` operators
only
when copying various dictionaries from a source document
to the builder any indirect references in the source document
would throw because the code expected the dictionary token
directly. now we follow the list of indirect references until we
find a non-indirect leaf token. also changes the exception type.
the file provided in issue #926 contains the following syntax
in pdf object streams:
```
% 750 0 obj
<< >>
```
currently we read the comment token and skip the rest
however this producer is writing nonsense to the stream.
comment tokens are only valid outside streams in pdf files
so we align to the behavior of pdfbox here by skipping the
entire line containing a comment inside a stream which fixes
parsing this file.
while nightly builds are useful they also cause
a large amount of spam on the main project
nuget. here we try to change the package id
so that it will be hosted as a separate package
while having all the same code and namespaces
this means people can opt into the nightly builds
while keeping the version history of the released
package tidy.
no idea if this will work because actions and
yaml is my idea of hell and is impossible to
debug, but let's give it a go
if skip missing fonts is set we want to read the file
as much as possible so we will also skip any missing
xobjects like images, forms or postscript code
default word extractor consumed words sorted
by descending y value order and had a check for
when the following letter had a value more than 0.5
units different to the current baseline (from first letter)
position. however we were checking if the new value was
more than the current baseline which it could never
be since the letter was always guaranteed to have an
equal or lower y value based on initial sort (since pdf
y dimension runs top to bottom)
if we're parsing a known dictionary (e.g. all keys are required
and there are no additional optional keys) and we encounter
an error we provide the possibility to recover by assuming
a dictionary end token after all required tokens are consumed
if parsing by looking for dictionary end failed due to a format
exception
these issues reported that parsing was failing due to a missing
token being reference in the tounicode entry. since neither
issue included a sample file it's impossible to determine the
right fix accurately, however since the tounicode entry is
optional in the spec we can try being more lenient here, this
might just result in more errors once we try to use the font
but the logger will at least prevent parsing the entire document
failing
this doesn't fix the reported issue since the pdf itself is corrupted on page 8 however it will
allow recovery in some scenarios where text content isn't important.
also adds more informative error when stream unintentionally passed with non zero offset
EliotJones