Commit Graph
100 Commits
Author SHA1 Message Date
Eliot Jones 79dea8d314 Merge pull request #179 from BobLd/master
Improve ContentStreamProcessor
2020-06-20 12:13:40 +01:00
Eliot Jones bf45602ac5 fix #176, allow startxref to appear earlier in the document 2020-05-31 17:01:38 +01:00
Eliot Jones a7a2ef0630 remove old text from the readme 2020-05-10 16:40:05 +01:00
Eliot Jones 256c2833ab 0.1.2-alpha002 2020-05-10 16:36:14 +01:00
Eliot Jones 0512bb1e4f handle indirect references appearing in cid font widths array #174 2020-05-10 15:46:38 +01:00
Eliot Jones 09b951f667 expose font details on individual letters
also fixes a regression for image extraction
2020-04-25 17:15:26 +01:00
Eliot Jones 98dd736f94 0.1.2-alpha001 2020-04-25 15:20:07 +01:00
Eliot Jones ae62197178 merge pull request #167 from bobld/master
Improve NearestNeighbourWordExtractor
2020-04-25 15:07:18 +01:00
Eliot Jones e264583c21 add merging to the readme 2020-04-25 12:06:11 +01:00
Eliot Jones 19047f62ae fix name output for merged documents 2020-04-25 11:23:37 +01:00
Eliot Jones 391b650e3c add more examples to the examples solution 2020-04-25 10:14:05 +01:00
Eliot Jones 27e251f921 make filter provider and filter public and use tryget for image bytes 2020-04-25 09:42:24 +01:00
Eliot Jones 635c4b4c5e formatting tidy-up 2020-04-25 09:11:16 +01:00
Eliot Jones 48aa9169c5 add whitespace size statistics to api scanner test 2020-04-19 17:10:46 +01:00
Eliot Jones 407ee5ca51 add content order text extractor and example of use 2020-04-19 17:06:34 +01:00
Eliot Jones f18bc0766a #161 handle zero point size by using rotated matrix 2020-04-19 10:28:11 +01:00
Eliot Jones 23087fe38c add extract images example 2020-04-18 18:55:44 +01:00
Eliot Jones 563985e2b4 add examples directory and first example 2020-04-18 18:46:26 +01:00
Eliot Jones 25314cc79d #161 change rotation to fix values and page size
this doesn't account for images and pdf paths yet.
2020-04-18 18:04:41 +01:00
Eliot Jones b122bf0ca6 inline transformation code and cache afm strings 2020-04-18 13:56:39 +01:00
Eliot Jones db442194c3 use a mutable struct 2020-04-18 12:10:17 +01:00
Eliot Jones e382e581ba add merge test for document with object stream 2020-04-16 20:57:57 +01:00
Eliot Jones 635ae13a77 add pdf/a2-a support 2020-04-16 20:50:21 +01:00
Eliot Jones 75c04eb81c fix namespace using order 2020-04-05 17:58:57 +01:00
Eliot Jones 2a0a3fae69 add test for svg exporter and escape xml characters 2020-04-05 17:58:57 +01:00
Eliot Jones 45ac8c8a60 default clipping to false for performance 2020-04-05 17:58:57 +01:00
Eliot Jones 2998b1b2eb tidy up clipper related code 2020-04-05 17:58:57 +01:00
Eliot Jones f1be6634a7 add a bunch more performance improvements
filter provider becomes single instance and no longer has constructor parameters.

tokenizers use list and stringbuilder pools to reduce allocations.

system font finder becomes static to preserve file cache across all documents.
2020-04-05 15:34:47 +01:00
Eliot Jones 7baa18b5dd add stringbuilder pool for tokenizers
we could replace these with spans in the next net core however for now our pools seem to increase performance by reducing gc load.
2020-04-04 18:31:55 +01:00
Eliot Jones cf46230c05 #127 add pdf/a2-b compliance to the builder 2020-04-04 17:49:27 +01:00
Eliot Jones 729234477a fix issue with null encodings for cid fonts 2020-04-04 17:30:06 +01:00
Eliot Jones 9abe9f4b2f #158 add strong naming to the solution 2020-04-04 16:59:51 +01:00
Eliot Jones 7f1bf094bc #127 pdf/a-1a compliance
adds struct tree and markinfo dictionaries to support pdf/a-1a compliance.
2020-03-29 17:55:02 +01:00
Eliot Jones 5f45ee53bd #127 add basic pdf/a-1b level compliance to the document builder
adds color profiles/output intents and an xmp metadata stream to the document in order to be compliant with pdf/a-1b (basic). this compliance level is toggled on the builder since it will generate larger files and set to 'off/none' by default. pdf/a documents are also not able to use standard fonts so using a font when the compliance level is not none will throw.
2020-03-29 16:43:52 +01:00
Eliot Jones 4ed1600cab version 0.1.1 2020-03-18 20:10:51 +00:00
Eliot Jones 0f91017613 fix issue with newlines in object start tokens #88
where we brute force the file and it contains newlines between object tokens we fix the parsing to prevent pseudo-infinite loops.
2020-03-17 20:09:47 +00:00
Eliot Jones 5094f9d9d0 remove debugging code
this code was used to check content streams but should not be committed.
2020-03-16 19:32:57 +00:00
Eliot Jones 98bcc16e11 fix width and height order in jpeg parsing
height is before width, incorrect order caused adobe reader to draw image strangely.
2020-03-16 19:32:57 +00:00
Eliot Jones 7212b9e38c enable re-use of jpeg images between or within pages
returns a reference to the added image object when calling addjpeg so that it can be shared between or within pages meaning the image is only written to the output file once but can appear multiple times.

this image doesn't seem to be displaying correctly in adobe reader.
2020-03-16 19:32:57 +00:00
Eliot Jones 19462d79f0 add support for jpeg images in pdf document builder
since jpegs can be trivially embedded in pdf documents without changes to the data stream this is the first image format we will support. currently this is a naive approach which doesn't share an image resources between pages. ideally we will either de-duplicated images when added, return a re-usable key once an image is added, or both.
2020-03-16 19:32:57 +00:00
Eliot Jones 8ac4195b83 0.1.1-alpha001 2020-03-15 16:52:28 +00:00
Eliot Jones 908d84ccc6 remove debug code from the test
i accidentally left this in when debugging the pdf merging.
2020-03-15 16:29:18 +00:00
Eliot Jones aa9df30722 handle invalid charstring sequences
it is possible for a file with an adobe type 1 font to contain an invalid charstring sequence, if this happens we handle it and return false from trygenerate.
2020-03-08 14:33:26 +00:00
Eliot Jones 8df2f9cf6b generate all xml docs and pack them #148
after we split the solution into multiple projects the xml doc comments were no longer packed in the generated nuget package. in addition they were only generated for the net standard 2.0 target framework.

this change generates comments for all target frameworks and makes sure they're included in the generated package. it also adds missing doc comments where they weren't included on the public api and clears up a couple of minor formatting issues in the affected files.
2020-03-08 13:44:09 +00:00
Eliot Jones 24c5cbea4b support custom page sizes for document builder #147
page size custom is not supported for the document builder so a new overload which supports user defined page sizes is provided.
2020-03-07 16:48:19 +00:00
Eliot Jones bef68a0654 avoid infinite loop in brute-force searcher #88
fixes the case where the brute-force searcher becomes stuck in an infinite loop, it may be the case that the problem pdf from #88 has a newline or some other whitespace between its object and generation number so this may cause a failure elsewhere.
2020-03-03 15:49:17 +00:00
Eliot Jones 58972de7cb begin to rework cross-reference parsing
most of the cross-reference code is the earliest code in the project and hasn't been revisited since then. the issue #88 has been reopened due to a bug with brute-force searching so this tidies up the code in this area ahead of trying to fix the bug.
2020-03-03 15:21:11 +00:00
Eliot Jones 4b5c8d510e add test for comment in dictionary from #145
check that we correctly handle the case where a comment appears inside a dictionary, this was handled by commit 3084a9. use list internally to dictionary tokenizer to avoid interface performance penalties.
2020-03-03 11:36:01 +00:00
Eliot Jones 14599521f5 test the simple document merge in reverse order
check we can create the document in both merge orders.
2020-03-02 17:00:16 +00:00
Eliot Jones 2effedd3c5 add inheritable keys back into the copied pages node
keys suchs as resources, mediabox, cropbox, etc can be inherited. we now copy them if they are present on the parent pages node.
2020-03-02 17:00:16 +00:00
Eliot Jones c596bef024 rename to resharper conventions and test 2nd page
renames fields to match the expected conventions for resharper. removes fully qualified names for using statements since resharper marks these as not-required.

adds a check to the pdf merger test to make sure the second page has the expected content. this is not currently valid since we lose the resources node on the pages tree.
2020-03-02 17:00:16 +00:00
Eliot Jones 2193063809 fix tests for public api and merge conflict
the cross reference parser tests behaviour had change, this fixes a compilation error from merge conflicts. also updates the merger tests to account for new version behaviour and checks the output document text. adds pdfmerger to the public api in the tests.
2020-03-02 17:00:16 +00:00
Eliot Jones b7a86f482f cache bounding boxes for composite fonts
cached the bounding box for a specific character code value for type 0 (composite) fonts to improve performance.
2020-02-28 16:36:06 +00:00
Eliot Jones 4442a69a97 use tryget rather than lambdas for union type
avoid the allocations caused by lambda expressions for performance reasons.
2020-02-28 16:02:20 +00:00
Eliot Jones 4d911fb9d1 use transform x for widths to improve performance
when transforming the advance width inside a font, we transform only the x coordinate rather than making a new point to transform.
2020-02-28 15:15:35 +00:00
Eliot Jones c864fa512c remove islenientparsing from page classes 2020-02-28 11:50:18 +00:00
Eliot Jones 48d166276d remove islenientparsing from contentstreamprocessor 2020-02-28 11:44:13 +00:00
Eliot Jones 6fdaf054cb remove islenientparsing from annotation provider 2020-02-28 11:39:56 +00:00
Eliot Jones 7b09999a3f remove islenientparsing from the font handlers
we're removing islenientparsing to make the code simpler to maintain and use as well as more resilient.
2020-02-28 11:37:18 +00:00
Eliot Jones 746cbfa30c remove lenient parsing from font related classes
lenient parsing gives us more code to maintain for no real benefit, parsing should always be as lenient as possible. remove the flag from some of the font code.
2020-02-27 18:10:02 +00:00
Eliot Jones ec9e425712 use length from stream dictionary if directly available
when brute forcing we use the length available in the stream's dictionary token if it is a direct number rather than an indirect reference.
2020-02-27 17:17:49 +00:00
Eliot Jones 420daaac6e handle unbalanced parentheses for string tokenization
when the close parenthesis is unbalanced and precedes a line break followed by '/' or '>' we assume the bracket to be unbalanced and finish reading the string.
2020-02-27 17:01:15 +00:00
Eliot Jones f7cabe5d12 ignore invalid postscript format type truetype
when reading the format type of a postscript table in a truetype font ignore an invalid format value.
2020-02-27 16:10:19 +00:00
Eliot Jones c033a0b3fe handle missing end bytes for cmap code
when the byte input does not contain a code of the full code length we return early.
2020-02-27 15:26:48 +00:00
Eliot Jones 25b6f1d46d change set version script to use correct path
rather than directory relative to the working directory use the path relative to the tools folder.
2020-02-27 13:29:15 +00:00
Eliot Jones f2d94413dc create script to increment project versions
each time we want to up the version number of the nuget package it involves opening every csproj and manually updating the version. this script updates the version for all projects, except the test project, in the 'src' folder.
2020-02-27 13:24:26 +00:00
Eliot Jones 0fcc4e54c8 add istestproject setting to all projects
indicates which projects are test projects to the test runner.
2020-02-27 12:35:40 +00:00
Eliot Jones 4150881be9 recover from invalid acro-form references
we add a try/catch to the direct object finder's tryget method so it returns false rather than throwing.

if we have an acro-form reference in the catalog but no corresponding object in the document we instead scan all objects in the document to find form fields and reconstruct the acro-form dictionary.
2020-02-27 12:08:40 +00:00
Eliot Jones f415c3116e cross reference offset is in the xref table we ignore the error
previously we checked the offset was not inside the table (correct thing to check), however this is only a special case of the more general issue (cross reference offsets are wrong). we move handling for this into the pdf token scanner. if we attempt to read an object at an offset and it fails we brute force the entire file to find correct offsets. we also needed to add handling to make sure we don't attempt to use stream length tokens if we're brute-forcing since we can't look up indirect references for length.
2020-02-26 14:03:46 +00:00
Eliot Jones 7d0d5806a9 fix reverse xref location search
when brute force searching for the start of the cross-reference table (xref) we read 5 byte buffers, previously if the 'x' of 'xref' was the first character of the buffer we skipped it. this checks when 'x' is the first character of the buffer.
2020-02-26 12:55:11 +00:00
Eliot Jones f07e2dfb84 more tolerant handling of endimage recovery
fixes the recorded offset when an endimage is recovered from the first time. it was off by one so if the subsequent end image was also the wrong tag then the second attempt at recovery failed.

also allows recovery when other tags appear after an endimage as long as they're not block ending operations (end text, perhaps pop/push in future).
2020-02-26 12:41:39 +00:00
Eliot Jones 43afac8f5d default to zero width characters in truetype for '.notdef'
when the character is not defined and the corresponding '.notdef' glyph isn't included in a truetype font we now default to a zero width character. it might be that we should use the default/missing width instead but this will work ok for most use-cases.
2020-02-26 12:39:12 +00:00
Eliot Jones 50c17f7951 make compact font format parser thread safe
the individual cff parser uses a cff dictionary reader inside it which has a per-instance operands list, for this reason it is not thread-safe and cannot be shared. this change creates a new individual font parser for each call to the top-level cff parser.
2020-02-25 14:24:29 +00:00
Eliot Jones 9f488809ac #141 cast adobe type 2 char string value to short
where the value is 28 the next two bytes indicate a short, not a 16 bit two's complement number, apparently, or i've misunderstood what the two's complement bit is about...
2020-02-25 13:56:26 +00:00
Eliot Jones 486ea446c5 #141 divide width by 1000 for adobe type 1 font
the width shouldn't be transformed by the font's matrix, instead the width is divided by 1000 by default.
2020-02-25 13:44:15 +00:00
Eliot Jones f7c6de4118 #141 fix two's complement in adobe type 2 charstring
the byte value of 28 indicates the next 2 bytes are a 16 bit two's complement number rather than just a short. this changes the calculation to generate the two's complement value correctly.
2020-02-25 13:19:47 +00:00
Eliot Jones d6d3869fe2 fix brute force searcher offsets
the brute force searcher offsets were off by one. this change means the offset returned is now aligned with the object number in the object number/generation/operator triple.
2020-02-24 12:24:18 +00:00
Eliot Jones 7ac3fb2a39 remove old clipping rule code and make enum byte
removes the unused set winding rule method and makes the clipping rule enum a byte which will save 3 bytes per pdfpath instance.
2020-02-24 11:29:06 +00:00
Eliot Jones c6dc4d9eb8 handle tokenizing invalid numeric string correctly
rather than throwing when an invalid numeric string is read, our tokenizer now returns false so that error recovery methods can be attempted.
2020-02-21 11:16:31 +00:00
Eliot Jones 8d415fd162 use type 1 font handler for mmtype1
multiple master fonts are an extension of the adobe type 1 font format. we don't have any special case handling for them so for now we default to attempting to use the adobe type 1 font handler. it may be that we need some special parsing logic but the test file using the mmtype1 fonts didn't actually specify any font bytes so we can't check.
2020-02-21 10:49:29 +00:00
Eliot Jones 28faf1c22c default to .notdef for type 2 charstrings
if the glyph with a specific name isn't found in the set of type 2 charstrings we default to using the .notdef glyph if present.
2020-02-21 10:37:58 +00:00
Eliot Jones c635b919d2 make numbers culture invariant in document builder
decimal numbers were dependent on the current thread culture for the output file. this meant values like '70.679' were output as '70,679' for cultures using a comma rather than period separator for the floating point (i.e. the whole world). this resulted in the file displaying incorrectly.
2020-02-20 13:06:12 +00:00
Eliot Jones ed894baffc Merge pull request #135 from BobLd/rxycut-stackoverflow
Fix RecursiveXYCut
2020-02-15 10:52:20 +00:00
Eliot Jones f3fcd1b3a1 ignore form dictionaries that do not contain fields #131
though a form dictionary should always contain fields (as required by the spec) it is possible for this entry to be missing. in this case we return false for trygetform.
2020-02-05 10:56:01 +00:00
Eliot Jones 40dc80c281 handle type 1 font with no descriptor information #132
though required by the spec an adobe type 1 font may be missing all width data. in this case we default to empty values and treat it like a normal adobe type 1 font.
2020-02-05 10:46:39 +00:00
Eliot Jones 10ca77a034 move values back to computed properties
the additional stored fields made the struct slower to copy and had an impact on the performance. this moves non-essential fields back to computed properties.
2020-01-31 16:24:59 +00:00
Eliot Jones b585fe9581 make width, height, area, rotation and centroid lazy
since the rectangle constructor is a hot path any calculations slow the library down considerably. for this reason we move calculations for the following properties into the property getter:
* width (cached)
* height (cached)
* rotation
* area
* centroid

where values are cached they set their backing field once calculated. this won't be thread safe if the same rectangle is accessed on multiple threads.
2020-01-31 16:24:59 +00:00
Eliot Jones 0e84fa34a8 fix usages of rectangle constructor.
now that rectangle constructor uses the order [ llx, lly, urx, ury ] and does not apply correction for points constructor parameters must be passed in the correct order. this change fixes the hyperlink factory which was passing them in the wrong order.

in addition the pdfpath bounding box was using left, right, top and bottom to calculate the minimum bounding box. this produced incorrect values now individual path operator bounding boxes are rotated, since for a rotated rectangle top may be less than bottom.

the performance seems to have taken a hit due to these changes however.
2020-01-31 16:24:59 +00:00
Eliot Jones 8ab2838063 recover from invalid cross reference position
if we are reading a cross reference offset which contains a number we assumed it was a stream object. if it's not we now brute-force the entire file looking for an 'xref' token. this should be combined with a search for cross-reference streams and should run when we read neither the numeric token or an 'xref' token but for now this fixes the observed issue.

also adds number of images to the page api to prevent consumers needing to enumerate.
2020-01-28 18:07:05 +00:00
Eliot Jones 29061b1fd2 handle unexpected adobe type 1 format
an encoding array in an adobe type 1 font may be missing its declaration ending in 'for', if we encounter 'dup' while looking for the 'for' token we have a special case to go straight into reading the encoding.

also handles a case where the page content stream contains a path-closing operator without any path being active.
2020-01-28 16:05:53 +00:00
Eliot Jones 6292fc256d handle direct font objects in the resource dictionary
fonts can appear as dicitionary objects rather than indirect references in the resource dictionary for a page. if we encounter this we parse and store the font by name for retrieval during content parsing.
2020-01-27 18:07:51 +00:00
Eliot Jones 6cf257a331 strings record encoding used to create them.
in order to recreate the valid bytes for use in decryption it is necessary to know which encoding was used to read a string token. this is because utf16-be encoding has a byte-order marker which should be included in the resulting bytes.
2020-01-26 17:07:58 +00:00
Eliot Jones 693a3d5958 use offset to file header to correct cross references
if the %pdf version header comment is offset from the start of the file the cross reference offsets will also be wrong by that amount. this change updates the cross reference location logic to use the offset from the located version header.
2020-01-26 15:30:20 +00:00
Eliot Jones a561c8954e handle the format header being preceded by nonsense
some files seem to have the format header preceded by large amounts of junk but this appears to be valid for chrome and acrobat reader. this change ups the amount of nonsense to be read prior to the version header.

also makes parsing of the version header culture invariant which may be related to #85.
2020-01-25 16:53:41 +00:00
Eliot Jones d9492ab2f8 handle empty encrypted portion in adobe type 1 font
the encrypted portion of an adobe type 1 font can be empty in which case we default to a blank private dictionary and charstrings set.
2020-01-25 16:41:54 +00:00
Eliot Jones 736f83e227 handle null charstring names
it appears charstring definitions in adobe type 1 fonts can omit the charstring name. in this case we set the name to the string value of the charstring index.
2020-01-25 16:35:08 +00:00
Eliot Jones ba09a13d08 more end image recovery logic
since inline image data may contain the end image "ei" token inside the data stream there's no reliable way to actually determine if we've read all the data. for this reason if we end up with an invalid state parsing operations after we've read the end image token we try to recover by reading from the previous token to the next end image token if any. we supply log information to let the consumer know this is what we're doing. it's still not bullet-proof but it should be good enough.

also support negative page rotation values by adding them to a 360 degree rotation so -90 degrees clockwise is 270 degrees clockwise.
2020-01-25 15:53:08 +00:00
Eliot Jones 3ac8d7ed91 update the github pages site
updates the information on the github pages site for the new api changes. includes some more seo friendly terms to improve discoverability, more engaging images as well as comprehensive code examples to improve onboarding.
2020-01-25 14:36:07 +00:00
Eliot Jones 3243be3ec5 change rectangle drawing logic for tests
support rotated output rectangles in the visual verification tests.
2020-01-22 13:45:52 +00:00
Eliot Jones 0ed4e58556 add test cases for rectangle transforms
our bounding rectangle values still seem to be wrong for rotated letters. this change adds some test cases for common transformation matrix operations on a rectangle, scale, translate and rotate.
2020-01-22 13:28:47 +00:00