Compare commits

...

121 Commits

Author SHA1 Message Date
davebrokit
f3e37eafae Introduce IBlock and ILettersBlock interfaces (Round 2) (#1263)
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Common Crawl Tests / build (0008-0009) (push) Has been cancelled
Run Common Crawl Tests / build (0010-0011) (push) Has been cancelled
Run Common Crawl Tests / build (0012-0013) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Tag Release / tag_if_version_changed (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
* Code review changes
- Keep the Bounds property on the image classes so this isn't a major breaking API change
- Don't expose letters collection

* Minor fix

* Switch to using BoundingBox in the library

---------

Co-authored-by: davmarksman <david@brokit.co.uk>
2026-02-28 16:25:51 +00:00
Eliot Jones
a4047247a8 Additional digital corpora testing (#1261)
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Common Crawl Tests / build (0008-0009) (push) Has been cancelled
Run Common Crawl Tests / build (0010-0011) (push) Has been cancelled
Run Common Crawl Tests / build (0012-0013) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Tag Release / tag_if_version_changed (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
* add additional testing pairs 0010-0011 for integration

some of these files required the skip missing fonts flag set to true

- propagate use lenient parsing for dictionaries inside arrays, handles a
corrupt file 0012710 not in this test set

* add pair 0012-0013
2026-02-22 21:01:03 +00:00
BobLd
34db05f735 Fix Benchmarks solution and add BruteForceBenchmarks 2026-02-22 20:05:57 +00:00
Eliot Jones
9c0d6893e0 revert flate decode handling to more lenient processing (#1254)
* revert flate decode handling to more lenient processing

the change to use zlib/a adler checksum verification flow meant that
invalid flate streams would not be decoded correctly. this caused
issues for files that included invalid/missing checksums. this reverts
the processing to the old approach for files like #1235

* fix object stream offset handling and track circular refs

* update tests

* normalize line endings for mac runner

* fixes for mac clownery

* add next pair to common crawl action

* add a test case for the root cause of the int overflow
2026-02-22 15:49:50 +00:00
BobLd
adb5713621 Make LinkAnnotation public
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Tag Release / tag_if_version_changed (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2026-02-18 14:09:20 +00:00
BobLd
665623561a Add UglyToad.PdfPig.Benchmarks and misc performance improvements
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Tag Release / tag_if_version_changed (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2026-02-17 11:53:09 +00:00
EliotJones
7e370a0eab confirm tag is on master before running publish
Some checks failed
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Tag Release / tag_if_version_changed (push) Has been cancelled
2026-02-15 18:39:45 +00:00
EliotJones
b9aa53166d replace release flow single job with pr process
since actions do not have permissions to push
directly to master and bot accounts to achieve
the same are hard to manage we will change the
release flow to work as follows.

1. manually invoke `prepare_release_pr.yml` action,
this creates a new branch with the version of all
project files updated and creates a pull request for
that version. this pr then should be merged using
rebase merge

2. `tag_release.yml` checks if the newest commit
name starts with the text "Release " and also verifies
if it changed the version of the package csproj. if
both those conditions are met it will create and
push a new tag, e.g. `v0.1.17` to master

3. `publish_nuget.yml` listens for new `v*` tags on
master and triggers the nuget deployment

this is all chat gpt code so who knows if it will work
2026-02-15 18:39:45 +00:00
BobLd
0a2b1e076f Improve HasFormXObjectCircularReference and fix #1250 2026-02-15 18:06:35 +00:00
EliotJones
f732718852 both Tj and TJ operators should increment text sequence #1241
the text sequence was added to group letters added by the same
content stream operation together for layout analysis. TJ allows for
individual letter positioning in the same operation which modifies
the text matrix while writing whereas Tj just writes the text at the current
location. both should be semantically equivalent for this sequence
number. this was originally added in #61
2026-02-15 17:50:07 +00:00
Alexander Vinnikov
d69e3fa6a9 make ext gstates stacked
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2026-02-11 18:39:33 +00:00
BobLd
d6e86b057e Handle empty encoding in Type1FontSimple and fix #1248
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2026-02-10 06:14:39 +00:00
BobLd
c27f1b6553 Only throw if ArrayToken length is less than 4 in ToRectangle() and fix #1238
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2026-02-01 16:10:24 +00:00
BobLd
629891e80e Make LinkAnnotation internal to fix unit tests 2026-02-01 15:50:21 +00:00
David Ochsner
13825b9424 Add Links to Pdf Generation 2026-02-01 15:13:55 +00:00
BobLd
7c4f5e2424 Introduce StackDepthGuard class to check for stack depth in CoreTokenScanner and fix #1217
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-12-23 16:24:04 +01:00
BobLd
bd573b2a1c Increment version to 0.1.14 2025-12-23 12:00:19 +01:00
BobLd
baeac0d0c3 Do not return glyph bbox and path in Type1Font if character name is '.notdef' 2025-12-23 11:09:00 +01:00
BobLd
ee0cb1dc4a Use file header offset when doing brute force find and fix #1223
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-12-07 13:43:22 +00:00
BobLd
c70b343caa Minor Type1FontParser optimisations
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-12-02 17:29:55 +00:00
Carlo Kok
ce563db133 Clarify handling of optional content group names
Updated comments to clarify handling of optional content group names in PDFs.
2025-11-23 10:52:21 +00:00
BobLd
37a5dffcaa Skip in ReadCompositeGlyph when glyphIndex is out of range and fix #1213 2025-11-21 21:31:18 +00:00
EliotJones
aef0a78ee6 update release logic to check out master before commit
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-11-14 09:50:07 +00:00
BobLd
005e52783e Ensure no key end up missing in ResolveInternal and fix #1209 2025-11-13 14:53:28 +00:00
BobLd
f4e7db5b5b Simply order by offset also when not doing brute force to fix #1208 2025-11-13 14:26:35 +00:00
BobLd
52ecef0e28 Increment version to 0.1.13 2025-11-13 09:34:17 +00:00
BobLd
599ce04bef Add test that fails before revert of e11dc6b
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-11-07 22:31:43 +00:00
BobLd
2a6ee918b7 Revert "Avoid a lot of seeks by making most tokenizers no longer read to far by using seek."
This reverts commit e11dc6bf40.
2025-11-07 22:31:43 +00:00
BobLd
9d3cd0a429 Check for array size before slice in ColorSpaceDetailsByteConverter.Convert() 2025-11-07 20:02:39 +00:00
BobLd
b49e5aa697 Check for index out of range in GlyphDataTable.ReadFlags() and fix #1199 2025-11-07 19:39:18 +00:00
Alexander Vinnikov
181fa9d837 make transform stack consistent
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-11-03 20:06:40 +00:00
BobLd
6ce6986d78 Update test run command to use Release configuration
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-29 20:29:42 +00:00
BobLd
37c9fef20b Do not slice the stream to the length breaks decoding in FlateDecode 2025-10-29 20:27:40 +00:00
Bert Huijben
e11dc6bf40 Avoid a lot of seeks by making most tokenizers no longer read to far by using seek.
Optimize the FirstPassParser to just fetch a final chunk before doing things char-by-char backwards.
2025-10-28 06:48:41 +00:00
BobLd
40bcc22ea1 Add CMap caching at document level and add MurmurHash3 hashing function
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-26 16:20:27 +00:00
Bert Huijben
94d515061e Update NameToUnicodeConvertAglSpecification to test what was intended
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-25 19:47:56 +01:00
BobLd
3555521634 Fix regression introduced in 3592fc8 where slicing the stream to the length breaks decoding
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-23 19:08:05 +01:00
Bert Huijben
6fba565d66 Avoid doing a true file seek for simple peeking the next char in the token parser
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-20 06:33:34 +01:00
Bert Huijben
3592fc8438 Use zlib information to verify compressed content before using it
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
2025-10-15 18:46:36 +01:00
BobLd
c9034f991c Only apply RemoveStridePadding() when bytes per pixel is one and fix #1183 2025-10-15 12:57:25 +01:00
BobLd
255e70f0a7 Set Type 3 font ascent to Top instead of Height, see #1164
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-14 12:18:52 +01:00
BobLd
2216ade1f2 Trim excess in long lived font collections
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
2025-10-14 10:26:41 +01:00
BobLd
cf0c33b1e0 Improve DfsIterative() performance
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-10-13 12:35:21 +01:00
BobLd
ffba176060 Improve GroupIndexes() performance with #1178 2025-10-13 12:35:21 +01:00
BobLd
b14f45f59f Add more tests to NearestNeighbourWordExtractorTests
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
2025-10-12 19:51:04 +01:00
ricflams
c28d114b79 Guard against circular references in XRef tables/streams
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
- Detect and prevent an xref table/stream at a certain offset from being read twice; malformed xref tables with circular references could otherwise cause the table-reading to loop forever.
- Another approach could be to prevent TryReadTableAtOffset from changing the bytes' CurrentOffset to the lastObjPosition in its attempt to read a table (eg restore CurrentOffset after the attempt to read a Table) so the outer bytes-loop could continue its search through the entire bytes unaffected.
2025-10-01 06:32:38 +01:00
Richard Flamsholt
d7d01f842e Update test Issue874: No longer missing a font
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
Including the stream-xref means that the formerly missing font is no longer missing, so simply run the two test-cases under the (stricter) assumption of SkipMissingFonts=false.
2025-09-30 18:35:45 +01:00
Richard Flamsholt
33a8d829ee Update test Issue874: Also more text on page 2
Page two has had four more characters added, which is now delected by this xref-stream fix
2025-09-30 18:35:45 +01:00
Richard Flamsholt
57921c7e9b Update test Issue874: Now finds more text on page 1
With the fix for including associated streams, this test now finds more text on the first page. I've verified using Aspose.PDF and by viewing the ErcotFacts.pdf file being tested that yes, it was indeed missing part of the text before.
2025-09-30 18:35:45 +01:00
ricflams
5a6b3970f0 Add table-xref's associated stream-xrefs
- If an XrefTable has an associated stream, as indicated via the XrefStm-property, then read and add that XrefStream
- Any table can have 0 or 1 such associated streams
- A caveat: such an associated stream might also theoretically be part of the Parts-sequence in which case it would be encountered both by looping through all those parts along with all the regular tables and now also by association to any of those tables. It doesn't seem harmful since the offsets are flattened eventually anyway and stored by their offset-key into a mapping-table.
2025-09-30 18:35:45 +01:00
ricflams
397ccb15d6 Add xref-streams tied to any parts, not just the first
On a large sample of pdf-files PdfPig failed to read the correct StructTree-object for about 1% of them. The StructTree object was simply missing in the CrossReferenceTable.CrossReferenceTable.
It turned out that the constructed CrossReferenceTable could miss Stream-parts if there were multiple Table-parts because a stream will only be added if it's associated with the very first Table-part. The remedy would seem to be to check for and add streams that are associated with any of the Table-parts, not just the first one.
On a sample of 72 files where this failed, this changed fixed the StructTree for all of them.
2025-09-30 18:35:45 +01:00
BobLd
ca284e0cb9 Use pageFactoryCache.Clear() in Pages dispose and fix #1170 2025-09-28 17:18:00 +01:00
BobLd
b2f4ca8839 Add GetDescent() and GetAscent() methods to IFont, improve font matrix for TrueTypeSimpleFont and TrueTypeStandard14FallbackSimpleFont and add loose bounding box to Letter
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-09-21 15:07:52 +01:00
BobLd
008959457a Expose letter's font via GetFont(), make Font property as obsolete and use FontDetails instead
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-09-20 17:11:38 +01:00
BobLd
a53d96cb73 Use record struct in FileHeaderOffset 2025-09-20 13:45:50 +01:00
EliotJones
efdedb9495 handle case where offsets are out of range
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (0000-0001) (push) Has been cancelled
Run Common Crawl Tests / build (0002-0003) (push) Has been cancelled
Run Common Crawl Tests / build (0004-0005) (push) Has been cancelled
Run Common Crawl Tests / build (0006-0007) (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
default to returning empty glyph where the offset is out of the
file length range, this fixes file 12623 where the truetype file
is completely broken
2025-09-14 15:26:12 +01:00
BobLd
eb906a776d Handle non seekable stream by copying it into a memory stream and fix #1146 2025-09-14 14:42:59 +01:00
BobLd
44e638ee4d Add initial support to process CFF fonts contained inside a TrueType font 2025-09-14 11:32:32 +01:00
BobLd
304d7dde5a Use correct font matrix when transforming the width in Type 0 font and fix #1156 2025-09-14 08:22:58 +01:00
Eliot Jones
07df6fd740 read last line of ignore file (#1155)
* read last line of ignore file

- do not cancel other matrix jobs if one test fails
- read all lines of the ignore list even if it doesn't end with a newline
- add ignore list for 0008 and 0009

* support missing object numbers when brute-forcing

the file 10404 (ironically) contains not found references with number 43 0
for its info dictionary. changes brute-force code so that objects can be
entirely missing

* fix test since document is now opened successfully but mediabox is broken
2025-09-13 16:57:35 +02:00
Eliot Jones
c96880ac61 handle case where xobjects use same key as fonts (#1154)
in document 10122 the font and xobject names are the same so the
xobject overwrote references to the font for the page content, separate
the dictionaries
2025-09-13 16:49:24 +02:00
EliotJones
77db6c6b54 add test jobs for common crawl 0000 to 0007 2025-09-13 14:52:04 +01:00
EliotJones
e886ae648f copy other parser behavior by treating end of stream as valid end inline image
this file cotains corrupt content following an inline image but other parsers
just treat this content as part of the image and parse the rest of the file
successfully
2025-09-13 14:36:14 +01:00
BobLd
c4f442c0cd Properly fix #1148 by always parsing optional tables in TrueTypeFontParser and remove Type 0 font hack 2025-09-13 12:48:20 +01:00
BobLd
0ef120dc5c Properly handle CompactFontFormatCidFont font matrix and fix #1149 2025-09-13 10:38:35 +01:00
BobLd
d5b97065bd Fix #1148 2025-09-13 10:38:35 +01:00
BobLd
22eab422a3 First create the StreamInputBytes in PdfDocument.Open() to check the stream CanRead and CanSeek
Some checks failed
Build, test and publish draft / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / Check if this commit has already been published (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
2025-09-09 19:12:58 +01:00
Eliot Jones
8408c98aec Draft release on master build (#1145)
* remove alpha postfix, releases will increment version

* update the master build job to draft a release

* add publish action to publish full release

* enable setting assembly and file version

* bump assembly and file version for package project

---------

Co-authored-by: BobLd <38405645+BobLd@users.noreply.github.com>
2025-09-08 20:07:36 +01:00
Eliot Jones
dd5aa46c75 File buffering read stream investigation (#1140)
Some checks failed
Build and test / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
* add test for filebufferingreadstream

* #1124 do not trust reported stream length if bytes can be read at end

the filebufferingreadstream input stream does not report more than the read
length. the change to seek the xref  in a sliding window from the end broke
with the assumption that the reported length was correct. here we switch to
reading the window or continue reading if we can read beyond the stream's
initially reported length while seeking the startxref marker

* remove rogue newlines
2025-09-07 14:39:46 +01:00
BobLd
e4ed4d1b39 Add early version of IOSSystemFontLister
Some checks failed
Build and test / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
2025-09-02 19:53:12 +01:00
Eliot Jones
0afe021ad3 move file parsing to single-pass static methods (#1102)
* move file parsing to single-pass static methods

for the file 0002973.pdf in the test corpus we need to completely overhaul
how initial xref parsing is done since we need to locate the xref stream by
brute-force and this is currently broken. i wanted to take this opportunity to
change the logic to be more imperative and less like the pdfbox methods with
instance data and classes.

currently the logic is split between the xref offset validator and parser methods
and we call the validator logic twice, followed by brute-force searching again
in the actual parser. we're going to move to a single method that performs
the following steps:

1. find the first (from the end) occurrence of "startxref" and pull out the location
in bytes. this will also support "startref" since some files in the wild have that
2. go to that offset if found and parse the chain of tables or streams by /prev
reference
3. if any element in step 2 fails then we perform a single brute-force over the
entire file and like pdfbox treat later in file-length xrefs as the ultimate arbiter
of the object positions. while we do this we potentially can capture the actual
object offsets since the xref positions are probably incorrect too.

the aim with this is to avoid as much seeking and re-reading of bytes as
possible. while this won't technically be single-pass it gets us much closer. it
also removes the more strict logic requiring a "startxref" token to exist and be
valid, since we can repair this by brute-force anyway.

we will surface as much information as possible from the static method so that
we could in future support an object explorer ui for pdfs.

this will also be more resilient to invalid xref formats with e.g. comment tokens
or missing newlines.

* move more parsing to the static classes

* plumb through the new parsing results

* plug in new parser and remove old classes, port tests to new classes

* update tests to reflect logic changes

* apply correction when file header has offset

* ignore console runner launch settings

* skip offsets outside of file bounds

* fix parsing tables missing a line break

* use brute forced locations if they're already present

* only treat line breaks and spaces as whitespace for stream content

* address review comments

---------

Co-authored-by: BobLd <38405645+BobLd@users.noreply.github.com>
2025-09-02 19:41:00 +01:00
Karl
3650e27432 add container node support for BookmarksProvider.cs (#1133)
* add container node support for BookmarksProvider.cs

* move position

* fixed unittest error

* revert package name

* remove duplicated package info.
2025-08-14 21:17:58 +01:00
BobLd
a43b968ea9 Lower max search depth in preventing StackOverflow in ParseTrailer 2025-08-10 10:06:23 +01:00
BobLd
1031dcc221 Prevent StackOverflow in ParseTrailer and fix #1122 2025-08-09 08:46:04 +01:00
BobLd
0f641774e6 Update build_and_test_macos.yml 2025-08-09 08:33:34 +01:00
BobLd
a3edc926c8 Update build_and_test_macos.yml 2025-08-09 08:21:21 +01:00
BobLd
f1923fcbcd Increase FlateFilter multiplier when preventing malicious OOM and fix #1125 2025-08-08 19:04:31 +01:00
EliotJones
7ff58893af only run tests if nightly publish needed 2025-08-04 21:46:13 -05:00
EliotJones
bee6f13888 fix tag fetching and parse behavior 2025-08-04 21:40:28 -05:00
EliotJones
e6dd2d15c2 use gemini to mark ched gpt's work and improve the action 2025-08-04 21:00:12 -05:00
EliotJones
7dd5d68be3 prevent duplicate package publish on manual run, attempt 1 2025-08-04 20:49:18 -05:00
BobLd
bdf3b8e2b4 Update nightly_release.yml 2025-08-03 20:03:13 +01:00
BobLd
c8dff885bd Update run_common_crawl_tests.yml 2025-08-03 08:56:17 +01:00
BobLd
0b228c57b7 Update run_integration_tests.yml 2025-08-03 08:52:27 +01:00
BobLd
ef21227b3c Update run_integration_tests.yml 2025-08-03 08:46:40 +01:00
BobLd
b9f2230a0a Add global.json in tools 2025-08-03 08:43:58 +01:00
BobLd
b6950a5fb0 Update run_integration_tests.yml (#1117) 2025-08-03 08:34:50 +01:00
Chuck B.
1ed9e017f4 Performance improvements and .Net 9 support (#1116)
* Refactor letter handling by orientation for efficiency

Improved the processing of letters based on their text orientation by preallocating separate lists for each orientation (horizontal, rotate270, rotate180, rotate90, and other). This change reduces multiple calls to `GetWords` and minimizes enumerations and allocations, enhancing performance and readability. Each letter is now added to the appropriate list in a single iteration over the `letters` collection.

* Update target frameworks to include net9.0

Expanded compatibility in `UglyToad.PdfPig.csproj` by adding
`net9.0` to the list of target frameworks, alongside existing
versions.

* Add .NET 9.0 support and refactor key components

Updated project files for UglyToad.PdfPig to target .NET 9.0, enhancing compatibility with the latest framework features.

Refactored `GetBlocks` in `DocstrumBoundingBoxes.cs` for improved input handling and performance.

Significantly optimized `NearestNeighbourWordExtractor.cs` by replacing multiple lists with an array of buckets and implementing parallel processing for better efficiency.

Consistent updates across `Fonts`, `Tests`, `Tokenization`, and `Tokens` project files to include .NET 9.0 support.

* Improve null checks and optimize list handling

- Updated null check for `words` in `DocstrumBoundingBoxes.cs` for better readability and performance.
- Changed from `ToList()` to `ToArray()` to avoid unnecessary enumeration.
- Added `results.TrimExcess()` in `NearestNeighbourWordExtractor.cs` to optimize memory usage.

---------

Co-authored-by: Chuck Beasley <CBeasley@kilpatricktownsend.com>
2025-08-01 22:24:16 +01:00
EliotJones
83d6fc6cc2 allow missing catalog type definition for catalog dictionary
Some checks failed
Build and test / build (push) Has been cancelled
Build and test [MacOS] / build (push) Has been cancelled
Run Common Crawl Tests / build (push) Has been cancelled
Run Integration Tests / build (push) Has been cancelled
Nightly Release / tests (push) Has been cancelled
Nightly Release / Check latest commit (push) Has been cancelled
Nightly Release / build_and_publish_nightly (push) Has been cancelled
as long as there is a pages entry we accept this in lenient parsing mode. this
is to fix document 006705.pdf in the corpus that had '/calalog' as the dictionary
entry.

also adds a test for some weird content stream content in 0006324.pdf where
numbers seem to get split in the content stream on a decimal place. this is
just to check that our parser doesn't hard crash
2025-07-27 02:55:29 +01:00
theolivenbaum
febfa4d4b3 Fix usage of List.Contains 2025-07-27 02:52:56 +01:00
Eliot Jones
0ebbe0540d add nullability to core projec (#1111) 2025-07-27 02:48:58 +01:00
EliotJones
52c0635273 support performance profiling information in console runner 2025-07-26 15:04:03 -05:00
EliotJones
b6bd0a3169 bump version to 0.1.12-alpha001 2025-07-26 13:43:28 -05:00
EliotJones
3d2e12cb16 version 0.1.11 2025-07-26 13:16:01 -05:00
Eliot Jones
9cb3b71e62 update readme to avoid people using page.Text or asking about editing docs (#1109)
* update readme to avoid people using `page.Text` or asking about editing docs

we need to be more clear because beloved chat gpt falls into the trap of
recommending `page.Text` when asked about the library even though this
text is usually the wrong field to use

* tabs to spaces

* rogue tab
2025-07-26 18:58:35 +01:00
EliotJones
27df4af5f9 handle additional broken pdf files in the common crawl set
- a file contained 2 indices pointing to '.notdef' for the character name so
we just take the first rather than requiring a single
- a file contained '/' (empty name) as the subtype declaration, so we fall back
to trying type 1 and truetype parsing in this situation
2025-07-26 18:55:29 +01:00
EliotJones
50f878b2ba restore copy link func logic 2025-07-25 18:18:22 +01:00
EliotJones
2a10b6c285 make link copying more tolerant when adding page
in #1082 and other issues relating to annotations we're running into
constraints of the current model of building a pdf document. currently
we skip all link type annotations, i think we can support copying of links
where the link destination is outside the current document. however the
more i look at this code the more i think we need a radical redesign of
how document building is done because it has been pushed far beyond
its current capabilities, i'll detail my thinking in the related pr in more
detail
2025-07-25 18:18:22 +01:00
EliotJones
85fc63d585 rework numeric tokenizer hot path
the existing numeric tokenizer involved allocations and string parsing. since
the number formats in pdf files are fairly predictable we can improve this
substantially
2025-07-25 18:12:43 +01:00
EliotJones
5abdfcb96c fix test case due to field renaming 2025-07-20 20:33:46 +01:00
EliotJones
00ca268092 move last uncovered operators to switch statement
in order to remove reflection from the core content stream operators
construction we ensure all types covered by the operations dictionary
have corresponding switch statement support. this moves the remaining
2 operators to the switch statement. fix for #1062
2025-07-20 20:33:46 +01:00
BobLd
813d3baa18 Track IndirectReference instead of only ObjectNumber when checking for cycles during indirect reference resolutionv and add test 2025-07-20 19:24:31 +01:00
EliotJones
2b11961c8c remove debug asserts causing test failures
we encountered a fence constructed in the middle of a field for an unknown
reason so we demolished it. i think this was intended to catch flaws in the
parser logic but the reality is in a pdf anything can happen so we no longer
want to catch these issues and this restores a green test run in debug mode.

fix for #915
2025-07-20 17:42:34 +01:00
EliotJones
efb8c2a803 i merged a pr which broke the build, this updates the build to work
move all arguments to add page to a setting object so it can be extended
in future in a non-breaking api change
2025-07-20 17:36:19 +01:00
jan-sutter
e636212ec8 check for cycles during indirect reference resolution (#1097)
Co-authored-by: Jan Sutter <jan@suttermail.de>
2025-07-20 11:12:55 -05:00
EnraH
3b318e1944 add option to strip annotation (#492)
* add option to strip annotation

* fix implementation and tests

---------

Co-authored-by: arne.hansen <arne.hansen@digitecgalaxus.ch>
Co-authored-by: Eliot Jones <elioty@hotmail.co.uk>
2025-07-20 11:10:15 -05:00
EliotJones
377eb507e8 when writing content to an existing page inverse any global transform #614
when adding a page to a builder from an existing document using either
addpage or copyfrom methods the added page's content stream can contain
a global transform matrix change that will subsequently change all the locations
of any modifications made by the user. here whenever using an existing stream
we apply the inverse of any active transformation matrix

there could be a bug here where if you use 'copy from' with a global transform
active, we then apply the inverse, and you use 'copy from' again to the same
destination page our inverse transform is now active and could potentially
affect the second stream, but I don't think it will
2025-07-20 00:53:03 +01:00
BobLd
ff4e763192 Update hack for 1bpc + DeviceGray 2025-07-19 21:45:41 +01:00
BobLd
6a06452103 Remove decode parameter application from Stencil color space for consistency 2025-07-19 13:45:06 +01:00
BobLd
a5e92cd11c Update run_common_crawl_tests.yml 2025-07-19 12:21:10 +01:00
EliotJones
4bf746c747 add new action to run integration against common crawl corpus 2025-07-19 11:49:34 +01:00
EliotJones
bffd51425d support bfrange having incorrect length in a cmap
the corpus file 0001413.pdf has an off-by-one error in its count
for cmap bfranges. here we exit early if an unexpected
endbfrange operator is encountered early. this matches the pdfbox
behavior:

067d56e4db/fontbox/src/main/java/org/apache/fontbox/cmap/CMapParser.java (L373)
2025-07-19 11:48:04 +01:00
Eliot Jones
e3388ec6b6 fix colorspace error when form xobject contains a transparency group (#1088)
* fix colorspace error when form xobject contains a transparency group

when a form xobject contains a reference to a group xobject this can only
be used to change attributes of the transparency imaging model. the old
code was setting the main colorspaces incorrectly causing errors when the
transparency component had a different number of channels. this was
causing #1071 in addition to the failure in file 0000355.pdf of the test corpus

* add master integration tests for corpus group 0000

* tidy up actions

* remove invalid reference in echo

* move new action to different branch
2025-07-19 11:46:56 +01:00
EliotJones
31658ca020 allow reading to continue if encountering an invalid surrogate pair
investigating the corpus at
https://digitalcorpora.s3.amazonaws.com/s3_browser.html#corpora/files/CC-MAIN-2021-31-PDF-UNTRUNCATED/zipfiles/0000-0999/

the input file 0000000.pdf contained a utf-16 surrogate pair in an input
defined as usc2. the approach of various parsers varies here, adobe
acrobat seems to hard crash, pdf js returns the same text we now
parse, chrome parses the intended text (2 invalid characters and
"ib exam"). we don't care too much about matching chrome exactly
so doing the same as firefox is fine here
2025-07-16 07:45:40 +01:00
EliotJones
1021729727 fall back to times-roman as standard 14 font when lenient
if parsing in lenient mode and encountering a malformed base name
(in this case 'helveticai') we fallback to times-roman as the adobe font
metrics file for a standard 14 font. this aligns with the behavior of pdfbox.
we also log a more informative error in non-lenient modes

this fixes document 0000086.pdf from the corpus
2025-07-16 07:43:49 +01:00
Eliot Jones
9503f9c137 fix off-by-one and optimize brute force xref search #1078 (#1079)
* fix off-by-one and optimize brute force xref search #1078

when performing a brute force xref search we were ending up
off-by-one, update the search to use a ring buffer to reduce
seeking and fix xref detection

* make method testable and add test coverage

* normalize test input on other platforms

* seal circular buffer class
2025-07-16 07:35:24 +01:00
Eliot Jones
016b754c5b back-calculate first char if last char and widths present (#1081)
* back-calculate first char if last char and widths present

when a truetype font has a last char and widths array in its font
dictionary the first char can be calculated #644

* fix off by 1 in last char calculation
2025-07-14 21:57:01 +01:00
Eliot Jones
de3b6ac6f4 use correct bounding boxes for standard 14 glyphs #850 (#1080)
* use correct bounding boxes for standard 14 glyphs #850

previously every bounding box for type 1 standard 14 fonts was assumed
to start at 0,0 and ignored the bounding box data in the font metrics file.
now we correctly read the glyph bounding box while preserving the
existing advance width values for advancing the renderer position

* update test case for new logic
2025-07-14 21:54:42 +01:00
EliotJones
b11f936f22 fix copying of sub-dictionary when keys collide
when copying from a ancestor node of a page's resource dictionary
we were incorrectly writing nested nodes of e.g. /fonts to the root
of the target dictionary, here we write to the intended target node
correctly
2025-07-10 18:32:20 +01:00
EliotJones
7fe60ff8c3 skip single letter final blocks
align with the behavior of pdfbox and c implementations where
single character final blocks are ignored rather than being written.
also makes the error more informative in case it is ever encountered
again.

add more test cases.

it is possible this is hiding the problem and will move the error elsewhere
but this matches the implementation behavior of the 2 reference
implementations. one other potential source for the error is if pdf supports
'<~' as a start of data marker which i can't find in the spec but wikipedia
says might be possible? without documents to trigger the error i think
this is the best fix for now
2025-07-09 07:33:12 +01:00
EliotJones
781991b6bf fix #670 by ignoring duplicate endstream definitions
when parsing a stream object with multiple endstream tokens
the last parsed token was selected instead of the actual stream
token so instead we just skip all following tokens if the first
is a stream and the following tokens are `endstream` operators
only
2025-07-07 20:34:26 +01:00
277 changed files with 150473 additions and 5402 deletions

View File

@@ -1,4 +1,4 @@
name: Build and test
name: Build, test and publish draft
on:
push:
@@ -10,7 +10,7 @@ jobs:
build:
runs-on: windows-2022
steps:
- uses: actions/checkout@master
- uses: actions/checkout@v4
- name: Set up dotnet core
uses: actions/setup-dotnet@v4
@@ -29,3 +29,14 @@ jobs:
- name: Run the tests
run: dotnet test -c Release src/UglyToad.PdfPig.sln
- name: Update draft release
if: github.ref == 'refs/heads/master'
uses: softprops/action-gh-release@v2
with:
draft: true
name: "Draft release"
tag_name: "unreleased" # fixed tag so this one draft is updated
generate_release_notes: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -19,10 +19,11 @@ jobs:
2.1.x
6.0.x
8.0.x
9.0.x
# Build the release build
- name: Build the solution
run: dotnet build -c Release src/UglyToad.PdfPig.sln
run: dotnet build -c Release src/UglyToad.PdfPig.sln -f net8.0
- name: Run the tests
run: dotnet test -c Release src/UglyToad.PdfPig.sln
run: dotnet test -c Release src/UglyToad.PdfPig.sln -f net8.0

View File

@@ -5,39 +5,55 @@ on:
- cron: "0 0 * * *"
workflow_dispatch:
jobs:
tests:
uses: ./.github/workflows/run_integration_tests.yml
check_date:
runs-on: ubuntu-latest
name: Check latest commit
outputs:
should_run: ${{ steps.should_run.outputs.should_run }}
steps:
- uses: actions/checkout@master
- name: print latest_commit
run: echo ${{ github.sha }}
permissions:
contents: write # Grant write permission for tagging
jobs:
check_publish_needed:
runs-on: ubuntu-latest
name: Check if this commit has already been published
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Fetch tags
run: git fetch --tags
- id: check
run: |
latest_commit=$(git rev-parse nightly-latest || echo "")
echo "Latest published commit: $latest_commit"
if [ "$latest_commit" = "${{ github.sha }}" ]; then
echo "No new commit since last publish."
echo "should_run=false" >> $GITHUB_OUTPUT
else
echo "New commit detected."
echo "should_run=true" >> $GITHUB_OUTPUT
fi
tests:
needs: check_publish_needed
if: ${{ needs.check_publish_needed.outputs.should_run == 'true' }}
uses: ./.github/workflows/run_integration_tests.yml
- id: should_run
continue-on-error: true
name: check latest commit is less than a day ago
if: ${{ github.event_name == 'schedule' }}
run: test -z $(git rev-list --after="24 hours" ${{ github.sha }}) && echo "::set-output name=should_run::false"
build_and_publish_nightly:
needs: [check_date, tests]
if: ${{ needs.check_date.outputs.should_run != 'false' }}
needs: [check_publish_needed, tests]
if: ${{ needs.check_publish_needed.outputs.should_run == 'true' }}
runs-on: windows-2022
name: build_and_publish_nightly
steps:
- uses: actions/checkout@master
- uses: actions/checkout@v4
- name: Set up dotnet core
uses: actions/setup-dotnet@v2
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
2.1.x
6.0.x
8.0.x
9.0.x
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v1.0.2
@@ -54,5 +70,10 @@ jobs:
- name: Publish Nuget to GitHub registry
run: dotnet nuget push **/*.nupkg --api-key ${{secrets.NUGET_API_KEY}} --source https://api.nuget.org/v3/index.json
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Tag latest nightly commit
run: |
git config user.name "github-actions"
git config user.email "github-actions@github.com"
git tag -f nightly-latest ${{ github.sha }}
git push origin nightly-latest --force

View File

@@ -0,0 +1,40 @@
name: Create Release PR
on:
workflow_dispatch
permissions:
contents: write
pull-requests: write
jobs:
bump_version:
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Calculate next version
id: version
run: |
$newVer = .\tools\get-next-main-version.ps1
echo "NEW_VERSION=$newVer" >> $env:GITHUB_ENV
.\tools\set-version.ps1 $newVer -UpdateAssemblyAndFileVersion
git config user.name "github-actions"
git config user.email "github-actions@github.com"
git checkout -b release/$newVer
git commit -am "Release $newVer"
- name: Create Pull Request
uses: peter-evans/create-pull-request@v6
with:
branch: release/${{ env.NEW_VERSION }}
base: master
title: "Release ${{ env.NEW_VERSION }}"
body: "Automated release PR."
delete-branch: true
merge-method: rebase

53
.github/workflows/publish_nuget.yml vendored Normal file
View File

@@ -0,0 +1,53 @@
name: Publish
on:
push:
tags:
- 'v*'
permissions:
contents: write
packages: write
jobs:
build_and_publish:
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- name: Verify tag points to master
id: verify
run: |
TAG_COMMIT=$(git rev-list -n 1 ${{ github.ref_name }})
if ! git branch -r --contains $TAG_COMMIT | grep -q 'origin/master'; then
echo "Tag is not on master — skipping publish"
exit 78 # 78 = neutral in GitHub Actions
fi
- name: Set up .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
2.1.x
6.0.x
8.0.x
9.0.x
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Run tests
run: dotnet test -c Release src/UglyToad.PdfPig.sln
- name: Build package
run: dotnet pack -c Release -o package tools/UglyToad.PdfPig.Package/UglyToad.PdfPig.Package.csproj -p:IncludeSymbols=true -p:SymbolPackageFormat=snupkg
- name: Publish to NuGet
run: dotnet nuget push package/*.nupkg --api-key ${{secrets.NUGET_API_KEY}} --source https://api.nuget.org/v3/index.json
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -0,0 +1,58 @@
name: Run Common Crawl Tests
on:
push:
branches: [master]
workflow_dispatch:
workflow_call:
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
pair: ["0000-0001", "0002-0003", "0004-0005", "0006-0007", "0008-0009", "0010-0011", "0012-0013"]
steps:
- uses: actions/checkout@v2
- name: Set up dotnet core
uses: actions/setup-dotnet@v3
with:
dotnet-version: |
8.0.x
9.0.x
- name: Restore corpus cache
id: restore-corpus
uses: actions/cache@v4
with:
path: corpus/
key: ${{ runner.os }}-pdf-corpus-${{ matrix.pair }}
- name: Download corpus if cache missed
if: steps.restore-corpus.outputs.cache-hit != 'true'
run: |
mkdir -p corpus/zipfiles
cd corpus/zipfiles
for file in $(echo "${{ matrix.pair }}" | tr '-' ' '); do
echo "Downloading $file.zip"
wget -nv "https://digitalcorpora.s3.amazonaws.com/corpora/files/CC-MAIN-2021-31-PDF-UNTRUNCATED/zipfiles/0000-0999/$file.zip" -O "$file.zip"
done
cd ..
unzip 'zipfiles/*.zip' -d extracted
# run: rm -f zipfiles/*.zip
- name: Remove unwanted test files
run: |
while read f || [ -n "$f" ]; do
full="corpus/extracted/$f"
if [ -f "$full" ]; then
echo "Removing $full"
rm "$full"
fi
done < tools/common-crawl-ignore.txt
- name: Run tests against corpus
run: dotnet run --project tools/UglyToad.PdfPig.ConsoleRunner/UglyToad.PdfPig.ConsoleRunner.csproj "corpus/extracted" --configuration Release

View File

@@ -15,7 +15,9 @@ jobs:
- name: Set up dotnet core
uses: actions/setup-dotnet@v3
with:
dotnet-version: "8.0.x"
dotnet-version: |
8.0.x
9.0.x
- name: Restore cached part 1
id: restore-cache-p1

43
.github/workflows/tag_release.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
name: Tag Release
on:
push:
branches:
- master
permissions:
contents: write
jobs:
tag_if_version_changed:
if: startsWith(github.event.head_commit.message, 'Release ')
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Detect version change
id: versioncheck
run: |
git fetch origin master --depth=2
$diff = git diff HEAD^ HEAD -- tools/UglyToad.PdfPig.Package/UglyToad.PdfPig.Package.csproj
if ($diff -match "<Version>") {
$version = (Select-String -Path tools/UglyToad.PdfPig.Package/UglyToad.PdfPig.Package.csproj -Pattern "<Version>(.*)</Version>").Matches.Groups[1].Value
echo "version=$version" >> $env:GITHUB_OUTPUT
echo "create=true" >> $env:GITHUB_OUTPUT
} else {
echo "create=false" >> $env:GITHUB_OUTPUT
}
- name: Create tag
if: steps.versioncheck.outputs.create == 'true'
run: |
git config user.name "github-actions"
git config user.email "github-actions@github.com"
git tag "v${{ steps.versioncheck.outputs.version }}"
git push origin "v${{ steps.versioncheck.outputs.version }}"

2
.gitignore vendored
View File

@@ -248,3 +248,5 @@ _Pvt_Extensions
/tools/ConsoleRunner/Properties/launchSettings.json
/docs/doxygen
/tools/UglyToad.PdfPig.ConsoleRunner/Properties/launchSettings.json
/src/UglyToad.PdfPig.Tests/Images/Files/Pdf/indexed-png-with-mask.png

View File

@@ -2,19 +2,11 @@
# PdfPig
[![Gitter](https://badges.gitter.im/pdfpig/community.svg)](https://gitter.im/pdfpig/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
[![nuget](https://img.shields.io/nuget/dt/PdfPig)](https://www.nuget.org/packages/PdfPig/)
[![Build and test](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml)
[![Build and test [MacOS]](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml)
This project allows users to read and extract text and other content from PDF files. In addition the library can be used to create simple PDF documents
containing text and geometrical shapes.
This project aims to port [PDFBox](https://github.com/apache/pdfbox) to C#.
## Wiki
Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
PdfPig supports reading text and content from PDF files. It also supports basic PDF file creation.
## Installation
@@ -32,29 +24,26 @@ While the version is below 1.0.0 minor versions will change the public API witho
See the [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples
### Read words in a page
### Reading text from a PDF
The simplest usage at this stage is to open a document, reading the words from every page:
```cs
// using UglyToad.PdfPig.DocumentLayoutAnalysis.TextExtractor;
// using UglyToad.PdfPig.DocumentLayoutAnalysis.WordExtractor;
using (PdfDocument document = PdfDocument.Open(@"C:\Documents\document.pdf"))
{
foreach (Page page in document.GetPages())
{
string pageText = page.Text;
foreach (Word word in page.GetWords())
{
Console.WriteLine(word.Text);
}
}
foreach (Page page in document.GetPages())
{
string text = ContentOrderTextExtractor.GetText(page);
IEnumerable<Word> words = page.GetWords(NearestNeighbourWordExtractor.Instance);
}
}
```
An example of the output of this is shown below:
You **should not** use `page.Text` directly, unless you know what you're doing. The `Text` property preserves the internal content order which is rarely ever the text in the order you want.
![Image shows three words 'Write something in' in 2 sections, the top section is the normal PDF output, the bottom section is the same text with 3 word bounding boxes in pink and letter bounding boxes in blue-green](https://raw.githubusercontent.com/UglyToad/Pdf/master/documentation/Letters/example-text-extraction.png)
Where for the PDF text ("Write something in") shown at the top the 3 words (in pink) are detected and each word contains the individual letters with glyph bounding boxes.
These layout analysis tools should get you the text you want in most cases.
### Create PDF Document
To create documents use the class `PdfDocumentBuilder`. The Standard 14 fonts provide a quick way to get started:
@@ -80,6 +69,12 @@ The output is a 1 page PDF document with the text "Hello World!" in Helvetica ne
Each font must be registered with the `PdfDocumentBuilder` prior to use enable pages to share the font resources. Only Standard 14 fonts and TrueType fonts (.ttf) are supported.
Document creation supports very limited changes to existing PDF documents. However it does not support any of the following:
- Editing forms
- Copying or changing annotations, metadata or document structure data
- Adding or removing text with existing fonts
### Advanced Document Extraction
In this example a more advanced document extraction is performed. `PdfDocumentBuilder` is used to create a copy of the pdf with debug information (bounding boxes and reading order) added.
@@ -259,7 +254,7 @@ string title = document.Information.Title;
### Document Structure
The document now has a Structure member:
The `PdfDocument` has a Structure member:
UglyToad.PdfPig.Structure structure = document.Structure;
@@ -283,7 +278,7 @@ PageSize size = Page.Size;
bool isA4 = size == PageSize.A4;
```
`Page` provides access to the text of the page:
`Page` provides access to the text of the page but you should use `ContentOrderTextExtractor` or alternatives if indexing the text, e.g. for RAG/LLMs:
string text = page.Text;
@@ -329,7 +324,7 @@ Retrieving annotations on each page is provided using the method:
page.GetAnnotations()
This call is not cached and the document must not have been disposed prior to use.
This call is not cached and the document must not have been disposed prior to use. Annotations cannot be edited.
### Bookmarks
@@ -357,6 +352,8 @@ A page has a method to extract hyperlinks (annotations of link type):
IReadOnlyList<UglyToad.PdfPig.Content.Hyperlink> hyperlinks = page.GetHyperlinks();
Hyperlinks cannot be added or edited when building documents.
### TrueType
The classes used to work with TrueType fonts in the PDF file are available for public consumption. Given an input file:
@@ -396,18 +393,17 @@ var resultFileBytes = PdfMerger.Merge(filePath1, filePath2);
File.WriteAllBytes(@"C:\pdfs\outputfilename.pdf", resultFileBytes);
```
## Wiki
Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
## Issues
Please do file an issue if you encounter a bug. See our [issue policy](https://github.com/UglyToad/PdfPig/issues/1095) and [contributing guide](https://github.com/UglyToad/PdfPig/blob/master/CONTRIBUTING.md) for details.
## API Reference
If you wish to generate doxygen documentation, run `doxygen doxygen-docs` and open `docs/doxygen/html/index.html`.
See also the [wiki](https://github.com/UglyToad/PdfPig/wiki) for a detailed documentation on parts of the API
## Issues
Please do file an issue if you encounter a bug.
However in order for us to assist you, you **must** provide the file which causes your issue. Please host this in a publically available place.
## Credit
This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.
This project started as an effort to port [PDFBox](https://github.com/apache/pdfbox) to C#. This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.

View File

@@ -1,5 +1,4 @@
using System;
using System.Buffers;
using System.Buffers;
namespace UglyToad.PdfPig.Core;

View File

@@ -71,7 +71,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return obj is IndirectReference other && Equals(other);
}

View File

@@ -16,7 +16,7 @@
/// <summary>
/// Convert the string to bytes using the ISO 8859-1 encoding.
/// </summary>
public static byte[] StringAsLatin1Bytes(string s)
public static byte[]? StringAsLatin1Bytes(string? s)
{
if (s == null)
{

View File

@@ -264,7 +264,7 @@
/// Try to convert raw bytes to a PdfDocEncoding encoded string. If unsupported characters are encountered
/// meaning we cannot safely round-trip the value to bytes this will instead return false.
/// </summary>
public static bool TryConvertBytesToString(ReadOnlySpan<byte> bytes, out string result)
public static bool TryConvertBytesToString(ReadOnlySpan<byte> bytes, out string? result)
{
result = null;
if (bytes.Length == 0)

View File

@@ -70,7 +70,7 @@
/// <summary>
/// Returns a value indicating whether this <see cref="PdfLine"/> is equal to a specified <see cref="PdfLine"/> .
/// </summary>
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return obj is PdfLine other && Equals(other);
}

View File

@@ -83,7 +83,7 @@
/// <summary>
/// Returns a value indicating whether this <see cref="PdfPoint"/> is equal to a specified <see cref="PdfPoint"/> .
/// </summary>
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return obj is PdfPoint other && Equals(other);
}

View File

@@ -177,7 +177,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return obj is PdfRectangle other && Equals(other);
}

View File

@@ -241,8 +241,8 @@
public bool IsClosed()
{
var filteredCount = 0;
IPathCommand last = null;
IPathCommand first = null;
IPathCommand? last = null;
IPathCommand? first = null;
for (int i = Commands.Count - 1; i >= 0; i--)
{
var cmd = Commands[i];
@@ -376,14 +376,14 @@
/// Gets a <see cref="PdfRectangle"/> which entirely contains the geometry of the defined path.
/// </summary>
/// <returns>For paths which don't define any geometry this returns <see langword="null"/>.</returns>
public static PdfRectangle? GetBoundingRectangle(IReadOnlyList<PdfSubpath> path)
public static PdfRectangle? GetBoundingRectangle(IReadOnlyList<PdfSubpath>? path)
{
if (path == null || path.Count == 0)
{
return null;
}
var bboxes = path.Select(x => x.GetBoundingRectangle()).Where(x => x.HasValue).Select(x => x.Value).ToList();
var bboxes = path.Select(x => x.GetBoundingRectangle()).Where(x => x.HasValue).Select(x => x!.Value).ToList();
if (bboxes.Count == 0)
{
return null;
@@ -433,7 +433,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return (obj is Close);
}
@@ -479,7 +479,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
if (obj is Move move)
{
@@ -545,7 +545,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
if (obj is Line line)
{
@@ -651,7 +651,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
if (obj is QuadraticBezierCurve curve)
{
@@ -809,7 +809,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
if (obj is CubicBezierCurve curve)
{
@@ -944,7 +944,7 @@
/// <summary>
/// Compares two <see cref="PdfSubpath"/>s for equality. Paths will only be considered equal if the commands which construct the paths are in the same order.
/// </summary>
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
if (!(obj is PdfSubpath path) || Commands.Count != path.Commands.Count)
{

View File

@@ -0,0 +1,63 @@
namespace UglyToad.PdfPig.Core
{
/// <summary>
/// Provides a guard for tracking and limiting the depth of nested stack operations, such as recursive calls or
/// nested parsing.
/// </summary>
/// <remarks>Use this class to prevent excessive stack usage by enforcing a maximum nesting depth. This is
/// particularly useful in scenarios where untrusted or deeply nested input could cause stack overflows or
/// performance issues.</remarks>
public sealed class StackDepthGuard
{
/// <summary>
/// Represents a stack depth guard with no effective limit on the allowed depth.
/// </summary>
/// <remarks>Use this instance when stack depth restrictions are not required.</remarks>
public static readonly StackDepthGuard Infinite = new StackDepthGuard(int.MaxValue);
private readonly int maxStackDepth;
private int depth;
/// <summary>
/// Initializes a new instance of the StackDepthGuard class with the specified maximum stack depth.
/// </summary>
/// <param name="maxStackDepth">The maximum allowed stack depth for guarded operations. Must be a positive integer.</param>
public StackDepthGuard(int maxStackDepth)
{
if (maxStackDepth <= 0)
{
throw new ArgumentOutOfRangeException(nameof(maxStackDepth));
}
this.maxStackDepth = maxStackDepth;
}
/// <summary>
/// Increments the current nesting depth and checks against the maximum allowed stack depth.
/// </summary>
/// <exception cref="PdfDocumentFormatException">Thrown if the maximum allowed nesting depth is exceeded.</exception>
public void Enter()
{
if (++depth > maxStackDepth)
{
depth--; // Decrement so Exit remains balanced if someone catches this
throw new PdfDocumentFormatException($"Exceeded maximum nesting depth of {maxStackDepth}.");
}
}
/// <summary>
/// Decreases the current depth level by one, ensuring that the depth does not become negative.
/// </summary>
/// <remarks>If the current depth is already zero, calling this method has no effect. This method
/// is typically used to track or manage nested operations or scopes where depth must remain
/// non-negative.</remarks>
public void Exit()
{
depth--;
if (depth < 0)
{
depth = 0;
}
}
}
}

View File

@@ -11,11 +11,12 @@
{
private readonly Stream stream;
private readonly bool shouldDispose;
private byte? peekByte;
private bool isAtEnd;
/// <inheritdoc />
public long CurrentOffset => stream.Position;
public long CurrentOffset => peekByte.HasValue ? stream.Position - 1 : stream.Position;
/// <inheritdoc />
public byte CurrentByte { get; private set; }
@@ -52,7 +53,8 @@
/// <inheritdoc />
public bool MoveNext()
{
var b = stream.ReadByte();
var b = peekByte ?? stream.ReadByte();
peekByte = null;
if (b == -1)
{
@@ -68,18 +70,21 @@
/// <inheritdoc />
public byte? Peek()
{
var current = CurrentOffset;
var b = stream.ReadByte();
stream.Seek(current, SeekOrigin.Begin);
if (b == -1)
if (!peekByte.HasValue)
{
return null;
var v = stream.ReadByte();
if (v >= 0)
{
peekByte = (byte)v;
}
else
{
return null;
}
}
return (byte)b;
return peekByte;
}
/// <inheritdoc />
@@ -92,6 +97,7 @@
public void Seek(long position)
{
isAtEnd = false;
peekByte = null;
if (position == 0)
{
@@ -112,9 +118,15 @@
{
return 0;
}
else if (peekByte.HasValue)
{
buffer[0] = peekByte.Value;
peekByte = null;
return Read(buffer.Slice(1)) + 1;
}
int read = stream.Read(buffer);
if (read > 0)
{
CurrentByte = buffer[read - 1];

View File

@@ -224,9 +224,18 @@
[Pure]
public double TransformX(double x)
{
var xt = A * x + C * 0 + E;
return A * x + E; // + C * 0
}
return xt;
/// <summary>
/// Transform an Y coordinate using this transformation matrix.
/// </summary>
/// <param name="y">The Y coordinate.</param>
/// <returns>The transformed Y coordinate.</returns>
public double TransformY(double y)
{
return D * y + F;
}
/// <summary>
@@ -463,7 +472,7 @@
}
/// <inheritdoc />
public override bool Equals(object obj)
public override bool Equals(object? obj)
{
return obj is TransformationMatrix other && Equals(other);
}

View File

@@ -1,23 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
<PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0;net9.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.14</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
<Nullable>enable</Nullable>
<WarningsAsErrors>nullable</WarningsAsErrors>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
<PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
</Project>

View File

@@ -89,7 +89,7 @@ namespace UglyToad.PdfPig.Core
/// <inheritdoc />
public override bool TryGetSecond(out B b)
{
b = default(B);
b = default!;
return false;
}
@@ -135,7 +135,7 @@ namespace UglyToad.PdfPig.Core
/// <inheritdoc />
public override bool TryGetFirst(out A a)
{
a = default(A);
a = default!;
return false;
}

View File

@@ -0,0 +1,20 @@
namespace UglyToad.PdfPig.Core;
/// <summary>
/// Indicates where an object is located in the Xref.
/// </summary>
public enum XrefEntryType : byte
{
/// <summary>
/// Free object.
/// </summary>
Free = 0,
/// <summary>
/// Located as an object in the file.
/// </summary>
File = 1,
/// <summary>
/// Located in a compressed object stream.
/// </summary>
ObjectStream = 2
}

View File

@@ -0,0 +1,42 @@
namespace UglyToad.PdfPig.Core;
/// <summary>
/// Information about where an object is located in the file according to the Xref (or brute force parsing).
/// </summary>
public readonly struct XrefLocation
{
/// <summary>
/// Which type of location is indicated.
/// </summary>
public readonly XrefEntryType Type;
/// <summary>
/// If <see cref="Type"/> is <see cref="XrefEntryType.File"/> then byte offset, otherwise <see cref="XrefEntryType.ObjectStream"/> this is the stream number.
/// </summary>
public readonly long Value1;
/// <summary>
/// If <see cref="Type"/> is <see cref="XrefEntryType.ObjectStream"/> then the index of the object in the stream.
/// </summary>
public readonly int Value2; // only used for ObjectStream
private XrefLocation(XrefEntryType type, long value1, int value2)
{
Type = type;
Value1 = value1;
Value2 = value2;
}
/// <summary>
/// Create a location mapped to a byte offset in the file.
/// </summary>
public static XrefLocation File(long offset)
=> new XrefLocation(XrefEntryType.File, offset, 0);
/// <summary>
/// Create a location mapped to an index inside and object stream.
/// </summary>
public static XrefLocation Stream(long objStream, int index)
=> new XrefLocation(XrefEntryType.ObjectStream, objStream, index);
}

View File

@@ -50,7 +50,15 @@
* (i,j,k) will form a group and (m,n) will form another group.
*************************************************************************************/
int[] indexes = Enumerable.Repeat(-1, elements.Count).ToArray();
int[] indexes = new int[elements.Count];
#if NET6_0_OR_GREATER
Array.Fill(indexes, -1);
#else
for (int k = 0; k < indexes.Length; k++)
{
indexes[k] = -1;
}
#endif
KdTree<T> kdTree = new KdTree<T>(elements, candidatesPoint);
ParallelOptions parallelOptions = new ParallelOptions() { MaxDegreeOfParallelism = maxDegreeOfParallelism };
@@ -118,7 +126,15 @@
* (i,j,k) will form a group and (m,n) will form another group.
*************************************************************************************/
int[] indexes = Enumerable.Repeat(-1, elements.Count).ToArray();
int[] indexes = new int[elements.Count];
#if NET6_0_OR_GREATER
Array.Fill(indexes, -1);
#else
for (int l = 0; l < indexes.Length; l++)
{
indexes[l] = -1;
}
#endif
KdTree<T> kdTree = new KdTree<T>(elements, candidatesPoint);
ParallelOptions parallelOptions = new ParallelOptions() { MaxDegreeOfParallelism = maxDegreeOfParallelism };
@@ -186,7 +202,15 @@
* (i,j,k) will form a group and (m,n) will form another group.
*************************************************************************************/
int[] indexes = Enumerable.Repeat(-1, elements.Count).ToArray();
int[] indexes = new int[elements.Count];
#if NET6_0_OR_GREATER
Array.Fill(indexes, -1);
#else
for (int k = 0; k < indexes.Length; k++)
{
indexes[k] = -1;
}
#endif
ParallelOptions parallelOptions = new ParallelOptions() { MaxDegreeOfParallelism = maxDegreeOfParallelism };
@@ -216,71 +240,37 @@
yield return group.Select(i => elements[i]).ToList();
}
}
/// <summary>
/// Group elements using Depth-first search.
/// <para>https://en.wikipedia.org/wiki/Depth-first_search</para>
/// </summary>
/// <param name="edges">The graph. edges[i] = j indicates that there is an edge between i and j.</param>
/// <returns>A List of HashSets containing the grouped indexes.</returns>
internal static List<HashSet<int>> GroupIndexes(int[] edges)
internal static List<List<int>> GroupIndexes(int[] edges)
{
int[][] adjacency = new int[edges.Length][];
// Improved thanks to https://github.com/UglyToad/PdfPig/issues/1178
var adjacency = new List<int>[edges.Length];
for (int i = 0; i < edges.Length; i++)
{
HashSet<int> matches = new HashSet<int>();
if (edges[i] != -1) matches.Add(edges[i]);
for (int j = 0; j < edges.Length; j++)
{
if (edges[j] == i) matches.Add(j);
}
adjacency[i] = matches.ToArray();
adjacency[i] = new List<int>();
}
List<HashSet<int>> groupedIndexes = new List<HashSet<int>>();
// one pass O(n)
for (int i = 0; i < edges.Length; i++)
{
int j = edges[i];
if (j != -1)
{
// i <-> j
adjacency[i].Add(j);
adjacency[j].Add(i);
}
}
List<List<int>> groupedIndexes = new List<List<int>>();
bool[] isDone = new bool[edges.Length];
for (int p = 0; p < edges.Length; p++)
{
if (isDone[p]) continue;
groupedIndexes.Add(DfsIterative(p, adjacency, ref isDone));
}
return groupedIndexes;
}
/// <summary>
/// Group elements using Depth-first search.
/// <para>https://en.wikipedia.org/wiki/Depth-first_search</para>
/// </summary>
/// <param name="edges">The graph. edges[i] = [j, k, l, ...] indicates that there is an edge between i and each element j, k, l, ...</param>
/// <returns>A List of HashSets containing the grouped indexes.</returns>
internal static List<HashSet<int>> GroupIndexes(int[][] edges)
{
int[][] adjacency = new int[edges.Length][];
for (int i = 0; i < edges.Length; i++)
{
HashSet<int> matches = new HashSet<int>();
for (int j = 0; j < edges[i].Length; j++)
if (isDone[p])
{
if (edges[i][j] != -1) matches.Add(edges[i][j]);
continue;
}
for (int j = 0; j < edges.Length; j++)
{
for (int k = 0; k < edges[j].Length; k++)
{
if (edges[j][k] == i) matches.Add(j);
}
}
adjacency[i] = matches.ToArray();
}
List<HashSet<int>> groupedIndexes = new List<HashSet<int>>();
bool[] isDone = new bool[edges.Length];
for (int p = 0; p < edges.Length; p++)
{
if (isDone[p]) continue;
groupedIndexes.Add(DfsIterative(p, adjacency, ref isDone));
}
return groupedIndexes;
@@ -290,22 +280,33 @@
/// Depth-first search
/// <para>https://en.wikipedia.org/wiki/Depth-first_search</para>
/// </summary>
private static HashSet<int> DfsIterative(int s, int[][] adj, ref bool[] isDone)
private static List<int> DfsIterative(int s, List<int>[] adj, ref bool[] isDone)
{
HashSet<int> group = new HashSet<int>();
Stack<int> S = new Stack<int>();
List<int> group = new List<int>();
Stack<int> S = new Stack<int>(4);
S.Push(s);
isDone[s] = true;
while (S.Count > 0)
{
var u = S.Pop();
if (!isDone[u])
group.Add(u);
#if NET
var currentAdj = System.Runtime.InteropServices.CollectionsMarshal.AsSpan(adj[u]);
int count = currentAdj.Length;
#else
var currentAdj = adj[u];
int count = currentAdj.Count;
#endif
for (int i = 0; i < count; ++i)
{
group.Add(u);
isDone[u] = true;
foreach (var v in adj[u])
var v = currentAdj[i];
ref bool done = ref isDone[v];
if (!done)
{
S.Push(v);
done = true;
}
}
}

View File

@@ -128,13 +128,13 @@
throw new ArgumentException("The algorithm cannot be used with a document of less than 2 pages.", nameof(pagesTextBlocks));
}
ConcurrentDictionary<int, List<TextBlock>> pageDecorations = new ConcurrentDictionary<int, List<TextBlock>>();
ConcurrentDictionary<int, OrderedSet<TextBlock>> pageDecorations = new ConcurrentDictionary<int, OrderedSet<TextBlock>>();
ParallelOptions parallelOptions = new ParallelOptions() { MaxDegreeOfParallelism = maxDegreeOfParallelism };
Parallel.For(0, pagesTextBlocks.Count, parallelOptions, p =>
{
if (!pageDecorations.TryAdd(p, new List<TextBlock>()))
if (!pageDecorations.TryAdd(p, new OrderedSet<TextBlock>()))
{
throw new ArgumentException("Cannot add element with index " + p + " in ConcurrentDictionary.");
}
@@ -165,7 +165,7 @@
var score = Score(current, previousPage, nextPage, minimumEditDistanceNormalised, similarityThreshold, n);
if (score >= similarityThreshold)
{
if (!pageDecorations[p].Contains(current)) pageDecorations[p].Add(current);
pageDecorations[p].TryAdd(current);
}
}
@@ -180,7 +180,7 @@
var score = Score(current, previousPage, nextPage, minimumEditDistanceNormalised, similarityThreshold, n);
if (score >= similarityThreshold)
{
if (!pageDecorations[p].Contains(current)) pageDecorations[p].Add(current);
pageDecorations[p].TryAdd(current);
}
}
@@ -195,7 +195,7 @@
var score = Score(current, previousPage, nextPage, minimumEditDistanceNormalised, similarityThreshold, n);
if (score >= similarityThreshold)
{
if (!pageDecorations[p].Contains(current)) pageDecorations[p].Add(current);
pageDecorations[p].TryAdd(current);
}
}
@@ -210,12 +210,12 @@
var score = Score(current, previousPage, nextPage, minimumEditDistanceNormalised, similarityThreshold, n);
if (score >= similarityThreshold)
{
if (!pageDecorations[p].Contains(current)) pageDecorations[p].Add(current);
pageDecorations[p].TryAdd(current);
}
}
});
return pageDecorations.OrderBy(x => x.Key).Select(x => x.Value).ToList();
return pageDecorations.OrderBy(x => x.Key).Select(x => x.Value.GetList()).ToList();
}
/// <summary>

View File

@@ -3,7 +3,6 @@
using Core;
using System;
using System.Collections.Generic;
using System.Linq;
/// <summary>
/// Contains helpful tools for distance measures.
@@ -169,12 +168,11 @@
distance = double.MaxValue;
int closestPointIndex = -1;
var candidatesPoints = candidates.Select(candidatePoint).ToList();
var pivot = pivotPoint(element);
for (var i = 0; i < candidates.Count; i++)
{
double currentDistance = distanceMeasure(pivot, candidatesPoints[i]);
double currentDistance = distanceMeasure(pivot, candidatePoint(candidates[i]));
if (currentDistance < distance && !candidates[i].Equals(element))
{
distance = currentDistance;
@@ -211,12 +209,11 @@
distance = double.MaxValue;
int closestLineIndex = -1;
var candidatesLines = candidates.Select(candidateLine).ToList();
var pivot = pivotLine(element);
for (var i = 0; i < candidates.Count; i++)
{
double currentDistance = distanceMeasure(pivot, candidatesLines[i]);
double currentDistance = distanceMeasure(pivot, candidateLine(candidates[i]));
if (currentDistance < distance && !candidates[i].Equals(element))
{
distance = currentDistance;

View File

@@ -3,7 +3,6 @@
using System.Collections.Generic;
using System.Linq;
using UglyToad.PdfPig.Content;
using UglyToad.PdfPig.PdfFonts;
/// <summary>
/// Checks if each letter is a duplicate and overlaps any other letter and remove the duplicate, and flag the remaining as bold.
@@ -24,41 +23,51 @@
return letters?.ToList();
}
var queue = new Queue<Letter>(letters);
var cleanLetters = new List<Letter>() { queue.Dequeue() }; // dequeue the first letter
// Use a dictionary keyed by (Value, FontName) to look up candidate duplicates in O(1)
var duplicateIndex = new Dictionary<(string, string), List<int>>();
var cleanLetters = new List<Letter>();
while (queue.Count > 0)
foreach (var letter in letters)
{
var letter = queue.Dequeue();
bool addLetter = true;
int duplicatesOverlappingIndex = -1;
var duplicates = cleanLetters.Where(l => l.Value.Equals(letter.Value) && l.FontName.Equals(letter.FontName)); // do other checks?
if (duplicates.Any())
var key = (letter.Value, letter.FontName);
if (duplicateIndex.TryGetValue(key, out var candidateIndices))
{
double tolerance = letter.GlyphRectangle.Width / (letter.Value.Length == 0 ? 1 : letter.Value.Length) / 3.0;
double minX = letter.GlyphRectangle.BottomLeft.X - tolerance;
double maxX = letter.GlyphRectangle.BottomLeft.X + tolerance;
double minY = letter.GlyphRectangle.BottomLeft.Y - tolerance;
double maxY = letter.GlyphRectangle.BottomLeft.Y + tolerance;
double tolerance = letter.BoundingBox.Width / (letter.Value.Length == 0 ? 1 : letter.Value.Length) / 3.0;
double minX = letter.BoundingBox.BottomLeft.X - tolerance;
double maxX = letter.BoundingBox.BottomLeft.X + tolerance;
double minY = letter.BoundingBox.BottomLeft.Y - tolerance;
double maxY = letter.BoundingBox.BottomLeft.Y + tolerance;
var duplicatesOverlapping = duplicates.FirstOrDefault(l => minX <= l.GlyphRectangle.BottomLeft.X &&
maxX >= l.GlyphRectangle.BottomLeft.X &&
minY <= l.GlyphRectangle.BottomLeft.Y &&
maxY >= l.GlyphRectangle.BottomLeft.Y);
if (duplicatesOverlapping != default)
for (int ci = 0; ci < candidateIndices.Count; ci++)
{
// duplicate overlapping letter was found, keeping the existing one and not adding this one.
addLetter = false;
duplicatesOverlappingIndex = cleanLetters.IndexOf(duplicatesOverlapping);
int idx = candidateIndices[ci];
var l = cleanLetters[idx];
if (minX <= l.BoundingBox.BottomLeft.X &&
maxX >= l.BoundingBox.BottomLeft.X &&
minY <= l.BoundingBox.BottomLeft.Y &&
maxY >= l.BoundingBox.BottomLeft.Y)
{
addLetter = false;
duplicatesOverlappingIndex = idx;
break;
}
}
}
if (addLetter)
{
int newIndex = cleanLetters.Count;
cleanLetters.Add(letter);
if (!duplicateIndex.TryGetValue(key, out var list))
{
list = new List<int>();
duplicateIndex[key] = list;
}
list.Add(newIndex);
}
else if (duplicatesOverlappingIndex != -1)
{
@@ -68,25 +77,11 @@
// update textSequence?
// update font details to bold
var fontDetails = new FontDetails(letter.Font.Name, true, letter.Font.Weight, letter.Font.IsItalic);
var newLetter = new Letter(letter.Value,
letter.GlyphRectangle,
letter.StartBaseLine,
letter.EndBaseLine,
letter.Width,
letter.FontSize,
fontDetails,
letter.RenderingMode,
letter.StrokeColor,
letter.FillColor,
letter.PointSize,
letter.TextSequence);
// update markedContentStack?
// update letters
cleanLetters[duplicatesOverlappingIndex] = newLetter;
cleanLetters[duplicatesOverlappingIndex] = letter.AsBold();
}
}

View File

@@ -229,7 +229,7 @@
private AltoDocument.AltoIllustration ToAltoIllustration(IPdfImage pdfImage, double height)
{
illustrationCount++;
var rectangle = pdfImage.Bounds;
var rectangle = pdfImage.BoundingBox;
return new AltoDocument.AltoIllustration
{
@@ -311,10 +311,10 @@
glyphCount++;
return new AltoDocument.AltoGlyph
{
VerticalPosition = (float)Math.Round((height - letter.GlyphRectangle.Top) * scale),
HorizontalPosition = (float)Math.Round(letter.GlyphRectangle.Left * scale),
Height = (float)Math.Round(letter.GlyphRectangle.Height * scale),
Width = (float)Math.Round(letter.GlyphRectangle.Width * scale),
VerticalPosition = (float)Math.Round((height - letter.BoundingBox.Top) * scale),
HorizontalPosition = (float)Math.Round(letter.BoundingBox.Left * scale),
Height = (float)Math.Round(letter.BoundingBox.Height * scale),
Width = (float)Math.Round(letter.BoundingBox.Width * scale),
Gc = 1.0f,
Content = invalidCharacterHandler(letter.Value),
Id = "P" + pageCount + "_ST" + stringCount.ToString("#00000") + "_G" + glyphCount.ToString("#00")

View File

@@ -277,7 +277,7 @@
private string GetCode(IPdfImage pdfImage, double pageHeight, int level)
{
imageCount++;
var bbox = pdfImage.Bounds;
var bbox = pdfImage.BoundingBox;
return GetIndent(level) + "<span class='ocr_image' id='image_" + pageCount + "_"
+ imageCount + "' title='" + GetCode(bbox, pageHeight) + "' />";
}

View File

@@ -280,7 +280,7 @@
private PageXmlDocument.PageXmlImageRegion ToPageXmlImageRegion(IPdfImage pdfImage, PageXmlData data, double pageWidth, double pageHeight)
{
data.RegionsCount++;
var bbox = pdfImage.Bounds;
var bbox = pdfImage.BoundingBox;
return new PageXmlDocument.PageXmlImageRegion()
{
Coords = ToCoords(bbox, pageWidth, pageHeight),
@@ -360,7 +360,7 @@
data.GlyphsCount++;
return new PageXmlDocument.PageXmlGlyph()
{
Coords = ToCoords(letter.GlyphRectangle, pageWidth, pageHeight),
Coords = ToCoords(letter.BoundingBox, pageWidth, pageHeight),
Ligature = false,
Production = PageXmlDocument.PageXmlProductionSimpleType.Printed,
TextStyle = new PageXmlDocument.PageXmlTextStyle()

View File

@@ -96,12 +96,12 @@
{
string fontFamily = GetFontFamily(l.FontName, out string style, out string weight);
string rotation = "";
if (l.GlyphRectangle.Rotation != 0)
if (l.BoundingBox.Rotation != 0)
{
rotation = $" transform='rotate({Math.Round(-l.GlyphRectangle.Rotation, Rounding)} {Math.Round(l.GlyphRectangle.BottomLeft.X, Rounding)},{Math.Round(height - l.GlyphRectangle.TopLeft.Y, Rounding)})'";
rotation = $" transform='rotate({Math.Round(-l.BoundingBox.Rotation, Rounding)} {Math.Round(l.BoundingBox.BottomLeft.X, Rounding)},{Math.Round(height - l.BoundingBox.TopLeft.Y, Rounding)})'";
}
string fontSize = l.FontSize != 1 ? $"font-size='{l.FontSize:0}'" : $"style='font-size:{Math.Round(l.GlyphRectangle.Height, 2)}px'";
string fontSize = l.FontSize != 1 ? $"font-size='{l.FontSize:0}'" : $"style='font-size:{Math.Round(l.BoundingBox.Height, 2)}px'";
var safeValue = XmlEscape(l, doc);
var x = Math.Round(l.StartBaseLine.X, Rounding);

View File

@@ -54,9 +54,6 @@
/// <typeparam name="T"></typeparam>
public class KdTree<T>
{
private readonly KdTreeComparerY kdTreeComparerY = new KdTreeComparerY();
private readonly KdTreeComparerX kdTreeComparerX = new KdTreeComparerX();
/// <summary>
/// The root of the tree.
/// </summary>
@@ -111,11 +108,11 @@
if (depth % 2 == 0)
{
P.Sort(kdTreeComparerX);
P.Sort((p0, p1) => p0.Value.X.CompareTo(p1.Value.X));
}
else
{
P.Sort(kdTreeComparerY);
P.Sort((p0, p1) => p0.Value.Y.CompareTo(p1.Value.Y));
}
if (P.Length == 2)
@@ -131,6 +128,26 @@
return new KdTreeNode<T>(vLeft, vRight, P[median], depth);
}
#else
private sealed class KdTreeComparerY : IComparer<KdTreeElement<T>>
{
public static readonly KdTreeComparerY Shared = new KdTreeComparerY();
public int Compare(KdTreeElement<T> p0, KdTreeElement<T> p1)
{
return p0.Value.Y.CompareTo(p1.Value.Y);
}
}
private sealed class KdTreeComparerX : IComparer<KdTreeElement<T>>
{
public static readonly KdTreeComparerX Shared = new KdTreeComparerX();
public int Compare(KdTreeElement<T> p0, KdTreeElement<T> p1)
{
return p0.Value.X.CompareTo(p1.Value.X);
}
}
private KdTreeNode<T> BuildTree(ArraySegment<KdTreeElement<T>> P, int depth = 0)
{
if (P.Count == 0)
@@ -145,13 +162,13 @@
if (depth % 2 == 0)
{
P.Sort(kdTreeComparerX);
P.Sort(KdTreeComparerX.Shared);
}
else
{
P.Sort(kdTreeComparerY);
P.Sort(KdTreeComparerY.Shared);
}
if (P.Count == 2)
{
return new KdTreeNode<T>(new KdTreeLeaf<T>(P.GetAt(0), depth + 1), null, P.GetAt(1), depth);
@@ -179,13 +196,14 @@
/// <returns>The nearest neighbour's element.</returns>
public T FindNearestNeighbour(T pivot, Func<T, PdfPoint> pivotPointFunc, Func<PdfPoint, PdfPoint, double> distanceMeasure, out int index, out double distance)
{
var result = FindNearestNeighbour(Root, pivot, pivotPointFunc, distanceMeasure);
var pivotPoint = pivotPointFunc(pivot);
var result = FindNearestNeighbour(Root, pivot, pivotPoint, distanceMeasure);
index = result.Item1 != null ? result.Item1.Index : -1;
distance = result.Item2 ?? double.NaN;
return result.Item1 != null ? result.Item1.Element : default;
}
private static (KdTreeNode<T>, double?) FindNearestNeighbour(KdTreeNode<T> node, T pivot, Func<T, PdfPoint> pivotPointFunc, Func<PdfPoint, PdfPoint, double> distance)
private static (KdTreeNode<T>, double?) FindNearestNeighbour(KdTreeNode<T> node, T pivot, PdfPoint pivotPoint, Func<PdfPoint, PdfPoint, double> distance)
{
if (node == null)
{
@@ -197,23 +215,22 @@
{
return (null, null);
}
return (node, distance(node.Value, pivotPointFunc(pivot)));
return (node, distance(node.Value, pivotPoint));
}
else
{
var point = pivotPointFunc(pivot);
var currentNearestNode = node;
var currentDistance = distance(node.Value, point);
var currentDistance = distance(node.Value, pivotPoint);
KdTreeNode<T> newNode = null;
double? newDist = null;
var pointValue = node.IsAxisCutX ? point.X : point.Y;
var pointValue = node.IsAxisCutX ? pivotPoint.X : pivotPoint.Y;
if (pointValue < node.L)
{
// start left
(newNode, newDist) = FindNearestNeighbour(node.LeftChild, pivot, pivotPointFunc, distance);
(newNode, newDist) = FindNearestNeighbour(node.LeftChild, pivot, pivotPoint, distance);
if (newDist.HasValue && newDist <= currentDistance && !newNode.Element.Equals(pivot))
{
@@ -223,13 +240,13 @@
if (node.RightChild != null && pointValue + currentDistance >= node.L)
{
(newNode, newDist) = FindNearestNeighbour(node.RightChild, pivot, pivotPointFunc, distance);
(newNode, newDist) = FindNearestNeighbour(node.RightChild, pivot, pivotPoint, distance);
}
}
else
{
// start right
(newNode, newDist) = FindNearestNeighbour(node.RightChild, pivot, pivotPointFunc, distance);
(newNode, newDist) = FindNearestNeighbour(node.RightChild, pivot, pivotPoint, distance);
if (newDist.HasValue && newDist <= currentDistance && !newNode.Element.Equals(pivot))
{
@@ -239,7 +256,7 @@
if (node.LeftChild != null && pointValue - currentDistance <= node.L)
{
(newNode, newDist) = FindNearestNeighbour(node.LeftChild, pivot, pivotPointFunc, distance);
(newNode, newDist) = FindNearestNeighbour(node.LeftChild, pivot, pivotPoint, distance);
}
}
@@ -258,7 +275,7 @@
/// <summary>
/// Get the k nearest neighbours to the pivot element.
/// Might return more than k neighbours if points are equidistant.
/// <para>Use <see cref="FindNearestNeighbour(KdTreeNode{T}, T, Func{T, PdfPoint}, Func{PdfPoint, PdfPoint, double})"/> if only looking for the (single) closest point.</para>
/// <para>Use <see cref="FindNearestNeighbour(T, Func{T, PdfPoint}, Func{PdfPoint, PdfPoint, double}, out int, out double)"/> if only looking for the (single) closest point.</para>
/// </summary>
/// <param name="pivot">The element for which to find the k nearest neighbours.</param>
/// <param name="k">The number of neighbours to return. Might return more than k neighbours if points are equidistant.</param>
@@ -267,13 +284,24 @@
/// <returns>Returns a list of tuples of the k nearest neighbours. Tuples are (element, index, distance).</returns>
public IReadOnlyList<(T, int, double)> FindNearestNeighbours(T pivot, int k, Func<T, PdfPoint> pivotPointFunc, Func<PdfPoint, PdfPoint, double> distanceMeasure)
{
var pivotPoint = pivotPointFunc(pivot);
var kdTreeNodes = new KNearestNeighboursQueue(k);
FindNearestNeighbours(Root, pivot, k, pivotPointFunc, distanceMeasure, kdTreeNodes);
return kdTreeNodes.SelectMany(n => n.Value.Select(e => (e.Element, e.Index, n.Key))).ToArray();
FindNearestNeighbours(Root, pivot, k, pivotPoint, distanceMeasure, kdTreeNodes);
var results = new List<(T, int, double)>();
for (int i = 0; i < kdTreeNodes.Count; i++)
{
double dist = kdTreeNodes.Keys[i];
foreach (var e in kdTreeNodes.Values[i])
{
results.Add((e.Element, e.Index, dist));
}
}
return results;
}
private static (KdTreeNode<T>, double) FindNearestNeighbours(KdTreeNode<T> node, T pivot, int k,
Func<T, PdfPoint> pivotPointFunc, Func<PdfPoint, PdfPoint, double> distance, KNearestNeighboursQueue queue)
PdfPoint pivotPoint, Func<PdfPoint, PdfPoint, double> distance, KNearestNeighboursQueue queue)
{
if (node == null)
{
@@ -286,7 +314,7 @@
return (null, double.NaN);
}
var currentDistance = distance(node.Value, pivotPointFunc(pivot));
var currentDistance = distance(node.Value, pivotPoint);
var currentNearestNode = node;
if (!queue.IsFull || currentDistance <= queue.LastDistance)
@@ -300,9 +328,8 @@
}
else
{
var point = pivotPointFunc(pivot);
var currentNearestNode = node;
var currentDistance = distance(node.Value, point);
var currentDistance = distance(node.Value, pivotPoint);
if ((!queue.IsFull || currentDistance <= queue.LastDistance) && !node.Element.Equals(pivot))
{
queue.Add(currentDistance, currentNearestNode);
@@ -313,12 +340,12 @@
KdTreeNode<T> newNode = null;
double newDist = double.NaN;
var pointValue = node.IsAxisCutX ? point.X : point.Y;
var pointValue = node.IsAxisCutX ? pivotPoint.X : pivotPoint.Y;
if (pointValue < node.L)
{
// start left
(newNode, newDist) = FindNearestNeighbours(node.LeftChild, pivot, k, pivotPointFunc, distance, queue);
(newNode, newDist) = FindNearestNeighbours(node.LeftChild, pivot, k, pivotPoint, distance, queue);
if (!double.IsNaN(newDist) && newDist <= currentDistance && !newNode.Element.Equals(pivot))
{
@@ -329,13 +356,13 @@
if (node.RightChild != null && pointValue + currentDistance >= node.L)
{
(newNode, newDist) = FindNearestNeighbours(node.RightChild, pivot, k, pivotPointFunc, distance, queue);
(newNode, newDist) = FindNearestNeighbours(node.RightChild, pivot, k, pivotPoint, distance, queue);
}
}
else
{
// start right
(newNode, newDist) = FindNearestNeighbours(node.RightChild, pivot, k, pivotPointFunc, distance, queue);
(newNode, newDist) = FindNearestNeighbours(node.RightChild, pivot, k, pivotPoint, distance, queue);
if (!double.IsNaN(newDist) && newDist <= currentDistance && !newNode.Element.Equals(pivot))
{
@@ -346,7 +373,7 @@
if (node.LeftChild != null && pointValue - currentDistance <= node.L)
{
(newNode, newDist) = FindNearestNeighbours(node.LeftChild, pivot, k, pivotPointFunc, distance, queue);
(newNode, newDist) = FindNearestNeighbours(node.LeftChild, pivot, k, pivotPoint, distance, queue);
}
}
@@ -395,9 +422,14 @@
if (this[key].Add(value))
{
var last = this.Last();
LastElement = last.Value.Last();
LastDistance = last.Key;
LastDistance = Keys[Count - 1];
var lastSet = Values[Count - 1];
KdTreeNode<T> lastElement = null;
foreach (var e in lastSet)
{
lastElement = e;
}
LastElement = lastElement;
}
}
}
@@ -418,23 +450,7 @@
public R Element { get; }
}
private sealed class KdTreeComparerY : IComparer<KdTreeElement<T>>
{
public int Compare(KdTreeElement<T> p0, KdTreeElement<T> p1)
{
return p0.Value.Y.CompareTo(p1.Value.Y);
}
}
private sealed class KdTreeComparerX : IComparer<KdTreeElement<T>>
{
public int Compare(KdTreeElement<T> p0, KdTreeElement<T> p1)
{
return p0.Value.X.CompareTo(p1.Value.X);
}
}
/// <summary>
/// K-D tree leaf.
/// </summary>

View File

@@ -0,0 +1,56 @@
namespace UglyToad.PdfPig.DocumentLayoutAnalysis
{
using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
internal class OrderedSet<T>
{
private readonly HashSet<T> _set;
private readonly List<T> _list;
public OrderedSet() : this(EqualityComparer<T>.Default)
{
}
public OrderedSet(IEqualityComparer<T> comparer)
{
_set = new HashSet<T>(comparer);
_list = new List<T>();
}
public int Count => _set.Count;
public bool TryAdd(T item)
{
if (_set.Contains(item)) return false;
_list.Add(item);
_set.Add(item);
return true;
}
public void Clear()
{
_list.Clear();
_set.Clear();
}
public bool Contains(T item)
{
return item is not null && _set.Contains(item);
}
public void CopyTo(T[] array, int arrayIndex)
{
_list.CopyTo(array, arrayIndex);
}
public List<T> GetList()
{
return _list;
}
}
}

View File

@@ -3,7 +3,6 @@
using Content;
using Core;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
@@ -48,12 +47,19 @@
/// <returns>The <see cref="TextBlock"/>s generated by the document spectrum method.</returns>
public IReadOnlyList<TextBlock> GetBlocks(IEnumerable<Word> words)
{
if (words?.Any() != true)
if (words is null)
{
return Array.Empty<TextBlock>();
}
return GetBlocks(words.ToList(),
// Avoid multiple enumeration and unnecessary ToArray() if already a list
var wordList = words as IReadOnlyList<Word> ?? words.ToArray();
if (wordList.Count == 0)
{
return Array.Empty<TextBlock>();
}
return GetBlocks(wordList,
options.WithinLineBounds, options.WithinLineMultiplier, options.WithinLineBinSize,
options.BetweenLineBounds, options.BetweenLineMultiplier, options.BetweenLineBinSize,
options.AngularDifferenceBounds,
@@ -147,57 +153,72 @@
{
ParallelOptions parallelOptions = new ParallelOptions() { MaxDegreeOfParallelism = maxDegreeOfParallelism };
var withinLineDistList = new ConcurrentBag<double>();
var betweenLineDistList = new ConcurrentBag<double>();
var withinLineDistList = new List<double>();
var betweenLineDistList = new List<double>();
// 1. Estimate within line and between line spacing
KdTree<Word> kdTreeBottomLeft = new KdTree<Word>(words, w => w.BoundingBox.BottomLeft);
Parallel.For(0, words.Count, parallelOptions, i =>
{
var word = words[i];
// Within-line distance
// 1.1.1 Find the 2 closest neighbours words to the candidate, using euclidean distance.
foreach (var n in kdTreeBottomLeft.FindNearestNeighbours(word, 2, w => w.BoundingBox.BottomRight, Distances.Euclidean))
Parallel.For(0, words.Count, parallelOptions,
() => (wl: new List<double>(), bl: new List<double>()),
(i, _, local) =>
{
// 1.1.2 Check if the neighbour word is within the angle of the candidate
if (wlBounds.Contains(AngleWL(word, n.Item1)))
var word = words[i];
// Within-line distance
// 1.1.1 Find the 2 closest neighbours words to the candidate, using euclidean distance.
foreach (var n in kdTreeBottomLeft.FindNearestNeighbours(word, 2, w => w.BoundingBox.BottomRight, Distances.Euclidean))
{
withinLineDistList.Add(Distances.Euclidean(word.BoundingBox.BottomRight, n.Item1.BoundingBox.BottomLeft));
}
}
// Between-line distance
// 1.2.1 Find the 2 closest neighbours words to the candidate, using euclidean distance.
foreach (var n in kdTreeBottomLeft.FindNearestNeighbours(word, 2, w => w.BoundingBox.TopLeft, Distances.Euclidean))
{
// 1.2.2 Check if the candidate words is within the angle
var angle = AngleBL(word, n.Item1);
if (blBounds.Contains(angle))
{
// 1.2.3 Compute the vertical (between-line) distance between the candidate
// and the neighbour and add it to the between-line distances list
double hypotenuse = Distances.Euclidean(word.BoundingBox.Centroid, n.Item1.BoundingBox.Centroid);
// Angle is kept within [-90, 90]
if (angle > 90)
// 1.1.2 Check if the neighbour word is within the angle of the candidate
if (wlBounds.Contains(AngleWL(word, n.Item1)))
{
angle -= 180;
}
var dist = Math.Abs(hypotenuse * Math.Cos((90 - angle) * Math.PI / 180))
- word.BoundingBox.Height / 2.0 - n.Item1.BoundingBox.Height / 2.0;
// The perpendicular distance can be negative because of the subtractions.
// Could occur when words are overlapping, we ignore that.
if (dist >= 0)
{
betweenLineDistList.Add(dist);
local.wl.Add(Distances.Euclidean(word.BoundingBox.BottomRight, n.Item1.BoundingBox.BottomLeft));
}
}
}
});
// Between-line distance
// 1.2.1 Find the 2 closest neighbours words to the candidate, using euclidean distance.
foreach (var n in kdTreeBottomLeft.FindNearestNeighbours(word, 2, w => w.BoundingBox.TopLeft, Distances.Euclidean))
{
// 1.2.2 Check if the candidate words is within the angle
var angle = AngleBL(word, n.Item1);
if (blBounds.Contains(angle))
{
// 1.2.3 Compute the vertical (between-line) distance between the candidate
// and the neighbour and add it to the between-line distances list
double hypotenuse = Distances.Euclidean(word.BoundingBox.Centroid, n.Item1.BoundingBox.Centroid);
// Angle is kept within [-90, 90]
if (angle > 90)
{
angle -= 180;
}
var dist = Math.Abs(hypotenuse * Math.Cos((90 - angle) * Math.PI / 180))
- word.BoundingBox.Height / 2.0 - n.Item1.BoundingBox.Height / 2.0;
// The perpendicular distance can be negative because of the subtractions.
// Could occur when words are overlapping, we ignore that.
if (dist >= 0)
{
local.bl.Add(dist);
}
}
}
return local;
},
local =>
{
lock (withinLineDistList)
{
withinLineDistList.AddRange(local.wl);
}
lock (betweenLineDistList)
{
betweenLineDistList.AddRange(local.bl);
}
});
// Compute average peak value of distribution
double? withinLinePeak = GetPeakAverageDistance(withinLineDistList, wlBinSize);
@@ -214,9 +235,9 @@
/// </summary>
/// <param name="distances">The set of distances to average.</param>
/// <param name="binLength"></param>
private static double? GetPeakAverageDistance(IEnumerable<double> distances, int binLength = 1)
private static double? GetPeakAverageDistance(List<double> distances, int binLength = 1)
{
if (!distances.Any())
if (distances.Count == 0)
{
return null;
}
@@ -226,7 +247,16 @@
throw new ArgumentException("DocstrumBoundingBoxes: the bin length must be positive when commputing peak average distance.", nameof(binLength));
}
double maxDbl = Math.Ceiling(distances.Max());
double maxDbl = distances[0];
for (int i = 1; i < distances.Count; i++)
{
if (distances[i] > maxDbl)
{
maxDbl = distances[i];
}
}
maxDbl = Math.Ceiling(maxDbl);
if (maxDbl > int.MaxValue)
{
throw new OverflowException($"Error while casting maximum distance of {maxDbl} to integer.");
@@ -242,30 +272,49 @@
binLength = binLength > max ? max : binLength;
}
var bins = Enumerable.Range(0, (int)Math.Ceiling(max / (double)binLength) + 1)
.Select(x => x * binLength)
.ToDictionary(x => x, _ => new List<double>());
foreach (var distance in distances)
int binCount = (int)Math.Ceiling(max / (double)binLength) + 1;
var bins = new List<double>[binCount];
for (int i = 0; i < binCount; i++)
{
bins[i] = new List<double>();
}
for (int i = 0; i < distances.Count; i++)
{
var distance = distances[i];
int bin = (int)Math.Floor(distance / binLength);
if (bin < 0)
{
throw new ArgumentOutOfRangeException(nameof(bin), "DocstrumBoundingBoxes: Negative distance found while commputing peak average distance.");
}
bins[bins.Keys.ElementAt(bin)].Add(distance);
if (bin >= binCount)
{
bin = binCount - 1;
}
bins[bin].Add(distance);
}
var best = default(List<double>);
foreach (var bin in bins)
List<double> best = null;
for (int i = 0; i < binCount; i++)
{
if (best == null || bin.Value.Count > best.Count)
var bin = bins[i];
if (best == null || bin.Count > best.Count)
{
best = bin.Value;
best = bin;
}
}
return best?.Average();
if (best == null || best.Count == 0)
{
return null;
}
double sum = 0;
for (int i = 0; i < best.Count; i++)
{
sum += best[i];
}
return sum / best.Count;
}
#endregion
@@ -410,28 +459,7 @@
return double.PositiveInfinity;
}
}
private sealed class PdfPointXYComparer : IComparer<PdfPoint>
{
public static readonly PdfPointXYComparer Instance = new();
public int Compare(PdfPoint p1, PdfPoint p2)
{
int comp = p1.X.CompareTo(p2.X);
return comp == 0 ? p1.Y.CompareTo(p2.Y) : comp;
}
}
private sealed class PdfPointYComparer : IComparer<PdfPoint>
{
public static readonly PdfPointYComparer Instance = new();
public int Compare(PdfPoint p1, PdfPoint p2)
{
return p1.Y.CompareTo(p2.Y);
}
}
/// <summary>
/// Get the structural blocking parameters.
/// </summary>
@@ -479,22 +507,30 @@
if (dXj != 0)
{
ps.Sort(PdfPointXYComparer.Instance);
ps.Sort((p1, p2) =>
{
int comp = p1.X.CompareTo(p2.X);
return comp == 0 ? p1.Y.CompareTo(p2.Y) : comp;
});
}
else if (dYj != 0)
{
ps.Sort(PdfPointYComparer.Instance);
ps.Sort((p1, p2) => p1.Y.CompareTo(p2.Y));
}
#else
PdfPoint[] ps = [j.Point1, j.Point2, Aj.Value, Bj.Value];
if (dXj != 0)
{
Array.Sort(ps, PdfPointXYComparer.Instance);
Array.Sort(ps, (p1, p2) =>
{
int comp = p1.X.CompareTo(p2.X);
return comp == 0 ? p1.Y.CompareTo(p2.Y) : comp;
});
}
else if (dYj != 0)
{
Array.Sort(ps, PdfPointYComparer.Instance);
Array.Sort(ps, (p1, p2) => p1.Y.CompareTo(p2.Y));
}
#endif

View File

@@ -372,7 +372,7 @@
public Func<IEnumerable<Letter>, double> DominantFontWidthFunc { get; set; } =
(letters) =>
{
var widths = letters.Select(x => Math.Max(Math.Round(x.Width, 3), Math.Round(x.GlyphRectangle.Width, 3)));
var widths = letters.Select(x => Math.Max(Math.Round(x.Width, 3), Math.Round(x.BoundingBox.Width, 3)));
var mode = widths.Mode();
if (double.IsNaN(mode) || mode == 0)
{
@@ -389,7 +389,7 @@
public Func<IEnumerable<Letter>, double> DominantFontHeightFunc { get; set; } =
(letters) =>
{
var heights = letters.Select(x => Math.Round(x.GlyphRectangle.Height, 3));
var heights = letters.Select(x => Math.Round(x.BoundingBox.Height, 3));
var mode = heights.Mode();
if (double.IsNaN(mode) || mode == 0)
{

View File

@@ -9,7 +9,7 @@
/// <summary>
/// A block of text.
/// </summary>
public class TextBlock
public class TextBlock: IBoundingBox
{
/// <summary>
/// The separator used between lines in the block.

View File

@@ -59,12 +59,13 @@
{
letter = new Letter(
" ",
letter.GlyphRectangle,
letter.BoundingBox,
letter.GlyphRectangleLoose,
letter.StartBaseLine,
letter.EndBaseLine,
letter.Width,
letter.FontSize,
letter.Font,
letter.GetFont()!,
letter.RenderingMode,
letter.StrokeColor,
letter.FillColor,

View File

@@ -9,7 +9,7 @@
/// <summary>
/// A line of text.
/// </summary>
public class TextLine
public class TextLine : IBoundingBox
{
/// <summary>
/// The separator used between words in the line.

View File

@@ -1,22 +1,22 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0;net9.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.14</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
</ItemGroup>
</Project>

View File

@@ -25,8 +25,8 @@
{
return GetWhitespaces(words,
images,
words.SelectMany(w => w.Letters).Select(x => x.GlyphRectangle.Width).Mode() * 1.25,
words.SelectMany(w => w.Letters).Select(x => x.GlyphRectangle.Height).Mode() * 1.25,
words.SelectMany(w => w.Letters).Select(x => x.BoundingBox.Width).Mode() * 1.25,
words.SelectMany(w => w.Letters).Select(x => x.BoundingBox.Height).Mode() * 1.25,
maxRectangleCount: maxRectangleCount,
maxBoundQueueSize: maxBoundQueueSize);
}
@@ -51,7 +51,7 @@
if (images?.Any() == true)
{
bboxes.AddRange(images.Where(w => w.Bounds.Width > 0 && w.Bounds.Height > 0).Select(o => o.Bounds));
bboxes.AddRange(images.Where(w => w.BoundingBox.Width > 0 && w.BoundingBox.Height > 0).Select(o => o.BoundingBox));
}
return GetWhitespaces(bboxes,

View File

@@ -51,34 +51,49 @@
if (options.GroupByOrientation)
{
// axis aligned
List<Word> words = GetWords(
letters.Where(l => l.TextOrientation == TextOrientation.Horizontal).ToList(),
options.MaximumDistance, options.DistanceMeasureAA, options.FilterPivot,
options.Filter, options.MaxDegreeOfParallelism);
var buckets = new List<Letter>[5];
for (int i = 0; i < buckets.Length; i++) buckets[i] = new List<Letter>();
words.AddRange(GetWords(
letters.Where(l => l.TextOrientation == TextOrientation.Rotate270).ToList(),
options.MaximumDistance, options.DistanceMeasureAA, options.FilterPivot,
options.Filter, options.MaxDegreeOfParallelism));
foreach (var l in letters)
{
switch (l.TextOrientation)
{
case TextOrientation.Horizontal: buckets[0].Add(l); break;
case TextOrientation.Rotate270: buckets[1].Add(l); break;
case TextOrientation.Rotate180: buckets[2].Add(l); break;
case TextOrientation.Rotate90: buckets[3].Add(l); break;
default: buckets[4].Add(l); break;
}
}
words.AddRange(GetWords(
letters.Where(l => l.TextOrientation == TextOrientation.Rotate180).ToList(),
options.MaximumDistance, options.DistanceMeasureAA, options.FilterPivot,
options.Filter, options.MaxDegreeOfParallelism));
// Use a thread-safe collection to avoid lock contention.
var results = new List<Word>(letters.Count); // Pre-allocate for performance
words.AddRange(GetWords(
letters.Where(l => l.TextOrientation == TextOrientation.Rotate90).ToList(),
options.MaximumDistance, options.DistanceMeasureAA, options.FilterPivot,
options.Filter, options.MaxDegreeOfParallelism));
// Limit parallelism to avoid oversubscription.
var parallelOptions = new System.Threading.Tasks.ParallelOptions
{
MaxDegreeOfParallelism = options.MaxDegreeOfParallelism > 0 ? options.MaxDegreeOfParallelism : Environment.ProcessorCount
};
// not axis aligned
words.AddRange(GetWords(
letters.Where(l => l.TextOrientation == TextOrientation.Other).ToList(),
options.MaximumDistance, options.DistanceMeasure, options.FilterPivot,
options.Filter, options.MaxDegreeOfParallelism));
return words;
// Use partitioner for better load balancing and avoid ConcurrentBag overhead
System.Threading.Tasks.Parallel.ForEach(
System.Collections.Concurrent.Partitioner.Create(0, buckets.Length),
parallelOptions,
range =>
{
for (int i = range.Item1; i < range.Item2; i++)
{
if (buckets[i].Count == 0) continue;
var measure = (i == 4) ? options.DistanceMeasure : options.DistanceMeasureAA;
var words = GetWords(buckets[i], options.MaximumDistance, measure, options.FilterPivot, options.Filter, options.MaxDegreeOfParallelism);
lock (results)
{
results.AddRange(words);
}
}
});
results.TrimExcess();
return results;
}
else
{
@@ -148,8 +163,8 @@
public Func<Letter, Letter, double> MaximumDistance { get; set; } = (l1, l2) =>
{
double maxDist = Math.Max(Math.Max(Math.Max(Math.Max(Math.Max(
Math.Abs(l1.GlyphRectangle.Width),
Math.Abs(l2.GlyphRectangle.Width)),
Math.Abs(l1.BoundingBox.Width),
Math.Abs(l2.BoundingBox.Width)),
Math.Abs(l1.Width)),
Math.Abs(l2.Width)),
l1.PointSize), l2.PointSize) * 0.2;

View File

@@ -770,6 +770,8 @@ namespace UglyToad.PdfPig.Fonts.CompactFontFormat.CharStrings
}
}
values.TrimExcess();
return new Type2CharStrings.CommandSequence(values, commandIdentifiers);
}

View File

@@ -32,7 +32,7 @@
public virtual string GetNameByStringId(int stringId)
{
return GlyphIdToStringIdAndName.SingleOrDefault(x => x.Value.stringId == stringId).Value.name;
return GlyphIdToStringIdAndName.FirstOrDefault(x => x.Value.stringId == stringId).Value.name;
}
public virtual int GetStringIdByGlyphId(int glyphId)

View File

@@ -257,6 +257,10 @@
gidToStringIdAndNameMap[gid++] = pair;
}
#if NET
gidToStringIdAndNameMap.TrimExcess();
#endif
glyphIdToStringIdAndName = gidToStringIdAndNameMap;
}

View File

@@ -27,7 +27,7 @@
/// <summary>
/// The font matrix for this font.
/// </summary>
public TransformationMatrix FontMatrix => TopDictionary.FontMatrix.HasValue ? TopDictionary.FontMatrix.Value : TransformationMatrix.FromValues(0.001, 0, 0, 0.001, 0, 0);
public TransformationMatrix FontMatrix => TopDictionary.FontMatrix ?? TransformationMatrix.FromValues(0.001, 0, 0, 0.001, 0, 0);
/// <summary>
/// The value of Weight from the top dictionary or <see langword="null"/>.
@@ -219,22 +219,26 @@
public override TransformationMatrix? GetFontMatrix(string characterName)
{
// BobLd: It seems PdfBox just returns TopDictionary.FontMatrix
// But see https://bugs.ghostscript.com/show_bug.cgi?id=690724
// and https://github.com/veraPDF/veraPDF-library/issues/1010
// TODO - We might need to multiply both matrices together
bool hasDictionary = TryGetFontDictionaryForCharacter(characterName, out var dictionary);
if (TopDictionary.FontMatrix is not null)
if (TopDictionary.FontMatrix.HasValue &&
hasDictionary &&
dictionary.FontMatrix.HasValue)
{
return TopDictionary.FontMatrix.Value.Multiply(dictionary.FontMatrix.Value);
}
if (TopDictionary.FontMatrix.HasValue)
{
return TopDictionary.FontMatrix;
}
if (!TryGetFontDictionaryForCharacter(characterName, out var dictionary))
if (hasDictionary && dictionary.FontMatrix.HasValue)
{
return null;
return dictionary.FontMatrix;
}
return dictionary.FontMatrix;
return null;
}
private bool TryGetPrivateDictionaryForCharacter(string characterName, out CompactFontFormatPrivateDictionary dictionary)

View File

@@ -0,0 +1,16 @@
namespace UglyToad.PdfPig.Fonts.SystemFonts
{
#if NET
using System.Collections.Generic;
internal sealed class IOSSystemFontLister : ISystemFontLister
{
// Very early version, intended to help developing support for iOS
public IEnumerable<SystemFontRecord> GetAllFonts()
{
yield break;
}
}
#endif
}

View File

@@ -102,6 +102,10 @@ namespace UglyToad.PdfPig.Fonts.SystemFonts
{
lister = new MacSystemFontLister();
}
else if (OperatingSystem.IsIOS())
{
lister = new IOSSystemFontLister();
}
#endif
else
{
@@ -113,7 +117,7 @@ namespace UglyToad.PdfPig.Fonts.SystemFonts
#error Missing ISystemFontLister for target framework
#endif
AvailableFonts = new Lazy<IReadOnlyList<SystemFontRecord>>(() => lister.GetAllFonts().ToList());
AvailableFonts = new Lazy<IReadOnlyList<SystemFontRecord>>(() => lister.GetAllFonts().ToArray());
}
private readonly ConcurrentDictionary<string, string> nameToFileNameMap = new ConcurrentDictionary<string, string>(StringComparer.OrdinalIgnoreCase);

View File

@@ -3,6 +3,7 @@
using System;
using System.Collections.Generic;
using Tables;
using UglyToad.PdfPig.Fonts.CompactFontFormat;
/// <summary>
/// Parses TrueType fonts.
@@ -59,7 +60,36 @@
private static TrueTypeFont ParseTables(float version, IReadOnlyDictionary<string, TrueTypeHeaderTable> tables, TrueTypeDataBytes data)
{
var isPostScript = tables.ContainsKey(TrueTypeHeaderTable.Cff);
bool isPostScript = false;
CompactFontFormatFontCollection? cffFontCollection = null;
if (tables.TryGetValue(TrueTypeHeaderTable.Cff, out var cffTable))
{
isPostScript = true;
try
{
/*
* The presence of a CFF (Compact Font Format) table in a TrueType font creates a hybrid situation where the font
* container uses TrueType structure but contains PostScript-based glyph descriptions. According to the OpenType
* specification, when a TrueType font contains a CFF table instead of a traditional glyf table, it indicates
* "an OpenType font with PostScript outlines". This creates what's known as an OpenType CFF font, which uses
* PostScript Type 2 charstrings for glyph descriptions rather than TrueType quadratic curves.
*
* This is to fix P2P-33713919.pdf
* See https://github.com/BobLd/PdfPig.Rendering.Skia/issues/46
* TODO - Add test coverage and need to review if the logic belongs here
*/
data.Seek(cffTable.Offset);
var buffer = data.ReadByteArray((int)cffTable.Length);
cffFontCollection = CompactFontFormatParser.Parse(new CompactFontFormatData(buffer));
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine(e);
// Ignore
}
}
var builder = new TableRegister.Builder();
@@ -102,7 +132,7 @@
{
builder.Os2Table = TableParser.Parse<Os2Table>(os2Table, data, builder);
}
if (!isPostScript)
{
if (!tables.TryGetValue(TrueTypeHeaderTable.Loca, out var indexToLocationHeaderTable))
@@ -121,11 +151,11 @@
// glyf
builder.GlyphDataTable = GlyphDataTable.Load(data, glyphHeaderTable, builder);
OptionallyParseTables(tables, data, builder);
}
return new TrueTypeFont(version, tables, builder.Build());
OptionallyParseTables(tables, data, builder);
return new TrueTypeFont(version, tables, builder.Build(), cffFontCollection);
}
internal static NameTable GetNameTable(TrueTypeDataBytes data)
@@ -134,7 +164,7 @@
{
throw new ArgumentNullException(nameof(data));
}
// Read these data points to move to the correct data location.
data.Read32Fixed();
int numberOfTables = data.ReadUnsignedShort();

View File

@@ -121,14 +121,23 @@
for (var i = 0; i < glyphCount; i++)
{
if (offsets[i + 1] <= offsets[i])
var offset = offsets[i];
if (offsets[i + 1] <= offset)
{
// empty glyph
result[i] = emptyGlyph;
continue;
}
data.Seek(offsets[i]);
// Invalid table, just sub in the empty glyph
if (offset >= data.Length)
{
result[i] = emptyGlyph;
continue;
}
data.Seek(offset);
var contourCount = data.ReadSignedShort();
@@ -231,9 +240,15 @@
flags = (CompositeGlyphFlags)data.ReadUnsignedShort();
var glyphIndex = data.ReadUnsignedShort();
var childGlyph = glyphs[glyphIndex];
if (glyphIndex >= glyphs.Length)
{
// Unsure why this happens but fixes #1213
continue; // TODO - Is there a better fix?
}
if (childGlyph == null)
IGlyphDescription? childGlyph = glyphs[glyphIndex];
if (childGlyph is null)
{
if (!compositeLocations.TryGetValue(glyphIndex, out var missingComposite))
{
@@ -310,7 +325,7 @@
}
}
builderGlyph = builderGlyph ?? emptyGlyph;
builderGlyph ??= emptyGlyph;
return new Glyph(false, builderGlyph.Instructions, builderGlyph.EndPointsOfContours, builderGlyph.Points, compositeLocation.Bounds);
}
@@ -329,7 +344,15 @@
for (int j = 0; j < numberOfRepeats; j++)
{
result[i + j + 1] = result[i];
int p = i + j + 1;
if (p >= result.Length)
{
// Unsure why this happens but fixes #1199
// TODO - Is there a better fix?
break;
}
result[p] = result[i];
}
i += numberOfRepeats;
@@ -409,7 +432,7 @@
}
}
private class CompositeComponent
private sealed class CompositeComponent
{
public int Index { get; }

View File

@@ -5,6 +5,7 @@
using Core;
using Parser;
using Tables.CMapSubTables;
using UglyToad.PdfPig.Fonts.CompactFontFormat;
/// <summary>
/// A TrueType font.
@@ -54,17 +55,33 @@
/// </summary>
public int NumberOfTables { get; }
// TODO - It would be better to use 'PdfCidCompactFontFormatFont' but the class is not accessible from here.
private readonly CompactFontFormatFontCollection? cffFontCollection;
/// <summary>
/// Create a new <see cref="TrueTypeFont"/>.
/// </summary>
internal TrueTypeFont(float version, IReadOnlyDictionary<string, TrueTypeHeaderTable> tableHeaders, TableRegister tableRegister)
internal TrueTypeFont(float version, IReadOnlyDictionary<string, TrueTypeHeaderTable> tableHeaders, TableRegister tableRegister, CompactFontFormatFontCollection? cffFontCollection)
{
Version = version;
TableHeaders = tableHeaders ?? throw new ArgumentNullException(nameof(tableHeaders));
TableRegister = tableRegister ?? throw new ArgumentNullException(nameof(tableRegister));
NumberOfTables = tableHeaders.Count;
if (TableRegister.CMapTable != null)
/*
* The presence of a CFF (Compact Font Format) table in a TrueType font creates a hybrid situation where the font
* container uses TrueType structure but contains PostScript-based glyph descriptions. According to the OpenType
* specification, when a TrueType font contains a CFF table instead of a traditional glyf table, it indicates
* "an OpenType font with PostScript outlines". This creates what's known as an OpenType CFF font, which uses
* PostScript Type 2 charstrings for glyph descriptions rather than TrueType quadratic curves.
*
* This is to fix P2P-33713919.pdf
* See https://github.com/BobLd/PdfPig.Rendering.Skia/issues/46
* TODO - Add test coverage and need to review if the logic belongs here
*/
this.cffFontCollection = cffFontCollection;
if (TableRegister.CMapTable is not null)
{
const int encodingSymbol = 0;
const int encodingUnicode = 1;
@@ -72,19 +89,19 @@
foreach (var subTable in TableRegister.CMapTable.SubTables)
{
if (WindowsSymbolCMap == null
if (WindowsSymbolCMap is null
&& subTable.PlatformId == TrueTypeCMapPlatform.Windows
&& subTable.EncodingId == encodingSymbol)
{
WindowsSymbolCMap = subTable;
}
else if (WindowsUnicodeCMap == null
else if (WindowsUnicodeCMap is null
&& subTable.PlatformId == TrueTypeCMapPlatform.Windows
&& subTable.EncodingId == encodingUnicode)
{
WindowsUnicodeCMap = subTable;
}
else if (MacRomanCMap == null
else if (MacRomanCMap is null
&& subTable.PlatformId == TrueTypeCMapPlatform.Macintosh
&& subTable.EncodingId == encodingMacRoman)
{
@@ -107,8 +124,36 @@
{
boundingBox = default(PdfRectangle);
if (TableRegister.GlyphTable == null)
if (TableRegister.GlyphTable is null)
{
if (cffFontCollection is not null)
{
/*
* The presence of a CFF (Compact Font Format) table in a TrueType font creates a hybrid situation where the font
* container uses TrueType structure but contains PostScript-based glyph descriptions. According to the OpenType
* specification, when a TrueType font contains a CFF table instead of a traditional glyf table, it indicates
* "an OpenType font with PostScript outlines". This creates what's known as an OpenType CFF font, which uses
* PostScript Type 2 charstrings for glyph descriptions rather than TrueType quadratic curves.
*
* This is to fix P2P-33713919.pdf
* See https://github.com/BobLd/PdfPig.Rendering.Skia/issues/46
* TODO - Add test coverage and need to review if the logic belongs here
*/
var name = cffFontCollection.FirstFont.GetCharacterName(characterCode, true); // TODO cid?
if (string.IsNullOrEmpty(name))
{
return false;
}
var bbox = cffFontCollection.FirstFont.GetCharacterBoundingBox(name);
if (bbox.HasValue)
{
boundingBox = bbox.Value;
return true;
}
}
return false;
}
@@ -143,8 +188,30 @@
{
path = null;
if (TableRegister.GlyphTable == null)
if (TableRegister.GlyphTable is null)
{
if (cffFontCollection is not null)
{
/*
* The presence of a CFF (Compact Font Format) table in a TrueType font creates a hybrid situation where the font
* container uses TrueType structure but contains PostScript-based glyph descriptions. According to the OpenType
* specification, when a TrueType font contains a CFF table instead of a traditional glyf table, it indicates
* "an OpenType font with PostScript outlines". This creates what's known as an OpenType CFF font, which uses
* PostScript Type 2 charstrings for glyph descriptions rather than TrueType quadratic curves.
*
* This is to fix P2P-33713919.pdf
* See https://github.com/BobLd/PdfPig.Rendering.Skia/issues/46
* TODO - Add test coverage and need to review if the logic belongs here
*/
var name = cffFontCollection.FirstFont.GetCharacterName(characterCode, true);
if (string.IsNullOrEmpty(name))
{
return false;
}
return cffFontCollection.FirstFont.TryGetPath(name, out path);
}
return false;
}
@@ -188,7 +255,7 @@
{
width = 0;
if (TableRegister.HorizontalMetricsTable == null)
if (TableRegister.HorizontalMetricsTable is null)
{
return false;
}
@@ -210,7 +277,7 @@
return true;
}
if (TableRegister.CMapTable == null)
if (TableRegister.CMapTable is null)
{
return false;
}

View File

@@ -18,16 +18,19 @@
private const int PfbFileIndicator = 0x80;
private static readonly char[] Separators = [' '];
private static readonly Type1EncryptedPortionParser EncryptedPortionParser = new Type1EncryptedPortionParser();
/// <summary>
/// Parses an embedded Adobe Type 1 font file.
/// </summary>
/// <param name="inputBytes">The bytes of the font program.</param>
/// <param name="length1">The length in bytes of the clear text portion of the font program.</param>
/// <param name="length2">The length in bytes of the encrypted portion of the font program.</param>
/// <param name="stackDepthGuard"></param>
/// <returns>The parsed type 1 font.</returns>
public static Type1Font Parse(IInputBytes inputBytes, int length1, int length2)
public static Type1Font Parse(IInputBytes inputBytes, int length1, int length2, StackDepthGuard stackDepthGuard)
{
// Sometimes the entire PFB file including the header bytes can be included which prevents parsing in the normal way.
var isEntirePfbFile = inputBytes.Peek() == PfbFileIndicator;
@@ -42,15 +45,15 @@
inputBytes = new MemoryInputBytes(ascii);
}
var scanner = new CoreTokenScanner(inputBytes, false);
var scanner = new CoreTokenScanner(inputBytes, false, stackDepthGuard);
if (!scanner.TryReadToken(out CommentToken comment) || !comment.Data.StartsWith("!"))
{
throw new InvalidFontFormatException("The Type1 program did not start with '%!'.");
}
string name;
var parts = comment.Data.Split(new[] { " " }, StringSplitOptions.RemoveEmptyEntries);
var parts = comment.Data.Split(Separators, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length == 3)
{
name = parts[1];
@@ -60,11 +63,9 @@
name = "Unknown";
}
var comments = new List<string>();
while (scanner.MoveNext() && scanner.CurrentToken is CommentToken commentToken)
while (scanner.MoveNext() && scanner.CurrentToken is CommentToken)
{
comments.Add(commentToken.Data);
// We ignore comments
}
var dictionaries = new List<DictionaryToken>();
@@ -441,7 +442,7 @@
return null;
}
private class PreviousTokenSet
private sealed class PreviousTokenSet
{
private readonly IToken[] tokens = new IToken[3];

View File

@@ -62,6 +62,11 @@
/// </summary>
public PdfRectangle? GetCharacterBoundingBox(string characterName)
{
if (string.Equals(characterName, GlyphList.NotDefined, StringComparison.OrdinalIgnoreCase))
{
return null;
}
var glyph = GetCharacterPath(characterName);
return PdfSubpath.GetBoundingRectangle(glyph);
}
@@ -94,8 +99,13 @@
/// <summary>
/// Get the pdfpath for the character with the given name.
/// </summary>
public IReadOnlyList<PdfSubpath> GetCharacterPath(string characterName)
public IReadOnlyList<PdfSubpath>? GetCharacterPath(string characterName)
{
if (string.Equals(characterName, GlyphList.NotDefined, StringComparison.OrdinalIgnoreCase))
{
return null;
}
if (!CharStrings.TryGenerate(characterName, out var glyph))
{
return null;

View File

@@ -1,40 +1,41 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\AdobeFontMetrics\*" />
<None Remove="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<Content Include="Resources\AdobeFontMetrics\MustRead.html">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0;net9.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.14</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
<Nullable>annotations</Nullable>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\AdobeFontMetrics\*" />
<None Remove="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<Content Include="Resources\AdobeFontMetrics\MustRead.html">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
</Project>

View File

@@ -1,6 +1,7 @@
namespace UglyToad.PdfPig.Tests.ContentStream
{
using PdfPig.Core;
using System.Globalization;
public class IndirectReferenceTests
{
@@ -33,50 +34,59 @@
[Fact]
public void IndirectReferenceHashTest()
{
var reference0 = new IndirectReference(1574, 690);
Assert.Equal(1574, reference0.ObjectNumber);
Assert.Equal(690, reference0.Generation);
CultureInfo lastCulture = CultureInfo.CurrentCulture;
CultureInfo.CurrentCulture = new CultureInfo("en-US");
try
{
var reference0 = new IndirectReference(1574, 690);
Assert.Equal(1574, reference0.ObjectNumber);
Assert.Equal(690, reference0.Generation);
var reference1 = new IndirectReference(-1574, 690);
Assert.Equal(-1574, reference1.ObjectNumber);
Assert.Equal(690, reference1.Generation);
var reference1 = new IndirectReference(-1574, 690);
Assert.Equal(-1574, reference1.ObjectNumber);
Assert.Equal(690, reference1.Generation);
var reference2 = new IndirectReference(58949797283757, 16);
Assert.Equal(58949797283757, reference2.ObjectNumber);
Assert.Equal(16, reference2.Generation);
var reference2 = new IndirectReference(58949797283757, 16);
Assert.Equal(58949797283757, reference2.ObjectNumber);
Assert.Equal(16, reference2.Generation);
var reference3 = new IndirectReference(-58949797283757, ushort.MaxValue);
Assert.Equal(-58949797283757, reference3.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference3.Generation);
var reference3 = new IndirectReference(-58949797283757, ushort.MaxValue);
Assert.Equal(-58949797283757, reference3.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference3.Generation);
var reference4 = new IndirectReference(140737488355327, ushort.MaxValue);
Assert.Equal(140737488355327, reference4.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference4.Generation);
var reference4 = new IndirectReference(140737488355327, ushort.MaxValue);
Assert.Equal(140737488355327, reference4.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference4.Generation);
var reference5 = new IndirectReference(-140737488355327, ushort.MaxValue);
Assert.Equal(-140737488355327, reference5.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference5.Generation);
var reference5 = new IndirectReference(-140737488355327, ushort.MaxValue);
Assert.Equal(-140737488355327, reference5.ObjectNumber);
Assert.Equal(ushort.MaxValue, reference5.Generation);
var ex0 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(140737488355328, 0));
Assert.StartsWith("Object number must be between -140,737,488,355,327 and 140,737,488,355,327.", ex0.Message);
var ex1 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(-140737488355328, 0));
Assert.StartsWith("Object number must be between -140,737,488,355,327 and 140,737,488,355,327.", ex1.Message);
var ex2 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(1574, -1));
Assert.StartsWith("Generation number must not be a negative value.", ex2.Message);
// We make sure object number is still correct even if generation is not
var reference6 = new IndirectReference(1574, int.MaxValue);
Assert.Equal(1574, reference6.ObjectNumber);
var reference7 = new IndirectReference(-1574, ushort.MaxValue + 10);
Assert.Equal(-1574, reference7.ObjectNumber);
var ex0 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(140737488355328, 0));
Assert.StartsWith("Object number must be between -140,737,488,355,327 and 140,737,488,355,327.", ex0.Message);
var ex1 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(-140737488355328, 0));
Assert.StartsWith("Object number must be between -140,737,488,355,327 and 140,737,488,355,327.", ex1.Message);
var reference9 = new IndirectReference(-140737488355327, ushort.MaxValue + 10);
Assert.Equal(-140737488355327, reference9.ObjectNumber);
var ex2 = Assert.Throws<ArgumentOutOfRangeException>(() => new IndirectReference(1574, -1));
Assert.StartsWith("Generation number must not be a negative value.", ex2.Message);
var reference10 = new IndirectReference(140737488355327, ushort.MaxValue * 10);
Assert.Equal(140737488355327, reference10.ObjectNumber);
// We make sure object number is still correct even if generation is not
var reference6 = new IndirectReference(1574, int.MaxValue);
Assert.Equal(1574, reference6.ObjectNumber);
var reference7 = new IndirectReference(-1574, ushort.MaxValue + 10);
Assert.Equal(-1574, reference7.ObjectNumber);
var reference9 = new IndirectReference(-140737488355327, ushort.MaxValue + 10);
Assert.Equal(-140737488355327, reference9.ObjectNumber);
var reference10 = new IndirectReference(140737488355327, ushort.MaxValue * 10);
Assert.Equal(140737488355327, reference10.ObjectNumber);
}
finally
{
CultureInfo.CurrentCulture = lastCulture;
}
}
[Fact]

View File

@@ -2,16 +2,23 @@
{
internal static class DlaHelper
{
private static readonly string DlaFolder = Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..", "..", "..", "Dla", "Documents"));
private static readonly string IntegrationFolder = Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..", "..", "..", "Integration", "Documents"));
public static string GetDocumentPath(string name, bool isPdf = true)
{
var documentFolder = Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..", "..", "..", "Dla", "Documents"));
if (!name.EndsWith(".pdf") && isPdf)
{
name += ".pdf";
}
return Path.Combine(documentFolder, name);
string doc = Path.Combine(DlaFolder, name);
if (File.Exists(doc))
{
return doc;
}
return Path.Combine(IntegrationFolder, name);
}
}
}

View File

@@ -0,0 +1 @@
Random Big Title Lorem Ipsum text with lists Lorem ipsum dolor sit amet, consectetur adipiscing elit. In sodales gravida felis, in rhoncus velit rutrum at. Curabitur hendrerit dapibus nulla, ut hendrerit diam imperdiet quis. Pellentesque id neque ali-quam, pulvinar neque in, vulputate elit. Pel-lentesque ut erat sit amet massa suscipit ullamcor-per. Sed porttitor viverra convallis. Duis vitae sem-per metus. Pellentesque eros purus, egestas eget velit eget, elementum aliquet velit. Suspendisse potenti. Nulla vitae massa rutrum, blandit erat vi-tae, aliquet arcu. Aenean feugiat leo sed enim sodales vehicula. Sus-pendisse tempus hendrerit magna sagittis dictum. Duis ultrices dapibus egestas. Cras eu felis eu lectus suscipit pharetra at at lacus. Nulla facilisi. Proin in-terdum faucibus elit nec rhoncus. Proin sodaless metus sed tincidunt hendrerit. • Duis leo enim, convallis sit amet orci eget, condimentum mattis mi ; • Etiam dolor erat, maximus nec mi sed, con-vallis convallis orci ; • Morbi viverra diam in diam cursus, vitae aliquet velit tempus ; • Donec at nisi fermentum, ultricies odio eget, egestas massa at nisi fermentum, ul-tricies odio eget, egestas massa. Donec ultricies cursus odio sed rutrum. Nam ven-enatis metus vitae elementum scelerisque. Ali-quam tempor sapien at turpis posuere eleifend. Sed placerat posuere nunc vel efficitur. Quisque auctor felis vel lectus dictum fringilla. Quisque vo-lutpat pulvinar© elit. Aliquam ultrices feugiat ali-quam. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Sus-pendisse imperdiet ex lorem, porta bibendum pu-rus ultricies id. Integer vel lacus sapien. Nam sodales ante eu risus facilisis placerat. Aliquam suscipit pulvinar ultricies. Aenean pulvinar, ex ac fermentum egestas, erat nisi feugiat velit, vitae suscipit tellus odio vitae quam. Morbi elementum sem in elit posuere, non rhoncus magna fringilla. Phasellus cursus in dolor laoreet rutrum. Curabitur tincidunt risus ullamcor-per, vehicula velit at, pulvinar metus. Donec quis ante leo. Vivamus pharetra, nisl ac vehi-cula tempor, tellus lacus aliquam sapien, eu congue nibh quam sit amet odio. Quisque metus arcu, sem-per nec consequat eu, pellentesque vel sem. Sed purus risus, tincidunt¹ sit amet dictum vitae, euis-mod id nibh. Praesent ultrices libero quis enim porta, sit amet pellentesque augue pretium. Viva-mus nec molestie nunc. Donec finibus enim nec tel-lus laoreet elementum. Curabitur efficitur placerat dolor et semper. Morbi laoreet dui eu tortor luctus, nec ultrices do-lor ullamcorper. Ut gravida sed nisl a efficitur. In tincidunt orci a condimentum semper. Suspendisse scelerisque fermentum lacinia. Vestibulum sit amet ornare tellus, aliquet euismod mauris. Cras suscipit venenatis ultrices. Sed diam erat, aliquet a tellus ut, viverra 12º ongue magna. Cras id justo tortor. Mauris in tortor vulputate, pellentesque nisl ac, facilisis ligula. Class aptent taciti² sociosqu ad li-tora torquent per conubia nostra³, per inceptos himenaeos. Aliquam eget dolor turpis. Mauris id molestie tellus. Sed elementum molestie nisi, at ali-quet sem vehicula nec. Morbi tempus nulla enim, a vulputate magna €51 luctus £66 eu. Fusce sodales, libero quis suscipit ultrices, metus erat auctor urna, sit amet dictum arcu tortor eu metus. 1. Ut volutpat, velit at interdum consectetur, nisl lorem consequat mauris, feugiat dignissim tellus massa ut nisl. 2. Praesent at est nisi. Pellentesque rutrum lorem sed dui accumsan gravida. 3. Pellentesque dictum nisl vitae urna luctus, congue pulvinar mi congue. Morbi vestibulum varius ipsum nec molestie. Proin auctor efficitur diam ut luctus. Phasellus cursus maximus ultricies. Mauris eu neque ut sem semper tempus. Curabitur non lorem eu nunc lobortis vi-verra at in diam. Pellentesque euismod purus a leo lobortis tempor. Maecenas mollis ligula at sem sus-cipit fringilla. Mauris sollicitudin tincidunt lectus id tempor. Etiam ut nisi est.

View File

@@ -4,19 +4,84 @@
public class NearestNeighbourWordExtractorTests
{
[Fact]
public void Words2559Doc()
public static IEnumerable<object[]> DataWords => new[]
{
// Microsoft Word count of words = 2559
new object[]
{
"2559 words.pdf",
5118,
2559
},
new object[]
{
"fseprd1102849.pdf",
12903,
11177
},
new object[]
{
"90 180 270 rotated.pdf",
589,
292
},
new object[]
{
"complex rotated.pdf",
805,
403
},
new object[]
{
"no horizontal distance.pdf",
4,
2
},
new object[]
{
"no vertical distance.pdf",
22,
10
},
new object[]
{
"no vertical horizontal distance.pdf",
4,
2
},
new object[]
{
"Random 2 Columns Lists Hyph - Justified.pdf",
1191,
607
},
new object[]
{
"caly-issues-56-1.pdf",
184,
156
},
new object[]
{
"caly-issues-58-2.pdf",
49,
49
},
};
using (var document = PdfDocument.Open(DlaHelper.GetDocumentPath("2559 words.pdf")))
[SkippableTheory]
[MemberData(nameof(DataWords))]
public void WordCount(string path, int wordCount, int noSpacesWordCount)
{
using (var document = PdfDocument.Open(DlaHelper.GetDocumentPath(path)))
{
var page = document.GetPage(1);
var words = NearestNeighbourWordExtractor.Instance.GetWords(page.Letters).ToArray();
Assert.Equal(wordCount, words.Length);
var noSpacesWords = words.Where(x => !string.IsNullOrEmpty(x.Text.Trim())).ToArray();
Assert.Equal(2559, noSpacesWords.Length);
Assert.Equal(noSpacesWordCount, noSpacesWords.Length);
}
}
}

View File

@@ -1,14 +1,12 @@
namespace UglyToad.PdfPig.Tests.Dla
{
using System;
using PdfFonts;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using UglyToad.PdfPig.Content;
using UglyToad.PdfPig.Core;
using UglyToad.PdfPig.DocumentLayoutAnalysis;
using UglyToad.PdfPig.DocumentLayoutAnalysis.ReadingOrderDetector;
using UglyToad.PdfPig.Core;
public class UnsupervisedReadingOrderTests
{
@@ -62,10 +60,11 @@
private static TextBlock CreateFakeTextBlock(PdfRectangle boundingBox)
{
var letter = new Letter("a",
boundingBox,
boundingBox,
boundingBox.BottomLeft,
boundingBox.BottomRight,
10, 1, null, TextRenderingMode.NeitherClip, null, null, 0, 0);// These don't matter
10, 1, (FontDetails)null, TextRenderingMode.NeitherClip, null, null, 0, 0);// These don't matter
var leftTextBlock = new TextBlock(new[] { new TextLine(new[] { new Word(new[] { letter }) }) });
return leftTextBlock;
}

View File

@@ -34,10 +34,39 @@ O<DJ+*.@<*K0@<6L(Df-\0Ec5e;DffZ(EZee.Bl.9pF""AGXBPCsi + DGm >@3BB / F * &OCAfu2
text);
}
[Fact]
public void ReplacesZWithEmptyBytes()
[Theory]
[InlineData("BE", "h")]
[InlineData("BOq", "he")]
[InlineData("BOtu", "hel")]
[InlineData("BOu!r", "hell")]
[InlineData("BOu!rDZ", "hello")]
[InlineData("BOu!rD]f", "hello ")]
[InlineData("BOu!rD]j6", "hello w")]
[InlineData("BOu!rD]j7B", "hello wo")]
[InlineData("BOu!rD]j7BEW", "hello wor")]
[InlineData("BOu!rD]j7BEbk", "hello worl")]
[InlineData("BOu!rD]j7BEbo7", "hello world")]
[InlineData("BOu!rD]j7BEbo80", "hello world!")]
public void DecodesHelloWorld(string encoded, string decoded)
{
var bytes = Encoding.ASCII.GetBytes("9jqo^zBlbD-");
var result = filter.Decode(
Encoding.ASCII.GetBytes(encoded),
dictionary,
TestFilterProvider.Instance,
0);
Assert.Equal(decoded, Encoding.ASCII.GetString(result.ToArray()));
}
[Theory]
[InlineData("9jqo^zBlbD-", "Man \0\0\0\0is d")]
[InlineData("", "")]
[InlineData("z", "\0\0\0\0")]
[InlineData("zz", "\0\0\0\0\0\0\0\0")]
[InlineData("zzz", "\0\0\0\0\0\0\0\0\0\0\0\0")]
public void ReplacesZWithEmptyBytes(string encoded, string decoded)
{
var bytes = Encoding.ASCII.GetBytes(encoded);
var result = filter.Decode(bytes, dictionary, TestFilterProvider.Instance, 1);
@@ -47,7 +76,7 @@ O<DJ+*.@<*K0@<6L(Df-\0Ec5e;DffZ(EZee.Bl.9pF""AGXBPCsi + DGm >@3BB / F * &OCAfu2
string text = Encoding.ASCII.GetString(result.Span);
#endif
Assert.Equal("Man \0\0\0\0is d", text);
Assert.Equal(decoded, text);
}
[Fact]
@@ -60,14 +89,17 @@ O<DJ+*.@<*K0@<6L(Df-\0Ec5e;DffZ(EZee.Bl.9pF""AGXBPCsi + DGm >@3BB / F * &OCAfu2
Assert.Throws<InvalidOperationException>(action);
}
[Fact]
public void SingleCharacterLastThrows()
[Theory]
[InlineData("@rH:%B", "cool")]
[InlineData("A~>", "")]
[InlineData("@rH:%A~>", "cool")]
public void SingleCharacterLastIgnores(string encoded, string decoded)
{
var bytes = Encoding.ASCII.GetBytes("9jqo^B");
var bytes = Encoding.ASCII.GetBytes(encoded);
Action action = () => filter.Decode(bytes, dictionary, TestFilterProvider.Instance, 1);
var result = filter.Decode(bytes, dictionary, TestFilterProvider.Instance, 1);
Assert.Throws<ArgumentOutOfRangeException>(action);
Assert.Equal(decoded, Encoding.ASCII.GetString(result.ToArray()));
}
private const string PdfContent = @"1 0 obj

View File

@@ -1,5 +1,6 @@
namespace UglyToad.PdfPig.Tests.Filters
{
using PdfPig.Core;
using PdfPig.Filters;
using PdfPig.Tokens;
@@ -11,15 +12,32 @@
public void EncodeAndDecodePreservesInput()
{
var parameters = new DictionaryToken(new Dictionary<NameToken, IToken>());
var input = new byte[] {67, 69, 69, 10, 4, 20, 6, 19, 120, 64, 64, 64, 32};
var input = new byte[] { 67, 69, 69, 10, 4, 20, 6, 19, 120, 64, 64, 64, 32 };
using (var inputStream = new MemoryStream(input))
{
inputStream.Seek(0, SeekOrigin.Begin);
var result = filter.Encode(inputStream, parameters, 0);
var result = filter.Encode(inputStream, parameters);
var decoded = filter.Decode(result, parameters, TestFilterProvider.Instance, 0);
Assert.Equal(input, decoded.ToArray());
}
}
[Fact]
public void CanDecodeCorruptedInputIssue1235()
{
const string hexStr =
"789C958D5D0AC2400C844FB077980B74BB7FD9D982F820B43E8B7B03C542C187EAFDC1F84B7D1164200999E49BD9044C6653D10E1E443DA1AF6636ED76EF315E7572968E1ECDAB7FB7506C4C59C0AEB3912EE270366AAAF4E36D364BF7911450DC274A5112B1AC9751D77A58680B51A4D8AE433D62953C037396E0F290FBE098B267A43051725AA34E77E44EF50B1B52B42C90E4ADF83FB94FDD0000000000";
var hex = new HexToken(hexStr.AsSpan());
var parameters = new DictionaryToken(new Dictionary<NameToken, IToken>());
var result = filter.Decode(hex.Bytes.ToArray(), parameters, TestFilterProvider.Instance, 0);
var text = OtherEncodings.BytesAsLatin1String(result.ToArray());
Assert.StartsWith("q", text);
}
}
}

View File

@@ -105,7 +105,7 @@
{
var parser = new CodespaceRangeParser();
var byteArrayInput = new MemoryInputBytes(OtherEncodings.StringAsLatin1Bytes("1 begincodespacerange\nendcodespacerange"));
var tokenScanner = new CoreTokenScanner(byteArrayInput, false);
var tokenScanner = new CoreTokenScanner(byteArrayInput, false, new StackDepthGuard(256));
Assert.True(tokenScanner.MoveNext());
Assert.True(tokenScanner.CurrentToken is NumericToken);

View File

@@ -1,5 +1,6 @@
namespace UglyToad.PdfPig.Tests.Fonts.Encodings
{
using System.Text;
using PdfPig.Fonts;
public class GlyphListTests
@@ -92,7 +93,7 @@
}
[Fact(Skip = "TODO - String don't match")]
[Fact]
public void NameToUnicodeConvertAglSpecification()
{
// https://github.com/adobe-type-tools/agl-specification?tab=readme-ov-file#3-examples
@@ -103,7 +104,22 @@
var result = list.NameToUnicode("Lcommaaccent_uni20AC0308_u1040C.alternate");
Assert.Equal("\u013B\u20AC\u0308\u1040C", result);
// This value is not encodable in single characters in UTF-16, so we get a surrogate pair for the final unicode character
Assert.Equal("\u013B\u20AC\u0308\uD801\uDC0C", result);
#if NET6_0_OR_GREATER
// But in .Net we can get the unicode rune values to verify that this is really the expected value
var runes = result.EnumerateRunes().ToList();
Assert.Equal(4, runes.Count);
Assert.Equal(0x013B, runes[0].Value);
Assert.Equal(0x20AC, runes[1].Value);
Assert.Equal(0x0308, runes[2].Value);
Assert.Equal(0x1040C, runes[3].Value);
#endif
// Ok, so we know this is what the actual string is. Now lets encode that last value the C# way
Assert.Equal("\u013B\u20AC\u0308\U0001040C", result);
}
}
}

View File

@@ -146,6 +146,72 @@ endbfchar";
Assert.Equal("10.003", cmap.Version);
}
[Fact]
public void CanParseFile0001413CMap()
{
const string cmap =
"""
/CIDInit /ProcSet findresource
begin
12 dict
begin
begincmap
/CIDSystemInfo <</Ordering (GB1) /Registry (Adobe) /Supplement 2 >> def
/CMapName /Adobe-GB1-2 def
/CMapType 2 def
1 begincodespacerange
<0000> <ffff> endcodespacerange
100 beginbfrange
<0001> <0002> <0022> <0003> <0019> <0025> <001b> <0023> <003f> <0024> <0025> <0049> <0026> <0033> <004c> <0034> <0036> <005d> <0037> <003d> <0061> <003e> <004b> <0069> <004c> <004d> <0079> <0053> <0054> <3001> <0055> <005a> <300a> <005c> <005d> <4e00> <0060> <0062> <4e09> <0063> <0064> <4e0d> <0066> <0067> <4e13> <006a> <006d> <4e1a> <006f> <0070> <4e24> <0076> <0079> <4e38> <007a> <007b> <4e3d> <007e> <007f> <4e48> <0080> <0085> <4e4b> <0089> <008b> <4e5f> <008d> <008e> <4e70> <0090> <0091> <4e88> <0092> <0093> <4e8b> <0094> <0095> <4e8e> <0097> <0098> <4e94> <0099> <009a> <4e9a> <009c> <00a0> <4ea4> <00a3> <00a4> <4ec0> <00a5> <00a6> <4ec5> <00a7> <00a8> <4eca> <00a9> <00aa> <4ecd> <00ac> <00af> <4ed3> <00b0> <00b1> <4ed8> <00b3> <00b5> <4ee3> <00b9> <00ba> <4ef6> <00bf> <00c1> <4f0f> <00c2> <00c5> <4f17> <00c6> <00c7> <4f1f> <00cb> <00cc> <4f2f> <00d2> <00d4> <4f4d> <00d8> <00d9> <4f5b> <00e2> <00e3> <4fa6> <00e7> <00e8> <4fc3> <00ef> <00f0> <5018> <0108> <010a> <516b> <010b> <010c> <5170> <010d> <010e> <5173> <010f> <0112> <5176> <0113> <0114> <517b> <0117> <0118> <518c> <011a> <011b> <519b> <011e> <011f> <51b2> <0124> <0125> <51e0> <0128> <012a> <51f9> <012b> <012c> <5200> <012d> <012e> <5206> <0132> <0134> <5219> <0138> <0139> <5228> <0144> <0147> <529e> <0149> <014b> <52a8> <0156> <0157> <5316> <0158> <0159> <533a> <015e> <015f> <534e> <0161> <0163> <5355> <0167> <0169> <5360> <016b> <016c> <5370> <016d> <016e> <5373> <0175> <0176> <538b> <017c> <0181> <53c8> <0183> <0186> <53d6> <0187> <018a> <53e3> <018b> <0193> <53e8> <0194> <0195> <53f2> <0196> <0197> <53f7> <019a> <019b> <5403> <019c> <019d> <5408> <019e> <01a0> <540c> <01a4> <01a5> <542b> <01ae> <01b0> <547b> <01be> <01bf> <552e> <01cb> <01cc> <56da> <01d5> <01d6> <56fd> <01e3> <01e4> <574e> <01e7> <01e8> <575a> <01e9> <01ea> <5760> <01f8> <01f9> <57f9> <0211> <0212> <5915> <0216> <0218> <5929> <0223> <0224> <5956> <0233> <0234> <5a03> <023e> <023f> <5b57> <0242> <0243> <5b63> <0248> <0249> <5b88> <024e> <0250> <5b9c> <0251> <0253> <5ba1> <0256> <0257> <5bb3> <025a> <025c> <5bbd> endbfrange
100 beginbfrange
<0260> <0264> <5bf8> <0268> <0269> <5c09> <0272> <0274> <5c3c> <0279> <027a> <5c4a> <0286> <0289> <5de5> <028c> <028f> <5df1> <0291> <0293> <5e01> <0298> <0299> <5e15> <029a> <029b> <5e1c> <029d> <029e> <5e2d> <02a2> <02a4> <5e72> <02ab> <02ad> <5e93> <02b0> <02b1> <5ea6> <02b3> <02b4> <5eb6> <02b6> <02b7> <5ef6> <02b9> <02ba> <5eff> <02bb> <02bc> <5f02> <02c1> <02c2> <5f17> <02c7> <02c8> <5f39> <02c9> <02ca> <5f52> <02cf> <02d0> <5f7b> <02d1> <02d2> <5f80> <02d3> <02d4> <5f84> <02e6> <02e7> <6000> <0300> <0301> <6167> <0308> <0309> <6210> <030e> <030f> <623f> <0311> <0312> <624d> <0315> <0316> <6266> <031c> <031e> <627e> <0321> <0324> <6295> <0327> <0328> <62a4> <0330> <0331> <62cc> <0332> <0333> <62d8> <0337> <0338> <62e8> <033b> <033c> <6301> <0340> <0341> <631a> <0355> <0356> <63a7> <0358> <0359> <63cf> <036b> <036c> <653e> <037d> <037e> <65af> <0387> <038b> <65e5> <038d> <038e> <65f6> <0393> <0394> <661f> <0397> <0398> <666e> <03a0> <03a2> <66fe> <03a3> <03a4> <6708> <03aa> <03ad> <672a> <03b2> <03b3> <6742> <03b4> <03b5> <6750> <03c2> <03c3> <67d0> <03ca> <03cc> <6837> <03db> <03de> <6b20> <03e2> <03e7> <6b62> <03e8> <03e9> <6b8a> <03f1> <03f2> <6bd4> <03fb> <03fc> <6c2e> <040d> <040e> <6cbb> <040f> <0410> <6cbe> <0421> <0422> <6d3d> <0425> <0426> <6d4a> <042d> <042e> <6d88> <043b> <043c> <6e20> <0457> <0458> <707e> <045a> <045b> <70b8> <045d> <045e> <70c2> <046d> <046e> <7247> <0476> <0477> <7279> <047b> <047c> <72ec> <0485> <0486> <73af> <0494> <0495> <7530> <0498> <0499> <7537> <04a9> <04aa> <767d> <04b0> <04b1> <76d0> <04ca> <04cb> <786b> <04d5> <04d6> <795d> <04e1> <04e2> <79ef> <04e9> <04ea> <7a76> <04f2> <04f4> <7ade> <0515> <0516> <7ea6> <0519> <051a> <7eb2> <051e> <051f> <7ec3> <0520> <0522> <7ec6> <0526> <0528> <7ed8> <0529> <052a> <7edc> <0531> <0532> <7f13> <0542> <0543> <800c> <0546> <0547> <804b> <0564> <0565> <81f3> <056f> <0570> <826f> <0579> <057a> <82e5> <0586> <0587> <83b1> <05b6> <05b7> <89c1> <05ba> <05bb> <89c8> <05c1> <05c2> <8ba1> <05c4> <05c5> <8ba8> <05c6> <05c9> <8bad> <05cd> <05ce> <8bbe> <05da> <05db> <8be5> <05dc> <05dd> <8bec> endbfrange
33 beginbfrange
<05df> <05e0> <8bf1> <05e2> <05e3> <8bf7> <05f4> <05f6> <8d21> <05f8> <05f9> <8d27> <05fa> <05fb> <8d2a> <05fc> <05fd> <8d2e> <05ff> <0601> <8d37> <060e> <060f> <8d8a> <061e> <061f> <8f90> <0623> <0624> <8fa8> <0629> <062a> <8fc4> <062d> <062e> <8fd0> <0630> <0631> <8fd8> <0632> <0636> <8fdb> <0637> <0638> <8fea> <063d> <0640> <9000> <0641> <0642> <9009> <0646> <0647> <901a> <0648> <0649> <901f> <0651> <0652> <906d> <0654> <0655> <907f> <0658> <0659> <90ae> <065d> <065e> <90e7> <0660> <0662> <914b> <0667> <066a> <91cc> <0671> <0672> <94c0> <067f> <0680> <95ed> <068d> <068f> <9644> <0694> <0695> <9676> <069c> <069d> <96c5> <069f> <06a0> <96f6> <06b0> <06b2> <9879> <06b6> <06b7> <9886> <06bc> <06bd> <98de> endbfrange
100 beginbfchar
<0000> <0020> <001a> <003d> <004e> <007d> <004f> <00a8> <0050> <2014> <0051> <2026> <0052> <25a1> <005b> <ff0c> <005e> <4e03> <005f> <4e07> <0065> <4e11> <0068> <4e16> <0069> <4e18> <006e> <4e22> <0071> <4e27> <0072> <4e2a> <0073> <4e2d> <0074> <4e30> <0075> <4e34> <007c> <4e43> <007d> <4e45> <0086> <4e53> <0087> <4e59> <0088> <4e5d> <008c> <4e66> <008f> <4e86> <0096> <4e92> <009b> <4ea2> <00a1> <4eab> <00a2> <4eba> <00ab> <4ed1> <00b2> <4edf> <00b6> <4eea> <00b7> <4eec> <00b8> <4ef2> <00bb> <4efb> <00bc> <4efd> <00bd> <4f01> <00be> <4f0a> <00c8> <4f24> <00c9> <4f26> <00ca> <4f2a> <00cd> <4f34> <00ce> <4f36> <00cf> <4f38> <00d0> <4f3c> <00d1> <4f46> <00d5> <4f53> <00d6> <4f55> <00d7> <4f59> <00da> <4f69> <00db> <4f6f> <00dc> <4f73> <00dd> <4f7f> <00de> <4f8b> <00df> <4f9b> <00e0> <4f9d> <00e1> <4fa3> <00e4> <4fae> <00e5> <4fb5> <00e6> <4fbf> <00e9> <4fdd> <00ea> <4fe1> <00eb> <4fee> <00ec> <4ff1> <00ed> <500d> <00ee> <5012> <00f1> <501f> <00f2> <5021> <00f3> <503c> <00f4> <503e> <00f5> <5047> <00f6> <505a> <00f7> <505c> <00f8> <5065> <00f9> <507f> <00fa> <5085> <00fb> <50a8> <00fc> <50b2> <00fd> <50f3> <00fe> <50fb> <00ff> <513f> <0100> <5141> <0101> <5143> <0102> <5145> <0103> <5148> <0104> <514b> <0105> <514d> <0106> <5165> <0107> <5168> <0115> <5180> <0116> <5185> <0119> <5199> <011c> <51ac> <011d> <51b0> <0120> <51b5> <0121> <51c0> <0122> <51c6> <0123> <51cf> <0126> <51ed> endbfchar
100 beginbfchar
<0127> <51ef> <012f> <520a> <0130> <5212> <0131> <5217> <0135> <521d> <0136> <5220> <0137> <5224> <013a> <522b> <013b> <5230> <013c> <5236> <013d> <523b> <013e> <5242> <013f> <524a> <0140> <524d> <0141> <5269> <0142> <526f> <0143> <529b> <0148> <52a3> <014c> <52b1> <014d> <52b3> <014e> <52bf> <014f> <52c3> <0150> <52d2> <0151> <52d8> <0152> <52e4> <0153> <52fa> <0154> <5300> <0155> <5305> <015a> <5341> <015b> <5343> <015c> <5347> <015d> <534a> <0160> <5353> <0164> <535a> <0165> <535c> <0166> <535e> <016a> <536b> <016f> <5378> <0170> <537f> <0171> <5382> <0172> <5384> <0173> <5386> <0174> <5389> <0177> <5398> <0178> <539a> <0179> <539f> <017a> <53bb> <017b> <53c2> <0182> <53d1> <0198> <53fc> <0199> <5401> <01a1> <5411> <01a2> <5417> <01a3> <5426> <01a6> <542f> <01a7> <5431> <01a8> <5446> <01a9> <544a> <01aa> <5458> <01ab> <5468> <01ac> <5473> <01ad> <5475> <01b1> <548c> <01b2> <5490> <01b3> <5496> <01b4> <54a8> <01b5> <54b1> <01b6> <54c1> <01b7> <54c8> <01b8> <54cd> <01b9> <54df> <01ba> <54e5> <01bb> <54ea> <01bc> <5510> <01bd> <5524> <01c0> <5531> <01c1> <5546> <01c2> <554a> <01c3> <5564> <01c4> <5566> <01c5> <5580> <01c6> <5584> <01c7> <558a> <01c8> <559c> <01c9> <5668> <01ca> <566a> <01cd> <56de> <01ce> <56e0> <01cf> <56e2> <01d0> <56e4> <01d1> <56ed> <01d2> <56f0> <01d3> <56f4> <01d4> <56fa> <01d7> <5703> <01d8> <5706> <01d9> <5708> <01da> <571f> <01db> <5723> <01dc> <5728> endbfchar
100 beginbfchar
<01dd> <572d> <01de> <5730> <01df> <573a> <01e0> <5740> <01e1> <5747> <01e2> <574a> <01e5> <5751> <01e6> <5757> <01eb> <5764> <01ec> <5766> <01ed> <5782> <01ee> <5784> <01ef> <578b> <01f0> <5792> <01f1> <579b> <01f2> <57a6> <01f3> <57ab> <01f4> <57c3> <01f5> <57ce> <01f6> <57d4> <01f7> <57df> <01fa> <5802> <01fb> <5806> <01fc> <5815> <01fd> <5821> <01fe> <5824> <01ff> <582a> <0200> <5851> <0201> <5854> <0202> <585e> <0203> <586b> <0204> <5883> <0205> <5893> <0206> <5899> <0207> <589e> <0208> <58a8> <0209> <58c1> <020a> <58e4> <020b> <58eb> <020c> <58f0> <020d> <5904> <020e> <5907> <020f> <590d> <0210> <590f> <0213> <591a> <0214> <591f> <0215> <5927> <0219> <592e> <021a> <5931> <021b> <5934> <021c> <5938> <021d> <593a> <021e> <5947> <021f> <5949> <0220> <594b> <0221> <594e> <0222> <5951> <0225> <5962> <0226> <5965> <0227> <5973> <0228> <5979> <0229> <597d> <022a> <5982> <022b> <5999> <022c> <59a5> <022d> <59a8> <022e> <59bb> <022f> <59c6> <0230> <59cb> <0231> <59d4> <0232> <5a01> <0235> <5a1c> <0236> <5a31> <0237> <5a74> <0238> <5a92> <0239> <5ac1> <023a> <5acc> <023b> <5ae9> <023c> <5b50> <023d> <5b54> <0240> <5b5c> <0241> <5b5f> <0244> <5b66> <0245> <5b6a> <0246> <5b83> <0247> <5b85> <024a> <5b8c> <024b> <5b8f> <024c> <5b97> <024d> <5b9a> <0254> <5baa> <0255> <5bb0> <0258> <5bb6> <0259> <5bb9> <025d> <5bc6> <025e> <5bcc> <025f> <5bdf> <0265> <5bff> <0266> <5c04> <0267> <5c06> endbfchar
100 beginbfchar
<026a> <5c0f> <026b> <5c11> <026c> <5c14> <026d> <5c1a> <026e> <5c1d> <026f> <5c24> <0270> <5c31> <0271> <5c38> <0275> <5c40> <0276> <5c42> <0277> <5c45> <0278> <5c48> <027b> <5c55> <027c> <5c5e> <027d> <5c65> <027e> <5c71> <027f> <5c7f> <0280> <5c97> <0281> <5c9b> <0282> <5cad> <0283> <5cb8> <0284> <5ddd> <0285> <5de1> <028a> <5deb> <028b> <5dee> <0290> <5dfe> <0294> <5e05> <0295> <5e08> <0296> <5e0c> <0297> <5e10> <029c> <5e26> <029f> <5e38> <02a0> <5e45> <02a1> <5e55> <02a5> <5e76> <02a6> <5e78> <02a7> <5e7f> <02a8> <5e86> <02a9> <5e8a> <02aa> <5e8f> <02ae> <5e9c> <02af> <5e9f> <02b2> <5ead> <02b5> <5ec9> <02b8> <5efa> <02bd> <5f0a> <02be> <5f0f> <02bf> <5f13> <02c0> <5f15> <02c3> <5f20> <02c4> <5f25> <02c5> <5f27> <02c6> <5f31> <02cb> <5f55> <02cc> <5f62> <02cd> <5f64> <02ce> <5f71> <02d5> <5f88> <02d6> <5f8b> <02d7> <5f92> <02d8> <5f97> <02d9> <5fa1> <02da> <5faa> <02db> <5fae> <02dc> <5fb5> <02dd> <5fb7> <02de> <5fc3> <02df> <5fc5> <02e0> <5fcd> <02e1> <5fd7> <02e2> <5fd9> <02e3> <5feb> <02e4> <5ff1> <02e5> <5ff5> <02e8> <600e> <02e9> <6015> <02ea> <601d> <02eb> <6025> <02ec> <6027> <02ed> <603b> <02ee> <6050> <02ef> <6062> <02f0> <6069> <02f1> <606f> <02f2> <6076> <02f3> <6089> <02f4> <60a0> <02f5> <60ac> <02f6> <60c5> <02f7> <60ca> <02f8> <60e7> <02f9> <60ef> <02fa> <60f3> <02fb> <6108> <02fc> <610f> <02fd> <611f> <02fe> <613f> <02ff> <614e> <0302> <6177> <0303> <61be> endbfchar
100 beginbfchar
<0304> <61c2> <0305> <61c8> <0306> <61d2> <0307> <6208> <030a> <6216> <030b> <6218> <030c> <621a> <030d> <6234> <0310> <624b> <0313> <6258> <0314> <6263> <0317> <6269> <0318> <6270> <0319> <6273> <031a> <6276> <031b> <6279> <031f> <628a> <0320> <6291> <0325> <629b> <0326> <62a2> <0329> <62b1> <032a> <62b5> <032b> <62b9> <032c> <62bd> <032d> <62c2> <032e> <62c5> <032f> <62c9> <0334> <62db> <0335> <62df> <0336> <62e5> <0339> <62ec> <033a> <62ff> <033d> <6307> <033e> <6309> <033f> <6311> <0342> <631d> <0343> <6325> <0344> <6328> <0345> <632a> <0346> <633d> <0347> <634e> <0348> <6350> <0349> <6355> <034a> <635f> <034b> <6362> <034c> <636e> <034d> <6377> <034e> <6388> <034f> <638c> <0350> <6392> <0351> <6398> <0352> <63a0> <0353> <63a2> <0354> <63a5> <0357> <63aa> <035a> <63d2> <035b> <63e1> <035c> <63ed> <035d> <63f4> <035e> <641c> <035f> <641e> <0360> <642d> <0361> <6458> <0362> <6469> <0363> <64a4> <0364> <64ad> <0365> <64bc> <0366> <64cd> <0367> <652f> <0368> <6536> <0369> <6539> <036a> <653b> <036d> <6545> <036e> <6548> <036f> <6551> <0370> <6559> <0371> <6563> <0372> <6566> <0373> <656c> <0374> <6570> <0375> <6574> <0376> <6587> <0377> <6597> <0378> <6599> <0379> <659f> <037b> <65a5> <037c> <65ad> <037f> <65b9> <0380> <65bd> <0381> <65c5> <0382> <65cb> <0383> <65cf> <0384> <65d7> <0385> <65e0> <0386> <65e2> <038c> <65f1> <038f> <65fa> <0390> <6606> <0391> <660e> <0392> <6613> endbfchar
100 beginbfchar
<0395> <662f> <0396> <663e> <0399> <667a> <039a> <6682> <039b> <66b4> <039c> <66dd> <039d> <66f2> <039e> <66f4> <039f> <66fc> <03a5> <670d> <03a6> <6717> <03a7> <671b> <03a8> <671f> <03a9> <6728> <03ae> <672f> <03af> <6734> <03b0> <673a> <03b1> <6740> <03b6> <675f> <03b7> <6761> <03b8> <6765> <03b9> <6770> <03ba> <677e> <03bb> <6781> <03bc> <6784> <03bd> <6790> <03be> <6797> <03bf> <679c> <03c0> <67af> <03c1> <67b6> <03c4> <67d3> <03c5> <67e5> <03c6> <6807> <03c7> <6811> <03c8> <6816> <03c9> <6821> <03cd> <683c> <03ce> <6848> <03cf> <6851> <03d0> <6863> <03d1> <6886> <03d2> <68a2> <03d3> <68c0> <03d4> <68da> <03d5> <68ee> <03d6> <690d> <03d7> <695a> <03d8> <6982> <03d9> <6a0a> <03da> <6a21> <03df> <6b27> <03e0> <6b32> <03e1> <6b3e> <03ea> <6b96> <03eb> <6bb5> <03ec> <6bc1> <03ed> <6bcb> <03ee> <6bcd> <03ef> <6bcf> <03f0> <6bd2> <03f3> <6bd7> <03f4> <6bdb> <03f5> <6beb> <03f6> <6c0f> <03f7> <6c11> <03f8> <6c14> <03f9> <6c1b> <03fa> <6c27> <03fd> <6c34> <03fe> <6c38> <03ff> <6c42> <0400> <6c47> <0401> <6c50> <0402> <6c57> <0403> <6c5f> <0404> <6c61> <0405> <6c7e> <0406> <6c89> <0407> <6c99> <0408> <6c9f> <0409> <6ca1> <040a> <6caa> <040b> <6cb3> <040c> <6cb9> <0411> <6cc4> <0412> <6cca> <0413> <6cd5> <0414> <6cdb> <0415> <6ce2> <0416> <6ce5> <0417> <6ce8> <0418> <6cf0> <0419> <6cfb> <041a> <6cfd> <041b> <6d01> <041c> <6d0b> <041d> <6d1b> <041e> <6d2a> <041f> <6d32> <0420> <6d3b> endbfchar
100 beginbfchar
<0423> <6d41> <0424> <6d46> <0427> <6d4e> <0428> <6d53> <0429> <6d66> <042a> <6d6a> <042b> <6d6e> <042c> <6d77> <042f> <6d95> <0430> <6da8> <0431> <6db2> <0432> <6dc0> <0433> <6dc6> <0434> <6de1> <0435> <6df1> <0436> <6df7> <0437> <6dfb> <0438> <6e05> <0439> <6e10> <043a> <6e14> <043d> <6e23> <043e> <6e29> <043f> <6e2f> <0440> <6e38> <0441> <6e56> <0442> <6e58> <0443> <6e7f> <0444> <6e89> <0445> <6e90> <0446> <6ecb> <0447> <6ee1> <0448> <6ee8> <0449> <6f0f> <044a> <6f14> <044b> <6f20> <044c> <6f2b> <044d> <6f5c> <044e> <6f6e> <044f> <6f84> <0450> <6fb3> <0451> <6fc0> <0452> <704c> <0453> <706b> <0454> <706d> <0455> <7070> <0456> <7075> <0459> <70ad> <045c> <70bc> <045f> <70c8> <0460> <70e7> <0461> <70ed> <0462> <711a> <0463> <7136> <0464> <7164> <0465> <7167> <0466> <719f> <0467> <71c3> <0468> <71e5> <0469> <7206> <046a> <722a> <046b> <722c> <046c> <7231> <046f> <7259> <0470> <725b> <0471> <7261> <0472> <7267> <0473> <7269> <0474> <7272> <0475> <7275> <0478> <72af> <0479> <72b6> <047a> <72de> <047d> <730e> <047e> <731b> <047f> <732e> <0480> <736d> <0481> <7387> <0482> <738b> <0483> <7396> <0484> <73ab> <0487> <73cd> <0488> <73d0> <0489> <73ed> <048a> <7403> <048b> <7406> <048c> <745e> <048d> <74a7> <048e> <74dc> <048f> <74e3> <0490> <74e6> <0491> <751a> <0492> <751f> <0493> <7528> <0496> <7533> <0497> <7535> <049a> <754c> <049b> <7559> <049c> <755c> <049d> <7565> <049e> <7574> endbfchar
100 beginbfchar
<049f> <7578> <04a0> <7591> <04a1> <75ab> <04a2> <75be> <04a3> <75c5> <04a4> <75c7> <04a5> <75d8> <04a6> <75db> <04a7> <764c> <04a8> <767b> <04ab> <7684> <04ac> <7687> <04ad> <76bf> <04ae> <76c6> <04af> <76ca> <04b2> <76d6> <04b3> <76d8> <04b4> <76df> <04b5> <76ee> <04b6> <76f2> <04b7> <76f4> <04b8> <76f8> <04b9> <76fe> <04ba> <770b> <04bb> <771f> <04bc> <773c> <04bd> <7740> <04be> <7763> <04bf> <77db> <04c0> <77e5> <04c1> <77ed> <04c2> <77ff> <04c3> <7801> <04c4> <780d> <04c5> <7814> <04c6> <7834> <04c7> <7838> <04c8> <7840> <04c9> <7845> <04cc> <786e> <04cd> <7877> <04ce> <788d> <04cf> <7891> <04d0> <78b0> <04d1> <78b3> <04d2> <78cb> <04d3> <793a> <04d4> <793e> <04d7> <7965> <04d8> <7968> <04d9> <7978> <04da> <7981> <04db> <798f> <04dc> <79bb> <04dd> <79c1> <04de> <79cd> <04df> <79d1> <04e0> <79d8> <04e3> <79fb> <04e4> <7a00> <04e5> <7a0b> <04e6> <7a0e> <04e7> <7a33> <04e8> <7a46> <04eb> <7a7a> <04ec> <7a81> <04ed> <7a84> <04ee> <7a92> <04ef> <7a96> <04f0> <7acb> <04f1> <7ad9> <04f5> <7ae5> <04f6> <7aed> <04f7> <7aef> <04f8> <7af9> <04f9> <7b14> <04fa> <7b26> <04fb> <7b2c> <04fc> <7b49> <04fd> <7b4f> <04fe> <7b51> <04ff> <7b54> <0500> <7b56> <0501> <7b79> <0502> <7b7e> <0503> <7b80> <0504> <7b97> <0505> <7ba1> <0506> <7c73> <0507> <7c7b> <0508> <7c92> <0509> <7c95> <050a> <7cae> <050b> <7cbe> <050c> <7cca> <050d> <7cfb> <050e> <7d20> <050f> <7d22> <0510> <7d27> <0511> <7d2f> endbfchar
100 beginbfchar
<0512> <7e41> <0513> <7e82> <0514> <7ea0> <0517> <7eaa> <0518> <7eaf> <051b> <7eb5> <051c> <7ebd> <051d> <7ebf> <0523> <7ecd> <0524> <7ecf> <0525> <7ed3> <052b> <7edf> <052c> <7ee7> <052d> <7ee9> <052e> <7eed> <052f> <7ef4> <0530> <7efc> <0533> <7f16> <0534> <7f29> <0535> <7f3a> <0536> <7f51> <0537> <7f57> <0538> <7f6e> <0539> <7f72> <053a> <7f8e> <053b> <7fa4> <053c> <7fc5> <053d> <7ff0> <053e> <7ffb> <053f> <8001> <0540> <8003> <0541> <8005> <0544> <8017> <0545> <8033> <0548> <8054> <0549> <8058> <054a> <805a> <054b> <806a> <054c> <8083> <054d> <8089> <054e> <8098> <054f> <809b> <0550> <80a5> <0551> <80a9> <0552> <80af> <0553> <80b2> <0554> <80c1> <0555> <80cc> <0556> <80ce> <0557> <80da> <0558> <80dc> <0559> <80fd> <055a> <8106> <055b> <811a> <055c> <8131> <055d> <814a> <055e> <8155> <055f> <8179> <0560> <818a> <0561> <818f> <0562> <81ea> <0563> <81ed> <0566> <8206> <0567> <820c> <0568> <8212> <0569> <821f> <056a> <822a> <056b> <822c> <056c> <8236> <056d> <8239> <056e> <8247> <0571> <8272> <0572> <827a> <0573> <8282> <0574> <82a6> <0575> <82ac> <0576> <82af> <0577> <82c7> <0578> <82cf> <057b> <82ef> <057c> <82f1> <057d> <8303> <057e> <8328> <057f> <8349> <0580> <835a> <0581> <8363> <0582> <836f> <0583> <8377> <0584> <8392> <0585> <83ab> <0588> <83b7> <0589> <83c7> <058a> <83cc> <058b> <83f2> <058c> <8404> <058d> <841d> <058e> <8425> <058f> <8428> <0590> <843d> <0591> <8461> endbfchar
100 beginbfchar
<0592> <848b> <0593> <8499> <0594> <84c4> <0595> <84dd> <0596> <8584> <0597> <85c9> <0598> <85cf> <0599> <8651> <059a> <866b> <059b> <867d> <059c> <8680> <059d> <8682> <059e> <86cb> <059f> <8718> <05a0> <8757> <05a1> <878d> <05a2> <87f9> <05a3> <8840> <05a4> <884c> <05a5> <8854> <05a6> <8857> <05a7> <8861> <05a8> <8863> <05a9> <8865> <05aa> <8868> <05ab> <8877> <05ac> <8896> <05ad> <88ab> <05ae> <88c1> <05af> <88f4> <05b0> <8902> <05b1> <8925> <05b2> <8944> <05b3> <897f> <05b4> <8981> <05b5> <8986> <05b8> <89c4> <05b9> <89c6> <05bc> <89e3> <05bd> <89e6> <05be> <8a00> <05bf> <8b66> <05c0> <8b6c> <05c3> <8ba4> <05ca> <8bb2> <05cb> <8bb8> <05cc> <8bba> <05cf> <8bc1> <05d0> <8bc4> <05d1> <8bc6> <05d2> <8bca> <05d3> <8bcd> <05d4> <8bd1> <05d5> <8bd5> <05d6> <8bd7> <05d7> <8bda> <05d8> <8bdd> <05d9> <8be2> <05de> <8bef> <05e1> <8bf4> <05e4> <8bfa> <05e5> <8bfe> <05e6> <8c03> <05e7> <8c08> <05e8> <8c0b> <05e9> <8c10> <05ea> <8c13> <05eb> <8c22> <05ec> <8c28> <05ed> <8c34> <05ee> <8c37> <05ef> <8c41> <05f0> <8c61> <05f1> <8c6a> <05f2> <8d1d> <05f3> <8d1f> <05f7> <8d25> <05fe> <8d35> <0602> <8d44> <0603> <8d4b> <0604> <8d4f> <0605> <8d54> <0606> <8d56> <0607> <8d5b> <0608> <8d5e> <0609> <8d66> <060a> <8d70> <060b> <8d75> <060c> <8d77> <060d> <8d85> <0610> <8da3> <0611> <8db3> <0612> <8dc3> <0613> <8dd1> <0614> <8ddd> <0615> <8def> <0616> <8e0a> <0617> <8eab> <0618> <8f67> <0619> <8f6c> endbfchar
100 beginbfchar
<061a> <8f7b> <061b> <8f7d> <061c> <8f83> <061d> <8f85> <0620> <8f93> <0621> <8f96> <0622> <8f9b> <0625> <8fb1> <0626> <8fb9> <0627> <8fbe> <0628> <8fc1> <062b> <8fc7> <062c> <8fce> <062f> <8fd4> <0639> <8ff0> <063a> <8ff7> <063b> <8ff9> <063c> <8ffd> <0643> <9010> <0644> <9012> <0645> <9014> <064a> <903b> <064b> <9047> <064c> <904d> <064d> <9053> <064e> <9057> <064f> <9063> <0650> <9065> <0653> <9075> <0656> <90a3> <0657> <90a6> <065a> <90b1> <065b> <90bb> <065c> <90d1> <065f> <90fd> <0663> <9192> <0664> <919a> <0665> <91c7> <0666> <91ca> <066b> <91d1> <066c> <9274> <066d> <9488> <066e> <94ae> <066f> <94b1> <0670> <94b3> <0673> <94f2> <0674> <94f6> <0675> <94fe> <0676> <9500> <0677> <9504> <0678> <9519> <0679> <9521> <067a> <9528> <067b> <952e> <067c> <953a> <067d> <957f> <067e> <95e8> <0681> <95f4> <0682> <95f7> <0683> <95fb> <0684> <95fd> <0685> <9602> <0686> <9610> <0687> <9614> <0688> <9632> <0689> <9634> <068a> <9636> <068b> <963b> <068c> <963f> <0690> <9650> <0691> <9662> <0692> <9664> <0693> <9669> <0696> <9686> <0697> <968f> <0698> <9694> <0699> <969c> <069a> <96be> <069b> <96c0> <069e> <96e8> <06a1> <9700> <06a2> <9707> <06a3> <970d> <06a4> <9713> <06a5> <9732> <06a6> <9752> <06a7> <9759> <06a8> <975e> <06a9> <9760> <06aa> <9762> <06ab> <978b> <06ac> <97e6> <06ad> <97e9> <06ae> <97f3> <06af> <9877> <06b3> <987e> <06b4> <9881> <06b5> <9884> <06b8> <9898> <06b9> <989d> endbfchar
24 beginbfchar
<06ba> <98a0> <06bb> <98ce> <06be> <9996> <06bf> <9a6c> <06c0> <9a71> <06c1> <9a7b> <06c2> <9a8c> <06c3> <9aa4> <06c4> <9aa8> <06c5> <9ad8> <06c6> <9c7c> <06c7> <9c81> <06c8> <9c9c> <06c9> <9ca4> <06ca> <9cb8> <06cb> <9cc3> <06cc> <9cd6> <06cd> <9e64> <06ce> <9ea6> <06cf> <9ece> <06d0> <9f13> <06d1> <9f3b> <06d2> <9f84> <06d3> <9f9a> endbfchar
endcmap
CMapName
currentdict
/CMap defineresource
pop
end
end
""";
var input = StringBytesTestConverter.Convert(cmap, false);
var result = cMapParser.Parse(input.Bytes);
Assert.Equal("Adobe", result.Info.Registry);
Assert.Equal("GB1", result.Info.Ordering);
Assert.Equal(2, result.Info.Supplement);
Assert.Equal("Adobe-GB1-2", result.Name);
Assert.Equal(2, result.Type);
}
private static byte[] ReadResourceBytes(string name)
{
using (var resource = typeof(CMapParser).Assembly.GetManifestResourceStream(name))

View File

@@ -4,6 +4,7 @@ using UglyToad.PdfPig.Tests.Dla;
namespace UglyToad.PdfPig.Tests.Fonts.SystemFonts
{
using PdfPig.Core;
using PdfPig.Geometry;
public class Linux
{
@@ -64,11 +65,14 @@ namespace UglyToad.PdfPig.Tests.Fonts.SystemFonts
var current = page.Letters[i];
Assert.Equal(expectedData.TopLeft.X, current.GlyphRectangle.TopLeft.X, 6);
Assert.Equal(expectedData.TopLeft.Y, current.GlyphRectangle.TopLeft.Y, 6);
Assert.Equal(expectedData.Width, current.GlyphRectangle.Width, 6);
Assert.Equal(expectedData.Height, current.GlyphRectangle.Height, 6);
Assert.Equal(expectedData.Rotation, current.GlyphRectangle.Rotation, 3);
Assert.Equal(expectedData.TopLeft.X, current.BoundingBox.TopLeft.X, 6);
Assert.Equal(expectedData.TopLeft.Y, current.BoundingBox.TopLeft.Y, 6);
Assert.Equal(expectedData.Width, current.BoundingBox.Width, 6);
Assert.Equal(expectedData.Height, current.BoundingBox.Height, 6);
Assert.Equal(expectedData.Rotation, current.BoundingBox.Rotation, 3);
Assert.True(current.BoundingBox.IntersectsWith(current.GlyphRectangleLoose));
Assert.Equal(current.BoundingBox.Rotation, current.GlyphRectangleLoose.Rotation, 3);
}
}
}

View File

@@ -229,5 +229,19 @@ namespace UglyToad.PdfPig.Tests.Fonts.TrueType.Parser
Assert.NotNull(font.TableRegister.NameTable);
Assert.NotEmpty(font.TableRegister.NameTable.NameRecords);
}
[Fact]
public void Parse12623CorruptFileAndGetGlyphs()
{
var bytes = TrueTypeTestHelper.GetFileBytes("corrupt-12623");
var input = new TrueTypeDataBytes(new MemoryInputBytes(bytes));
var font = TrueTypeFontParser.Parse(input);
Assert.NotNull(font);
font.TryGetPath(1, out _);
}
}
}

View File

@@ -20,14 +20,14 @@
// check 'm'
var m = letters[0];
Assert.Equal("m", m.Value);
Assert.Equal(new PdfPoint(253.4458, 658.431), m.GlyphRectangle.BottomLeft, pointComparer);
Assert.Equal(new PdfPoint(261.22659, 662.83446), m.GlyphRectangle.TopRight, pointComparer);
Assert.Equal(new PdfPoint(253.4458, 658.431), m.BoundingBox.BottomLeft, pointComparer);
Assert.Equal(new PdfPoint(261.22659, 662.83446), m.BoundingBox.TopRight, pointComparer);
// check 'p'
var p = letters[1];
Assert.Equal("p", p.Value);
Assert.Equal(new PdfPoint(261.70778, 656.49825), p.GlyphRectangle.BottomLeft, pointComparer);
Assert.Equal(new PdfPoint(266.6193, 662.83446), p.GlyphRectangle.TopRight, pointComparer);
Assert.Equal(new PdfPoint(261.70778, 656.49825), p.BoundingBox.BottomLeft, pointComparer);
Assert.Equal(new PdfPoint(266.6193, 662.83446), p.BoundingBox.TopRight, pointComparer);
}
}
}

View File

@@ -11,7 +11,7 @@
{
var bytes = GetFileBytes("AdobeUtopia.pfa");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
[Fact]
@@ -20,7 +20,7 @@
// TODO: support reading in these pfb files
var bytes = GetFileBytes("Raleway-Black.pfb");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
[Fact]
@@ -28,7 +28,7 @@
{
var bytes = GetFileBytes("CMBX10.pfa");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
[Fact]
@@ -36,7 +36,7 @@
{
var bytes = GetFileBytes("CMCSC10");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
[Fact]
@@ -44,7 +44,7 @@
{
var bytes = GetFileBytes("CMBX12");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
[Fact]
@@ -52,7 +52,7 @@
{
var bytes = GetFileBytes("CMBX10");
var result = Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
var result = Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
var builder = new StringBuilder("<!DOCTYPE html><html><head></head><body>");
foreach (var charString in result.CharStrings.CharStrings)
@@ -71,7 +71,7 @@
{
var bytes = GetFileBytes("CMR10");
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0);
Type1FontParser.Parse(new MemoryInputBytes(bytes), 0, 0, new StackDepthGuard(256));
}
private static byte[] GetFileBytes(string name)

View File

@@ -68,12 +68,20 @@
[Fact]
public void ReflectionGraphicsStateOperationFactoryKnowsAllOperations()
{
var operationsField = typeof(ReflectionGraphicsStateOperationFactory).GetField("operations", BindingFlags.NonPublic | BindingFlags.Static);
var operationDictionary = operationsField.GetValue(null) as IReadOnlyDictionary<string, Type>;
Assert.NotNull(operationDictionary);
var operationsField = typeof(ReflectionGraphicsStateOperationFactory)
.GetField("Operations", BindingFlags.NonPublic | BindingFlags.Static);
Assert.NotNull(operationsField);
var operationDictionaryRaw = operationsField.GetValue(null) as IReadOnlyDictionary<string, Type>;
Assert.NotNull(operationDictionaryRaw);
var operationDictionary = new Dictionary<string, Type>(operationDictionaryRaw!.ToDictionary(x => x.Key, x => x.Value));
var allOperations = GetOperationTypes();
Assert.Equal(allOperations.Count(), operationDictionary.Count);
Assert.Equal(allOperations.Count, operationDictionary.Count);
var mapped = allOperations.Select(o =>
{
@@ -84,14 +92,14 @@
Assert.Equivalent(operationDictionary, mapped, strict: true);
}
private static IEnumerable<Type> GetOperationTypes()
private static IReadOnlyList<Type> GetOperationTypes()
{
var assembly = Assembly.GetAssembly(typeof(IGraphicsStateOperation));
var operationTypes = assembly.GetTypes().Where(x => typeof(IGraphicsStateOperation).IsAssignableFrom(x)
&& !x.IsInterface);
return operationTypes;
return operationTypes.ToList();
}
private static object[] GetConstructorParameters(ParameterInfo[] parameters)

View File

@@ -270,10 +270,10 @@
GetCurrentState().FontState.WordSpacing = spacing;
}
public void ModifyCurrentTransformationMatrix(double[] value)
public void ModifyCurrentTransformationMatrix(TransformationMatrix value)
{
var ctm = GetCurrentState().CurrentTransformationMatrix;
GetCurrentState().CurrentTransformationMatrix = TransformationMatrix.FromArray(value).Multiply(ctm);
var state = GetCurrentState();
state.CurrentTransformationMatrix = value.Multiply(state.CurrentTransformationMatrix);
}
public void SetCharacterSpacing(double spacing)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

View File

@@ -172,11 +172,12 @@
var decodedBytes = ImageHelpers.LoadFileBytes("ccittfax-decoded.bin");
var image = new TestPdfImage
{
ColorSpaceDetails = IndexedColorSpaceDetails.Stencil(DeviceGrayColorSpaceDetails.Instance, new[] { 1.0, 0 }),
ColorSpaceDetails = IndexedColorSpaceDetails.Stencil(DeviceGrayColorSpaceDetails.Instance),
DecodedBytes = decodedBytes,
WidthInSamples = 1800,
HeightInSamples = 3113,
BitsPerComponent = 1
BitsPerComponent = 1,
Decode = [1.0, 0]
};
Assert.True(PngFromPdfImageFactory.TryGenerate(image, out var bytes));

View File

@@ -32,4 +32,18 @@ public class AccentedCharactersInBookmarksTests
},
nodes);
}
[Fact]
public void CanReadContainerBookmarksCorrectly()
{
var path = IntegrationHelpers.GetDocumentPath("dotnet-ai.pdf");
using var document = PdfDocument.Open(path);
var isFound = document.TryGetBookmarks(out var bookmarks, false);
Assert.True(isFound);
Assert.True(bookmarks.Roots.Count == 3);
isFound = document.TryGetBookmarks(out bookmarks, true);
Assert.True(isFound);
Assert.True(bookmarks.Roots.Count > 3);
}
}

View File

@@ -50,7 +50,7 @@
Assert.Equal(Width, letter.Width, 1);
if (includeHeight)
{
Assert.Equal(Height, letter.GlyphRectangle.Height, 1);
Assert.Equal(Height, letter.BoundingBox.Height, 1);
}
}

View File

@@ -0,0 +1,55 @@
namespace UglyToad.PdfPig.Tests.Integration
{
using System;
using System.Text;
public class CMapLocalCachingTests
{
private static readonly Lazy<string> DocumentFolder = new Lazy<string>(() => Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..", "..", "..", "Integration", "Documents")));
private static readonly Lazy<string> DlaFolder = new Lazy<string>(() => Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..", "..", "..", "Dla", "Documents")));
public static object[][] DocumentsData = new object[][]
{
["68-1990-01_A.pdf"],
["Type0 Font.pdf"],
["11194059_2017-11_de_s.pdf"],
["2108.11480.pdf"],
["reference-2-numeric-error.pdf"],
["MOZILLA-3136-0.pdf"],
["FICTIF_TABLE_INDEX.pdf"],
["Approved_Document_B__fire_safety__volume_2_-_Buildings_other_than_dwellings__2019_edition_incorporating_2020_and_2022_amendments.pdf"],
["dotnet-ai.pdf"],
["Old Gutnish Internet Explorer.pdf"],
["Random 2 Columns Lists Hyph - Justified.pdf"]
};
[Theory]
[MemberData(nameof(DocumentsData))]
public void CheckText(string documentName)
{
string fullPath = Path.Combine(DocumentFolder.Value, documentName);
if (!File.Exists(fullPath))
{
fullPath = Path.Combine(DlaFolder.Value, documentName);
}
Assert.True(File.Exists(fullPath));
var sb = new StringBuilder();
using (var document = PdfDocument.Open(fullPath, new ParsingOptions { UseLenientParsing = true }))
{
for (var i = 0; i < document.NumberOfPages; i++)
{
var page = document.GetPage(i + 1);
sb.Append(page.Text);
}
}
//File.WriteAllText(Path.ChangeExtension(fullPath, "txt"), sb.ToString());
string expected = File.ReadAllText(Path.ChangeExtension(fullPath, "txt"));
Assert.Equal(expected, sb.ToString());
}
}
}

View File

@@ -9,5 +9,17 @@
using var document = PdfDocument.Open(path);
Assert.Equal(3, document.NumberOfPages);
}
[Fact]
public void CanReadDocumentWithCircularXRef()
{
string path = IntegrationHelpers.GetSpecificTestDocumentPath("B17-2000-transportation-fuels.pdf");
// If parser can't deal with xrefs that have circular references then
// opening the document will loop forever
using var document = PdfDocument.Open(path);
Assert.Equal(1, document.NumberOfPages);
}
}
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
TypeWeekly newspaperEditorJoe StuverFoundedOctober, 1918 (asthe BroadusIndependent), 1935(as the PowderRiver CountyExaminer), and1965 (as thePowder RiverExaminer)Headquarters119 1/2 N. ParkAve.Broadus, MT 59317United StatesPowder River ExaminerPowder River ExaminerThe Powder River Examiner, originally established in October, 1918 asthe Broadus Independent, is the only newspaper printed in Powder RiverCounty, Montana, and is located in the county seat of Broadus.The Broadus Independent was first published in Broadus, Montana inOctober, 1918, and continued until February, 1919.From March 6, 1919 until April 17, 1919, the paper was published inOlive, Montana as the Olive Branch.The Broadus Independent was published weekly from April 24, 1919until 1935.The Powder River County Examiner replaced the BroadusIndependent in 1935, beginning publication and continuing weekly until1965.In 1965 the newspaper's name was shortened to Powder RiverExaminer, and remains that today.Broadus Independent, Broadus, Montana, October, 1918-February, 1919.Olive Branch, Olive, Montana, March 6, 1919 April 17, 1919.Broadus Independent, Broadus, Montana, April 24, 1919 1935.Powder River County Examiner, Broadus, Montana, 1935-1965.Powder River Examiner, Broadus, Montana, 1965-current.Joe Stuver, (current editor)Retrieved from "https://en.wikipedia.org/w/index.php?title=Powder_River_Examiner&oldid=747264669"This page was last edited on 1 November 2016, at 11:53.Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By usingthis site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the WikimediaFoundation, Inc., a non-profit organization.HistoryPreceding TitlesNotable contributorsPowder River Examiner - Wikipediahttps://en.wikipedia.org/wiki/Powder_River_Examiner1 of 130/03/2018, 03:50

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
(Jl σ3 法海阔』咱国司被NA咿清峭阉且啊辛茹海耀表3-19车辆齿轮用钢系列牌号化学成分质量分数%国内牌号国外牌号c Si Mn p s Ni Cr Mo Al Cu Ti B 16CrMnTiH 0.130.18 0. \Kl -1. 20 20CrMnTiHI 20CrMnTiH2 。.80 -1. 10 1.00-1.30 20CrMnTiH3 0. 17 -0. 37军三0.0350. 04 -0. 10 0.18句0.2320CrMnTiH4 20CrMnTiH5 0. \Kl -1. 25 I.I。”1.4520CrMnTiH6 16MnCrH 16MnCr5 0. 14 -0. 20 I. 001.40 O.\KJ-1.20 20MnCrH 20MnCr5 0.170.23 1.10-1.50 1.00-1.30 0.02 罢王0.20运0.120.020.仍525MnCrH 25MnCr5 0.230.28 0.055 o. ro -o. so0. 80-1.10 28MnCrH 28MnCr5 0. 25 -0. 30 运0.15运0.1016CrMnBH ZF6 0. 13 -0. 18。.80-1.100.001 -18CrMnBH ZF7 0.15 -0. 40 1.00-1.30 0. 15 -0. 20军军aα丑。0. 015 -0. 035 1.00-1.30 0.α)317CrMnBH ZF7B 17Cr2Ni2H ZFI 0.15 -0.19 0.15 -0. 40 0.40o.ro1.40I. 70 1.40-1.70 16CrNiH 16CrNi4 0.13“0.18 0.02-0.04 0.150.35 0.70句1.100. 80-1. 200. 801.20 :;;;0.10 0.02-0. 05 19CrNiH 19CrNi5 0.160. 21 0. 02 -0. 035 17Cr2Ni2MoH ZFlA 0.15”0.19 0.150.40 o. 40 -o. ro0.0150.035 I. 4。”I.70 1.50-1.80 0. 25 -0. 35 20CrNiMoHI 8620Hl 0.02-0.17”0.23 0.15町0.35o. ro -o. 95 0.0170.032 0.350. 75 0.35”0.65 0. 15 -0. 25 20CrNiMoH2 8620田0.045 15CrMoH 0.13”0.18 0.250.45 0.170. 37 0.4。”0.70 髦。但50.8。”1.1020CrMo 0. 18 -0. 230.15 -0. 25 20CrMoH SCM420 0.17 -0. 23 0.17町0.350.550. \Kl 0.85句I.25 0.15”0.35 0.02-0.0击:;;;0.15 35CrMo 0. 32 -0. 40 0.400. 70 0.80斗100. 15 -0. 25 运0.03520CrH 0.170.37 0.70-1.00 运0.200.500.80 40Cr 0. 18 -0. 23 0.370.440. 80 -1.10

Some files were not shown because too many files have changed in this diff Show More