rework numeric tokenizer hot path

the existing numeric tokenizer involved allocations and string parsing. since
the number formats in pdf files are fairly predictable we can improve this
substantially

README.md

@@ -2,11 +2,19 @@
 
 # PdfPig
 
+[![Gitter](https://badges.gitter.im/pdfpig/community.svg)](https://gitter.im/pdfpig/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 [![nuget](https://img.shields.io/nuget/dt/PdfPig)](https://www.nuget.org/packages/PdfPig/)
+
 [![Build and test](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml)
 [![Build and test [MacOS]](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml)
 
-PdfPig supports reading text and content from PDF files. It also supports basic PDF file creation.
+This project allows users to read and extract text and other content from PDF files. In addition the library can be used to create simple PDF documents
+containing text and geometrical shapes.
+
+This project aims to port [PDFBox](https://github.com/apache/pdfbox) to C#.
+
+## Wiki
+Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
 
 ## Installation
 
@@ -24,26 +32,29 @@ While the version is below 1.0.0 minor versions will change the public API witho
 
 See the [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples 
 
-### Reading text from a PDF
-
+### Read words in a page
 The simplest usage at this stage is to open a document, reading the words from every page:
 
 ```cs
-// using UglyToad.PdfPig.DocumentLayoutAnalysis.TextExtractor;
-// using UglyToad.PdfPig.DocumentLayoutAnalysis.WordExtractor;
 using (PdfDocument document = PdfDocument.Open(@"C:\Documents\document.pdf"))
 {
-    foreach (Page page in document.GetPages())
-    {
-        string text = ContentOrderTextExtractor.GetText(page);
-        IEnumerable<Word> words = page.GetWords(NearestNeighbourWordExtractor.Instance);
-    }
+	foreach (Page page in document.GetPages())
+	{
+		string pageText = page.Text;
+
+		foreach (Word word in page.GetWords())
+		{
+			Console.WriteLine(word.Text);
+		}
+	}
 }
 ```
 
-You **should not** use `page.Text` directly, unless you know what you're doing. The `Text` property preserves the internal content order which is rarely ever the text in the order you want.
+An example of the output of this is shown below:
 
-These layout analysis tools should get you the text you want in most cases.
+![Image shows three words 'Write something in' in 2 sections, the top section is the normal PDF output, the bottom section is the same text with 3 word bounding boxes in pink and letter bounding boxes in blue-green](https://raw.githubusercontent.com/UglyToad/Pdf/master/documentation/Letters/example-text-extraction.png)
+
+Where for the PDF text ("Write something in") shown at the top the 3 words (in pink) are detected and each word contains the individual letters with glyph bounding boxes.
 
 ### Create PDF Document
 To create documents use the class `PdfDocumentBuilder`. The Standard 14 fonts provide a quick way to get started:
@@ -69,12 +80,6 @@ The output is a 1 page PDF document with the text "Hello World!" in Helvetica ne
 
 Each font must be registered with the `PdfDocumentBuilder` prior to use enable pages to share the font resources. Only Standard 14 fonts and TrueType fonts (.ttf) are supported.
 
-Document creation supports very limited changes to existing PDF documents. However it does not support any of the following:
-
-- Editing forms
-- Copying or changing annotations, metadata or document structure data
-- Adding or removing text with existing fonts
-
 ### Advanced Document Extraction
 In this example a more advanced document extraction is performed. `PdfDocumentBuilder` is used to create a copy of the pdf with debug information (bounding boxes and reading order) added.
 
@@ -254,7 +259,7 @@ string title = document.Information.Title;
 
 ### Document Structure
 
-The `PdfDocument` has a Structure member:
+The document now has a Structure member:
 
     UglyToad.PdfPig.Structure structure = document.Structure;
 
@@ -278,7 +283,7 @@ PageSize size = Page.Size;
 bool isA4 = size == PageSize.A4;
 ```
 
-`Page` provides access to the text of the page but you should use `ContentOrderTextExtractor` or alternatives if indexing the text, e.g. for RAG/LLMs:
+`Page` provides access to the text of the page:
 
     string text = page.Text;
 
@@ -324,7 +329,7 @@ Retrieving annotations on each page is provided using the method:
 
     page.GetAnnotations()
 
-This call is not cached and the document must not have been disposed prior to use. Annotations cannot be edited.
+This call is not cached and the document must not have been disposed prior to use.
 
 ### Bookmarks
 
@@ -352,8 +357,6 @@ A page has a method to extract hyperlinks (annotations of link type):
 
     IReadOnlyList<UglyToad.PdfPig.Content.Hyperlink> hyperlinks = page.GetHyperlinks();
 
-Hyperlinks cannot be added or edited when building documents.
-
 ### TrueType
 
 The classes used to work with TrueType fonts in the PDF file are available for public consumption. Given an input file:
@@ -393,17 +396,18 @@ var resultFileBytes = PdfMerger.Merge(filePath1, filePath2);
 File.WriteAllBytes(@"C:\pdfs\outputfilename.pdf", resultFileBytes);
 ```
 
-## Wiki
-Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
-
-## Issues
-
-Please do file an issue if you encounter a bug. See our [issue policy](https://github.com/UglyToad/PdfPig/issues/1095) and [contributing guide](https://github.com/UglyToad/PdfPig/blob/master/CONTRIBUTING.md) for details.
-
 ## API Reference
 
 If you wish to generate doxygen documentation, run `doxygen doxygen-docs` and open `docs/doxygen/html/index.html`.
 
+See also the [wiki](https://github.com/UglyToad/PdfPig/wiki) for a detailed documentation on parts of the API
+
+## Issues
+
+Please do file an issue if you encounter a bug.
+
+However in order for us to assist you, you **must** provide the file which causes your issue. Please host this in a publically available place.
+
 ## Credit
 
-This project started as an effort to port [PDFBox](https://github.com/apache/pdfbox) to C#. This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.
+This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.

src/UglyToad.PdfPig.Core/ArrayPoolBufferWriter.cs

@@ -1,4 +1,5 @@
-using System.Buffers;
+using System;
+using System.Buffers;
 
 namespace UglyToad.PdfPig.Core;
 

src/UglyToad.PdfPig.Core/IndirectReference.cs

@@ -71,7 +71,7 @@
         }
 
         /// <inheritdoc />
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             return obj is IndirectReference other && Equals(other);
         }

src/UglyToad.PdfPig.Core/OtherEncodings.cs

@@ -16,7 +16,7 @@
         /// <summary>
         /// Convert the string to bytes using the ISO 8859-1 encoding.
         /// </summary>
-        public static byte[]? StringAsLatin1Bytes(string? s)
+        public static byte[] StringAsLatin1Bytes(string s)
         {
             if (s == null)
             {

src/UglyToad.PdfPig.Core/PdfDocEncoding.cs

@@ -264,7 +264,7 @@
         /// Try to convert raw bytes to a PdfDocEncoding encoded string. If unsupported characters are encountered
         /// meaning we cannot safely round-trip the value to bytes this will instead return false.
         /// </summary>
-        public static bool TryConvertBytesToString(ReadOnlySpan<byte> bytes, out string? result)
+        public static bool TryConvertBytesToString(ReadOnlySpan<byte> bytes, out string result)
         {
             result = null;
             if (bytes.Length == 0)

src/UglyToad.PdfPig.Core/PdfLine.cs

@@ -70,7 +70,7 @@
         /// <summary>
         /// Returns a value indicating whether this <see cref="PdfLine"/> is equal to a specified <see cref="PdfLine"/> .
         /// </summary>
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             return obj is PdfLine other && Equals(other);
         }

src/UglyToad.PdfPig.Core/PdfPoint.cs

@@ -83,7 +83,7 @@
         /// <summary>
         /// Returns a value indicating whether this <see cref="PdfPoint"/> is equal to a specified <see cref="PdfPoint"/> .
         /// </summary>
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             return obj is PdfPoint other && Equals(other);
         }

src/UglyToad.PdfPig.Core/PdfRectangle.cs

@@ -177,7 +177,7 @@
         }
 
         /// <inheritdoc />
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             return obj is PdfRectangle other && Equals(other);
         }

src/UglyToad.PdfPig.Core/PdfSubpath.cs

@@ -241,8 +241,8 @@
         public bool IsClosed()
         {
             var filteredCount = 0;
-            IPathCommand? last = null;
-            IPathCommand? first = null;
+            IPathCommand last = null;
+            IPathCommand first = null;
             for (int i = Commands.Count - 1; i >= 0; i--)
             {
                 var cmd = Commands[i];
@@ -376,14 +376,14 @@
         /// Gets a <see cref="PdfRectangle"/> which entirely contains the geometry of the defined path.
         /// </summary>
         /// <returns>For paths which don't define any geometry this returns <see langword="null"/>.</returns>
-        public static PdfRectangle? GetBoundingRectangle(IReadOnlyList<PdfSubpath>? path)
+        public static PdfRectangle? GetBoundingRectangle(IReadOnlyList<PdfSubpath> path)
         {
             if (path == null || path.Count == 0)
             {
                 return null;
             }
 
-            var bboxes = path.Select(x => x.GetBoundingRectangle()).Where(x => x.HasValue).Select(x => x!.Value).ToList();
+            var bboxes = path.Select(x => x.GetBoundingRectangle()).Where(x => x.HasValue).Select(x => x.Value).ToList();
             if (bboxes.Count == 0)
             {
                 return null;
@@ -433,7 +433,7 @@
             }
 
             /// <inheritdoc />
-            public override bool Equals(object? obj)
+            public override bool Equals(object obj)
             {
                 return (obj is Close);
             }
@@ -479,7 +479,7 @@
             }
 
             /// <inheritdoc />
-            public override bool Equals(object? obj)
+            public override bool Equals(object obj)
             {
                 if (obj is Move move)
                 {
@@ -545,7 +545,7 @@
             }
 
             /// <inheritdoc />
-            public override bool Equals(object? obj)
+            public override bool Equals(object obj)
             {
                 if (obj is Line line)
                 {
@@ -651,7 +651,7 @@
             }
 
             /// <inheritdoc />
-            public override bool Equals(object? obj)
+            public override bool Equals(object obj)
             {
                 if (obj is QuadraticBezierCurve curve)
                 {
@@ -809,7 +809,7 @@
             }
 
             /// <inheritdoc />
-            public override bool Equals(object? obj)
+            public override bool Equals(object obj)
             {
                 if (obj is CubicBezierCurve curve)
                 {
@@ -944,7 +944,7 @@
         /// <summary>
         /// Compares two <see cref="PdfSubpath"/>s for equality. Paths will only be considered equal if the commands which construct the paths are in the same order.
         /// </summary>
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             if (!(obj is PdfSubpath path) || Commands.Count != path.Commands.Count)
             {

src/UglyToad.PdfPig.Core/TransformationMatrix.cs

@@ -463,7 +463,7 @@
         }
 
         /// <inheritdoc />
-        public override bool Equals(object? obj)
+        public override bool Equals(object obj)
         {
             return obj is TransformationMatrix other && Equals(other);
         }

src/UglyToad.PdfPig.Core/UglyToad.PdfPig.Core.csproj

@@ -1,25 +1,23 @@
 <Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-		<Nullable>enable</Nullable>
-    <WarningsAsErrors>nullable</WarningsAsErrors>
-  </PropertyGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
-    <PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
-    <PackageReference Include="System.Memory" Version="4.6.0" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+	</PropertyGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
+		<PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
+		<PackageReference Include="System.Memory" Version="4.6.0" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig.Core/Union.cs

@@ -89,7 +89,7 @@ namespace UglyToad.PdfPig.Core
             /// <inheritdoc />
             public override bool TryGetSecond(out B b)
             {
-                b = default!;
+                b = default(B);
                 return false;
             }
 
@@ -135,7 +135,7 @@ namespace UglyToad.PdfPig.Core
             /// <inheritdoc />
             public override bool TryGetFirst(out A a)
             {
-                a = default!;
+                a = default(A);
                 return false;
             }
 

src/UglyToad.PdfPig.DocumentLayoutAnalysis/UglyToad.PdfPig.DocumentLayoutAnalysis.csproj

@@ -1,22 +1,22 @@
 <Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-  </PropertyGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
-  </ItemGroup>
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+	</PropertyGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig.Fonts/CompactFontFormat/Charsets/CompactFontFormatCharset.cs

@@ -32,7 +32,7 @@
 
         public virtual string GetNameByStringId(int stringId)
         {
-            return GlyphIdToStringIdAndName.FirstOrDefault(x => x.Value.stringId == stringId).Value.name;
+            return GlyphIdToStringIdAndName.SingleOrDefault(x => x.Value.stringId == stringId).Value.name;
         }
 
         public virtual int GetStringIdByGlyphId(int glyphId)

src/UglyToad.PdfPig.Fonts/UglyToad.PdfPig.Fonts.csproj

@@ -1,40 +1,40 @@
 <Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-  </PropertyGroup>
-  <ItemGroup>
-    <None Remove="Resources\AdobeFontMetrics\*" />
-    <None Remove="Resources\GlyphList\*" />
-  </ItemGroup>
-  <ItemGroup>
-    <Content Include="Resources\AdobeFontMetrics\MustRead.html">
-      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
-    </Content>
-  </ItemGroup>
-  <ItemGroup>
-    <EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
-  </ItemGroup>
-  <ItemGroup>
-    <EmbeddedResource Include="Resources\GlyphList\*" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
-    <PackageReference Include="System.Memory" Version="4.6.0" />
-  </ItemGroup>
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+	</PropertyGroup>
+	<ItemGroup>
+		<None Remove="Resources\AdobeFontMetrics\*" />
+		<None Remove="Resources\GlyphList\*" />
+	</ItemGroup>
+	<ItemGroup>
+		<Content Include="Resources\AdobeFontMetrics\MustRead.html">
+			<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+		</Content>
+	</ItemGroup>
+	<ItemGroup>
+		<EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
+	</ItemGroup>
+	<ItemGroup>
+		<EmbeddedResource Include="Resources\GlyphList\*" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
+		<PackageReference Include="System.Memory" Version="4.6.0" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig.Tokenization/UglyToad.PdfPig.Tokenization.csproj

@@ -1,21 +1,21 @@
 <Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-  </PropertyGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
-  </ItemGroup>
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+	</PropertyGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig.Tokens/NameToken.Constants.cs

@@ -310,7 +310,6 @@
         public static readonly NameToken Last = new NameToken("Last");
         public static readonly NameToken LastChar = new NameToken("LastChar");
         public static readonly NameToken LastModified = new NameToken("LastModified");
-        public static readonly NameToken Launch = new NameToken("Launch");
         public static readonly NameToken Lc = new NameToken("LC");
         public static readonly NameToken Le = new NameToken("LE");
         public static readonly NameToken Leading = new NameToken("Leading");

src/UglyToad.PdfPig.Tokens/UglyToad.PdfPig.Tokens.csproj

@@ -1,23 +1,23 @@
 <Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-  </PropertyGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
-    <PackageReference Include="System.Memory" Version="4.6.0" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
-  </ItemGroup>
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+	</PropertyGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
+		<PackageReference Include="System.Memory" Version="4.6.0" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig/PdfFonts/FontFactory.cs

@@ -44,25 +44,6 @@
                 return handler.Generate(dictionary);
             }
 
-            // Try simple font recovery:
-            NameToken[] orderedFallbacks = [NameToken.Type1, NameToken.TrueType];
-            foreach (var fallback in orderedFallbacks)
-            {
-                if (!handlers.TryGetValue(fallback, out handler))
-                {
-                    continue;
-                }
-
-                try
-                {
-                    return handler.Generate(dictionary);
-                }
-                catch (Exception ex)
-                {
-                    log?.Error($"Tried to parse font as fallback type: {fallback}", ex);
-                }
-            }
-
             throw new NotImplementedException($"Parsing not implemented for fonts of type: {subtype}, please submit a pull request or an issue.");
         }
     }

src/UglyToad.PdfPig/UglyToad.PdfPig.csproj

@@ -1,41 +1,41 @@
-<Project Sdk="Microsoft.NET.Sdk">
-  <PropertyGroup>
-    <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
-    <LangVersion>12</LangVersion>
-    <Version>0.1.12-alpha001</Version>
-    <IsTestProject>False</IsTestProject>
-    <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <SignAssembly>true</SignAssembly>
-    <AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
-    <Nullable>enable</Nullable>
-  </PropertyGroup>
-  <ItemGroup>
-    <None Remove="Resources\CMap\*" />
-    <None Remove="Resources\ICC\*" />
-  </ItemGroup>
-  <ItemGroup>
-    <EmbeddedResource Include="Resources\CMap\*" />
-    <EmbeddedResource Include="Resources\ICC\*" />
-  </ItemGroup>
-  <ItemGroup>
-    <None Include="..\pdfpig.snk" Link="pdfpig.snk" />
-  </ItemGroup>
-  <ItemGroup>
-    <PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0">
-      <PrivateAssets>all</PrivateAssets>
-      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
-    </PackageReference>
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='net462'">
-    <PackageReference Include="System.ValueTuple" Version="4.5.0" />
-  </ItemGroup>
-  <ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
-    <PackageReference Include="System.Memory" Version="4.6.0" />
-  </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
-    <ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
-  </ItemGroup>
+<Project Sdk="Microsoft.NET.Sdk">
+	<PropertyGroup>
+		<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
+		<LangVersion>12</LangVersion>
+		<Version>0.1.11-alpha001</Version>
+		<IsTestProject>False</IsTestProject>
+		<GenerateDocumentationFile>true</GenerateDocumentationFile>
+		<SignAssembly>true</SignAssembly>
+		<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
+		<Nullable>enable</Nullable>
+	</PropertyGroup>
+	<ItemGroup>
+		<None Remove="Resources\CMap\*" />
+		<None Remove="Resources\ICC\*" />
+	</ItemGroup>
+	<ItemGroup>
+		<EmbeddedResource Include="Resources\CMap\*" />
+		<EmbeddedResource Include="Resources\ICC\*" />
+	</ItemGroup>
+	<ItemGroup>
+		<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
+	</ItemGroup>
+	<ItemGroup>
+		<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0">
+			<PrivateAssets>all</PrivateAssets>
+			<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
+		</PackageReference>
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='net462'">
+		<PackageReference Include="System.ValueTuple" Version="4.5.0" />
+	</ItemGroup>
+	<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
+		<PackageReference Include="System.Memory" Version="4.6.0" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
+		<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
+	</ItemGroup>
 </Project>

src/UglyToad.PdfPig/Writer/PdfDocumentBuilder.cs

@@ -1,26 +1,26 @@
 
 namespace UglyToad.PdfPig.Writer
 {
-    using Actions;
-    using Content;
-    using Core;
-    using Filters;
-    using Fonts;
-    using Graphics;
-    using Logging;
-    using Outline;
-    using Outline.Destinations;
-    using Parser;
-    using Parser.Parts;
-    using PdfPig.Fonts.Standard14Fonts;
-    using PdfPig.Fonts.TrueType;
-    using PdfPig.Fonts.TrueType.Parser;
     using System;
     using System.Collections.Generic;
     using System.IO;
     using System.Linq;
     using System.Runtime.CompilerServices;
     using System.Xml.Linq;
+    using Content;
+    using Core;
+    using Fonts;
+    using Actions;
+    using Filters;
+    using Graphics;
+    using Logging;
+    using PdfPig.Fonts.TrueType;
+    using PdfPig.Fonts.Standard14Fonts;
+    using PdfPig.Fonts.TrueType.Parser;
+    using Outline;
+    using Outline.Destinations;
+    using Parser;
+    using Parser.Parts;
     using Tokenization.Scanner;
     using Tokens;
 
@@ -307,6 +307,7 @@ namespace UglyToad.PdfPig.Writer
         /// </summary>
         /// <param name="document">Source document.</param>
         /// <param name="pageNumber">Page to copy.</param>
+        /// <param name="options">Control how copying for the page occurs.</param>
         /// <returns>A builder for editing the page.</returns>
         public PdfPageBuilder AddPage(PdfDocument document, int pageNumber)
         {
@@ -457,16 +458,72 @@ namespace UglyToad.PdfPig.Writer
                     {
                         continue;
                     }
+                    
+                    var val = kvp.Value;
+                    if (kvp.Value is IndirectReferenceToken ir)
+                    {
+                        ObjectToken tk = document.Structure.TokenScanner.Get(ir.Data);
+                        if (tk is null)
+                        {
+                            // malformed
+                            continue;
+                        }
+                        val = tk.Data;
+                    }
 
-                    var copiedTokens = CopyAnnotationsFromPageSource(
-                        kvp.Value,
-                        document.Structure.TokenScanner,
-                        refs,
-                        page,
-                        options.CopyLinkFunc,
-                        x => links.Add(x));
+                    if (!(val is ArrayToken arr))
+                    {
+                        // should be array... ignore and remove bad dict
+                        continue;
+                    }
 
-                    copiedPageDict[NameToken.Annots] = new ArrayToken(copiedTokens);
+                    // if copyLink is unset, ignore links to resolve issues with refencing non-existing pages
+                    var toAdd = new List<IToken>();
+                    foreach (var annot in arr.Data)
+                    {
+                        DictionaryToken? tk = GetRemoteDict(annot);
+                        if (tk is null)
+                        {
+                            // malformed
+                            continue;
+                        }
+
+                        if (tk.TryGet(NameToken.Subtype, out var st) && st is NameToken nm && nm == NameToken.Link)
+                        {
+                            if (options.CopyLinkFunc is null)
+                            {
+                                // ignore link if don't know how to copy
+                                continue;
+                            }
+
+                            var link = page.annotationProvider.GetAction(tk);
+                            if (link is null)
+                            {
+                                // ignore unknown link actions
+                                continue;
+                            }
+
+                            var copiedLink = options.CopyLinkFunc(link);
+                            if (copiedLink is null)
+                            {
+                                // ignore if caller wants to skip the link
+                                continue;
+                            }
+
+                            if (copiedLink != link)
+                            {
+                                // defer to write links when all pages are added
+                                var copiedToken = (DictionaryToken)WriterUtil.CopyToken(context, tk, document.Structure.TokenScanner, refs);
+                                links.Add((copiedToken, copiedLink));
+                                continue;
+                            }
+
+                            // copy as is if caller returns the same link
+                        }
+                        toAdd.Add(WriterUtil.CopyToken(context, tk, document.Structure.TokenScanner, refs));
+                    }
+                    // copy rest
+                    copiedPageDict[NameToken.Annots] = new ArrayToken(toAdd);
                     continue;
                 }
 
@@ -568,161 +625,6 @@ namespace UglyToad.PdfPig.Writer
             }
         }
 
-        private IReadOnlyList<IToken> CopyAnnotationsFromPageSource(
-            IToken val,
-            IPdfTokenScanner sourceScanner,
-            IDictionary<IndirectReference, IndirectReferenceToken> refs,
-            Page page,
-            Func<PdfAction, PdfAction?>? linkCopyFunc = null,
-            Action<(DictionaryToken, PdfAction)>? deferredActionUpdate = null)
-        {
-            var permittedLinkActionTypes = new HashSet<NameToken>
-            {
-                // A web URI.
-                NameToken.Uri,
-                // A page in a different non-embedded document.
-                NameToken.GoToR,
-                // Launch an external application.
-                NameToken.Launch,
-            };
-
-            if (!DirectObjectFinder.TryGet(val, sourceScanner, out ArrayToken? annotationsArray))
-            {
-                return [];
-            }
-
-            var copiedAnnotations = new List<IToken>();
-            foreach (var annotEntry in annotationsArray.Data)
-            {
-                if (!DirectObjectFinder.TryGet(annotEntry, sourceScanner, out DictionaryToken? annotDict))
-                {
-                    continue;
-                }
-
-                var removedKeys = new List<NameToken>();
-
-                /*
-                 * An indirect reference to the page object with which this annotation is associated.
-                 * Note: This entry is required for screen annotations associated with rendition actions.
-                 */
-                if (annotDict.TryGet(NameToken.P, out _))
-                {
-                    // If we have a page reference we should update it when this page is written.
-                    // For now, we'll remove it. This will corrupt screen annotations as noted above.
-                    removedKeys.Add(NameToken.P);
-                }
-
-                // We don't copy the struct tree so skip this for now.
-                if (annotDict.TryGet(NameToken.StructParent, out _))
-                {
-                    removedKeys.Add(NameToken.StructParent);
-                }
-
-                // We treat non-link annotations as ok for now, we should revisit this.
-                if (!annotDict.TryGet(NameToken.Subtype, sourceScanner, out NameToken? subtype)
-                    || subtype != NameToken.Link)
-                {
-                    var copiedRef = WriterUtil.CopyToken(
-                        context,
-                        CopyWithSkippedKeys(annotDict, removedKeys),
-                        sourceScanner,
-                        refs);
-
-                    copiedAnnotations.Add(copiedRef);
-
-                    continue;
-                }
-
-                if (linkCopyFunc != null && deferredActionUpdate != null)
-                {
-                    var action = page.annotationProvider.GetAction(annotDict);
-
-                    if (action != null)
-                    {
-                        var copiedLink = linkCopyFunc(action);
-                        if (copiedLink != action && copiedLink != null)
-                        {
-                            // defer to write links when all pages are added
-                            var copiedToken = (DictionaryToken)WriterUtil.CopyToken(context, annotDict, sourceScanner, refs);
-                            deferredActionUpdate((copiedToken, copiedLink));
-                            continue;
-                        }
-                    }
-                }
-
-                // If the link has an action then this link can point elsewhere in this document, maybe not to a page we copied?
-                if (annotDict.TryGet(NameToken.A, sourceScanner, out DictionaryToken? actionDict))
-                {
-                    // If the link annotation points somewhere inside our document we can't currently maintain validity on-copy.
-                    if (!actionDict.TryGet(NameToken.S, sourceScanner, out NameToken? actionType)
-                        || !permittedLinkActionTypes.Contains(actionType))
-                    {
-                        continue;
-                    }
-
-                    var copiedRef = WriterUtil.CopyToken(
-                        context,
-                        CopyWithSkippedKeys(annotDict, removedKeys),
-                        sourceScanner,
-                        refs);
-
-                    copiedAnnotations.Add(copiedRef);
-
-                    continue;
-                }
-
-                // A dest can point elsewhere in this document, maybe not to a page we copied?
-                if (annotDict.TryGet(NameToken.Dest, out _))
-                {
-                    // Skip for now.
-                    continue;
-                }
-
-                // If neither /A nor /Dest are present then I don't really know what this link does, so it should be safe to copy:
-                var finalCopiedRef = WriterUtil.CopyToken(
-                    context,
-                    CopyWithSkippedKeys(annotDict, removedKeys),
-                    sourceScanner,
-                    refs);
-
-                copiedAnnotations.Add(finalCopiedRef);
-            }
-
-            return copiedAnnotations;
-        }
-
-        private static DictionaryToken CopyWithSkippedKeys(
-            DictionaryToken source,
-            IReadOnlyList<NameToken> skipped)
-        {
-            var dict = new Dictionary<NameToken, IToken>();
-
-            foreach (var kvp in source.Data)
-            {
-                var name = NameToken.Create(kvp.Key);
-
-                var ignore = false;
-
-                foreach (var skippedName in skipped)
-                {
-                    if (skippedName == name)
-                    {
-                        ignore = true;
-                        break;
-                    }
-                }
-
-                if (ignore)
-                {
-                    continue;
-                }
-
-                dict[name] = kvp.Value;
-            }
-
-            return new DictionaryToken(dict);
-        }
-
         private void CompleteDocument()
         {
             // write fonts to reserved object numbers

tools/UglyToad.PdfPig.Package/UglyToad.PdfPig.Package.csproj

@@ -1,4 +1,4 @@
-<Project Sdk="Microsoft.NET.Sdk">
+<Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
     <TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
     <PackageId>PdfPig</PackageId>
@@ -11,9 +11,9 @@
     <PackageTags>PDF;Reader;Document;Adobe;PDFBox;PdfPig;pdf-extract;pdf-to-text;pdf;file;text;C#;dotnet;.NET</PackageTags>
     <RepositoryUrl>https://github.com/UglyToad/PdfPig</RepositoryUrl>
     <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <Version>0.1.12-alpha001</Version>
-    <AssemblyVersion>0.1.12.0</AssemblyVersion>
-    <FileVersion>0.1.12.0</FileVersion>
+    <Version>0.1.11-alpha001</Version>
+    <AssemblyVersion>0.1.10.0</AssemblyVersion>
+    <FileVersion>0.1.10.0</FileVersion>
     <PackageIconUrl>https://raw.githubusercontent.com/UglyToad/PdfPig/master/documentation/pdfpig.png</PackageIconUrl>
     <PackageIcon>pdfpig.png</PackageIcon>
     <Product>PdfPig</Product>