Compare commits

..

1 Commits

Author SHA1 Message Date
EliotJones
45cf9745a3 rework numeric tokenizer hot path
the existing numeric tokenizer involved allocations and string parsing. since
the number formats in pdf files are fairly predictable we can improve this
substantially
2025-07-24 21:16:25 -05:00
15 changed files with 322 additions and 709 deletions

View File

@@ -2,11 +2,19 @@
# PdfPig
[![Gitter](https://badges.gitter.im/pdfpig/community.svg)](https://gitter.im/pdfpig/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
[![nuget](https://img.shields.io/nuget/dt/PdfPig)](https://www.nuget.org/packages/PdfPig/)
[![Build and test](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test.yml)
[![Build and test [MacOS]](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml/badge.svg)](https://github.com/UglyToad/PdfPig/actions/workflows/build_and_test_macos.yml)
PdfPig supports reading text and content from PDF files. It also supports basic PDF file creation.
This project allows users to read and extract text and other content from PDF files. In addition the library can be used to create simple PDF documents
containing text and geometrical shapes.
This project aims to port [PDFBox](https://github.com/apache/pdfbox) to C#.
## Wiki
Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
## Installation
@@ -24,26 +32,29 @@ While the version is below 1.0.0 minor versions will change the public API witho
See the [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples
### Reading text from a PDF
### Read words in a page
The simplest usage at this stage is to open a document, reading the words from every page:
```cs
// using UglyToad.PdfPig.DocumentLayoutAnalysis.TextExtractor;
// using UglyToad.PdfPig.DocumentLayoutAnalysis.WordExtractor;
using (PdfDocument document = PdfDocument.Open(@"C:\Documents\document.pdf"))
{
foreach (Page page in document.GetPages())
{
string text = ContentOrderTextExtractor.GetText(page);
IEnumerable<Word> words = page.GetWords(NearestNeighbourWordExtractor.Instance);
}
foreach (Page page in document.GetPages())
{
string pageText = page.Text;
foreach (Word word in page.GetWords())
{
Console.WriteLine(word.Text);
}
}
}
```
You **should not** use `page.Text` directly, unless you know what you're doing. The `Text` property preserves the internal content order which is rarely ever the text in the order you want.
An example of the output of this is shown below:
These layout analysis tools should get you the text you want in most cases.
![Image shows three words 'Write something in' in 2 sections, the top section is the normal PDF output, the bottom section is the same text with 3 word bounding boxes in pink and letter bounding boxes in blue-green](https://raw.githubusercontent.com/UglyToad/Pdf/master/documentation/Letters/example-text-extraction.png)
Where for the PDF text ("Write something in") shown at the top the 3 words (in pink) are detected and each word contains the individual letters with glyph bounding boxes.
### Create PDF Document
To create documents use the class `PdfDocumentBuilder`. The Standard 14 fonts provide a quick way to get started:
@@ -69,12 +80,6 @@ The output is a 1 page PDF document with the text "Hello World!" in Helvetica ne
Each font must be registered with the `PdfDocumentBuilder` prior to use enable pages to share the font resources. Only Standard 14 fonts and TrueType fonts (.ttf) are supported.
Document creation supports very limited changes to existing PDF documents. However it does not support any of the following:
- Editing forms
- Copying or changing annotations, metadata or document structure data
- Adding or removing text with existing fonts
### Advanced Document Extraction
In this example a more advanced document extraction is performed. `PdfDocumentBuilder` is used to create a copy of the pdf with debug information (bounding boxes and reading order) added.
@@ -254,7 +259,7 @@ string title = document.Information.Title;
### Document Structure
The `PdfDocument` has a Structure member:
The document now has a Structure member:
UglyToad.PdfPig.Structure structure = document.Structure;
@@ -278,7 +283,7 @@ PageSize size = Page.Size;
bool isA4 = size == PageSize.A4;
```
`Page` provides access to the text of the page but you should use `ContentOrderTextExtractor` or alternatives if indexing the text, e.g. for RAG/LLMs:
`Page` provides access to the text of the page:
string text = page.Text;
@@ -324,7 +329,7 @@ Retrieving annotations on each page is provided using the method:
page.GetAnnotations()
This call is not cached and the document must not have been disposed prior to use. Annotations cannot be edited.
This call is not cached and the document must not have been disposed prior to use.
### Bookmarks
@@ -352,8 +357,6 @@ A page has a method to extract hyperlinks (annotations of link type):
IReadOnlyList<UglyToad.PdfPig.Content.Hyperlink> hyperlinks = page.GetHyperlinks();
Hyperlinks cannot be added or edited when building documents.
### TrueType
The classes used to work with TrueType fonts in the PDF file are available for public consumption. Given an input file:
@@ -393,17 +396,18 @@ var resultFileBytes = PdfMerger.Merge(filePath1, filePath2);
File.WriteAllBytes(@"C:\pdfs\outputfilename.pdf", resultFileBytes);
```
## Wiki
Check out our [wiki](https://github.com/UglyToad/PdfPig/wiki) for more examples and detailed guides on the API.
## Issues
Please do file an issue if you encounter a bug. See our [issue policy](https://github.com/UglyToad/PdfPig/issues/1095) and [contributing guide](https://github.com/UglyToad/PdfPig/blob/master/CONTRIBUTING.md) for details.
## API Reference
If you wish to generate doxygen documentation, run `doxygen doxygen-docs` and open `docs/doxygen/html/index.html`.
See also the [wiki](https://github.com/UglyToad/PdfPig/wiki) for a detailed documentation on parts of the API
## Issues
Please do file an issue if you encounter a bug.
However in order for us to assist you, you **must** provide the file which causes your issue. Please host this in a publically available place.
## Credit
This project started as an effort to port [PDFBox](https://github.com/apache/pdfbox) to C#. This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.
This project wouldn't be possible without the work done by the [PDFBox](https://pdfbox.apache.org/) team and the Apache Foundation.

View File

@@ -1,23 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
<PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' OR '$(TargetFramework)'=='net471'">
<PackageReference Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
</Project>

View File

@@ -1,22 +1,22 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig\UglyToad.PdfPig.csproj" />
</ItemGroup>
</Project>

View File

@@ -32,7 +32,7 @@
public virtual string GetNameByStringId(int stringId)
{
return GlyphIdToStringIdAndName.FirstOrDefault(x => x.Value.stringId == stringId).Value.name;
return GlyphIdToStringIdAndName.SingleOrDefault(x => x.Value.stringId == stringId).Value.name;
}
public virtual int GetStringIdByGlyphId(int glyphId)

View File

@@ -1,40 +1,40 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\AdobeFontMetrics\*" />
<None Remove="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<Content Include="Resources\AdobeFontMetrics\MustRead.html">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\AdobeFontMetrics\*" />
<None Remove="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<Content Include="Resources\AdobeFontMetrics\MustRead.html">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\AdobeFontMetrics\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\GlyphList\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
</Project>

View File

@@ -1,21 +1,21 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
</Project>

View File

@@ -310,7 +310,6 @@
public static readonly NameToken Last = new NameToken("Last");
public static readonly NameToken LastChar = new NameToken("LastChar");
public static readonly NameToken LastModified = new NameToken("LastModified");
public static readonly NameToken Launch = new NameToken("Launch");
public static readonly NameToken Lc = new NameToken("LC");
public static readonly NameToken Le = new NameToken("LE");
public static readonly NameToken Leading = new NameToken("Leading");

View File

@@ -1,23 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
</ItemGroup>
</Project>

View File

@@ -44,25 +44,6 @@
return handler.Generate(dictionary);
}
// Try simple font recovery:
NameToken[] orderedFallbacks = [NameToken.Type1, NameToken.TrueType];
foreach (var fallback in orderedFallbacks)
{
if (!handlers.TryGetValue(fallback, out handler))
{
continue;
}
try
{
return handler.Generate(dictionary);
}
catch (Exception ex)
{
log?.Error($"Tried to parse font as fallback type: {fallback}", ex);
}
}
throw new NotImplementedException($"Parsing not implemented for fonts of type: {subtype}, please submit a pull request or an issue.");
}
}

View File

@@ -1,41 +1,41 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.12-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\CMap\*" />
<None Remove="Resources\ICC\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\CMap\*" />
<EmbeddedResource Include="Resources\ICC\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<LangVersion>12</LangVersion>
<Version>0.1.11-alpha001</Version>
<IsTestProject>False</IsTestProject>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\pdfpig.snk</AssemblyOriginatorKeyFile>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<None Remove="Resources\CMap\*" />
<None Remove="Resources\ICC\*" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\CMap\*" />
<EmbeddedResource Include="Resources\ICC\*" />
</ItemGroup>
<ItemGroup>
<None Include="..\pdfpig.snk" Link="pdfpig.snk" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='net462'">
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='net462' or '$(TargetFramework)'=='net471'">
<PackageReference Include="System.Memory" Version="4.6.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\UglyToad.PdfPig.Core\UglyToad.PdfPig.Core.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Fonts\UglyToad.PdfPig.Fonts.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokenization\UglyToad.PdfPig.Tokenization.csproj" />
<ProjectReference Include="..\UglyToad.PdfPig.Tokens\UglyToad.PdfPig.Tokens.csproj" />
</ItemGroup>
</Project>

View File

@@ -1,26 +1,26 @@

namespace UglyToad.PdfPig.Writer
{
using Actions;
using Content;
using Core;
using Filters;
using Fonts;
using Graphics;
using Logging;
using Outline;
using Outline.Destinations;
using Parser;
using Parser.Parts;
using PdfPig.Fonts.Standard14Fonts;
using PdfPig.Fonts.TrueType;
using PdfPig.Fonts.TrueType.Parser;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Xml.Linq;
using Content;
using Core;
using Fonts;
using Actions;
using Filters;
using Graphics;
using Logging;
using PdfPig.Fonts.TrueType;
using PdfPig.Fonts.Standard14Fonts;
using PdfPig.Fonts.TrueType.Parser;
using Outline;
using Outline.Destinations;
using Parser;
using Parser.Parts;
using Tokenization.Scanner;
using Tokens;
@@ -307,6 +307,7 @@ namespace UglyToad.PdfPig.Writer
/// </summary>
/// <param name="document">Source document.</param>
/// <param name="pageNumber">Page to copy.</param>
/// <param name="options">Control how copying for the page occurs.</param>
/// <returns>A builder for editing the page.</returns>
public PdfPageBuilder AddPage(PdfDocument document, int pageNumber)
{
@@ -457,16 +458,72 @@ namespace UglyToad.PdfPig.Writer
{
continue;
}
var val = kvp.Value;
if (kvp.Value is IndirectReferenceToken ir)
{
ObjectToken tk = document.Structure.TokenScanner.Get(ir.Data);
if (tk is null)
{
// malformed
continue;
}
val = tk.Data;
}
var copiedTokens = CopyAnnotationsFromPageSource(
kvp.Value,
document.Structure.TokenScanner,
refs,
page,
options.CopyLinkFunc,
x => links.Add(x));
if (!(val is ArrayToken arr))
{
// should be array... ignore and remove bad dict
continue;
}
copiedPageDict[NameToken.Annots] = new ArrayToken(copiedTokens);
// if copyLink is unset, ignore links to resolve issues with refencing non-existing pages
var toAdd = new List<IToken>();
foreach (var annot in arr.Data)
{
DictionaryToken? tk = GetRemoteDict(annot);
if (tk is null)
{
// malformed
continue;
}
if (tk.TryGet(NameToken.Subtype, out var st) && st is NameToken nm && nm == NameToken.Link)
{
if (options.CopyLinkFunc is null)
{
// ignore link if don't know how to copy
continue;
}
var link = page.annotationProvider.GetAction(tk);
if (link is null)
{
// ignore unknown link actions
continue;
}
var copiedLink = options.CopyLinkFunc(link);
if (copiedLink is null)
{
// ignore if caller wants to skip the link
continue;
}
if (copiedLink != link)
{
// defer to write links when all pages are added
var copiedToken = (DictionaryToken)WriterUtil.CopyToken(context, tk, document.Structure.TokenScanner, refs);
links.Add((copiedToken, copiedLink));
continue;
}
// copy as is if caller returns the same link
}
toAdd.Add(WriterUtil.CopyToken(context, tk, document.Structure.TokenScanner, refs));
}
// copy rest
copiedPageDict[NameToken.Annots] = new ArrayToken(toAdd);
continue;
}
@@ -568,161 +625,6 @@ namespace UglyToad.PdfPig.Writer
}
}
private IReadOnlyList<IToken> CopyAnnotationsFromPageSource(
IToken val,
IPdfTokenScanner sourceScanner,
IDictionary<IndirectReference, IndirectReferenceToken> refs,
Page page,
Func<PdfAction, PdfAction?>? linkCopyFunc = null,
Action<(DictionaryToken, PdfAction)>? deferredActionUpdate = null)
{
var permittedLinkActionTypes = new HashSet<NameToken>
{
// A web URI.
NameToken.Uri,
// A page in a different non-embedded document.
NameToken.GoToR,
// Launch an external application.
NameToken.Launch,
};
if (!DirectObjectFinder.TryGet(val, sourceScanner, out ArrayToken? annotationsArray))
{
return [];
}
var copiedAnnotations = new List<IToken>();
foreach (var annotEntry in annotationsArray.Data)
{
if (!DirectObjectFinder.TryGet(annotEntry, sourceScanner, out DictionaryToken? annotDict))
{
continue;
}
var removedKeys = new List<NameToken>();
/*
* An indirect reference to the page object with which this annotation is associated.
* Note: This entry is required for screen annotations associated with rendition actions.
*/
if (annotDict.TryGet(NameToken.P, out _))
{
// If we have a page reference we should update it when this page is written.
// For now, we'll remove it. This will corrupt screen annotations as noted above.
removedKeys.Add(NameToken.P);
}
// We don't copy the struct tree so skip this for now.
if (annotDict.TryGet(NameToken.StructParent, out _))
{
removedKeys.Add(NameToken.StructParent);
}
// We treat non-link annotations as ok for now, we should revisit this.
if (!annotDict.TryGet(NameToken.Subtype, sourceScanner, out NameToken? subtype)
|| subtype != NameToken.Link)
{
var copiedRef = WriterUtil.CopyToken(
context,
CopyWithSkippedKeys(annotDict, removedKeys),
sourceScanner,
refs);
copiedAnnotations.Add(copiedRef);
continue;
}
if (linkCopyFunc != null && deferredActionUpdate != null)
{
var action = page.annotationProvider.GetAction(annotDict);
if (action != null)
{
var copiedLink = linkCopyFunc(action);
if (copiedLink != action && copiedLink != null)
{
// defer to write links when all pages are added
var copiedToken = (DictionaryToken)WriterUtil.CopyToken(context, annotDict, sourceScanner, refs);
deferredActionUpdate((copiedToken, copiedLink));
continue;
}
}
}
// If the link has an action then this link can point elsewhere in this document, maybe not to a page we copied?
if (annotDict.TryGet(NameToken.A, sourceScanner, out DictionaryToken? actionDict))
{
// If the link annotation points somewhere inside our document we can't currently maintain validity on-copy.
if (!actionDict.TryGet(NameToken.S, sourceScanner, out NameToken? actionType)
|| !permittedLinkActionTypes.Contains(actionType))
{
continue;
}
var copiedRef = WriterUtil.CopyToken(
context,
CopyWithSkippedKeys(annotDict, removedKeys),
sourceScanner,
refs);
copiedAnnotations.Add(copiedRef);
continue;
}
// A dest can point elsewhere in this document, maybe not to a page we copied?
if (annotDict.TryGet(NameToken.Dest, out _))
{
// Skip for now.
continue;
}
// If neither /A nor /Dest are present then I don't really know what this link does, so it should be safe to copy:
var finalCopiedRef = WriterUtil.CopyToken(
context,
CopyWithSkippedKeys(annotDict, removedKeys),
sourceScanner,
refs);
copiedAnnotations.Add(finalCopiedRef);
}
return copiedAnnotations;
}
private static DictionaryToken CopyWithSkippedKeys(
DictionaryToken source,
IReadOnlyList<NameToken> skipped)
{
var dict = new Dictionary<NameToken, IToken>();
foreach (var kvp in source.Data)
{
var name = NameToken.Create(kvp.Key);
var ignore = false;
foreach (var skippedName in skipped)
{
if (skippedName == name)
{
ignore = true;
break;
}
}
if (ignore)
{
continue;
}
dict[name] = kvp.Value;
}
return new DictionaryToken(dict);
}
private void CompleteDocument()
{
// write fonts to reserved object numbers

View File

@@ -1,10 +1,5 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Text;
using Console = System.Console;
@@ -12,128 +7,6 @@ namespace UglyToad.PdfPig.ConsoleRunner
{
public static class Program
{
private class OptionalArg
{
public required string ShortSymbol { get; init; }
public required string Symbol { get; init; }
public required bool SupportsValue { get; init; }
public string? Value { get; set; }
}
private class ParsedArgs
{
public required IReadOnlyList<OptionalArg> SuppliedArgs { get; init; }
public required string SuppliedDirectoryPath { get; init; }
}
private const string FileSymbol = "f";
private const string RepeatSymbol = "r";
private static IReadOnlyList<OptionalArg> GetSupportedArgs() =>
[
new OptionalArg
{
SupportsValue = false,
ShortSymbol = "nr",
Symbol = "no-recursion"
},
new OptionalArg
{
SupportsValue = true,
ShortSymbol = "o",
Symbol = "output"
},
new OptionalArg
{
SupportsValue = true,
ShortSymbol = "l",
Symbol = "limit"
},
new OptionalArg
{
SupportsValue = true,
ShortSymbol = "f",
Symbol = "file"
},
new OptionalArg
{
SupportsValue = true,
ShortSymbol = "r",
Symbol = "repeats"
}
];
private static bool TryParseArgs(
string[] args,
[NotNullWhen(true)] out ParsedArgs? parsed)
{
parsed = null;
string? path = null;
var suppliedOpts = new List<OptionalArg>();
var opts = GetSupportedArgs();
for (var i = 0; i < args.Length; i++)
{
var str = args[i];
var isOptFlag = str.StartsWith('-');
if (!isOptFlag)
{
if (path == null)
{
path = str;
}
else
{
return false;
}
}
else
{
var item = opts.SingleOrDefault(x =>
string.Equals("-" + x.ShortSymbol, str, StringComparison.OrdinalIgnoreCase)
|| string.Equals("--" + x.Symbol, str, StringComparison.OrdinalIgnoreCase));
if (item == null)
{
return false;
}
if (item.SupportsValue)
{
if (i == args.Length - 1)
{
return false;
}
i++;
item.Value = args[i];
}
suppliedOpts.Add(item);
}
}
if (path == null)
{
return false;
}
parsed = new ParsedArgs
{
SuppliedArgs = suppliedOpts,
SuppliedDirectoryPath = path
};
return true;
}
public static int Main(string[] args)
{
if (args.Length == 0)
@@ -142,140 +15,71 @@ namespace UglyToad.PdfPig.ConsoleRunner
return 7;
}
if (!TryParseArgs(args, out var parsed))
var path = args[0];
if (!Directory.Exists(path))
{
var strJoined = string.Join(" ", args);
Console.WriteLine($"Unrecognized arguments passed: {strJoined}");
Console.WriteLine($"The provided path is not a valid directory: {path}.");
return 7;
}
if (!Directory.Exists(parsed.SuppliedDirectoryPath))
{
Console.WriteLine($"The provided path is not a valid directory: {parsed.SuppliedDirectoryPath}.");
return 7;
}
var maxCount = default(int?);
int? maxCount = null;
var limit = parsed.SuppliedArgs.SingleOrDefault(x => x.ShortSymbol == "l");
if (limit?.Value != null && int.TryParse(limit.Value, CultureInfo.InvariantCulture, out var maxCountArg))
if (args.Length > 1 && int.TryParse(args[1], out var countIn))
{
Console.WriteLine($"Limiting input files to first: {maxCountArg}");
maxCount = maxCountArg;
maxCount = countIn;
}
var noRecursionMode = parsed.SuppliedArgs.Any(x => x.ShortSymbol == "nr");
var outputOpt = parsed.SuppliedArgs.SingleOrDefault(x => x.ShortSymbol == "o" && x.Value != null);
var fileOpt = parsed.SuppliedArgs.SingleOrDefault(x => x.ShortSymbol == FileSymbol && x.Value != null);
var hasError = false;
var errorBuilder = new StringBuilder();
var fileList = Directory.GetFiles(
parsed.SuppliedDirectoryPath,
"*.pdf",
noRecursionMode ? SearchOption.TopDirectoryOnly : SearchOption.AllDirectories)
.OrderBy(x => x).ToList();
var fileList = Directory.GetFiles(path, "*.pdf", SearchOption.AllDirectories);
var runningCount = 0;
if (fileOpt?.Value != null)
Console.WriteLine($"Found {fileList.Length} files.");
Console.WriteLine($"{GetCleanFilename("File")}| Size\t| Words\t| Pages");
foreach (var file in fileList)
{
fileList = fileList.Where(x => x.EndsWith(fileOpt.Value, StringComparison.OrdinalIgnoreCase)).ToList();
}
var repeatOpt = parsed.SuppliedArgs.SingleOrDefault(x => x.ShortSymbol == RepeatSymbol);
var repeats = 1;
if (repeatOpt?.Value != null && int.TryParse(repeatOpt.Value, CultureInfo.InvariantCulture, out repeats))
{
}
Console.WriteLine($"Found {fileList.Count} files.");
Console.WriteLine();
PrintTableColumns("File", "Size", "Words", "Pages", "Open cost (μs)", "Total cost (μs)", "Page cost (μs)");
var dataList = new List<DataRecord>();
var sw = new Stopwatch();
for (int i = 0; i < repeats; i++)
{
foreach (var file in fileList)
if (maxCount.HasValue && runningCount >= maxCount)
{
if (maxCount.HasValue && runningCount >= maxCount)
break;
}
try
{
var numWords = 0;
var numPages = 0;
using (var pdfDocument = PdfDocument.Open(file))
{
break;
}
try
{
var numWords = 0;
var numPages = 0;
long openMicros;
long totalPageMicros;
sw.Reset();
sw.Start();
using (var pdfDocument = PdfDocument.Open(file))
foreach (var page in pdfDocument.GetPages())
{
sw.Stop();
openMicros = sw.Elapsed.Microseconds;
sw.Start();
foreach (var page in pdfDocument.GetPages())
numPages++;
foreach (var word in page.GetWords())
{
numPages++;
foreach (var word in page.GetWords())
if (word != null)
{
if (word != null)
{
numWords++;
}
numWords++;
}
}
sw.Stop();
totalPageMicros = sw.Elapsed.Microseconds;
}
var filename = Path.GetFileName(file);
var size = new FileInfo(file);
var item = new DataRecord
{
FileName = filename,
OpenCostMicros = openMicros,
Pages = numPages,
Size = size.Length,
Words = numWords,
TotalCostMicros = totalPageMicros + openMicros,
PerPageMicros = Math.Round(totalPageMicros / (double)Math.Max(numPages, 1), 2)
};
dataList.Add(item);
PrintTableColumns(
item.FileName,
item.Size,
item.Words,
item.Pages,
item.OpenCostMicros,
item.TotalCostMicros,
item.PerPageMicros);
}
catch (Exception ex)
{
hasError = true;
errorBuilder.AppendLine($"Parsing document {file} failed due to an error.")
.Append(ex)
.AppendLine();
}
runningCount++;
var filename = Path.GetFileName(file);
var size = new FileInfo(file);
Console.WriteLine($"{GetCleanFilename(filename)}| {size.Length}\t| {numWords}\t| {numPages}");
}
catch (Exception ex)
{
hasError = true;
errorBuilder.AppendLine($"Parsing document {file} failed due to an error.")
.Append(ex)
.AppendLine();
}
runningCount++;
}
if (hasError)
@@ -284,71 +88,12 @@ namespace UglyToad.PdfPig.ConsoleRunner
return 5;
}
if (outputOpt != null && outputOpt.Value != null)
{
WriteOutput(outputOpt.Value, dataList);
}
Console.WriteLine("Complete! :)");
return 0;
}
private static void WriteOutput(string outPath, IReadOnlyList<DataRecord> records)
{
using var fs = File.OpenWrite(outPath);
using var sw = new StreamWriter(fs);
sw.WriteLine("File,Size,Words,Pages,Open Cost,Total Cost,Per Page");
foreach (var record in records)
{
var sizeStr = record.Size.ToString("D", CultureInfo.InvariantCulture);
var wordsStr = record.Words.ToString("D", CultureInfo.InvariantCulture);
var pagesStr = record.Pages.ToString("D", CultureInfo.InvariantCulture);
var openCostStr = record.OpenCostMicros.ToString("D", CultureInfo.InvariantCulture);
var totalCostStr = record.TotalCostMicros.ToString("D", CultureInfo.InvariantCulture);
var ppcStr = record.PerPageMicros.ToString("F2", CultureInfo.InvariantCulture);
var numericPartsStr = string.Join(",",
[
sizeStr,
wordsStr,
pagesStr,
openCostStr,
totalCostStr,
ppcStr
]);
sw.WriteLine($"\"{record.FileName}\",{numericPartsStr}");
}
sw.Flush();
}
private static void PrintTableColumns(params object[] values)
{
for (var i = 0; i < values.Length; i++)
{
var value = values[i];
var valueStr = value.ToString();
var cleaned = GetCleanStr(valueStr ?? string.Empty);
var padChars = 16 - cleaned.Length;
var padding = padChars > 0 ? new string(' ', padChars) : string.Empty;
var padded = cleaned + padding;
Console.Write("| ");
Console.Write(padded);
}
Console.WriteLine();
}
private static string GetCleanStr(string name, int maxLength = 16)
private static string GetCleanFilename(string name, int maxLength = 30)
{
if (name.Length <= maxLength)
{
@@ -360,21 +105,4 @@ namespace UglyToad.PdfPig.ConsoleRunner
return name.Substring(0, maxLength);
}
}
internal class DataRecord
{
public required string FileName { get; init; }
public required long Size { get; init; }
public required int Words { get; init; }
public required int Pages { get; init; }
public required long OpenCostMicros { get; init; }
public required long TotalCostMicros { get; init; }
public required double PerPageMicros { get; init; }
}
}

View File

@@ -1,8 +1,8 @@
{
"profiles": {
"UglyToad.PdfPig.ConsoleRunner": {
"commandName": "Project",
"commandLineArgs": "\"C:\\temp\\pdfs\\archive\""
"commandName": "Project",
"commandLineArgs": "\"C:\\temp\\pdfs\\archive\""
}
}
}

View File

@@ -4,7 +4,6 @@
<LangVersion>latest</LangVersion>
<OutputType>Exe</OutputType>
<TargetFramework>net8</TargetFramework>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>

View File

@@ -1,4 +1,4 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net462;net471;net6.0;net8.0</TargetFrameworks>
<PackageId>PdfPig</PackageId>
@@ -11,9 +11,9 @@
<PackageTags>PDF;Reader;Document;Adobe;PDFBox;PdfPig;pdf-extract;pdf-to-text;pdf;file;text;C#;dotnet;.NET</PackageTags>
<RepositoryUrl>https://github.com/UglyToad/PdfPig</RepositoryUrl>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<Version>0.1.12-alpha001</Version>
<AssemblyVersion>0.1.12.0</AssemblyVersion>
<FileVersion>0.1.12.0</FileVersion>
<Version>0.1.11-alpha001</Version>
<AssemblyVersion>0.1.10.0</AssemblyVersion>
<FileVersion>0.1.10.0</FileVersion>
<PackageIconUrl>https://raw.githubusercontent.com/UglyToad/PdfPig/master/documentation/pdfpig.png</PackageIconUrl>
<PackageIcon>pdfpig.png</PackageIcon>
<Product>PdfPig</Product>