Skip to content

SharePoint OCR: making scanned PDFs searchable

SharePoint search does not find scanned PDFs without a text layer. How OCR processing on upload is set up and what to bear in mind in operation.

Updated
Diagram: a scanned PDF without a text layer becomes searchable through OCR, a set of minutes receives filing level and phase, an email its header data as metadata

A delivery note is scanned on the construction site and filed in the project workspace. Three months later, someone searches for the delivery note number and finds nothing. The file is there. But it consists only of an image, and search finds no text in an image.

This post describes how we make scanned PDFs searchable on upload for one customer and what to bear in mind in operation.

What SharePoint finds on its own

Search in SharePoint searches the text of documents: Word, Excel, PowerPoint and PDFs that have a text layer. A PDF that was exported from Word is found.

A scan is a photo in a PDF wrapper. Without text recognition, OCR for short, its content remains invisible to search. The same applies to photos of rating plates, delivery notes or whiteboards.

Microsoft offers text recognition for Microsoft 365 that is billed by consumption. Whether it covers your file types and volumes and what it costs is best checked against the current documentation. For organisations with occasional scans, this can be the simplest route.

With large volumes and the wish to make the file itself searchable, a setup of your own is worthwhile.

The setup: react, check, replace

1. React to new files. The libraries of the project workspace report changes to a service. SharePoint offers webhooks for this. They expire after 180 days at the latest and are therefore checked and renewed daily.

2. Acknowledge immediately, work later. The service accepts the notification, places it in a queue and responds immediately. The actual work happens afterwards. SharePoint expects the acknowledgement within a few seconds.

3. Filter out duplicates. A file often triggers several notifications, for example on upload and when the metadata is set. A table remembers what has already been processed.

4. Check whether OCR is needed. The service opens the PDF and checks whether it contains text. Most files have a text layer and are skipped.

5. Recognise and replace. PDFs without text go through text recognition. The result is a PDF that looks exactly the same but has an invisible text layer. It replaces the original file. Author and editor are preserved.

6. Treat images differently. With photos, the file is not changed. The recognised text ends up in a metadata field and can be found through it.

Diagram of the automation after upload: OCR, filing level and phase, header data from emails

Figures from operation

For a construction group, the service checked around two million files in 30 days, about 68,000 a day. For 92.8 per cent, no text recognition was needed.

Two things follow from this. The check has to be cheap because it runs for every file. And the actual recognition concerns only a small share, which does, however, need computing time. Handling the two separately keeps the costs within limits.

Seven stumbling blocks

The loop. Replacing the file is itself a change and triggers a new notification. Without protection, the service processes its own results. The table from step 3 and the check from step 4 prevent that.

The versions. Every replacement creates a version. For large scans, that doubles the storage the file requires. Cleaning up old versions is part of the concept.

Who was it? If a service replaces the file, then without precautions the service account appears as the last editor in the library. The details of the original author have to be preserved explicitly.

Protected PDFs. Encrypted and password-protected PDFs cannot be processed. They are skipped.

Making errors visible. Every file carries two fields: processed and processing error. A view shows what has got stuck. Without it, nobody notices that nothing has been recognised for a week.

Drawings. Large-format drawings with little text are demanding for text recognition. It is still worthwhile because title blocks and drawing numbers become findable this way.

Quality of the scan. Text recognition is as good as the original. Skewed photos in poor light produce patchy text. Search then finds some things, but not everything. Users should know that.

What else the same service handles

If you are reacting to every new file anyway, you can do more than recognise text. For the same customer, the service handles two further tasks:

  • Filing level and phase. From the file’s storage location, it writes filing levels 1 to 3 and the current project phase as metadata. Why that helps is described in the post on the folder structure for projects.
  • Emails. For filed emails, it reads out sender, recipient, date received and attachments and writes them to columns. A library full of emails can then be sorted and filtered by them.

When the setup is worthwhile

  • Yes, if scans regularly end up in project workspaces: delivery notes, signed minutes, official notices, as-built documents.
  • Yes, if the files are passed on later and should also be searchable outside SharePoint.
  • No, if almost all documents are created digitally. Then the text layer is already there.
  • Maybe, if only a few scans are involved. Then the scanner itself, which can produce searchable PDFs, is often enough.

Searchable PDFs are one building block of Smarter Project Portal.

Are there scans in your project workspaces that nobody can find again? Write to us and tell us which volumes and document types are involved.

All parts of the series:

  1. Provisioning SharePoint project workspaces automatically: where templates end
  2. Folder structure for projects in SharePoint: template, not copy
  3. Teams template with Planner: project teams with standard tasks
  4. SharePoint permission concept for projects: roles, not names
  5. Project data from ERP and SAP in SharePoint: four patterns
  6. SharePoint OCR: making scanned PDFs searchable (this article)
  7. Drawing management in SharePoint: drawing code, index and versions
  8. Project phases in SharePoint: from quotation to project closure
  9. Managing SharePoint permissions in Excel: the permission matrix of a project workspace

Related articles

Drawing management in SharePoint: drawing code, index
Diagram: a file name is broken down into project, trade, drawing type, number, index and status, with the versions of a drawing with index A to C below

Drawing management in SharePoint: drawing code, index and versions

Managing drawings in SharePoint via a drawing code in the file name: metadata from the name, every index a version and where the limits lie.

Read more
Project data from ERP and SAP in SharePoint: four
Diagram: project workspace in the centre, connected to ERP and CRM, SAP, controlling, site apps, BIM repository and signature service

Project data from ERP and SAP in SharePoint: four patterns

How SharePoint project workspaces are connected to ERP, CRM, SAP and site apps: four patterns from practice and what keeps them stable in operation.

Read more
Folder structure for projects in SharePoint: template,
Diagram: on the left the template with six numbered areas and subfolders, on the right three projects with the same structure

Folder structure for projects in SharePoint: template, not copy

Three real-world filing structures, from the reference structure to a filing plan with over 90 libraries, and the rules for building them in SharePoint.

Read more

Questions about this topic?

We are happy to help you put this into practice in your environment.