OCRRouter

OCR on iOS: Apple's Vision framework or a cloud OCR API

Two ways to add OCR to an iPhone app: the on-device Vision framework, and a cloud OCR API called from Swift. Code for both, and when to use each.

Updated

TL;DR

  • Apple's Vision framework is the built-in iOS OCR SDK: free, offline, and good for printed text and live camera text.
  • A cloud OCR API is better for tables, handwriting, long PDFs and fields such as invoice totals.
  • Never put a cloud API key in the app. Call the OCR API from your own server and let the app call your server.
  • Many apps use both: Vision for instant text, a cloud API when the user saves a document.

There are two ways to read text from an image in an iPhone app. The Vision framework runs OCR on the device and ships with iOS. A cloud OCR API sends the image to a hosted model and returns text, Markdown or JSON. This guide shows both in Swift and when each one is the better choice.

Vision framework or cloud API?

Vision framework (on device)Cloud OCR API
CostFreePer page, from $1 per 1,000 pages
Works offlineYesNo
SpeedInstant on recent iPhonesAbout 1 to 3 seconds per page
Printed textGoodGood
HandwritingLimitedStrong on the better models
TablesLines with positions, no tableMarkdown or JSON tables
PDFs with many pagesYou render each page yourselfSend the PDF
PrivacyImage never leaves the phoneImage goes to the provider

Use Vision for live camera text, short labels, and anything that must work offline or stay on the device. Use a cloud API when the result has to be structured (a table, an invoice total, a form) or the input is hard: handwriting, poor photos, long PDFs.

On-device OCR with the Vision framework

Vision has been Apple's OCR SDK since iOS 13. This is the classic request, which works on every supported iOS version:

import Vision
import UIKit

func recognizeText(in image: UIImage, completion: @escaping ([String]) -> Void) {
    guard let cgImage = image.cgImage else { return completion([]) }

    let request = VNRecognizeTextRequest { request, _ in
        let observations = request.results as? [VNRecognizedTextObservation] ?? []
        let lines = observations.compactMap { $0.topCandidates(1).first?.string }
        DispatchQueue.main.async { completion(lines) }
    }
    request.recognitionLevel = .accurate      // .fast for live camera frames
    request.usesLanguageCorrection = true
    request.recognitionLanguages = ["en-US"]  // add others your users need

    let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
    DispatchQueue.global(qos: .userInitiated).async {
        try? handler.perform([request])
    }
}

From iOS 18 there is also a Swift-native API, RecognizeTextRequest, which you call with try await request.perform(on: image). It returns the same kind of observations.

What you get back is a list of lines, each with a bounding box and a confidence. That is enough for search, copy and paste, or finding a phone number. It is not a table and it does not know that a number is an invoice total.

Cloud OCR from Swift

A cloud OCR API is one HTTP request. The example below uses OCRRouter, which puts many OCR models behind the same request, so you can switch models by changing one string.

Do not call the API with your key from the app. Put a small endpoint on your own server that adds the key and forwards the request, and call that endpoint from the app. The Swift side then looks like this:

import UIKit

struct OCRResult: Decodable { let markdown: String }

func cloudOCR(_ image: UIImage) async throws -> String {
    guard let jpeg = image.jpegData(compressionQuality: 0.85) else { return "" }

    // Your backend, which adds the OCRRouter key and forwards to /v1/ocr.
    var req = URLRequest(url: URL(string: "https://your-backend.example.com/ocr")!)
    req.httpMethod = "POST"
    req.setValue("application/json", forHTTPHeaderField: "Content-Type")
    req.httpBody = try JSONEncoder().encode([
        "model": "mistral/mistral-ocr",
        "file_base64": jpeg.base64EncodedString(),
        "format": "markdown",
    ])

    let (data, _) = try await URLSession.shared.data(for: req)
    return try JSONDecoder().decode(OCRResult.self, from: data).markdown
}

On the server, forward the same JSON body to POST https://api.ocrrouter.ai/v1/ocr with the header Authorization: Bearer <your key>. The response has a markdown field, and with "format": "json" a pages array instead. Add "fallbacks": ["google/gemini-flash"] so a failed page is retried on another model.

Reading PDFs on iOS

Vision reads images, not PDFs. To OCR a PDF on the device, render each page to an image with PDFKit first:

import PDFKit

func pageImages(from url: URL, scale: CGFloat = 2) -> [UIImage] {
    guard let doc = PDFDocument(url: url) else { return [] }
    return (0..<doc.pageCount).compactMap { i in
        guard let page = doc.page(at: i) else { return nil }
        let box = page.bounds(for: .mediaBox)
        let size = CGSize(width: box.width * scale, height: box.height * scale)
        return UIGraphicsImageRenderer(size: size).image { ctx in
            UIColor.white.fill(ctx.format.bounds)
            ctx.cgContext.translateBy(x: 0, y: size.height)
            ctx.cgContext.scaleBy(x: scale, y: -scale)
            page.draw(with: .mediaBox, to: ctx.cgContext)
        }
    }
}

A scale of 2 gives roughly 144 dpi, which is enough for normal print; use 3 or 4 for small text. If the PDF already has a text layer, page.string returns the text without any OCR at all, so check that first.

With a cloud API you skip this step. Send the PDF itself as file_base64, and add "pages": "1-5" to read only some pages. Only the pages that are read are charged.

Languages

Vision supports a fixed set of languages that grows with each iOS release. Ask the request which ones the current device supports instead of hard-coding a list:

let request = VNRecognizeTextRequest()
let languages = (try? request.supportedRecognitionLanguages()) ?? []
print(languages)

Set recognitionLanguages to the languages you expect, most likely first. Accuracy drops when the list is long or wrong. For scripts Vision does not cover, or for mixed-language documents, cloud models such as PaddleOCR-VL and Google Document AI are built for many languages.

What a cloud API costs in an app

Cloud OCR is priced per page, so the cost depends on how many documents your users save, not on how often they open the camera. Some round numbers at OCRRouter prices:

Pages per month$1 per 1,000 pages model$6 per 1,000 pages model
10,000$10$60
100,000$100$600
1,000,000$1,000$6,000

Two things keep the bill down. Send only pages the user chose to keep, and cache results on your server: sending the same file to the same model again is answered from cache at no cost. The cheapest OCR page lists the price of every model.

Which cloud model for which document

From our published tests (each linked post has the file):

The best OCR API comparison has prices and results for all of them, and the OCR model benchmark roundup explains how they rank on public tests.

Privacy and App Store review

On-device OCR with Vision keeps the image on the phone, which makes the privacy story simple. As soon as you send images to a cloud API, the image leaves the device, and that has consequences you should plan for:

  • Tell users. Say in the app, at the moment they scan, that the document is sent to a server for reading.
  • Declare it. Your App Store privacy details must list the data you collect, and documents can contain names, addresses and financial details.
  • Keep retention short. Delete the image on your server once you have the result, and check how long the OCR provider keeps files. OCRRouter keeps uploads privately to debug results and deletes them when you email support@ocrrouter.ai.

Using both in one app

A common pattern:

  1. Run Vision on camera frames so the user sees text highlighted instantly, with no cost and no network.
  2. When the user taps Save or Scan document, send the captured page to the cloud API for a clean Markdown or JSON result.
  3. If the phone is offline, keep the Vision text and upload later.

This keeps the camera fast and free, and pays per page only for documents the user actually keeps.

FAQ

Read next