OpenCV Panorama Stitching in an Expo Native Module

Key takeaways: Don't vendor heavy native SDKs — declare them through SPM and Maven and let package managers do the work. Keep the C++ surface tiny (ours is ~140 lines on iOS, zero on Android). Design one symmetric API so JavaScript never knows which platform it's on. Release every OpenCV Mat explicitly — the JS garbage collector can't see native buffers.

Why We Built This

Panorama stitching keeps coming up — camera apps, real-estate and property tours, mapping and inspection tools. The engine for it is almost always OpenCV. The problem is getting OpenCV into a React Native app without losing a few days to native build plumbing.

So we built and open-sourced expo-panoramic-stitcher: an Expo native module that takes a set of overlapping photos and returns a stitched panorama — 360° equirectangular, a wide horizontal sweep, or a flat scan — with one typed API that behaves identically on iOS and Android. It's MIT-licensed and targets Expo SDK 56+ (React Native 0.85+).

This post is the engineering story behind it.

The Real Problem Isn't Stitching — It's Integration

OpenCV already solves stitching well. Its Stitcher class does feature detection, matching, warping, and multi-band blending out of the box. The hard part is everything around it:

• Vendoring the SDK. OpenCV is large. The usual approach is to download a platform SDK by hand and check binaries into the repo.
• Two native build systems. Android wants CMake/NDK and JNI glue. iOS wants an XCFramework and the right build settings.
• API drift. Once you have two native code paths, they diverge — different option names, different return shapes, different bugs — and your JavaScript ends up special-casing each platform.

We wanted installation to feel like adding any other dependency, and we wanted the JavaScript layer to never know which platform it's running on.

Decision 1: Don't Vendor OpenCV — Let the Package Managers Do It

The biggest simplification was refusing to vendor OpenCV at all.

On iOS, the module declares a Swift Package Manager dependency on a prebuilt OpenCV XCFramework (yeatse/opencv-spm) straight from the podspec. On Android, it adds org.opencv:opencv:4.13.0 from Maven Central in build.gradle. Both package managers already know how to fetch and link those artifacts.

The payoff: no checked-in binaries, no NDK or CMake project to maintain, and no manual SDK download step in the README. A consumer installs the module and builds — the native dependency resolves like anything else.

Decision 2: Keep the Native Surface as Thin as Possible

The less native code you own, the less can break. We pushed almost everything into ordinary Swift and Kotlin and kept the C++ to a minimum.

On iOS, all the OpenCV-touching C++ lives in a single Objective-C++ shim of about 140 lines. Its core is nothing more than this:

// PanoramaStitcher.mm (simplified)
std::vector<cv::Mat> images = loadImages(inputPaths);

cv::Ptr<cv::Stitcher> stitcher = cv::Stitcher::create(mode); // PANORAMA or SCANS
stitcher->setPanoConfidenceThresh(matchConfidence);

cv::Mat pano;
cv::Stitcher::Status status = stitcher->stitch(images, pano);
if (status != cv::Stitcher::OK) {
  return errorResult(statusMessage(status)); // readable error, not a crash
}

resizeToMaxWidth(pano, outputWidth);
cv::imwrite(outputPath, pano, jpegQuality);

The Swift module above it handles option records, temp files, and base64 — normal app code, easy to read.

On Android there's no C++ at all. OpenCV's Maven package ships a complete Java API, so the module is pure Kotlin calling Stitcher.create and stitch directly — no JNI, no CMake, no NDK.

That's the whole native footprint: one small shim and one Kotlin file. The part that can actually go wrong is small enough to audit in a sitting.

Decision 3: One Symmetric API

Both platforms expose the same three functions, consumed from TypeScript like this:

import { stitchImagePaths } from "@notchip/expo-panoramic-stitcher";

const result = await stitchImagePaths(photoUris, {
  warpMode: "spherical", // "cylindrical" | "plane"
  outputWidth: 4096,
  jpegQuality: 95,
});

if (result.success) {
  // result.outputPath, result.width, result.height, result.aspectRatio
}

• stitchImagePaths takes file paths and writes a JPEG to disk. It's the lowest-memory path because images never leave the filesystem.
• stitchBase64 takes and returns base64 JPEGs — convenient when you already hold images in memory, and the payload is identical on both platforms.
• stitchIncrementalBase64 folds one new frame into an existing panorama, so you can build a panorama progressively from a live camera feed.

Options are the same everywhere — warp mode, blend strength, match confidence, output width, auto-resize, and JPEG quality — and defaults are applied once in TypeScript. Results share one shape too: success, the output, dimensions, aspect ratio, and an error message. App code calls the same function with the same options and gets the same result, whatever the OS.

There's also a web stub that reports unavailable, plus an isStitchingAvailable check, so universal Expo code compiles and degrades gracefully where OpenCV isn't present.

Decision 4: File Paths at the Core, Base64 at the Edge

A subtle but important choice: the C++ core only ever reads and writes image files. Base64 is handled at the Swift/Kotlin boundary — decoded to a temp JPEG on the way in, encoded back on the way out.

This keeps the expensive native path identical no matter how the caller hands images in, and it keeps memory predictable. Stitching is memory-heavy, and buffers that linger compound fast in the incremental case where frames arrive continuously.

The Gotcha: OpenCV Redefines YES and NO

A fun one. OpenCV's C++ headers define their own YES and NO macros. Objective-C already uses YES and NO as its boolean literals. The moment you import OpenCV into an Objective-C++ file, those macros collide and compilation breaks in confusing ways.

The fix is small but necessary:

#pragma push_macro("YES")
#pragma push_macro("NO")
#undef YES
#undef NO

#import <opencv2/opencv.hpp>
#import <opencv2/stitching.hpp>

#pragma pop_macro("NO")
#pragma pop_macro("YES")

That quarantines the conflict to a couple of lines instead of letting it leak into the rest of the iOS module.

Memory: Release Everything, Clean Up Everything

OpenCV's Mat objects hold large native buffers that aren't governed by the JavaScript garbage collector, so they have to be released explicitly. The Android module releases every Mat it creates in finally blocks; the iOS side cleans up temp files in defer blocks. Combined with the file-path core, that keeps memory flat across repeated and incremental stitches instead of climbing with every frame.

Warp Modes, and Picking the Right One

Stitching quality depends on choosing the right projection for the shot:

One practical note worth putting in your own UI: OpenCV needs roughly 30–40% overlap between adjacent frames to find enough matching features. When it can't, it returns a non-OK status, which the module surfaces as a readable error rather than a silent failure. For live capture, the onStitchProgress events (a 0–1 value plus a stage name) let you show real feedback while a stitch runs.

What Shipped

expo-panoramic-stitcher is open source under the MIT license, published to npm and GitHub Packages as @notchip/expo-panoramic-stitcher. Install it with npx expo install @notchip/expo-panoramic-stitcher. It runs on Expo SDK 56+ / React Native 0.85+, supports iOS 16.4+ and Android API 24+, and exposes the same panorama pipeline — paths, base64, and incremental — through one typed API.

The broader lesson generalizes past stitching: when you wrap a heavy native library for React Native, lean on prebuilt packages instead of vendoring, keep the C++ surface tiny, and make the two platforms symmetric so your JavaScript never has to care which one it's on. See the engineering write-up in our Panoramic Stitcher case study.

Built by NOTchip](/) — a [mobile and web app development agency. If you need a React Native team comfortable down at the native layer, get in touch.