OpenCV Panorama Stitching in an Expo Native Module

Key takeaways: Package a heavy native SDK once, inside the module, so consumers never touch it. We tried letting SPM and Maven do that job and it broke in real apps; the fix is below. Keep the C++ surface tiny (ours is one ~180-line shim on iOS and one ~190-line JNI file on Android, same core in both). Design one symmetric API so JavaScript never knows which platform it's on. Release every OpenCV Mat explicitly, because the JS garbage collector can't see native buffers.

*Updated September 2026 for version 0.6.0. The original June post described the 0.1 release; the sections on OpenCV packaging and the API have been rewritten to match what ships now, and the original wording is preserved in the git history of this page.*

Why We Built This

Panorama stitching keeps coming up — camera apps, real-estate and property tours, mapping and inspection tools. The engine for it is almost always OpenCV. The problem is getting OpenCV into a React Native app without losing a few days to native build plumbing.

So we built and open-sourced expo-panoramic-stitcher: an Expo native module that takes a set of overlapping photos and returns a stitched panorama — 360° equirectangular, a wide horizontal sweep, or a flat scan — with one typed API that behaves identically on iOS and Android. It's MIT-licensed and targets Expo SDK 56+ (React Native 0.85+).

This post is the engineering story behind it, including the part where our first packaging decision turned out to be wrong. We now use the module ourselves, as the capture engine for FieldDojo's upcoming Walkthrough feature, which is how most of the bugs got found.

The Real Problem Isn't Stitching — It's Integration

OpenCV already solves stitching well. Its Stitcher class does feature detection, matching, warping, and multi-band blending out of the box. The hard part is everything around it:

• Getting the SDK. OpenCV is large. The usual approach is to download a platform SDK by hand and check binaries into the repo, per app.
• Two native build systems. Android wants CMake/NDK and JNI glue. iOS wants an XCFramework and the right build settings.
• API drift. Once you have two native code paths, they diverge — different option names, different return shapes, different bugs — and your JavaScript ends up special-casing each platform.

We wanted installation to feel like adding any other dependency, and we wanted the JavaScript layer to never know which platform it's running on.

Decision 1: Package OpenCV Once, Inside the Module (Second Attempt)

The first version of this section said: don't vendor OpenCV, let the package managers do it. On iOS the podspec declared a Swift Package Manager dependency on a prebuilt XCFramework (yeatse/opencv-spm). On Android, build.gradle pulled org.opencv:opencv:4.13.0 from Maven Central. Zero binaries, zero build scripts, and it worked in the example app. I was pleased with it.

Then we started consuming the module from FieldDojo, and both halves failed within a week of each other.

On iOS, FieldDojo builds with static frameworks (expo-build-properties, useFrameworks: "static"), which a lot of Expo apps do because some popular pods need it. Under that setting the SPM-provided OpenCV never got linked into the shim, and the app failed to link with a wall of undefined cv:: symbols. Nothing in the module was wrong on its own. It was only wrong inside a real app.

On Android the failure was simpler and more embarrassing. The Maven AAR ships libopencv_java4.so without the stitching module compiled in. The one module we needed. It had been working in development because a stale local build still had it.

So versions 0.3 and 0.4 did what the June post said not to do, with two constraints that keep it from being the bad kind of vendoring:

• iOS: a postinstall script downloads the prebuilt opencv2.xcframework (about 191 MB) from the same yeatse release, checks its SHA-256, and drops it into the pod as a vendored static framework. It runs on macOS only, skips itself if the marker matches, and can be pointed at a local zip for offline machines.
• Android: a Gradle task downloads the official OpenCV 4.13.0 Android SDK (about 303 MB) once per machine into the Gradle cache, hash-checked, and a CMake project links just the stitching, calib3d, features2d, flann, imgcodecs, imgproc and core modules statically into one libpanostitcher.so. The zip is deleted after unpacking and the cache survives a clean.

Nothing is committed to git in either case, and the consumer's install command didn't change: npx expo install, then prebuild. What we gave up was the tidy story. What we got was a module that links in apps we don't control.

Decision 2: Keep the Native Surface as Thin as Possible

The less native code you own, the less can break. We pushed almost everything into ordinary Swift and Kotlin and kept the C++ to a minimum.

On iOS, all the OpenCV-touching C++ lives in a single Objective-C++ shim of about 180 lines. Its core is nothing more than this:

// PanoramaStitcherShim.mm (simplified)
std::vector<cv::Mat> images = loadImages(inputPaths);

cv::Ptr<cv::Stitcher> stitcher = cv::Stitcher::create(mode); // PANORAMA or SCANS
stitcher->setFeaturesMatcher(matcherFor(mode, matchConf));
stitcher->setPanoConfidenceThresh(panoConfidence);

cv::Mat pano;
cv::Stitcher::Status status = stitcher->stitch(images, pano);
if (status != cv::Stitcher::OK) {
  return errorResult(statusMessage(status)); // readable error, not a crash
}

resizeToMaxWidth(pano, outputWidth);
cv::imwrite(outputPath, pano, jpegQuality);

The Swift module above it handles option records, temp files, and base64 — normal app code, easy to read.

Android used to have no C++ at all; the Kotlin module called OpenCV's Java Stitcher directly. When the Maven AAR went away (see Decision 1), so did the Java API, so Android now has one JNI file of about 190 lines. It's the same stitch core as the iOS shim, copied line for line, with a thin JNI wrapper that takes a list of paths and returns a small pipe-delimited string the Kotlin side parses. The Kotlin module still does everything else: options, temp files, base64, the executor.

That's the whole native footprint: one shim, one JNI file, and the Swift and Kotlin around them. The part that can actually go wrong is small enough to audit in a sitting, and because the two C++ cores are identical, a fix lands in both by copy-paste, which is not elegant but has been reliable.

Decision 3: One Symmetric API

Both platforms expose the same functions, consumed from TypeScript like this:

import { stitchImagePaths } from "@notchip/expo-panoramic-stitcher";

const result = await stitchImagePaths(photoUris, {
  warpMode: "spherical", // "cylindrical" | "plane"
  outputWidth: 4096,
  jpegQuality: 95,
});

if (result.success) {
  // result.path, result.width, result.height, result.aspectRatio
  // result.usedIndices — which inputs OpenCV actually kept
}

• stitchImagePaths takes file paths and writes a JPEG to disk. It's the lowest-memory path because images never leave the filesystem.
• stitchBase64 takes and returns base64 JPEGs — convenient when you already hold images in memory, and the payload is identical on both platforms.
• stitchIncrementalBase64 folds one new frame into an existing panorama, so you can build a panorama progressively from a live camera feed.
• stitchSweep (added in 0.6) takes photos tagged with the compass heading they were shot at. It stitches cylindrical by default, detects when the sweep covers a full turn and closes the loop, and if OpenCV drops a run of frames it stitches the dropped run separately and reports the gaps in degrees instead of pretending the panorama is complete.

There's also a separate entry point, @notchip/expo-panoramic-stitcher/capture, with a useGuidedSweep hook and a GuidedSweepCapture screen. It reads the gyroscope, tells the user to keep turning, slow down, hold, or level the phone, and takes a shot every 15° with a haptic tick. expo-camera and expo-sensors are optional peers, so apps that only want the stitcher don't pay for them.

Options are the same everywhere — warp mode, blend strength, match confidence, panorama confidence, output width, auto-resize, and JPEG quality — and defaults are applied once in TypeScript. Results share one shape too: success, the output, dimensions, aspect ratio, the used indices, and an error message. App code calls the same function with the same options and gets the same result, whatever the OS.

There's also a web stub that reports unavailable, plus an isStitchingAvailable check, so universal Expo code compiles and degrades gracefully where OpenCV isn't present.

Decision 4: File Paths at the Core, Base64 at the Edge

A subtle but important choice: the C++ core only ever reads and writes image files. Base64 is handled at the Swift/Kotlin boundary — decoded to a temp JPEG on the way in, encoded back on the way out.

This keeps the expensive native path identical no matter how the caller hands images in, and it keeps memory predictable. Stitching is memory-heavy, and buffers that linger compound fast in the incremental case where frames arrive continuously.

The Gotcha: OpenCV Redefines YES and NO

A fun one. OpenCV's C++ headers define their own YES and NO macros. Objective-C already uses YES and NO as its boolean literals. The moment you import OpenCV into an Objective-C++ file, those macros collide and compilation breaks in confusing ways.

The fix is small but necessary:

#pragma push_macro("YES")
#pragma push_macro("NO")
#undef YES
#undef NO

#import <opencv2/opencv.hpp>
#import <opencv2/stitching.hpp>

#pragma pop_macro("NO")
#pragma pop_macro("YES")

That quarantines the conflict to a couple of lines instead of letting it leak into the rest of the iOS module.

Memory: Release Everything, Clean Up Everything

OpenCV's Mat objects hold large native buffers that aren't governed by the JavaScript garbage collector, so they have to be released explicitly. The C++ core releases every Mat it creates before returning; Kotlin cleans temp files in finally blocks and Swift in defer blocks. Combined with the file-path core, that keeps memory flat across repeated and incremental stitches instead of climbing with every frame.

Warp Modes, and Picking the Right One

Stitching quality depends on choosing the right projection for the shot:

One practical note worth putting in your own UI: OpenCV needs roughly 30–40% overlap between adjacent frames to find enough matching features. When it can't, it returns a non-OK status, which the module maps to a readable message on both platforms (ERR_NEED_MORE_IMGS becomes "not enough matched images, ensure 30–40% overlap or lower panoConfidence", and so on). For live capture, the onStitchProgress events (a 0–1 value plus a stage name) let you show real feedback while a stitch runs.

A quieter failure mode took longer to notice. OpenCV will happily return success after throwing away most of your inputs. In one FieldDojo test room, 6 of 9 photos were discarded on blank walls at the default confidence and the "panorama" was three frames wide. That's why 0.5 added the panoConfidence option and the usedIndices field on every result. Success and complete are different things, and the app needs to know which one it got.

What Shipped

expo-panoramic-stitcher is open source under the MIT license, published to npm and GitHub Packages as @notchip/expo-panoramic-stitcher, currently 0.6.0 with provenance attached via npm trusted publishing. Install it with npx expo install @notchip/expo-panoramic-stitcher. It runs on Expo SDK 56+ / React Native 0.85+ (developed and verified against SDK 57), supports iOS 16.4+ and Android API 24+, and exposes the same panorama pipeline — paths, base64, incremental, and guided sweep — through one typed API.

The broader lesson changed a little between June and September. Keep the C++ surface tiny and make the two platforms symmetric, that part held. On packaging, the honest version is: pull the heavy SDK in at install time, verify it, and link it yourself, because the package-manager artifacts you don't control will eventually not have the thing you need. Do that work once in the module so nobody downstream has to. The product-side breakdown is in the Panoramic Stitcher case study.

Built by Notchip](/) — a [mobile and web app development agency. If you need a React Native team comfortable down at the native layer, get in touch.