Start on-device if your app needs low latency or handles sensitive images, and move to a cloud API when you need broader features or large-scale custom training. Tools like ML Kit, Qualcomm FastCV and ESP-VISION cover most on-device scenarios. Whichever route looks right, run a small proof of concept on your own images before committing engineering time to full integration.
TL;DR:
- On-device SDKs are suitable for applications requiring low latency and offline operation, but increase binary size and require periodic updates.
- Cloud APIs offer advanced features and centralized updates but depend on network quality and incur per-call costs, making them less ideal for privacy-sensitive or latency-critical cases.
- Running a representative dataset on multiple device tiers and measuring latency, precision, and environmental factors is essential for realistic performance evaluation before full integration.
- Bundled models provide offline consistency but delay updates, whereas hosted models enable central management at the expense of initial download and connectivity dependence.
- Pricing complexities arise from storage, bandwidth, and model training costs, which often surpass recognition call expenses, emphasizing the need for detailed cost modeling.
Table of Contents
- On-device, cloud or hybrid: choosing the right SDK architecture
- Platform support and packaging: what to confirm before you prototype
- Performance and accuracy: a POC checklist that holds up in production
- Model customisation and deployment: bundled or hosted, and when to train your own
- Pricing and cost drivers you need in your ROI model
- Privacy, security and compliance for image data
- SDK ergonomics: what good developer experience looks like
- How Pocket App approaches image recognition projects
- Simple rules for deciding whether to build or buy
- Turning your POC into a shipped feature with Pocket App
- Sources
- FAQ
On-device, cloud or hybrid: choosing the right SDK architecture
An on-device SDK runs inference directly on the phone or embedded chip, using a bundled or locally stored model. A cloud API sends images to a remote service and returns results over the network. A hybrid approach does quick on-device filtering, then escalates uncertain cases to the cloud for a fuller answer.
Each model carries different costs for your team:
- On-device SDKs give you low latency and offline capability, but add to your binary size and tie you to periodic app updates for model improvements.
- Cloud APIs offer richer features and centralised model updates, but depend on network quality and introduce per-call costs.
- Hybrid setups balance both, though they add architectural complexity and more testing surface.
Qualcomm's FastCV SDK is a good example of the on-device route: it offers hardware-accelerated primitives such as face detection and text recognition tuned for Snapdragon chips, aimed squarely at mobile-first experiences that need to run broadly across consumer devices. For constrained edge hardware, ESP-VISION takes a similar approach with a low-code Python framework for capture, processing and inference on Espressif SoCs.
Platform support and packaging: what to confirm before you prototype
Before you write a line of integration code, confirm the platform and language bindings your project actually needs. That means checking Android and iOS support, whether a cross-platform wrapper exists for Flutter or React Native, and whether you need a web or server runtime for backend processing.
Packaging varies by SDK. Some ship as native Android or iOS libraries, others as TensorFlow Lite or ONNX models you load yourself, and some expose functionality purely through REST endpoints. ML Kit provides image labelling APIs and supports both approaches: you can bundle a custom TensorFlow Lite model directly in the app, or host it remotely through Play Services and download it on demand, which affects both app size and update speed.
That bundled-versus-hosted choice matters early. A bundled model means every user gets the same version on install and updates only ship with a new app release. A hosted model can be updated centrally, but adds a download step and a dependency on connectivity the first time it runs. Decide this before you scope your sprint, not after your POC is already working.
Performance and accuracy: a POC checklist that holds up in production
Vendor benchmarks rarely reflect what happens on a mid-range Android phone in a warm pocket. A cross-regional benchmark of enterprise cloud vision services found meaningful variation in accuracy and latency between providers, and that local inference latency can be far lower than cloud round-trip time, which makes a dataset-specific POC essential rather than optional.
Before you commit to an SDK, run this checklist:
- Build a representative dataset from your own images, not stock photos or vendor samples.
- Run repeatable latency tests across at least three device tiers, low, mid and high spec.
- Measure precision, recall and F1 score against a labelled ground truth, not just overall accuracy.
- Log environmental factors: lighting, camera angle, occlusion and device temperature.
Practical testing notes back this up. Instrumented on-device profiling is necessary because thermal throttling and camera pipeline contention change real-world latency in ways a host machine benchmark never shows. If you're testing a cloud API, measure full round-trip time end to end, not just server-side processing time, since network variability often dominates the user experience.
Model customisation and deployment: bundled or hosted, and when to train your own
Base models handle general object and scene recognition well. You need a custom model when your domain is narrow, product SKUs, specific defect types, or faces of known individuals, and base models cannot separate your classes reliably.
Deployment then comes down to a short set of trade-offs:
- Bundled models keep the app fully offline but grow your binary and lock the model version to your release cycle.
- Hosted models allow centralised updates and A/B testing, but need a reliable first-run download and add a maintenance pipeline.
- Edge frameworks with low-code interfaces, such as ESP-VISION, suit constrained devices where a full ML pipeline is impractical.
For face matching specifically, capturing multiple reference photos per person across varied angles and using centroid averaging produces noticeably more robust embeddings than relying on a single image.
Pricing and cost drivers you need in your ROI model
Cloud vision pricing typically follows a per-call or per-1,000-transaction model, often with a free tier for low-volume testing, scaling up through paid tiers as usage grows. Google Cloud Vision's pricing illustrates the pattern: costs vary by feature, with OCR, product recognition and video processing each billed differently, which means a flat per-call estimate rarely reflects your real bill.
The costs that catch teams out are rarely the per-call fees themselves. Storage and bandwidth for uploaded images, model training time if you're building a custom classifier, and long-running video or streaming workloads can dominate total cost of ownership at scale, often more than the recognition calls themselves. Before signing off on an architecture, run a small-volume scenario through the provider's own pricing calculator using your expected monthly image count and feature mix, then extrapolate.
Privacy, security and compliance for image data
Image data is personal data whenever it can identify someone, so treat it accordingly. Where possible, filter or crop on-device before anything leaves the phone, and avoid uploading images you don't strictly need for the result you're returning.
Encrypt images in transit and at rest, and set an explicit retention policy rather than leaving old images sitting in storage indefinitely. If you're processing personal data at scale, particularly faces, check your lawful basis for processing, consider whether a data protection impact assessment applies, and keep audit logs of what was processed and when.
SDK ergonomics: what good developer experience looks like
Good SDK documentation includes a working sample app you can run in minutes, not just an API reference. If the vendor doesn't offer that, budget extra time for integration and add your own smoke tests to CI early.
Hardware acceleration is where real performance gains come from: look for NEON support on ARM, Apple's Neural Engine on iOS, or Qualcomm's own optimised libraries on Snapdragon devices. Beyond raw speed, build in resilience: batch requests where possible, throttle frame processing rather than running inference on every camera frame, and design a degraded but functional UX for when recognition fails or times out.
How Pocket App approaches image recognition projects
Our process for these features follows the same shape every time: discovery to scope the use case, a focused POC to validate latency and accuracy on the client's own data, integration into the production app, then monitoring and optimisation once it's live. This catches the usual failure points early, model drift, privacy assumptions that don't hold up, and performance that looked fine on a test bench but not on a customer's device. Our AI in mobile app development work and our barcode scanning commissioning guide cover related delivery patterns in more depth.
Simple rules for deciding whether to build or buy
My rule of thumb: choose on-device when privacy or latency are non-negotiable, choose cloud when you need scale or feature depth you can't build yourself. Weigh it against your team's actual ML skills, who maintains the model after launch, and how fast you need to ship.
— Paul
Turning your POC into a shipped feature with Pocket App
Once your proof of concept shows the accuracy and latency you need, the harder part is turning it into a maintained production feature, and that's where a dedicated partner earns its place. Pocket App's Discover, Design, Develop and Deploy phases take a validated POC through to a shipped app, with the same team handling architecture decisions, on-device testing and post-launch monitoring so nothing gets lost between prototype and release.
If you want a faster read on your specific use case before committing to a full build, our Mobile App Clinics offer a focused technical review of your architecture and data.
Pro Tip: Bring your own sample images to a clinic session rather than relying on vendor demo data, the feedback will match what you'll actually see in production.
Sources
- Image labeling | ML Kit | Google for Developers
- Qualcomm FastCV SDK
- espressif/esp-vision
- Cross-Regional Benchmarking of Enterprise Cloud Object Recognition Services: A Comparative Analysis of AWS, Azure, Google Cloud Vision, and YOLO12
FAQ
Which API is best for image search?
There's no single best option, it depends on whether you need broad web-scale image search or matching against your own catalogue. Cloud vision APIs typically offer stronger general-purpose search features, while a custom-trained model suits narrow, domain-specific catalogues better.
What is the best free image recognition software?
Several SDKs offer free tiers suitable for prototyping, including ML Kit, which is free to use for its on-device APIs. Cloud vision providers also commonly offer a limited free tier before per-call charges begin, as seen in Google Cloud Vision's pricing structure.
What is image recognition used for?
Image recognition is used for tasks like object and scene labelling, face matching, text extraction from photos, product identification in retail apps, and quality inspection in manufacturing. The right SDK choice depends on whether the task needs to run offline, in real time, or at large scale in the cloud.
Can ChatGPT do image recognition?
Some general-purpose AI chat tools can describe and analyse images, but they are not purpose-built image recognition SDKs and lack the on-device latency, offline support and fine-grained customisation that dedicated tools like ML Kit or FastCV offer. For production apps needing reliable, testable accuracy on your own data, a dedicated SDK or vision API remains the more practical choice.
How do I test an image recognition SDK before committing to it?
Run a proof of concept using your own representative images rather than vendor demo data, measuring latency across multiple device tiers and calculating precision, recall and F1 score against your own labelled set. This approach reflects real-world performance far more reliably than published vendor benchmarks, as cross-regional testing has shown.