Gemini Nano for Android: An offline AI model

What is Gemini Nano

Gemini Nano is an efficient AI model for on device tasks according to Google. With Gemini Nano, developers will be able to deliver rich generative AI experiences without requiring a network connection or moving data off-device. Gemini Nano is ideal for use cases where low latency, low cost, and privacy safeguards are paramount.

Architecture

To handle AI services, Android OS has introduced a new module called AICore. It is available in Android 14.

AICore, a brand new system service that simplifies using AI features in your apps. AICore takes care of everything behind the scenes, from managing AI models to ensuring their safe operation on your device. AICore lets your apps run powerful AI models like Gemini Nano directly on your device. This means faster processing compared to sending data to the cloud.

Gemini Nano and AICore Architecture - Image from Google

AICore prioritizes Privacy

• Limited Connections:  AICore operates in a restricted environment, only interacting with essential system functions. This isolation helps prevent unauthorized access to your data by other other apps.

• Secure Downloads: AICore doesn't directly access the internet. Instead, it relies on a separate, privacy-focused app (Private Compute Services) to download any required models. This ensures your information stays secure during the download process.
  

Getting Started with Gemini Nano

You will need Google AI Edge SDK for Android to work with Gemini Nano. This SDK provides APIs for interacting with the model and managing its functionalities.

Currently the SDK is available only for private review and is not available for public, but soon it will be.

Compatibility Check

Not all Android devices support Gemini Nano. Hence you need to run a compatibility check in your code.

fun isDeviceSupported(): Boolean {
  val edgeManager = AiEdgeManager.getInstance(context)
  return edgeManager.isModelSupported(GeminiNano.MODEL_NAME)
}

Accessing Gemini Nano

Once you've confirmed compatibility, you can obtain an instance of the GeminiNano class using the AIEdgeManager in AI Edge SDK:

val geminiNano = AiEdgeManager.getInstance(context).getAiModel(GeminiNano.MODEL_NAME) as GeminiNano

Example: Sentiment classification

Gemini Nano claims it is good in performing inference on data. Let's run a piece of text and infer the sentiment score.

val text = "I'm happy to meet you again."
val i = TextInput.Builder().setText(text).build()
val o = geminiNano.runInference(i) as TextClassificationOutput
val sentimentClassification = o.classification

Pretty straight forward. The model classifies the sentiment and stores in a variable.

Gemini Nano UseCases

Comming back to Gemini Nano, here are some of the use cases of the model that can be put to use by developer once it is out of private beta.

• Get smarter summaries:  Gemini Nano can condense lengthy articles or reports into easy-to-understand summaries, saving you time.

• Find answers fast:  Have questions? Ask away! Gemini Nano can analyze text and provide relevant answers.

• Write with confidence:  Gemini Nano helps you polish your writing with features like grammar correction, proofreading, and writing suggestions. It can even generate smart replies based on the context.

• Analyze emotions:  Understand the sentiment behind text.  Gemini Nano can detect the overall mood or feeling expressed in writing.

• Built-in privacy:  Use powerful AI features without compromising your data.  Gemini Nano is designed with privacy in mind.

Already on Pixel:  Several Google Pixel apps, like Voice Recorder and Gboard, leverage the power of AICore to enhance your experience.