Text Recognition from Image using Mobile Vision API in Android

3 min readMar 22, 2020

Hello World!

I am writing my first post on Medium!:). I am developing a platform that organise and store all Invoices/Bills at location, I called it expense.AI . For one functionality I am using DL approach to extract Invoices parameter from heterogeneous structures of I/B. In this post, I am describing how to make Text Recognition apk using Mobile vision in Android.

Introduction to Mobile Vision

Mobile Vision is powerful API available in most Android devices. Some use-cases of Mobile Vision are 1. Detect faces 2. Barcode scanner 3. Text recognition. In this post we are developing android application for text recognition using Mobile Vision.

Use cases of Text Recognition

Extracting Invoices parameters from Invoice and Bills
Digitising documents
Scanning vouchers, cards and etc.

Text Recognition Mobile Vision

Mobile Vision API segments text into three parts; Blocks, Lines, and words.

Souce: https://developers.google.com/vision/android/text-overview

Text Recognition API Overview | Mobile Vision | Google Developers

Text recognition is the process of detecting text in images and video streams and recognizing the text contained…

developers.google.com

Github repo :

Tutorial

Start new project. Select empty activity and give preferred to the project.
Go to Gradle scripts. In build.gradle( Module :app we have to add dependencies for given project). Since we are using Google mobile vision api we have to add its dependancies. Add line : implementation ‘com.google.android.gms:play-services-vision:19.0.0’

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'

    implementation 'com.google.android.gms:play-services-vision:19.0.0'
}

3. Get camera permission from Manifest file

<uses-feature
    android:name="android.hardware.camera"
    android:required="true" />

4. Layout file:

<LinearLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">
<Button
    android:id="@+id/btn"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_margin="20dp"
    android:text="Start the camera" />

<ImageView
    android:id="@+id/imageView"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_margin="20dp"
    android:visibility="visible"
    tools:srcCompat="@tools:sample/avatars" />

<TextView
    android:id="@+id/text"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_alignParentBottom="true"
    android:layout_margin="20dp"
    android:textColor="#000000"
    android:textSize="20sp" />
</LinearLayout>

4. MainActivity file, recogniser

recognizer=  new TextRecognizer.Builder(getApplicationContext()).build();

if(recognizer.isOperational())
{
   // bitmap.;
    Frame frame =new Frame.Builder().setBitmap(bitmap).build();

    final SparseArray<TextBlock> items = recognizer.detect(frame);
    if (items.size() != 0)
    {
        StringBuilder stringBuilder = new StringBuilder();
        for (int i=0 ; i<items.size(); i++)
        {
            TextBlock item = items.valueAt(i);
            stringBuilder.append(item.getValue());
            stringBuilder.append("\n");
        }
        textView.setText(stringBuilder.toString());
    }
}

Github Repository

shreeshiv/Image2Text