Android TV Voice Search

“Every good conversation starts with good listening”. So wouldn’t be amazing if you could make your smart tv listen 👂 and give you intents accordingly 😉

Voice Search

This is what future looks like !! Today we all are busy n too lazy to type and search things on gadgets. Just imagine if you have to type “X-MEN Days of Future Past”, that too on a keyboard different from mobile, something like this:

Early Text Search UI

This is early search UI where you can just enter text. Can’t imagine how much time you have to spend for searching a required title 🤯🤯. So today voice search is an essential keep in all Smart TV’s and it’s getting smarter and smarter day by day.

Now let’s step towards creating a smarter better search. #VoiceSearch

In this conversation we will focus on how to create a custom voice search. Leaving the text keyboard creation for other time 😃

Here are some pre-requisites for creating voice search UI:

• Custom Voice Search UI with clearly defined user states.
• Handling MIC input from Remote.
• Reliable, working across all Android TV Versions and no OS restrictions.

So, let’s begin building our custom voice search:

• Firstly, add permission and feature requirement in AndroidManifest.xml

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-feature      
    android:name="android.hardware.microphone" 
    android:required="false" />

• Now create a SearchActivity. When user uses recording/mic feature for first time, check for dynamic audio permission. Request for permission, if not granted.

private fun checkRunTimePermission() {
    Log.e(TAG, "==== checkRuntime Permission")
    val permissionArrays = arrayOf(Manifest.permission.RECORD_AUDIO)
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
        requestPermissions(permissionArrays, 11111)
    } else {
        Log.e(TAG, "==== checkRuntime Permission else")
        mFragment.setRecognitionListener()
    }
}

Voice Permission Screen

So, now activity is set with all the required checks.

• Create Search Fragment, which acts like a wrapper on the CustomSearchSupportFragment(base class). Do set up recognition listener before next step.

class SearchFragment : CustomSearchSupportFragment(),    SearchResultProvider {
fun setRecognitionListener() {
    Log.d(TAG, "==== setRecognitionListener in Fragment")
    if (DEBUG) {
        Log.d(
                TAG, "User is initiating a search. Do we have RECORD_AUDIO permission? " +
                hasPermission(Manifest.permission.RECORD_AUDIO)
        )
    }
    if (!hasPermission(Manifest.permission.RECORD_AUDIO)) {
        if (DEBUG) {
            Log.d(TAG, "Does not have RECORD_AUDIO, using SpeechRecognitionCallback")
        }
        Log.d(
                TAG, "==== Don't have permission setting recognition call back ," +
                " open intent"
        )
        setSpeechRecognitionCallback(SpeechRecognitionCallback {
            try {
                this@SearchFragment.startActivityForResult(recognizerIntent, REQUEST_SPEECH)
            } catch (e: ActivityNotFoundException) {
                Log.e(TAG, "Cannot find activity for speech recognizer", e)
            }
        })
    } else {
        setSpeechRecognizer()
        Log.d(TAG, "We DO have RECORD_AUDIO")
    }
}
}

In order to make robust Voice Search handle all cases such as permission denial where voice search happens through internal speech recognizer.

Create CustomSearchFragment, base class with responsibilities such as:

• Creation of recognizerIntent()

fun getRecognizerIntent(): Intent {
    val recognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
    recognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
            RecognizerIntent.LANGUAGE_MODEL_FREE_FORM)
    recognizerIntent.putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true)
    if (mSearchBar != null && mSearchBar.getHint() != null) {
        recognizerIntent.putExtra(RecognizerIntent.EXTRA_PROMPT, mSearchBar.getHint())
    }
    return recognizerIntent
}

• Setting up the searchProvider()

fun setSearchResultProvider(searchResultProvider: SearchResultProvider) {
    if (mProvider !== searchResultProvider) {
        mProvider = searchResultProvider
        onSetSearchResultProvider()
    }
}

• In onSetSearchResultProvider(), start speech recognition.

private val mStartRecognitionRunnable = Runnable {
  mAutoStartRecognition = false
  mSearchBar.startRecognition()
}

Here mSearch bar is a CustomSearchBar, with all defined states such as listening and non listening. Refer Github for more details.

Rendered UI :

Listening State !!

When user says any particular keyword for example “Urvashi” in this case, on successful recognition output will be :

Speech to text successfully recognised

• Add stop implementation on back press or any close gestures.
• CustomSearchSupportFragment delegates results of voice search. Feel free to customise them. Refer Github for more details.

Do let me know, if this blogs helps you out to create custom voice search UI.

To know more about building apps for Android TV do check this out.

Happy Coding !!