With the immense potential of artificial intelligence and its applications in our daily lives, it is no surprise that smartphone manufacturers are doubling down on machine learning and the field of AI. Developing digital assistants that get smarter the more you use it is not a distant dream, but a reality of today.

Google has been leading the field of AI and using machine learning models in truly interesting ways, but other companies are working on AI solutions of their own. Earlier this year, Samsung introduced Bixby, the company’s own digital assistant, alongside its flagship Galaxy S8 smartphone. Samsung aims to differentiate by making Bixby capable not just of handling basic voice commands, but lets you customise how it can help get things done faster.

What is Bixby?

But first, let’s start with the basics. Bixby is a virtual assistant developed by Samsung that’s designed to make device interaction easier. It is specifically designed to reduce the complexity of increasingly fully-featured devices. Bixby encompasses a range of different features, which includes Bixby Voice, Bixby Vision, Bixby Home, and Bixby Reminder.

Get things done easier

Like any other digital assistant, Bixby aims to let users get things done more efficiently. It naturally supports voice commands, and can be activated in two ways: pressing the dedicated Bixby button on the left side of the phone (the dedicated button is only available on Galaxy S8, S8+, and Galaxy Note8) underneath the volume control buttons, or by simply saying, “Hi Bixby”.

From there, users can use Bixby to perform simple tasks, such as checking the weather forecast for today, or doing simple web searches. It can also do more complex tasks with some system settings or default apps on your phone. Besides the usual tasks of setting up an alarm or set a reminder for something, Bixby also understands trickier requests.

Bixby’s strongest asset is its ability to delve deep into the phone settings, without you going through several layers of menus to reach them. You can get Bixby to do simple toggles such as “turn on performance mode/power saving/blue light filter, do not disturb”, which may sound simple, but is something that is unique to Bixby at the moment. You can also ask “What’s the Android version on this phone” and you’ll be directed to the About Phone screen.

On top of that, you can go be a little more adventurous – though you will have to be more specific. You can say things like “Open Settings and increase font size”, “Take a screenshot and share it”, or even “How much data have I used this month?”, and Bixby delivers. As long as the instructions are clear, Bixby will usually understand and execute the right actions; otherwise, it may request you to repeat your sentence.

As mentioned earlier, Bixby is great at executing requests that requires the default apps on your phone. It understands general commands like “close recent apps”, “take a screenshot”, and even “open camera and take a selfie”. And, as expected, you can also use Bixby to call anyone on your phonebook – though Bixby may have issues with our local names. That said, one useful feature I recently used was “Block the last number” after a call from a telemarketer.

And, odd as it may seem, Bixby also allows you to navigate your phone using voice commands instead of swiping and tapping the screen. You can “swipe left/right”, or when reading an article on a website, “scroll up/down”. Bixby also recognises “go back to Home Screen”. This certainly sounds unnecessary under normal circumstances, but may be useful for situations when you can’t navigate the phone with your hands (driving, carrying groceries, etc). After all, Bixby is designed to mimic a user’s swipes and taps when navigating the phone.

In all, there are thousands of actions you can get Bixby to do – and software updates may add thousands more. It is an exhaustive list, from the basic to oddly specific, and this video from Samsung gives a taste of what you can do with Bixby:

https://www.youtube.com/watch?v=34-LYLuq-aI

Set up a string of actions with Quick Commands

Essentially, Bixby is designed to perform tasks that would require a number of taps to complete. Voice commands from the section above are fairly straightforward, so Bixby has another trick up its sleeve: Quick Commands.

After using Bixby to perform an action, you may get an option to “Add quick command”. Or, you can set up Quick Commands by accessing Bixby Home > My Bixby > Quick Commands, and tap on “Add”. Here, you’ll see a list of tasks that you’ve previously asked Bixby to do; tapping on any of them allows you to say a word or phrase to do the same action again (without the long command). So, something like “Call (name)” can be a Quick Command for “Open Phone and call (name)”.

You can be a lot more creative, of course. Bixby allows you to create your own quick command using a single word or phrase to perform many tasks in sequence. For example, you can set “Good night” as a quick command to perform the following actions: turn the brightness level to minimum, turn on Do Not Disturb, turn on Always On Display, turn on the 6am alarm.

With thousands of commands at your disposal, you can set up some very specific tasks that may take several taps to accomplish – and the end result is actually really satisfying. There’s even a thread of Bixby users sharing their Quick Commands to help you get along the way.

Quick Commands makes certain things you do every day a lot faster to complete. It is similar in nature to the IFTTT app, which automates a string of action when a trigger factor is met. The only difference here is that you use your voice to trigger the actions.

Use Bixby on popular third party apps

Bixby is great for navigating around system settings and default apps, but Samsung is also working to integrate third party app support. It’s understandable, simply because Bixby would be quite restricted in its abilities.

Bixby Labs is an opt-in addition to Bixby, where the digital assistant accepts tasks associated with third party apps. The library of supported apps is small, but includes popular apps such as Facebook, WhatsApp, Instagram, Facebook, Facebook Messenger, Twitter, and also the Google Play Store. Bixby Labs is “experimental”, so to say, meaning some actions may not be completed as intended, or may not be understood at all.

However, at this stage Bixby understands basic tasks on these apps, so you can do a number of things. You can say “Open Facebook and show me my notifications”, or “Open WhatsApp and show me the messages from (name)”, and our personal favourite: “Open the Play Store and download (app)” simply because of the automated actions you see below.

Adapts to you

As you use Bixby, the AI assistant continuously learns and adapts to your speech. For example, you may begin using Bixby by saying “Open Phone and call (name)” or “Open Messages and send a text to (name), but as you use it, it will understand the more natural “Call (name)” and “Message (name) commands.

Because Bixby uses machine learning models, it gets better the more you use it. To encourage users to keep using Bixby, you get awarded with XP points which increases your level based on what you do with Bixby. You get XP for performing tasks with Bixby Voice, creating Quick Commands, and more XP for “teaching” Bixby if it performs a task incorrectly. As you level up, you may unlock rewards (need clarification if this applies in Malaysia).

Finally, there’s Bixby Home. This is a dedicated panel that sits on the left of the Home screen, and dynamically adapts its content to show you the most relevant content. Depending on your usage, Bixby Home may show you your calendar appointments, reminders, alarms, the weather, news articles that you may find interesting, and also updates from your social apps.

A smarter way to use your phone

There may already be several digital assistants around, but Bixby is different and unique in its ability to perform complex and multi-step tasks – something the other digital assistants are not capable of. The deep integration with core apps – as well as improving ability to work on third party apps – allows Bixby to be genuinely useful beyond something to impress your friends with.

Plus, you can also interact with Bixby by typing your commands instead of using your voice. And, with Quick Commands, you can set a string of tasks that suit your needs with just a few words.

With Bixby, Samsung is aiming to create a new, smarter way to use your phone. It is still a new product with some hiccups, but we’ve been genuinely surprised at some of the complex tasks it has managed to complete successfully. And, because it gets better the more you use it, Bixby definitely has the potential to be a core feature on your smartphone in the future.

Disclosure: This article was sponsored by Samsung Malaysia.