Google’s iron grip on Android: Controlling open source by any means necessary

Six years ago, in November 2007, the Android Open Source Project (AOSP) was announced. The original iPhone came out just a few months earlier, capturing people's imaginations and ushering in the modern smartphone era. While Google was an app partner for the original iPhone, it could see what a future of unchecked iPhone competition would be like. Vic Gundotra, recalling Andy Rubin's initial pitch for Android, stated:

He argued that if Google did not act, we faced a Draconian future, a future where one man, one company, one device, one carrier would be our only choice.

Google was terrified that Apple would end up ruling the mobile space. So, to help in the fight against the iPhone at a time when Google had no mobile foothold whatsoever, Android was launched as an open source project.

In that era, Google had nothing, so any adoption—any shred of market share—was welcome. Google decided to give Android away for free and use it as a trojan horse for Google services. The thinking went that if Google Search was one day locked out of the iPhone, people would stop using Google Search on the desktop. Android was the "moat" around the Google Search "castle"—it would exist to protect Google's online properties in the mobile world.

Android's rocketing market share

Credit: Smartmo / Ron Amadeo

Today, things are a little different. Android went from zero percent of the smartphone market to owning nearly 80 percent of it. Android has arguably won the smartphone wars, but "Android winning" and "Google winning" are not necessarily the same thing. Since Android is open source, it doesn't really "belong" to Google. Anyone is free to take it, clone the source, and create their own fork or alternate version.

As we've seen with the struggles of Windows Phone and Blackberry 10, app selection is everything in the mobile market, and Android's massive install base means it has a ton of apps. If a company forks Android, the OS will already be compatible with millions of apps; a company just needs to build its own app store and get everything uploaded. In theory, you'd have a non-Google OS with a ton of apps, virtually overnight. If a company other than Google can come up with a way to make Android better than it is now, it would be able to build a serious competitor and possibly threaten Google's smartphone dominance. This is the biggest danger to Google's current position: a successful, alternative Android distribution.

And a few companies are taking a swing at separating Google from Android. The most successful, high-profile alternative version of Android is Amazon's Kindle Fire. Amazon takes AOSP, skips all the usual Google add-ons, and provides its own app store, content stores, browser, cloud storage, and e-mail. The entire country of China skips the Google part of Android, too. Most Google services are banned, so the only option there is an alternate version. In both of these cases, Google's Android code is used, and it gets nothing for it.

It's easy to give something away when you're in last place with zero market share, precisely where Android started. When you're in first place though, it's a little harder to be so open and welcoming. Android has gone from being the thing that protects Google to being something worth protecting in its own right. Mobile is the future of the Internet, and controlling the world's largest mobile platform has tons of benefits. At this point, it's too difficult to stuff the open source genie back into the bottle, which begs the question: how do you control an open source project?

Google has always given itself some protection against alternative versions of Android. What many people think of as "Android" actually falls into two categories: the open parts from the Android Open Source Project (AOSP), which are the foundation of Android, and the closed source parts, which are all the Google-branded apps. While Google will never go the entire way and completely close Android, the company seems to be doing everything it can to give itself leverage over the existing open source project. And the company's main method here is to bring more and more apps under the closed source "Google" umbrella.

Closed source creep

There have always been closed source Google apps. Originally, the group consisted mostly of clients for Google's online services, like Gmail, Maps, Talk, and YouTube. When Android had no market share, Google was comfortable keeping just these apps and building the rest of Android as an open source project. Since Android has become a mobile powerhouse though, Google has decided it needs more control over the public source code.

For some of these apps, there might still be an AOSP equivalent, but when the proprietary Google version was launched, the AOSP version is usually deprecated. Less open source code means more work for Google's competitors. While you can't kill an open source app, you can turn it into abandonware by moving future development to a closed source app. When Google rebrands an app or releases a new piece of Android onto the Play Store, it's often a sign that the source has been closed and the AOSP version is dead.

Search

We'll start with the Search app, which is an excellent example of what happens when Google duplicates AOSP functionality.

In August 2010, Google launched Voice Actions. With it, the company introduced "Google Search" into the (then) Android Market. These were the days of Froyo. The above picture shows the latest version of AOSP Search and Google Search running on Android 4.3. As you can see, AOSP Search is still stuck in the days of Froyo (Android 2.2). Once Google had its closed source app up and running, it immediately abandoned the open source version. The Google version has search by voice, audio search, text-to-speech, an answer service, and it contains Google Now, the company's predictive assistant feature. The AOSP version can do Web and local searches and... that's it.

Music

Google first demoed its cloud music service at Google I/O 2010, and sure enough, that's about when the AOSP music app was frozen in time. To this day, it still looks and acts like a Froyo app.

Play Music has gained access to Google's cloud music storage, along with a huge music store and subscription option. Play Music has also gone through several user interface redesigns, gaining Equalizer and Chromecast support. The two apps are so different now, it's hard to imagine that they once were the same thing.

Calendar

Google Calendar was one of the more recent apps to get the closed source treatment. The way this process is pitched to the Android community is always rather amusing: The stock calendar is now available to everyone! We can now do updates from the Play Store! There are more features! (Oh, and by the way, it's closed source now.)

Since this was a recent split, there isn't much of a difference between the two versions. Google Calendar will sync notifications across devices, and it's gotten a cool new icon. I wouldn't expect the AOSP calendar to get these updates anytime soon.

Keyboard

The keyboard settings screens showing the missing features.

Even the keyboard is not safe from closed source creep. A few months ago, Google added Swype-like gesture typing to the stock keyboard, which was released as a new app in the Play Store called "Google Keyboard." Guess where the source code for that is? Not in AOSP. Above, you can see the settings for the two keyboards. The Google Keyboard has options for swipe typing, and AOSP doesn't—it was deprecated as soon as Google Keyboard was released.

Gallery/Camera

The Camera and Gallery are actually a single APK (Android application package file). The AOSP version is called "Gallery2.apk," and Google's version is called "GalleryGoogle.apk." As you can see in the above picture, Photospheres are exclusive to the Google version—the innovative camera mode is not available on AOSP. The open source version also omits any Google+ album integration. The normal behavior is to display cloud-based Google+ albums alongside local ones.

Here, though, we've got to give Google some credit. While the AOSP version hasn't kept up in terms of features, the new design introduced in 4.3 has made it to the Android source code.

The future

While it hasn't yet been released, the next app out the door is the stock SMS app. Although folks are clamoring for Google Hangouts to integrate text messaging and really go after iMessage, that would mean you'd be moving Android's SMS functionality to a closed source app. Once Google does make the switch, I predict that in one or two Android versions, you'll see the SMS app disappear as a default app, similar to what Google did when it killed the stock web browser in favor of Chrome (though most of Chrome is still open source).

When Hangouts does integrate SMS, the AOSP messaging app will be completely abandoned. Messaging already seems halfway down the path to retirement. (It hasn't seen a significant updating since its big redesign in Android 4.0.) So when this finally comes to pass, you'll know what the subtext will be: the open source texting app will be dead.

Left: KitKat, showing "Google Photos." Right: The current "G+ Photos" icon.

Credit: TuttoAndroid/Ron Amadeo

Also next on the chopping block is the open source Gallery. In leaked pictures of KitKat, the next version of Android, there is a new icon called "Google Photos." "Gallery," which alphabetically should be between "E-mail" and "Gmail," is suspiciously absent. While we've never seen Google Photos before, it shares the same icon as a current Google app called "G+ Photos." It looks like the AOSP Gallery is going to die and be replaced by a service with a closed source app that heavily depends on Google+. It's the ultimate expression of Google's new walled garden.

• The Google Search app has been deeply integrated into Android as the always-on, always listening Google Assistant. And on Google devices, the Google Search app even renders the home screen and app drawer.
• The Google Keyboard is now "Gboard" and more Googley than ever, with integrated Google Search, gif search, and other things.
• The open source Gallery really did die in favor of Google Photos.
• Google's camera app for the Pixels comes with proprietary camera algorithms that produce superior pictures.
• The AOSP e-mail app was retired in favor of making the Gmail app fetch mail from non-Gmail accounts.
• Since this article, Google has moved from Hangouts to Allo to Android Messages, but you won't be texting on an AOSP app anytime soon. 

Locking-in manufacturers

While Google is out to control the open source codebase as much as possible, owning the consumer-facing app side of the equation isn't the company's only power play.

If a company does ever manage to fork AOSP, clone the Google apps, and create a viable competitor to Google's Android, it's going to have a hard time getting anyone to build a device for it. In an open market, it would be as easy as calling up an Android OEM and convincing them to switch, but Google is out to make life a little more difficult than that. Google's real power in mobile comes from control of the Google apps—mainly Gmail, Maps, Google Now, Hangouts, YouTube, and the Play Store. These are Android's killer apps, and the big (and small) manufacturers want these apps on their phones. Since these apps are not open source, they need to be licensed from Google. It is at this point that you start picturing a scene out of The Godfather, because these apps aren't going to come without some requirements attached.

While it might not be an official requirement, being granted a Google apps license will go a whole lot easier if you join the Open Handset Alliance. The OHA is a group of companies committed to Android—Google's Android—and members are contractually prohibited from building non-Google approved devices. That's right, joining the OHA requires a company to sign its life away and promise to not build a device that runs a competing Android fork.

Acer was bit by this requirement when it tried to build devices that ran Alibaba's Aliyun OS in China. Aliyun is an Android fork, and when Google got wind of it, Acer was told to shut the project down or lose its access to Google apps. Google even made a public blog post about it:

While Android remains free for anyone to use as they would like, only Android compatible devices benefit from the full Android ecosystem. By joining the Open Handset Alliance, each member contributes to and builds one Android platform—not a bunch of incompatible versions.

This makes life extremely difficult for the only company brazen enough to sell an Android fork in the west: Amazon. Since the Kindle OS counts as an incompatible version of Android, no major OEM is allowed to produce the Kindle Fire for Amazon. So when Amazon goes shopping for a manufacturer for its next tablet, it has to immediately cross Acer, Asus, Dell, Foxconn, Fujitsu, HTC, Huawei, Kyocera, Lenovo, LG, Motorola, NEC, Samsung, Sharp, Sony, Toshiba, and ZTE off the list. Currently, Amazon contracts Kindle Fire manufacturing out to Quanta Computer, a company primarily known for making laptops. Amazon probably doesn't have many other choices.

For OEMs, this means they aren't allowed to slowly transition from Google's Android to a fork. The second they ship one device that runs a competing fork, they are given the kiss of death and booted out of the Android family—it must be a clean break. This, by design, makes switching to forked Android a terrifying prospect to any established Android OEM. You must jump off the Google cliff, and there's no going back.

Any OEM hoping to license Google Apps will need to pass Google's "compatibility" tests in order to be eligible. Compatibility ensures that all the apps in the Play Store will run on your device. And to Google, "compatibility" is also a fluid concept that an Android engineer once internally described as "a club to make [OEMs] do what we want." While Google now has automated tools that will test your device's "compatibility," getting a Google apps license still requires a company to privately e-mail Google and "kiss the ring" so to speak. Most of this is done through backroom agreements and secret contracts, so the majority of the information we have comes from public spats and/or lawsuits between Google and potential Android deserters (see: Acer).

Another point of control is that the Google apps are all licensed as a single bundle. So if you want Gmail and Maps, you also need to take Google Play Services, Google+, and whatever else Google feels like adding to the package. A company called Skyhook found this out the hard way when it tried to develop a competing location service for Android. Switching to Skyhook's service meant Google would not be able to collect location data from users. This was bad for Google, so Skyhook was declared "incompatible." OEMs that wanted the Google Apps were not allowed to use them. Skyhook sued, and the lawsuit is still pending.

Testing the waters with bloatware

For most OEMs, leaving the Google ecosystem and still being successful is nothing more than a pipe dream. One way for an OEM to experiment with a Google-free existence without incurring the wrath of Mountain View is to produce alternative versions of Google's apps. This is what most of us dismiss as "bloatware." Bloatware works as a software engineering "what if" thought exercise, where OEMs set out to replicate all of Google's core apps to see just how hard life outside of the walled garden would be.

Samsung dreams of a Google-free existence.

Samsung does a particularly "good" job of this, going as far as having its own user account system, backend syncing, and app store. It also maintains the most complete set of alternatives to Google apps. A lot of these, like Internet, E-mail, and Calendar, have roots in AOSP, but Samsung continued to add features long after Google abandoned them for closed alternatives.

On a phone with Google apps, it seems silly and redundant to have two calendar apps. But many OEMs view bloatware as an important strategic fallback—a "Plan B"—for if things ever get really bad. If Google does something out of line and an OEM is forced to leave, the company needs at least something to show prospective customers. OEMs include them with their shipping phones—because, hey, why not?—and gain valuable feedback. While this creates redundancy and adds to user confusion, a few users might even like the OEM's version of a core app.

With such a huge list of alternative apps, it might seem like Samsung is poised to jump ship at any moment, but replicating the Google apps is only a small portion of the massive effort it would take to break free of the Google ecosystem. The aspect of Android that an OEM really wants is the gigantic third-party app selection. Google knows this is its biggest weakness, and the company has started working to make the app ecosystem Google-dependent as well.

Locking in third-party apps

We previously explored Play Service's update implications, but it is a huge weapon in the fight against Android forks. Play Services is a closed source app owned by Google and licensed as part of the Google Apps package. Any feature you see move from "normal" Android to Google Play Services is also moving from open source to closed source. This app pulls off the neat trick of not only enticing users with exclusive, closed source features, but locking in third-party developers with Google's proprietary APIs as well.

Taking the Android app ecosystem from Google seems easy: just get your own app store up and running, convince developers to upload their apps to it, and you're on your way. But the Google APIs that ship with Play Services are out to stop this by convincing developers to weave dependence on Google into their apps. Google's strategy with Google Play Services is to turn the "Android App Ecosystem" into the "Google Play Ecosystem" by making a developer's life as easy as possible on a Google-approved device—and as difficult as possible on a non-Google-approved device.

If you use any Google APIs and try to run your app on a Kindle, or any other non-Google version of AOSP: surprise! Your app is broken. Google's Android is a very high percentage of the Android market, and developers only really care about making their app easily, making it work well, and reaching a wide audience. Google APIs accomplish all that, with the side effect that your app is now dependent on the device having a Google Apps license.

Google Maps API

The Google Maps API allows you to use Google's map data in your application. It's extremely handy for things like overlying the weather on top of a map or showing location in a travel app. The only problem is, it's part of Google services and not part of Android. Relying on the Maps API means your app will not work on a non-Google-approved device.

In response to this, Amazon was forced to license mapping data from Nokia and build a working clone of the Google Maps API. The company even has an instruction page dedicated to migrating your app from Google Maps. Again, Google is all about making life easy in its ecosystem and extremely difficult outside of it. If you want to run on the Kindle, you now need to support two different Maps APIs.

It's a terrible situation for the Android forker, in this case Amazon, who now has to deal with either paying license fees to Nokia forever or going out and mapping the entire planet on its own. Amazon is also now required to keep up with Google's breakneck pace of development: Amazon's Maps API supports Google Maps API v1, but Google is already up to v2. If you're a developer and depend on some new feature in the Maps v2 API, Amazon doesn't support it yet. Now you have even more work to do.

Google Cloud Messaging

Google Cloud Messaging (GCM) is the easiest way to do push notifications on Android, but you'll never see it on AOSP. GCM was recently added to Play Services at I/O 2013, and it now includes not only receiving notifications, but also pushing messages upstream. It's responsible for the newly added ability to sync notifications across devices. Developers often use GCM to push breaking news out to devices or to notify an app that new data is available and a sync should be performed.

While Google Maps may seem like it would be used in a small amount of apps, many more apps need push messaging in order to be any good. This is another feature that Amazon was forced to copy in order to not be left behind. Its version is called "Amazon Device Messaging," and it only works on Amazon devices. Just like the Maps API, you'll be doing extra work and testing for a very small subset of users. Every feature of GCM might not be in Amazon's version, so you'll have extra work to figure out ways around that.

Location APIs

At Google I/O 2013, Google revamped the Android location APIs and released them as part of Google Play Services. In other words, Android's top-tier location services are now closed source. If the above history is any indication, the open source location stack will be left to rot. The added features include the Fused Location Provider, a "complete rewrite" of Android's location algorithms, Geofencing (which lets you define locations on a map that will trigger events in an app when the user enters them), and Activity recognition, which uses accelerometer data and fancy algorithms to determine if the user is walking, biking, or driving—all without turning on the GPS.

It made complete sense to put the Maps API and Google Cloud Messaging into a proprietary app, as those services depend on Google servers to function. However, moving over the entire location stack feels like a massive power grab on Google's part. There are now two methods to get location: the good, low power, closed source Google way, and the crappy, battery expensive, open source way.

In-app purchasing

The best in-app purchasing on Android is done through the Google Play Store. If a developer wants their app to work on a Kindle or in China, however, they'll be stuck having to find another solution. This is another feature where, if you want to have a viable AOSP fork, you'll have to replicate it, which is just what Amazon did with the Amazon In-App Purchasing API. Samsung is even in on the party, having introduced an in-app purchasing API two years ago.

Play Games

Play Games is another proprietary API that solves a lot of difficult problems for mobile developers. It provides easy access to user accounts, leaderboards, achievements, cloud saves, anti-piracy, and (on Android) real-time multiplayer. The best part is that works on just about everything: Web apps, iOS, and Android. Well, everything except AOSP, which is not supported. This is yet another thing a third-party app could depend on and an alternate Android distribution would have to replicate.

Amazon has a set of game APIs called "GameCircle," but it's not a drop-in replacement for Play Games, the way the Amazon Maps API is. A developer will have to spend time making a completely separate multiplayer implementation work.

Supporting lock-in by supporting iOS

The borderline-evil-genius part of Google's strategy is that 90 percent of the Google APIs are also supported on iOS. Now, put yourself in the shoes of a developer deciding whether or not to use Google's APIs: many of Google's solutions offer best-in-class usability, functionality, and ease-of-implementation. Google supports both major mobile platforms, so it will cover a very high percentage of your potential user base. The only bad part is that it won't work with an Android fork, but any AOSP fork is going to be a tiny sliver of your possible target devices.

Most developers probably say "yes" to Google APIs, and the next question is what should they do about the Kindle and other Android forks? Developers are largely on their own to find a replacement API solution, which might be out of date and might not work perfectly with their existing app. If this other solution isn't a perfect drop-in replacement, the developer will have to figure out how to design their app around the missing feature. Since this is such a small amount of users compared to their current iOS + Android user base, is it even worth it to try to figure out this separate ecosystem? Will they get a return on their time investment? It would be easy to say "the hell with forked Android" and skip all the extra work and Q/A that would entail.

Samsung isn't going anywhere

This is the section that shows why Amazon can live without Google and Samsung can't. While Amazon is a Google-API-copying machine, Samsung doesn't have many answers for third-party developers that currently rely on Google. Any speculation about Samsung leaving the Google ecosystem is premature until you see it licensing map data or building a cloud messaging API.

Amazon has done a decent job of keeping up, but the company was born on the Internet. Servers and software are the company's forte, so building out a bunch of cloud services isn't a huge change. Samsung Electronics is, well, an electronics company—building a cloud infrastructure and a bunch of APIs isn't in its DNA. So while Amazon can whip this together in a few years on the back of its cloud services platform, Samsung has much more of an uphill climb ahead of it.