Background restrictions in Android
Once upon a time, running in the background was easy. Now Android has a lot of restrictions: running in the background, access to the file system, permissions that need to be approved by Google Play moderators, and others. Developers have to work with all of them. Let’s remember where it all started and how the requirements changed with each new version of Android
How it started
Before Android 5.0, we had the set of tools to run tasks in the background:
- Alarm Manager. The tool that allows you to set an alarm on your system and receive notifications.
- Broadcast Intent. It sends notifications about events that are happening on the system (new messages, for example).
- Service — Background, Foreground & Bound.
- Sync Adapter. A special ancient stuff, which is connected to account management for data synchronization. A clear example: when you set up a Google account in Android, you can set up synchronization of your calendar, contacts and other services. Under the hood, it works through the Sync Adapter. The tool has not lived up to our days.
- Download Manager. A utility that allows you to download files. It is not very smart, but it is simple and convenient. Google Play, for example, is what downloads files with it.
The problems appeared because developers did not follow Spider-Man’s rule: “With great opportunities for developers comes responsibility for the speed of the system”. Google gave them huge opportunities from the first versions of Android, but did not warn them that they were working in a system that was responsible for quality.
When users broke something, they didn’t blame third-party developers, they said, “Android is lagging, but everything is fast on the iPhone.” It turned out that the developers were good and Google was bad.
Google decided to deal with this, but to do this, they had to find “enemies” affecting the quality of applications. They turned out to be:
- Battery size vs. device size. Everyone wants a thin device that weighs little and fits nicely in the hand. At the same time everyone wants a big and powerful battery. It is difficult to fight with this enemy.
- Screen size & quality. Everyone wants high brightness and color rendering, but the screen is the main place where a smartphone’s energy is spent.
- Firmware optimizations. That’s what vendors do.
- Third-party developers. I mean you and me.
- How to sell smartphone. A cool smartphone has powerful hardware, and it also eats a lot of power. You can’t sell a cool smartphone with a budget processor. You have to put one that heats up like a stove, and uses crazy amounts of energy, but it does everything fast.
- Marketing, which is trying to fool our expectations. With the new generation of smartphones, companies say, “We went to four nanometers, three nanometers, two nanometers, so we have more power. It went up, but everything we saved on power consumption, we gave to the new transistors. And in the end, we only increased the power consumption.
Android 5.0
Google got sick of it all, and they decided that they couldn’t rely on third-party developers, because they are the biggest pirates in the Android ecosystem. And then Project Volta came along.
The idea of the project was to give new tools to do work and operations in the key of energy efficiency. That is, to keep developers from doing too much unnecessary stuff so they don’t eat too much energy. So the big optimization began, which is what we’re going to talk about.
JobScheduler API. New system service, single entry point for most background operations. Also, the new service allowed you to set comfortable conditions. For example, the operation is performed when the device is connected to the charger, not used by the owner or there is access to the network with the set requirements.
The best part: JobScheduler was not used in Android 5.0, because it was terribly buggy. It only worked in Android 6.0.
Battery Saver, the second important new feature. Before that, similar tools were available to vendors as their own features. Now the tool has become a standardized part of the system. Here’s what it brought:
- Reduced CPU frequency
- Reducing display refresh rate
- Limiting data consumption in the background
Other optimizations from vendor can be applied
Android 6.0
Doze Mode. This version of Android is getting serious. And it brings an idea to turn off everything you don’t need when you’re not using the device.
How does the system know to turn Doze Mode on? First it was like this: if the user puts the phone down and doesn’t use it, you could turn off network access and other unnecessary features. For example, the mode was turned on while the smartphone owner was asleep. Google claimed that the Nexus 5 in the Doze Mode state ate only 1% of the charge overnight. Really, I’ve never seen that.
All of this has paid off. Aside from the fact that Doze Mode turns off maximum functions, there is such a thing as a maintenance window. That’s when the device comes out of Doze Mode and allows the developer to do or check something. And here again the JobScheduler starts to play an important role, because everything is controlled through it.
App Standby. There is a second power saving mode — suspending the activity of applications that a user is not using. If we minimize an app and do not use it, there are restrictions imposed on it. They are not as severe as in Doze Mode, but they interfere with going online and reduce the frequency of triggers. In what exactly are the restrictions, Google, as always, does not specify, because the vendor can do anything.
FCM High Priority is the type of push that can wake up a device in Doze Mode and App Standby. And it’s an important one, but you have to remember that if you make all the pushes high-priority, they won’t necessarily get there. Some of them will start to be ignored sooner or later. So, you should use them only for really important events. For example, when a user receives a chat message in which he has enabled notifications.
Android 7.0
Doze on the Go, the second version of Doze Mode. If previously the device came to Doze Mode only when it was still for some time, now it is enough to turn off the screen and put the smartphone in your pocket.
In Android 7.0 we got a slightly different experience with maintenance windows. The windows started to appear more often, the restrictions became softer. But applications still could not work properly in the background.
Project Svelte. The second project to reduce memory consumption and optimize the way applications work in the background.
No Broadcasts. The first thing we did in this project was to remove all the broadcasts. We began with CONNECTIVITY_ACTION. This was the first and most important one, because this broadcast is sent out for any network change. For example, when the user switches from a mobile network to Wi-Fi. Android has a special feature: even if the app is now killed, but the forecast receiver is subscribed to some forecast, the system will pick up the app to deliver it.
Imagine we have one such app. Pulling the process up is not a cheap operation, but it’s generally not terrible. But what if we have 20 such applications? Each change will cause processes to be brought up and killed 20 times. But this is not so dramatic. The worst thing is that CONNECTIVITY_ACTION is called not only when leaving the network to Wi-Fi, but also, for example, from 3G to 4G. And the type of connection is constantly jumping when walking around the city — imagine how many events are sent to the system. That’s why it was disabled first.
ACTION_NEW_PICTURE. Then we switched off action_new_picture and action_new_video. Not a critical thing, because this could be done in other ways, for example through JobScheduler. The ability to react to changes in the content provider appeared. You can pass some Uri with ContentProvider and, when a change notification happens, call the job.
Android 8. 0
NoBroadcasts+, a ban on all system broadcasts. Either the developer doesn’t need it, or he can subscribe to them when the app is running. This cannot be done implicitly, only through JobScheduler. There are exceptions left, but very few:
- ACTION_BOOT_COMPLETED;
- ACTION_LOCAL_CHANGED;
- ACTION_PACKAGE_DATA_CLEARED
- ACTION_PACKAGE_ FULLY_REMOVED;
- ACTION_NEW_OUTGOING_CALL;
- ACTION_MEDIA_***;
- SMS_RECEIVED_ACTION and WAP_PUSH_RECIEVED_ACTION.
This list is up to date for summer 2022.
No Background Service. Not allowed to run a Service in the background. If a developer starts this Service and quit the application, it will not live very long and will be forcibly killed. If you try to start a normal Service when applications are in the background, it will also crash.
JobScheduler update. First, there is not enough free memory (that’s what the condition sounds like). Google doesn’t say how much memory is enough. Not low battery — what exactly is meant by this, again not explained. Apparently, it’s about the critical percentages when the battery turns red.
Metered network. A cool new thing, linked to the cost. When we connect to the network in settings, the cellular network comes by default as chargeable, which means we pay for traffic. Wi-Fi by default is considered non-tariffable, but you can change that in the settings.
Foreground Service. Background services now need to be explicitly invoked. Previously, you just had to call such a service, and then say that it was Foreground. Now, because the background services are killed, you have to say explicitly: I’m going to run the Foreground Service. And the code becomes something like this.
// Old way
context.startService(Intent(context, MediaService::class))
// New way in Android 8.0
context.startForegroundService(Intent(context, MediaService::class))
class MediaService : Service() {
override fun onCreate() {
super.onCreate()
// Must be called up to 5 seconds after startForegroundService
startForeground(NOTIFICATION_ID, newOngoingNotification())
}
fun newOngoingNotification() : Notification
companion object {
const val NOTIFICATION_ID
}
}An important feature: no more than five seconds should pass between the call to startForegroundService and startForeground. Otherwise the application will crash because we haven’t fulfilled the contract.
One important thing to remember is that in any application, when a service, activation or anything else is started, the application starts loading first. How many different nits do you make in the application? We usually put all the libraries in there. And the point of it is to minimize the volume.
Roughly speaking, if you have an initialization needed for the UI, you don’t need to do it in the application because it’s optional and could run some service. You either have to take everything into the background or do a lazy thing, which is a new approach.
It may seem like five seconds is a lot, but trust me, for budget Android 8.0 devices it’s an eternity. The code might just not execute, and there will be a crash. Especially if there is some network operation in the app. Therefore, you should not forget to optimize it.
Location access restriction. Another limitation related to security and battery saving. If the application is in the background, it will receive updates no more than a few times per hour. We don’t know exactly how many times.
WakeLock removal by the system. When the app goes into cached state (no active Android components), all the WakeLocks captured by the system will be released. WakeLock is a special-API that allows the system to block unnecessary things. For example, if the system sees that nothing is being used right now and the device goes to sleep, you can turn off the CPU. It is like WakeLock is telling the system: do not turn off the processor, I am going to use it. Consequently, there is a load on the hardware. And if the application goes into cached state, it means that there are no active Service and Activity in it, no one is calling its content provider or broadcast receivers. All WakeLock for it will be automatically removed.
Android 9.0
App StandBy Buckets. The so-called buckets are a new approach to what apps should be stopped and how. If an app used to go into the background, the system assumed that it had to be stopped. Now the system analyzes how applications are used and categorizes them:
- Active — the application is being worked with by a user or another application in Active state;
- Working Set — the application is used frequently, but the user is not currently working with it;
- Frequent — the app is used regularly, but not necessarily every day;
- Rare — the application is rarely used;
- Never — The application has been installed, but the user has never run it. Disable all
Improved energy saving mode. It has been made stronger by introducing more restrictions:
- the system more aggressively puts apps into App Standby mode;
- restrictions are applied to all apps, regardless of targetSdk;
- access to the app may be gone when the screen is turned off;
- background apps have no access to the network;
- additional items — vendors can add or change something.
Permission to run Foreground Service. It is not critical because it falls into the normal category. It only needs to be declared in the manifest and there is no need to ask for it in the runtime. In fact, it helps the system know that you have requested a permissions, so you can be tracked. If you try to run Foreground Service without permission, the system crashes everything and says you have SecurityException.
Restrictions on sensor access. The applications in the background cannot access the microphone and the camera, the sensors either don’t send any data at all or do it at a slow frequency. We don’t know how slow it is.
Update JobScheduler. Now provides information about the size of the downloaded file. Based on this info the system might understand: if you load a small file, you can be skipped quickly. And if the file is big, it is better to wait for a non-targeted network and high charge level.
There is also an important thing for pre-loading content. If the app needs to pre-load data to display, for example, the home screen, you can mark a job with this new flag and JobScheduler will raise its priority.
Android 10
Foreground Service. A new feature that allows the system to understand why a Foreground Service is needed and to prioritize it. There are eight types in total, there shouldn’t be any others:
- Camera — uses the camera, takes photos and records video;
- ConnectedDevice — works with bluetooth devices, cars and other things;
- DataSync — transmits data over the network, backups/restores, downloads files;
- Location — GPS, map or navigation;
- MediaPlayback — playing audio and video;
- MediaProjection — Controls media projection, such as recording video from the screen, taking screenshots, and so on;
- Microphone — uses a microphone or records audio;
- PhoneCall — operations related to phone and video calls and similar communications.
Importantly, a single Foreground Service can have several declared types. They can now be specified clearly at startup. For example, one of the types declared in the manifest will not work at all in operations of that type or will run all at once.
class MediaService : Service() {
override fun onCreate() {
super.onCreate()
startForeground(NOTIF_ID, newNotification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK)
startForeground(NOTIF_ID, newNotification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_MANIFEST)
startForeground(NOTIF_ID, newNotification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_NONE)
}
}Prohibit starting an Activity from the background. Previously, if a push came, it was possible to start an Activity from the Broadcast Receiver. Now you can’t do that: any display of UI must be initiated by the user. For example, a notification can be shown. If the user clicks on it, we get permission to run the Activity. If it doesn’t, there will be a crash.
Accessing the location from the background. There is a third permission for locality, and now we have to notify that we are going to work in the background as well. In addition, we need to get a separate permission to work in the background from the user.
Android 11
The most interesting and most frustrating thing for me is that Android 11 has not restricted anything. So we move on to…
Android 12
Restricted App Standby Bucket. The toughest bucket: if an app consumes a lot of system resources or works in an undesirable way, the system will put it there. The wording is unclear, but most likely it’s about malware.
Restrictions on running the Foreground Service. Applications in the background can’t run the Foreground Service unless there are a few exceptions. The list of exceptions is very short.
Permission to Exact Alarm. Previously, you could do this as you like, but now there is a permission. And it’s not some runtime permission, but an interesting thing like Picture-in-Picture. So we have to tell the user: go to the system settings and allow my application to do this. On the one hand, it’s inconvenient, but on the other hand, it’s useful. Many developers had to deal with the fact that Exact Alarms worked unpredictably or didn’t work at all. And a permutation is the standardization and explicit signal of the systems. The main thing is to remember that an Exact Alarm will not necessarily be triggered at the right time. It may happen a few minutes or more later.
Expedited Job. A new Job type for important tasks that need to be completed immediately. You can’t abuse it. Here are the signs of such a Job:
- Completed as soon as possible;
- Has fewer restrictions in Doze and Battery Saving modes;
- Not affected by network access restrictions imposed by Doze, App Standby, and Battery Saving mode;
- Less likely to be killed than the regular Job;
- Location access restrictions in the background remain in place;
- Only requires network availability, amount of free space, and saving Job between device reboots to run.
This type has features that are useful to be aware of:
- The system allocates a quota for the number of simultaneously running Expedited Job for the application;
- Quotas do not apply to the active application;
- May not start immediately if the device is too busy or if the Job specified for execution cannot be performed;
- Has a limited execution time of at least one minute;
- If the application has an available quota left, the Expedited Job may be able to run longer.
Android 13
Foreground Service Task Manager. The user can now see in a special window in the notification bar which applications are currently doing something in Foreground. That is, they are minimized, but are trying to do something. The user can manually stop them.
A special button stops the application forcefully. Not like force stop in the settings, but it still stops it completely. Even if it is just minimized.
Battery usage setting for each application. By default, all applications are optimized, i.e. a balance is maintained between runtime in the background and battery consumption. The user can choose one of two settings:
- Unrestricted — “do whatever you want, I don’t care”.
- Restricted — when the application can do almost nothing in the background.
In Restricted mode, the following restrictions will be applied:
- Foreground Service cannot be run;
- Foreground Services that are running will be removed from the Foreground;
- no Alarms will be triggered;
- Job will not be started;
ACTION_BOOT_COMPLETED and ACTION_LOCKED_BOOT_COMPLETED broadcasts will not be delivered.
Notification of Foreground Service being too long. The system monitors if the app has been running too long in this mode and recommends that you stop it. For some reason, Google thinks that long is 20 hours in a window of 24 hours. That’s weird; I have a hard time imagining an app running like that. Features of working in this mode:
- Notification can be shown no more than once every 30 days;
- The alert will not show if the notification associated with a Foreground Service is visible;
- Exclusions are services with MediaPlayback and Location types;
- System applications, target-specific applications, and devices in demo mode are also exceptions.
JobScheduler update. There is now support for uploading files in addition to downloading. If the server you’re downloading files from supports uploading, you can specify this data in Job. And if JobScheduler decides that Job needs to be stopped, then it will start it, and you will know from which part to continue execution.
Other than that, we’ve added Job priority. This is a special constant that helps to sort all the Jobs within the same application and prioritize their execution.
For the last eight years Google has been trying to limit developers and force them to use special APIs, because the freedom to choose tools has led to chaos on devices.
The second line of protection is Google Play, which does not allow apps to be published with individual permissions without a special check from the store. It’s only going to get tougher from here on, with all of the new default APIs focused on restrictions, battery saving and clear control of the app by the system.
This is logical, because at the beginning Google had inherited the source code. It needed to attract developers and vendors to populate Android. By 2014, the company had solved this problem and changed its vector to ensure reliability, security and a pleasant user experience with Android devices.
