md_chunk_alignment = 1
    data_alignment_detection = 1
    data_alignment = 0
    data_alignment_offset_detection = 1

I assume the reader starts with a stock unmodified factory image, and knows how to use fastboot.

Step 1: Getting root access

The first step is getting root access, to accomplish this the easiest way is to temporarily unlock the bootloader (don’t worry, we will re-lock it later).

Open a shell prompt and type fastboot oem unlock, all data on the phone will be lost as after oem unlock the bootloader performs a factory data reset (also called hard-reset or reset to factory default).

Once the bootloader is unlocked, we can flash “unsigned” data through fastboot, this allows us to flash a customized recovery image, which will allow to flash an usable ‘su’ binary with the proper suid permissions and a superuser app into the system.

There are a few custom recovery images out there to choose from, the most popular being ClockworkMod recovery and TeamWin Recovery Project (TWRP). I recommend the later, because it supports decrypting an encrypted data partition on Galaxy Nexus since version 2.2.0.

To successfully root the device, first flash the custom recovery image using fastboot: fastboot flash recovery recovery-maguro.img, then reboot into recovery without booting the system (this is important, as during boot the recovery image checksum is verified and if it doesn’t match the stock recovery the system will overwrite the custom recovery with the stock one).

Once we’re into the custom recovery, we need to flash the ‘su’ binary and the superuser apk, again there are multiple possibilities out there, the most popular being ChainsDD Superuser (flashable zip) and Chainfire SuperSU (flashable zip).

Finally, when the phone is booted again the stock recovery will be automatically flashed and you will have a rooted phone with unlocked bootloader and stock recovery.

Step 2: Relocking the bootloader

Re-locking the bootloader is important, because with an unlocked bootloader any unauthorized user could access your private data by flashing a custom recovery and backing up or mounting your storage and data partitions. This is why the ‘oem unlock‘ process wipes your data, and this is why you should keep the bootloader locked.

The Galaxy Nexus bootloader stores the lock status at position 0x000007C (124 decimal) of the param partition of the device’s internal storage. When the byte is set to ’0′, bootloader is unlocked, when it is set to ’1′ bootloader is locked. As you can guess now, if you have root access you can manually change the bootloader status from the system, thus it is possible to unlock and relock it without using fastboot, and without wiping your data. You could do this process manually using ‘dd‘, but of course the good folks at XDA have created an open source app to automate this process for you. Install BootUnlocker for Galaxy Nexus from the play store, give it root permissions and re-lock your bootloader now. If you want to have a look at the application source code, check the google code project page.

You now have a rooted phone with stock recovery and locked bootloader.

Step 3: Encrypting the phone

Encryption on Android uses the dm-crypt layer in the Linux kernel, to enable encryption go to Settings, Security, Encryption and click on “Encrypt phone”, for the encryption process to start battery should be fully charged and the phone AC adapter must be plugged in. For more details on how encryption works, read Notes on the implementation of encryption in Android.

Once the phone is encrypted, you need to type a numeric PIN or password to decrypt it each time you power it on. The master key to decrypt the filesystem is encrypted with a hash of the user’s lock screen password (that’s why you can only use pin or password in the lockscreen when encryption is enabled).

Currently, there is only one password for both the encryption and lock screen. This is especially bad because you cannot turn off screen lock and therefore have to type it rather frequently, which makes it easier to get a glance on the screen while typing it. Until Google provides a way to use different passwords for encryption and screen lock, you can manually change password for encryption by issuing the following shell command as root:

vdc cryptfs changepw <new_password>

This command changes only encryption password requested at phone boot. The lock screen PIN or password remains unchanged. Please see (and star) Android issue 29468 for Google to implement this in the UI.

Again, there’s an app for that too! check out Cryptfs Password (github source) if you want to change the password without messing with the command line.

Make sure to choose a good password not based on a dictionary word, as the encryption can be cracked using brute force (see Into the Droid – Gaining Access to Android User Data presentation from Thomas Cannon at Defcon 2012, slide 26 and slide 27).

You have now an encrypted rooted phone with stock recovery and locked bootloader.

Step 4: Keeping root access after OTA updates

When Over The Air updates are applied, the function set_perm_recursive is called from recovery, this removes the setuid bits on the ‘su’ binary and disables your root access after the OTA has been applied.

To prevent this form happening, one could change the file attributes for the ‘su’ binary to immutable using chattr +i /system/xbin/su, again there’s an app that will do this for you automatically (and also keep a backup of your su binary, allowing to “temporarily unroot” your phone): OTA RootKeeper, the source code is available on github.

The same functionality from OTA RootKeper has been recently included in recent versions of ChainsDD Superuser app.

You have now an encrypted rooted phone with stock recovery, locked bootloader and capable of keeping root access through OTAs.

Some notes concerning OTA updates and backups

With this setup, you have not modified any system files so your phone should be able to automatically get OTA updates and apply them cleanly. If you are impatient and want to manually flash OTAs as soon as they are available you should keep in mind your setup is a bit different and instructions posted everywere won’t work exactly for your phone.

To manually apply an OTA update, you should flash a custom recovery first, you can do it either from system using ‘dd‘ (you are root now), or through fastboot (but remember to oem-unlock your bootloader first using the BootUnlocker app, and re-lock when done).

To flash the custom recovery from system, you can use the following command as root:

$ su
# dd if=/sdcard/twrp-2.2.0-maguro.img of=/dev/block/platform/omap/omap_hsmmc.0/by-name/recovery

You can then reboot into the custom recovery to apply the OTA from sdcard, or to backup your system, remember to use a custom recovery with support for encrypted partitions like TWRP. Again, remember that when you boot your phone again, the custom recovery will be overwritten with the stock recovery on boot.

Update: An avid reader (thanks Marc!) noticed that factory images don’t include the files /system/etc/install-recovery.sh and /system/recovery-from-boot.p which are responsible of re-installing the factory image on boot when the installed recovery checksum doesn’t match the stock recovery, these files are only present if you have updated through an OTA. This means that if you come from a factory image, you’ll need to reflash the stock recovery manually at the end of the process. To do this first extract the stock recovery from a factory image:

$ tar zxvf takju-factory.tgz
$ cd takju-*/
$ unzip image-takju-*.zip
[...]
  inflating: recovery.img

Then reboot into fastboot mode and flash it using fastboot: fastboot flash recovery recovery.img

sudo apt-get install vbetool

$ sudo add-apt-repository ppa:poliva/opengalax
$ sudo apt-get update
$ sudo apt-get install opengalax xinput-calibrator

$ sudo add-apt-repository ppa:poliva/pof
$ sudo apt-get update
$ sudo apt-get install indicator-sysbat
$ echo "coretemp" |sudo tee -a /etc/modules

TL;DR – the broadcom sta linux driver always fails in the first scan request after the interface is brought up, this produces a long delay when connecting to a wireless network. There’s an open source driver which does not have this problem, but is not good with power management. In this post I describe the steps I took to pinpoint the problem in the proprietary driver and to fix it.

The story begins when I updated Ubuntu from 11.10 to 12.04 on my MacBook Air, everything worked fine after upgrading except one thing that bothered me a lot: when resuming the laptop after suspending it, it took around 30 seconds to connect to my wireless network. It wouldn’t have bothered me if it had been the same in 11.10, but in 11.10 the time to connect was barely 5 or 6 seconds, so having to wait 30 seconds was totally unacceptable.

Initially I thought it was a bug in NetworkManager, and increased the debug level in the config file to finally come out to the conclusion that I was using a different driver in 12.04 than in 11.10.

There are two drivers available for the Broadcom BCM4353 802.11 Wireless Controller:

• Broadcom brcmsmac (mac80211-based softmac PCIe): the completely open source drivers, included in the kernel
• Broadcom 802.11 Linux STA driver: the broadcom mixed GPL source + a proprietary hybrid binary file agnostic to the specific version of the Linux kernel

Both wl (broadcom proprietary driver) and brcmsmac (the open source driver) were installed in my Ubuntu 11.10 but the open source driver was used by default, and this driver connected to the wifi network in 5 seconds.

In Ubuntu 12.04, the wl proprietary driver provided by the package bcmwl-kernel-source has been updated from version 5.100.82.38+bdcom-0ubuntu4 to version 5.100.82.38+bdcom-0ubuntu6.1 which includes the following fix:

---------------
bcmwl (5.100.82.38+bdcom-0ubuntu6.1) precise-proposed; urgency=low
	
  * debian/bcmwl-kernel-source.postinst:
    - Blacklist brcmfmac, brcmsmac and bcma so that they don't
      conflict with the closed driver (LP: #873117)
 -- Alberto Milone  Mon, 23 Apr 2012 16:11:56 +0200

Which basically blacklists the open source brcmsmac module, forcing the wl proprietary driver to be in use. When the brcmsmac was not blacklisted, even if the wl driver was loaded it failed silently and brcmsmac was used instead.

So, the easy path to solve my problem would have been to blacklist the wl module, and add the brcmsmac to /etc/modules and live happy with my 5 seconds needed to associate when resuming, *BUT* I compared both drivers and the proprietary driver has better signal and way better power management, which makes my battery last longer, so I decided to go the long route. My goal was to achieve the lowest delay possible to connect to a wireless network when coming from a suspend using the proprietary driver.

I went “down” one level and started looking at wpasupplicant, as NetworkManager communicates with it using the DBus control interface (dbus-monitor showed the problem was not in dbus communication) so, increased the debug level in wpa-supplicant by adding ‘-dd‘ switch in /usr/share/dbus-1/system-services/fi.w1.wpa_supplicant1.service, and looked through the logs, which quickly revealed the following:

May 20 11:49:46 maco wpa_supplicant[12610]: Scan requested (ret=0) - scan timeout 5 seconds
May 20 11:49:52 maco wpa_supplicant[12610]: Scan timeout - try to get results
May 20 11:49:52 maco wpa_supplicant[12610]: Failed to get scan results
May 20 11:49:52 maco wpa_supplicant[12610]: Failed to get scan results - try scanning again
May 20 11:50:07 maco wpa_supplicant[12610]: Scan requested (ret=0) - scan timeout 5 seconds

The first scan request (SIOCSIWSCAN), after the wireless interface was brought up always failed (?), and wpa_supplicant tried to get the scan results (SIOCGIWSCAN) because some drivers do not deliver SIOCGIWSCAN events to notify when scan is complete, but this failed too, so wpasupplicant requested a second scan after a timeout, which properly delivered the results this time. This first failing scan was adding 21 seconds of delay to the network association process.

I googled the error and found I was not the only soul affected by this problem, Kalle Valo submitted 4 different patches to the hostap mailing list between October 2010 and March 2011, but the patches were never accepted upstream, nor included in the ubuntu package. The wpasupplicant code has changed a bit since Kalle submitted his patches, so I adapted them to the current wpa_supplicant version in Ubuntu. If you are curious, you can dig through ubuntu bug #994739.

In short, the version 4 patch from Kalle basically patches the WEXT driver from wpasupplicant to check the return value when trying to get scan results (SIOCGIWSCAN) from the wl driver, if the number of last error (errno) is EINVAL on the first scan, it requests another scan, so this one will go through (as only the first one fails) and return the scan results next time wpasupplicant tries to get them. This is far from perfect, but it works (and doesn’t seem to break anything), reducing the time needed to associate to the wireless network after the interface has been brought up from 30 seconds to 12 seconds.

But I was still unhappy with this result, so I patched the wpasupplicant code to request a scan right after the driver init function, so this would be the first “failing” scan, and the real scan requested a bit later would return results. This was a very ugly patch, because it made wpasupplicant request a scan in INACTIVE state (when it should be SCANNING), but it worked and reduced the time from 30 seconds to 10 seconds.

So, still unhappy with the results, I decided to go down one more level and have a look at the GPL’d source of the Broadcom’s Linux STA proprietary driver, and BINGO! this is how the wl_iw_set_scan() function ends:

        (void) dev_wlc_ioctl(dev, WLC_SCAN, &ssid, sizeof(ssid));
	
        return 0;

They always return 0, even when the dev_wlc_ioctl() function fails!! and WTH is it casted to void?? It would have been easier to just return the result of this function!. Patching this shows that the first scan after the interface is up fails with errno EBUSY (device or resource busy), so I added a workaround here to make it request the scan to the underlying hardware until it returned something different than EBUSY and could be correctly handled by wpasupplicant, et voilà, time reduced to 10 seconds.

But hey, now that I looked at their source, it turns out that there’s a newer version available in broadcom’s website: 5.100.82.112. This version now supports the new linux cfg80211 wireless configuration API in addition to the older Wireless Extensions (WEXT), you can choose between CFG80211 or WEXT at compile time, the ubuntu package broadcom-sta-dkms in the development release for 12.10 ‘Quantal Quetzal’ has been updated to this version but still uses the old WEXT which is still broken (always returns 0, remember above?). But, guess what they have done it correctly this time in the new CFG80211 code, see the end of the function __wl_cfg80211_scan():

        err = wl_dev_ioctl(ndev, WLC_SCAN, &sr->ssid, sizeof(sr->ssid));
        if (err) {
                if (err == -EBUSY) {
                        WL_INF((\"system busy : scan for \\"%s\\" \"
                                \"canceled\n\", sr->ssid.SSID));
                } else {
                        WL_ERR((\"WLC_SCAN error (%d)\n\", err));
                }
                goto scan_out;
        }
	
        return 0;
	
scan_out:
        clear_bit(WL_STATUS_SCANNING, &wl->status);
        wl->scan_request = NULL;
        return err;

As you can see, they now return EBUSY when the driver cannot perform the scan, and wpa_supplicant can manage this situation correctly, so I quickly backported the broadcom-sta-dkms package from ubuntu 12.10 to ubuntu 12.04 and added a patch to compile it with CFG80211 enabled, and finally I CAN HAS ONLY 8 SECONDS DELAY!!!!1 to associate to the wifi network after my laptop resumes from suspend using the wl driver, and I’m a happy camper!

$ sudo apt-get install bzr ubuntu-dev-tools
$ bzr whoami "Your Name <you@example.org>"
$ bzr launchpad-login poliva

$ bzr branch lp:ubuntu/precise/wpasupplicant
$ cd wpasupplicant
$ patch -p1 < /path/to/your.patch

$ dpkg-source --commit
$ dpkg-buildpackage -us -uc

$ apt-get source wpasupplicant 

$ dch -i
$ bzr commit

bzr push lp:~<your-launchpad-id>/ubuntu/<release>/<package>/<branchname>
bzr lp-open

ANDROID_SDK="/home/user/android-sdk-linux_x86"
export PATH="${PATH}:${ANDROID_SDK}/tools:${ANDROID_SDK}/platform-tools"

If your SDK installation is not up to date, you can update it using the android update sdk command. To generate a list of system image targets use android list targets, this will produce output similar to this (stripped down for better readability):

id: 1 or "android-3"
     Name: Android 1.5
     API level: 3
id: 3 or "android-4"
     Name: Android 1.6
     API level: 4
id: 5 or "android-7"
     Name: Android 2.1-update1
     API level: 7
id: 7 or "android-8"
     Name: Android 2.2
     API level: 8
id: 10 or "android-9"
     Name: Android 2.3.1
     API level: 9
id: 12 or "android-10"
     Name: Android 2.3.3
     API level: 10
id: 14 or "android-11"
     Name: Android 3.0
     API level: 11

First thing you want to do is create an AVD (Android Virtual Device), this will create an emulator image where you can test your apps without using a real device, the android version used is specified with the -t command switch, and must be one of the targets available in the command we used avobe.

android create avd -n <name> -t <targetID> [-<option> <value>] ...

For example, to create an AVD running android 2.3.3, you would use the command android create avd -n Android233 -t 12.

To list all existing AVDs, you can use the command android list avd, this will produce output similar to this:

Available Android Virtual Devices:
    Name: Android233
  Target: Android 2.3.3 (API level 10)
    Skin: WVGA800
  Sdcard: 15M
---------
    Name: Android16
  Target: Android 1.6 (API level 4)
    Skin: WVGA800

To delete an AVD we no longer need, we can use the command android delete avd -n <name>.
To launch the AVD, just use the command emulator -avd <name>:

Now it’s time to create our first project, we will use the command android create project with the following command line parameters:

android create project --target 1 --name MyApp --path ./MyProject --activity MyActivity --package com.example.myapp

Target specifies the minimum android version our project will be able to run, name is just the name of the application, path specifies the path in your hard drive where the project will be stored, activity specifies the name of the first Activity that will be run when we launch the program (an Activity is an application component that provides a screen with which users can interact in order to do something), and finally package specifies the namespace that will be used (it must be a unique package name across all packages installed on the Android system, following the same rules as for packages in Java).

Now it’s time to fire up your favorite text editor (vim FTW!!), and start coding… if you don’t know where to start, a good starting point is reading the Android Application Fundamentals from the Android Developers guide.

If you’re upgrading a project from an older version of the Android SDK or want to create a new project from existing code, we will use the command android update project as folows:

android update project --name MyApp --target 2 --path ./MyProject

The following commands are useful when you have finished coding your application and want to compile, run or debug it:

Building in debug mode:

cd ./MyProject && ant debug

Building in debug mode and install on already running emulator:

cd ./MyProject && ant install

Automatically launch app on the running emulator after building it:

adb shell 'am start -n com.example.myapp/.MyActivity'

And now, all together: build in debug mode, install and run on emulator:

cd ./MyProject && ant install && adb shell 'am start -n com.example.myapp/.MyActivity'

When you are happy with the results, you might want to build your app in release mode, for example to publish it on the android market:

cd ./MyProject && ant release

Note that output needs to be signed and zipaligned before it can be uploaded on the Android Market, you might want to read the Signing Your Applications chapter in the Development guide for instructions on how to generate your private key for signing your applications.

If you want to build in release mode, with the output signed and aligned, you can do it by modifying the ant.properties file in your project as follows:

echo "key.store=path/to/my.keystore" >> ./MyProject/ant.properties
echo "key.alias=mykeystore" >> ./MyProject/ant.properties
cd ./MyProject && ant release

Finally, to wrap up everything we will create a hello world app from command line:

	
android create project --package com.example.helloandroid --activity HelloAndroid --target 2 --path ./HelloAndroid
	
tee ./HelloAndroid/src/com/example/helloandroid/HelloAndroid.java <<-EOF
package com.example.helloandroid;
	
import android.app.Activity;
import android.os.Bundle;
import android.widget.TextView;
	
public class HelloAndroid extends Activity
{
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        TextView tv = new TextView(this);
        tv.setText("Hello, Android");
        setContentView(tv);
    }
}
EOF
android list avd
emulator -avd XXXXX &
cd ./MyProject
adb devices
ant install
adb shell 'am start -n com.example.helloandroid/.HelloAndroid'

Happy coding!

Option 1: PTP

Option 2: MTP

$ sudo aptitude install mtpfs
$ mkdir ~/android/
$ mount.mtpfs ~/android/

Option 3: adbfs

$ adbfs /media/android/ -o modules=subdir -o subdir=/mnt/sdcard/

I’ve written a small daemon named lightum, source is on github and licensed under GPL-2+.

Usage:  lightum [-m value] [-p value] [-f]
        -m 0..255 : maximum brightness value between 1 and 255 (default=255)
        -p num    : number of seconds between light sensor polls (default=8)
        -f        : run in foreground (do not daemonize)
        -v        : verbose mode, useful for debugging with -f

For it to work you need dbus installed, and your MacBook should have the light sensor located in /sys/devices/platform/applesmc.768/light (should be available on all MacBookAir and MacBookPro versions which have a backlight on keyboard, as far as I know).

If you are running Ubuntu, you can install it by adding lightum-mba ppa to your system:

sudo add-apt-repository ppa:poliva/lightum-mba
sudo apt-get update
sudo apt-get install lightum


Otherwise, you can build it from source.

Edit the file ~/.gconf/apps/compiz-1/general/screen0/options/%gconf.xml and add two new gconf entries as follows:

<?xml version="1.0"?>
<gconf>
        <entry name="hsize" mtime="1317329603" type="int" value="4"/>
        <entry name="vsize" mtime="1317329604" type="int" value="1"/>
</gconf>