Friday, August 19, 2011

Using Robolectric with Real Android Jars

Overview

"Why doesn't Robolectric just use the real Android jars?" This one of the most commonly asked questions. The short answer is that it's hard: hard to get the real Android jars, hard to hook them up, hard to deal with native code, hard to know what the heck is going to happen when real code is called, hard to change from the Shadow-way of doing things, hard hard hard.

But, many of you know that we have a branch at http://github.com/pivotal/robolectric named "reviscerated" that backs Robolectric with real Android implementations. This branch has languished due to lack of time and resources.

This week we spent time reviving this branch, merging the master branch into it and getting it working with APK 10/2.3.3 jars.

Robolectric tests pass in both SDK and real-jars mode, as do the tests in RobolectricSample. Now we need you to help us test this code. If you find it valuable then help us fix what's broken.

So, what happens?

All non-Shadowed entities, such as methods and classes, call through to the real Android implementations. No more "Stub!" exceptions from Android, no more null return values or no-op implementations from Robolectric -- if you call Android code, you get Android code. Note that any methods implemented in a Shadow are still called -- shadowed things win.

Essential Android Links:

Terminology:

  • SDK jars: These are the jars installed by android using AVD or command line tools. Implementations replaced with "Stub!" exceptions and entities annotated @hide are either non-public or otherwise unavailable.
  • real jars: Jars generated by manually building Android. Full implementations present including entities annotated @hide.

Getting The Real Jars

Google does not distribute the real Android jars -- you must build them yourself.


Setting up your Workstation

Follow Mac setup instructions here: http://source.android.com/source/initializing.html

After create and mount (double-click) the sparse image, make a directory specifically for the version of Android you want to build.

/Volumes/android $ ls
gingerbread-233 honeycomb-30

Note: once you have built one version of android it is is hard to use that same dir to build another version. I don't know how to clean up from the last version. It's hit-or-miss.

# choose a specific branch found on http://source.android.com/source/build-numbers.html
/Volumes/android/gingerbread-233 $ repo init -u git://android.git.kernel.org/platform/manifest.git -b android-2.3.3_r1
/Volumes/android/gingerbread-233 $ repo sync # give this 40+ minutes

Compiling, Building

See the build instructions here: http://source.android.com/source/building.html

Note: make sure the lunch output looks right! If the numbers don't match what you expect from http://source.android.com/source/build-numbers.html then you have synced to the wrong version and will need to repo sync -b [some other tag]

$ lunch full-eng

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.3 <===== gingerbread
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=false
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
HOST_ARCH=x86
HOST_OS=darwin
HOST_BUILD_TYPE=release
BUILD_ID=GRI40 <===== matches gingerbread on http://source.android.com/source/build-numbers.html
============================================

Make It

$ make -j8 # 4-cpu, 8-core machine


Don't Wait: Watch for classes.jar


The build takes about 20-30+ minutes, but you don't have to wait for the full build. Watch for out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar. When it appears you can grab it and move it somewhere handy:

$ mkdir -p ~/android-real-jars/gingerbread-233
$ cp /Volumes/android/gingerbread-233/out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/*.jar. ~/android-real-jars/gingerbread-233/

Also: download kxml2-2.3.0.jar and put it in the same directory you moved classes.jar. http://www.findjar.com/jar/net/sf/kxml/kxml2/2.3.0/kxml2-2.3.0.jar.html


Using the Jars with Robolectric

From Maven 2.0-SNAPSHOT



From Robolectric Source

$ git clone git://github.com/pivotal/robolectric.git
$ cd robolectric
$ git checkout origin/reviscerated # this is the "use real jars" branch

  • Open RobolectricTestRunner.USE_REAL_ANDROID_SOURCES and make sure it is set to true.
  • Look at RobolectricTestRunner.getDefaultLoader() and make sure the paths point to the real jars you copied above. You can also use environment variables for these. See Setting ANDROID_HOME and ANDROID_REAL_JARS_HOME below.
  • Rebuild the jars in real-jar mode:
    $ ant clean jar-all
  • Replace your robojectric.jar with target/robolectric.jar or target/robolectric-all.jar

If you are using IntelliJ you might need to fix the Project Structure => Modules => Dependencies.

Setting ANDROID_HOME and ANDROID_REAL_JARS_HOME

You need to tell Robolectric some extra information when in real-jars mode. At this time you can either hand-edit RobolectricTestRunner and change the defaults or set environment variables:

# in .bash_profile or something

export ANDROID_HOME=/Users/pivotal/android-sdk-mac_x86
export ANDROID_REAL_JARS_HOME=/Users/pivotal/android-real-jars/gingerbread-233


Intellij most likely won't see those variables. Set them in the run configruation or default run configuration for JUnit:





Issues

  • Real jars mode seems to be incompatible with Roboguice at the moment. Sometimes it works, somtimes not.
  • Running ant clean test results in MethodGenerator test failures -- javassist.NotFoundException for Views. mvn clean test succeeds.
  • We would like to find the minimum time and effort to generate real jars.
  • There are likely errors in this blog post. We will be updating it as issues are found.

3 comments:

  1. If you want to download sun-java5-jdk to compile the android jars on ubuntu the google page: http://source.android.com/source/initializing.html is incorrect.

    instead of:

    deb http://archive.ubuntu.com/ubuntu dapper main multiverse
    deb http://archive.ubuntu.com/ubuntu dapper-updates main multiverse

    use:

    deb http://old-releases.ubuntu.com/ubuntu dapper main multiverse
    deb http://old-releases.ubuntu.com/ubuntu dapper-updates main multiverse

    ReplyDelete
  2. If you want to compile with JAVA 1.5 (not in your PATH environment variable) and JAVA 1.6 is on your PATH.

    you can do this when you execute make to override java 1.6 with 1.5: 'sudo env PATH=/usr/lib/jvm/java-1.5.0-sun/bin:$PATH make -j8'

    Also Ubuntu uses dash shell by default and android requires bash to compile, you can execute 'dpkg-reconfigure dash' to switch the system to the bash shell.
    see https://wiki.ubuntu.com/DashAsBinSh

    ReplyDelete
  3. Thank you for writing this in-depth article. You have covered every angle, the good problem is that you will be able to reference completely different components. These are heaps of key pieces that make it work.

    ReplyDelete