Run Android CTS

Recently, I was asked to look into CTS report of my work. So, I downloaded CTS on Android website and started running it. After hours of experiments, these are my experiences that should save you some time.
  • Use SDK 1.6r1 against your CTS. Don't use SDK r10.
  • Modify maxTestCount to -1 in your $CTS_ROOT/repository/host_config.xml
  • Don't run large test plan. Predefined test plans in the CTS are,
    • CTS plan
      • VM plan
      • Java plan
      • Android plan
        • RefApp plan
        • AppSecurity plan
      • Signature plan
    • Performance plan
  • If you run into errors complaining this "Installing met .... due to unknown reasons", try to run a customized smaller test plan composed of nonExecuted packages.
    • add --plan reset_of_packages. It prompts you available packages to add into this plan and you can add packages that haven't run yet since the error happens.
  • You can run CTS in Linux, Windows or Ubuntu. The whole CTS suite is just a script wrapped on top of "java -cp". It should be platform-independent. If you are running CTS on Windows/cygwin system, 
    • Move the definition of ${JARS} in your startcts to the following,
      JARS=`cygpath -w -p ${CTS_LIB}:${DDM_LIB}:${JUNIT_LIB}:${HOSTTEST_LIB}`
    • and, this line
      java ${JAVA_OPTS} -cp ${JARS} com.android.cts.TestHost `cygpath -w ${CONFIG}` "$@" ${DDCONFIG}