Robbie Mosaic's Web Log

• 

Decheng (Robbie) Fan

2014-08-26

I wanted to install GCC (GNU Compiler Collection) on my Android phone.
Yes, you read it correctly, my Android phone! Not a Linux PC or Windows
PC. Not a Windows 8 tablet.

The reason that I want to do it is to exploit more power out of my
Android phone. I understand that Android is not Linux, so to use
Android without rooting, I must perform all the installation "the
Android way".

Android controls security based on application level. Each application
is run in its own user account, with limited access to system resources
and to other applications. An application can access system resources
according to the security requirements of it, which are confirmed by
the user during installation. It cannot break the barrier the system
sets on it. By default, it cannot access other applications through the
file system or through process manipulation. It can only call another
application's "Activity", or "Share" contents with another other
application. "Activity" and "Share" are Android UI concepts for
connecting applications together.

This strict security model means that, a non-rooted Android device
cannot install Linux applications in the traditional way--because
traditional Linux applications require access to some common
directories such as /usr/lib, which is forbidden in Android. In
addition, the default directory locations are not usually available on
Android.

So I went on exploring a way to get GCC installed on my KBox
environment. What is KBox, you might ask? It is a minimalist
command-line environment introduced by Kevin Boone. Visit his website
at http://kevinboone.net/index.html to get documentation on KBox and
how to install it. The current version is KBox2, by the way.

Although there is a package for GCC on KBox2, my environment is the
first generation KBox. I don't have an alternate Android device so I
could only do with 1st-gen KBox. I downloaded the KBox2 manual
installation package and installed it into a temporary directory
(within my real KBox environment). The installation is not
global--manual configuration is required to use it as the default shell
environment. However, because I already have a first-gen KBox, what I
need is the `dpkg*' utility only. So I examined the `dpkg*' commands
available, and found two symbolic links--dpkg and dpkg-deb, pointing to
the busybox executable of KBox2. I copied the busybox of KBox2 and
renamed it to kbox2_busybox, and then manually created the dpkg and
dpkg-deb symbolic links.

Then I used the dpkg command to extract the gcc 4.7 package downloaded
from Kevin Boone's website. It successfully installs gcc to my KBox
installation: /data/data/jackpal.androidterm/app_HOME/kbox/usr/lib/gcc.
By the way, my $KBOX variable points to
/data/data/jackpal.androidterm/app_HOME/kbox.

After installation, there is already an executable shell script called
gcc under the $KBOX/usr/bin directory. I tried to adapt it and copy it
into my actual bin directory--$KBOX/bin--to release it to
manufactoring, but it doesn't work out of the box.

The original script has one inconvenience: it puts all environment
variables on one line and then call the gcc executable file. It looks
like (below is one line):

TMPDIR=$KBOX/tmp PATH=$PATH:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7
/bin $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin/gcc "$@"

For convenience, I changed it to export the environment variables, and
then call the gcc executable, like:

export TMPDIR=$KBOX/tmp
export PATH=$PATH:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin
$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin/gcc "$@"

Another benefit of using `export' is that in some (older) shells,
environment variables specified on the command line without exporting
won't be propagated to child processes, which would cause problems.

But it didn't work. To fix it, I started investigating. Later I also
found out how to print debugging information and how to display verbose
information of the compiler. The discovery was later than I tried
several ways, but it is helpful, so I put the tips here:

- Use "gcc --print-search-dir" to show its search directories,
  including compiler program paths, include paths and library paths.
- Use "gcc -v" to print all sub-commands that have been run during a
  compilation process.
- gcc option "-Wl,-option" will pass an option to the linker, such as
  "-Wl,--debug".

With the help of the debugging commands above, I did the things below
to make the configuration correct, for compiling a program written in
the C programming language (C++ compilation--g++--is not covered here).

First of all, the include directories are not specified yet. I found
that two include paths are necessary: $KBOX/usr/lib/gcc/arm-linux-
androideabi/4.7/arm-linux-androideabi/include contains stdio.h and the
`sys' directory; $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/lib/gcc/
arm-linux-androideabi/4.7/include contains stdarg.h. A third directory,
$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/lib/gcc/arm-linux-
androideabi/4.7/include-fixed, contains stdio.h, but is optional.

GCC_EXEC_PREFIX=$KBOX/usr/lib/gcc/ is necessary. It must end with a
slash (the path separator). It will be used to append pre-configured
directories after it. However most pre-configured directories don't
match the KBox2 package installation, so it's just for reference.

liblto_plugin.so is used for link time optimization. Note that it
locates under $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/libexec/gcc/
arm-linux-androideabi/4.7. It is a shared library (`.so' means "shared
object") for the linker. Together with the directory containing the
compiler executables, two directories need to be included in both PATH
and COMPILER_PATH:

- $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin
- $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/libexec/gcc/arm-linux-
  androideabi/4.7

Then I found that, gcc still doesn't work. By using "gcc -v test.c", I
got much more debug information. It contains the command lines it uses
to call sub-programs. Firstly it calls `cc1' to compile the C program
into assembly (a `.s' file). Then it calls `as' to assemble the
assembly code into machine code (a `.o' file, meaning "object"). Later,
it calls `collect2' to link the object file with standard stubs (two .o
files, crtbegin_dynamic.o and crtend_android.o) and standard libraries
(lib*.a files, here libgcc.a, libc.a and libdl.a).

I can run the individual commands to see the output of each step. What
I observed is that the first two commands ran successfully--`cc1' and
`as' do generate expected outputs. The third command `collect2',
however, doesn't generate any output. No error messages are issued
either. I checked online what the command is for, and got the
information that it is a wrapper (frontend) for the `ld' command. In
addition to `ld', `collect2' supports C++ global object
constructor/destructor calling code generation. So it is a necessary
tool, but why doesn't it generate any output?

During my testing I accidentally found that the individual command of
`collect2' reaches the length limit of a shell command on the command
line. But later I found that this is not the reason, because the length
limit only has effect on the terminal command line. It doesn't apply
when running a shell script, as I tried adding a lot of dummy non-space
characters to the command line, whether it works or not doesn't depend
on this.

I found another interesting thing: when I call `collect2' with its
full path, /data/data/jackpal.androidterm/app_HOME/kbox/usr/lib/gcc/
arm-linux-androideabi/4.7/libexec/gcc/arm-linux-androideabi/4.7/
collect2, it doesn't work. But if I set up all necessary environment
variables and call it by its name alone, `collect2', it works fine!

I downloaded gcc 4.7.4 source code and read through the main() function
of collect2. I found that in some step it gets its executable file name
by examining the last path component of argv[0]. I'm not sure whether
there is any problem regarding this mechanism in the version of gcc
contained in KBox2, but for safety I renamed collect2 to real_collect2,
and wrote a shell script `collect2' to replace it. In the shell script,
I just put the lines below:

#!/system/bin/sh
real_collect2 "$@"

By using this shell script, the linker, `ld', seems to get invoked
correctly. The only problem left here is that it mentions:

ld: cannot find -lgcc

Then I looked up the `ld' documentation and found that the `-l' switch
is used to specify the library name. So `-lgcc' means libgcc.a or
libgcc.so. I searched through the gcc installation directory tree and
found libgcc.a located under two places. I chose one place, plus the
location for standard C library (libc.a), I added two paths to
LIBRARY_PATH: $KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/arm-linux-
androideabi/lib:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/lib/gcc/
arm-linux-androideabi/4.7.

Then, my gcc installation can compile C programs! How exciting! So it's
the end of this story. Later I will of course investigate how to make
g++ work. But for now I can already use a huge base of C programs.

The final `gccrun' shell script contains the lines below (backslash at
the end of lines are used to connect two lines due to line length
limit of SJTU BBS):

#!/system/bin/sh
export PATH=$PATH:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin:\
$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/libexec/gcc/arm-linux-\
androideabi/4.7
export GCC_EXEC_PREFIX=$KBOX/usr/lib/gcc/
export COMPILER_PATH=$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/bin\
:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/libexec/gcc/arm-linux-
androideabi/4.7
export LIBRARY_PATH=$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/arm-\
linux-androideabi/lib:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/lib\
/gcc/arm-linux-androideabi/4.7
export C_INCLUDE_PATH=$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7/arm-\
linux-androideabi/include:$KBOX/usr/lib/gcc/arm-linux-androideabi/4.7\
/lib/gcc/arm-linux-androideabi/4.7/include
"$@"

To compile a C program, I just type "gccrun gcc program.c" and I will
get an "a.out". I will also make programs using "gccrun make" as long
as there is a Makefile available.

Terminologies:

busybox - a multi-call binary program that supports many useful
    commands. It also provides the default shell for 1st-gen KBox (run
    "busybox sh"). 1st-gen KBox busybox doesn't have dpkg support but
    KBox2 busybox does.
chmod - changes access rights--read, write and execute--to a file: for
    the file owner, for a specified group, and for all others.
dpkg - within the context of KBox2, it is a command used to extract
    `.deb' packages, such as those on Kevin Boone's website, including
    GCC. This command is a symbolic link to the KBox2 busybox
    executable.
GCC - GNU Compiler Collection. Here the GCC is built for Android and
    runs on the ARMv7 CPU. It supports C and C++ languages, standard
    libraries and Android-specific libraries.
KBox/KBox2 - a busybox command line environment created by Kevin Boone.
    These environments don't require rooting the Android device, and
    works with Jack Palevich's Android Terminal.
ld - the static linker contained in gcc. The linker links object files
    together to make an executable file.
.so files - shared object files. Like DLLs on Windows, they are
    dynamically-linked libraries loaded for an executable file at
    runtime.
shell script - a script program file that is interpreted by a Linux or
    Unix shell, such as /system/bin/sh or "busybox sh". It can be set
    to be executable by using the `chmod' command.
symbolic link - a mechanism supported by most Linux file systems,
    including Android's internal flash memory file system. Use command
    `ln' with `-s' switch to create symbolic links.
/system/bin/linker - the dynamic linker, which is used to load and link
    to `.so' files. When used during the static link stage, it can help
    the linker generate function calls to the shared object file. It's
    somewhat like the feature of import libraries in Win32. When used
    during runtime, it helps the executable file to load the shared
    object file into virtual memory and gets the address of the
    function in the shared object. dlopen/dlsym/dlclose/dlerror
    functions are used for dynamic loading, just like Win32 API
    LoadLibrary/GetProcAddress/FreeLibrary functions.