Saturday, August 2, 2014

Problem on linking OpenSSL into your NDK application

This week, I tried to compile a simple NDK application and link it with the OpenSSL library. Most of libraries (including OpenSSL) are not supported by the NDK, what makes it a bit more complicated to use. So, in this post, I describe what I usually do to properly compile applications that need external libs.

The project structure is as follows:

my_project/
    + jni/my_code.c
    + jni/Android.mk
    + jni/Application.mk
    + libs/system

Problem #01: How my Android.mk looks like?

In this example, I need two libraries: libcrypto.so and libssl.so. So, the final Android.mk looks like this

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)

LOCAL_MODULE := my_exec
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := my_code.c
LOCAL_C_INCLUDES += $(ANDROID_SRC)/external/openssl/include \ 
                    $(ANDROID_SRC)/external/openssl/crypto \ 
                    $(KERNEL_SRC)/kernel/module
LOCAL_LDLIBS += -L$(LOCAL_PATH)/../libs/system
LOCAL_SHARED_LIBRARIES := libandroid libdl libz libcrypto libssl
LOCAL_LDLIBS += -landroid -ldl -lz


include $(BUILD_EXECUTABLE) 

See, as an example, that we also include the headers for the library (in ANDROID_SRC/external/openssl/include).

NDK does not provide support for libcrypto.so and libssl.so -- we need to have access to the libraries somehow. So, you should create a folder (for example, my_project/libs/system) and push the files /system/lib/libcrypto.so and /system/lib/libssl.so from the device to such folder.

Problem #02: Where do I get the libs from?

You shall get all of them (including libc.so) from your rooted device. The point is that, as I said, NDK does not provide support for libcrypto.so and libssl.so. Therefore, you need to get such libraries from the device. However, there's another problem: most likely, the libcrypto.so and libssl.so libraries don't recognize the symbols from the NDK libc.so and libstdc++.so libraries. Regarding this problem, it will be discussed on item #04.

Problem #03: Where do I get the headers from?

Usually, you get them from the android source code. For OpenSSL, they are located inside the folder ANDROID_SRC/external.

Problem #04: Why does my lib complain about libc symbols?

As I described in topic #02, you need to copy the libraries from the device. However, libcrypto.so and libssl.so do not recognize some symbols from the libc.so and libstdc++.so libraries (most likely, the libraries provided by NDK are not compatible with the ones in the device). Usually, you will have the following compilation error:

/home/raul/android-ndk-r10/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /home/raul/my_project/jni/../libs/system/libcrypto.so: error: undefined reference to '__strlen_chk'
/home/raul/android_development/android-ndk-r10/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /home/raul/my_project/jni/../libs/system/libcrypto.so: error: undefined reference to '__memcpy_chk'
/home/raul/android_development/android-ndk-r10/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /home/raul/my_project/jni/../libs/system/libcrypto.so: error: undefined reference to '__memset_chk'
/home/raul/android_development/android-ndk-r10/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: 
/home/raul/my_project/jni/../libs/system/libcrypto.so: error: undefined reference to '__strchr_chk'
/home/raul/android_development/android-ndk-r10/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-
x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /home/raul/my_project/jni/../libs/system/libcrypto.so: error: undefined reference to '__strcat_chk'

That's easy to solve. Here comes the trick: you have to replace NDK's libraries by those ones from the device. NDK contains several libraries for distinct platforms, as we can see from the path NDK_FOLDER/platforms:

@desktop:$ ls NDK_FOLDER/platforms
android-12  android-13  android-14  android-15  android-16  android-17  
android-18  android-19  android-3   android-4   android-5   android-8  android-9

 Considering that your application is using NDK for platform android-17 (you can define that in file Application.mk), replace the main libraries of folder NDK_PATH/platforms/android-17

@desktop:$ ls NDK_FOLDER/platforms/android-17/arch-arm/usr/lib
crtbegin_dynamic.o  crtend_android.o  libc.a    libEGL.so       libjnigraphics.so libm_hard.a         libOpenSLES.so    libthread_db.so
crtbegin_so.o       crtend_so.o       libc.so   libGLESv1_CM.so  liblog.so       libm.so             libstdc++.a       libz.so

crtbegin_static.o   libandroid.so     libdl.so  libGLESv2.so     libm.a           libOpenMAXAL.so     libstdc++.so

So, push the libraries /system/lib/libc.so, and /system/lib/libstdc++.so from the device to the folder NDK_FOLDER/platforms/android-17/arch-arm/usr/lib. After that, compile the application again and voilĂ  -- all problems solved :-) 


Monday, July 28, 2014

Android image/kernel building/flashing - A *VERY* short guide :-)

This week, I had to go through the process of Android OS/Kernel building/installation. And it was a lot much better and 6 months ago (maybe, because I built it for a device and not for the emulator?). I compiled the images in Ubuntu 12.04 and I used a Samsung Galaxy Nexus device (maguro with tuna as kernel). Therefore, I decided to summarize the steps that I took. This mini-tutorial is a lot shorter and simpler (and really works!!).

1. Android OS

1.0 Setting up the building environment

Check this instructions (here and here) to set up the basic environment and download the code. I used the branch [android-4.3_r1.1].

1.1 Compiling the Android OS

a. Download and unpack the manufacturer drivers from this link. They have to be unpacked into the directory [android_source_code]/vendors -- but don't worry, as the .zip files contain a script that does all the work for you.

b. Once the drivers are in the proper place, run the following commands:

  @desktop:$ cd [android_source_code]
  @desktop:$ make clobber
  @desktop:$ lunch full_maguro-userdebug
  @desktop:$ make -j4

It takes a long time to compile the image.

After these steps, the Android OS is ready.

1.2 Flashing the device with the new Android OS

Now, you need two tools from the Android SDK: adb and fastboot. These tools are located in the folder [androis_sdk]/platform-tools.

a. Reboot the device in the bootloader mode -- hold VolumeDown and VolumeUp and then press the PowerUp button.

b. Connect the USB cable.

c. Run the following commands:

  @desktop:$ export PATH=$PATH:[android_sdk]/platform-tools
  @desktop:$ cd [android_source_code]
  @desktop:$ sudo fastboot format cache
  @desktop:$ sudo fastboot format userdata
  @desktop:$ sudo ANDROID_PRODUCT_OUT=[android_source_code]/out/target/product/maguro/ fastboot -w flashall

After these steps, reboot the device. A clean installation will take place. To check the new version of you device, go to "Settings" - - > "About Phone" and check "Model number": now, it should be "AOSP on Maguro" (check attached image)



2. Android Kernel

Ok. Now, we have the AOSP in place and we need to compile a new kernel. But why do you need to compile and install a new kernel? Oh, well, let's say that you want to apply some patches or that you need to change the kernel to enable Linux module support (the default Android Linux Kernel does not support modules).

2.0 Setting up the building environment

If you have built the Android OS before, you don't need anything special for the kernel building. I used the official code from https://android.googlesource.com/kernel/omap.git, branch android-omap-tuna-3.0-jb-mr2.

2.1 Compiling the Kernel

First, you need to set some variables that are important for the building process (ARCH and CROSS_COMPILE):

  @desktop:$ export ARCH=arm
  @desktop:$ export CROSS_COMPILE=[android_source_code]/prebuilts/gcc/linux-x86/arm/arm-eabi-4.7/bin/arm-eabi-

Now, you have to generate a .config which contains all the options for the kernel building. By running the following command you generate a basic .config file for Android.

  @desktop:$ cd [android_kernel_code]
  @desktio:$ make tuna_defconfig

Sometimes, you need to set some specific entries of the .config to enable/disable certain features of the kernel. For this specific example, let's set the option CONFIG_MODULES to y (the entry in the .config file should be CONFIG_MODULES=y). With CONFIG_MODULES set to y, it is possible to insert/remove kernel modules. Now, let's build the kernel

  @desktop:$ cd [android_kernel_code]
  @desktop:$ make

(it takes some time to compile the kernel)

2.2 Preparing the kernel for installation

The kernel image is almost ready: it's still necessary to wrap it up properly to flash it into the device. The Android source code contains scripts that do the work for us. Consider that the image was generated at [android_kernel_code]/arch/arm/boot/zImage.

  @desktop:$ cd [android_source_code]
  @desktop:$ export TARGET_PREBUILT_KERNEL= [android_kernel_code]/arch/arm/boot/zImage
  @desktop:$ make bootimage

At the end, a custom image is ready for installation at [android_source_code]/out/target/product/maguro/boot.img

2.3 Flashing the device with the new Kernel
 
Now, everything is in place and we can finally flash our kernel image. To do so:

a. You need to boot the device in bootloader mode (hold VolumeDown and VolumeUp and then press the PowerUp button)

b. Connect the USB cable

c. Run the following commands

  @desktop:$ cd [android_source_code]
  @desktop:$ sudo ANDROID_PRODUCT_OUT=[android_source_code]/out/target/product/maguro/ fastboot flash boot [android_source_code]/out/target/product/maguro/boot.img

After these steps, reboot the device. A clean installation will take place. To check the new version of you kernel, go to "Settings" - - > "About Phone" and check "Kernel version": you will see a different name for you kernel image (as for the previuos image).


Monday, August 26, 2013

Building/Running Android Goldfish kernel

Among the tasks for build an Android kernel development, building the Android Goldfish kernel is the easiest :-) (I mean, no compiling errors...)

First, you need to have access to the cross-compiling tools. The Android source code provides them in folder /prebuilt/linux-x86/toolchain/. 

Steps


1. Set the path to include the pre-build toolchain
   # export PATH=/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/:$PATH
2. Set the Kernel config file. In this case, we are compiling for the ARM emulator (armv7).
   # make ARCH=arm goldfish_armv7_defconfig
3. Run make
   # CROSS_COMPILE=/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi- ARCH=arm make
4. Change var ANDROID_PRODUCT_OUT
   # export ANDROID_PRODUCT_OUT=/out/target/product/generic/
5. Run the emulator
   # /out/host/linux-x86/bin/emulator-arm -system /out/target/product/generic/system.img -kernel /goldfish/arch/arm/boot/zImage -data /out/target/product/generic/userdata.img -ramdisk /out/target/product/generic/ramdisk.img -skindir /sdk/emulator/skins/ -skin HVGA -verbose -show-kernel

Issues


  • The option CONFIG_MODULES is NOT enabled by default. If you want to insert modules into the kernel, make sure to set CONFIG_MODULES;
  • Android emulator does NOT work with Kernels > 2.6.X So, make sure to use the proper Kernel. For devices, any support Kernel will work.


Building Android ICS


To compile ICS (and probably any new version of Android platform >= 4.0), you
might have some problems with the compilation tools.

  • Follow the instructions described on http://source.android.com/source/building.html
    • It also provides information about Android Kernel building.
  • Check which compiler best fits into the version of Android platform you are using.
    • Usually, due to new features (for example, C++11 support on GCC/++ 4.7), it is a bit more complicated to build it with new compilers... you will see several compiling errors. For example, for the ICS, GCC/++ 4.6 works better than GCC/++ 4.7.
  • You may need some changes in order to compile it properly (mostly on Makefile). Again, depending on your compiler, you may apply more or less changes on your code.
  • For ICS, GCC/++ 4.6 works better.
This link provides a good discussion about it (and solutions!). In my case, for compiling ICS, I made only a few changes (see below) and I also used GCC/++ 4.6.

=================================================================

project build/
diff --git a/core/combo/HOST_linux-x86.mk b/core/combo/HOST_linux-x86.mk
index 5ae4972..7df2893 100644
--- a/core/combo/HOST_linux-x86.mk
+++ b/core/combo/HOST_linux-x86.mk
@@ -53,6 +53,6 @@ HOST_GLOBAL_CFLAGS += \
  -include $(call select-android-config-h,linux-x86)
 # Disable new longjmp in glibc 2.11 and later. See bug 2967937.
-HOST_GLOBAL_CFLAGS += -D_FORTIFY_SOURCE=0
+HOST_GLOBAL_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0
 HOST_NO_UNDEFINED_LDFLAGS := -Wl,--no-undefined

project dalvik/
diff --git a/vm/native/dalvik_system_Zygote.cpp b/vm/native/dalvik_system_Zygote.cpp
index 31fecfd..b3b745a 100644
--- a/vm/native/dalvik_system_Zygote.cpp
+++ b/vm/native/dalvik_system_Zygote.cpp
@@ -19,6 +19,7 @@
  */
 #include "Dalvik.h"
 #include "native/InternalNativePriv.h"
+#include "sys/resource.h"
 #include
 #include

project development/
diff --git a/tools/emulator/opengl/host/renderer/Android.mk b/tools/emulator/opengl/host/renderer/Android.mk
index 55fcb80..613179c 100644
--- a/tools/emulator/opengl/host/renderer/Android.mk
+++ b/tools/emulator/opengl/host/renderer/Android.mk
@@ -6,6 +6,8 @@ $(call emugl-import,libOpenglRender)
 LOCAL_SRC_FILES := main.cpp
 LOCAL_CFLAGS    += -O0 -g
+LOCAL_LDLIBS += -lX11
+
 #ifeq ($(HOST_OS),windows)
 #LOCAL_LDLIBS += -lws2_32
 #endif

project external/gtest/
diff --git a/include/gtest/internal/gtest-param-util.h b/include/gtest/internal/gtest-param-util.h
index 5559ab4..0af13b0 100644
--- a/include/gtest/internal/gtest-param-util.h
+++ b/include/gtest/internal/gtest-param-util.h
@@ -39,6 +39,7 @@
 #include
 #include
+#include
 #if GTEST_HAS_PARAM_TEST
diff --git a/src/Android.mk b/src/Android.mk
index 2465e16..1d93ac8 100644
--- a/src/Android.mk
+++ b/src/Android.mk
@@ -49,7 +49,7 @@ LOCAL_SRC_FILES := gtest-all.cc
 LOCAL_C_INCLUDES := $(libgtest_host_includes)
-LOCAL_CFLAGS += -O0
+LOCAL_CFLAGS += -O0 -fpermissive
 LOCAL_MODULE := libgtest_host
 LOCAL_MODULE_TAGS := eng
@@ -67,7 +67,7 @@ LOCAL_SRC_FILES := gtest_main.cc
 LOCAL_C_INCLUDES := $(libgtest_host_includes)
-LOCAL_CFLAGS += -O0
+LOCAL_CFLAGS += -O0 -fpermissive
 LOCAL_STATIC_LIBRARIES := libgtest

project external/llvm/
diff --git a/llvm-host-build.mk b/llvm-host-build.mk
index 5219efd..cc91ee0 100644
--- a/llvm-host-build.mk
+++ b/llvm-host-build.mk
@@ -44,6 +44,8 @@ LOCAL_C_INCLUDES := \
 LOCAL_IS_HOST_MODULE := true
+LOCAL_LDLIBS := -lpthread -ldl
+
 ###########################################################
 ## Commands for running tblgen to compile a td file
 ###########################################################

project external/oprofile/
diff --git a/libpp/format_output.h b/libpp/format_output.h
index b6c4592..8e527d5 100644
--- a/libpp/format_output.h
+++ b/libpp/format_output.h
@@ -91,7 +91,7 @@ protected:
  symbol_entry const & symbol;
  sample_entry const & sample;
  size_t pclass;
- mutable counts_t & counts;
+ counts_t & counts;
  extra_images const & extra;
  double diff;
  };

project frameworks/base/
diff --git a/libs/utils/Android.mk b/libs/utils/Android.mk
index 638f72f..127cad9 100644
--- a/libs/utils/Android.mk
+++ b/libs/utils/Android.mk
@@ -61,7 +61,7 @@ LOCAL_SRC_FILES:= $(commonSources)
 LOCAL_MODULE:= libutils
-LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1 $(TOOL_CFLAGS)
+LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1 $(TOOL_CFLAGS) -fpermissive
 LOCAL_C_INCLUDES += external/zlib
 ifeq ($(HOST_OS),windows)
diff --git a/tools/aapt/Android.mk b/tools/aapt/Android.mk
index a3e5d9a..4c415d6 100644
--- a/tools/aapt/Android.mk
+++ b/tools/aapt/Android.mk
@@ -28,7 +28,7 @@ LOCAL_SRC_FILES := \
     ZipFile.cpp
-LOCAL_CFLAGS += -Wno-format-y2k
+LOCAL_CFLAGS += -Wno-format-y2k -fpermissive
 LOCAL_C_INCLUDES += external/expat/lib
 LOCAL_C_INCLUDES += external/libpng

project frameworks/compile/slang/
diff --git a/Android.mk b/Android.mk
index fce3637..8ffe68f 100644
--- a/Android.mk
+++ b/Android.mk
@@ -19,7 +19,7 @@ ifeq ($(TARGET_BUILD_APPS),)
 LOCAL_PATH := $(call my-dir)
-local_cflags_for_slang := -Wno-sign-promo -Wall -Wno-unused-parameter -Werror
+local_cflags_for_slang := -Wno-sign-promo -Wall -Wno-unused-parameter
 ifneq ($(TARGET_BUILD_VARIANT),eng)
 local_cflags_for_slang += -D__DISABLE_ASSERTS
 endif

Android (Goldfish) kernel development

Finally, back to this :-)

Lately, I have being developing a new project which basically needs some kernel changes on Android Goldfish kernel. Initially, I thought that this would be something like "traditional" kernel development, but NOT. Android kernel is, of course, a Linux kernel but it has some peculiarities that impact on the development phase.

[NOTE] I'd rather provide a link to the source that I used than replicating the same information.

So, I split my experience into the following topics:
  1. Building Android ICS
  2. Building/Running Android Goldfish kernel 
  3. Creating a Goldfish kernel module
  4. Adding new syscalls into the Goldfish kernel

Thursday, January 24, 2013

Moving...

My life has changed upside down recently: I have been accepted to Saarland University Graduate School and I will move to Germany soon. I will try to keep this updated as possible.

Thursday, November 29, 2012

LLVM - ARM cross-compilation using Raspberry!!

Well, a short break on continuous integration posts!

I bought my Raspberry Pi (http://www.raspberrypi.org/) board some weeks ago mainly to use it as a developer board (pretty affordable and efficient). So, I decided to use it to cross-compile LLVM (http://www.llvm.org).

So, here you go.

Building

First of all, download LLVM (from either SVN or GIT repositories). I got the following information from  LLVM documentation. For more details, check this out!

  • Checkout LLVM:
    $ cd where-you-want-llvm-to-live
    $ svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm

  • Checkout Clang:
    $ cd where-you-want-llvm-to-live
    $ cd llvm/tools
    $ svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
  • Checkout Compiler-RT:
    $ cd where-you-want-llvm-to-live
    $ cd llvm/projects
    $ svn co http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt
  • Get the Test Suite Source Code [Optional]
    $ cd where-you-want-llvm-to-live
    $ cd llvm/projects
    $ svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suite

To cross-compile it to ARM platform, you need a toolchain. I use Sourcery G++ Lite 2011.03-41 (MentorGraphics Sourcery CodeBench Lite Edition) - it is free and contains everything we need. Install it and also remember to set the PATH variable so that you can use your toolchain.
$ export $PATH=$PATH:/home/user/CodeSourcery/Sourcery_G++_Lite/bin
Now, it´s time to build the application (and remember: for further details, check this out!)

$ cd where-you-want-to-build-llvm
$ mkdir build
(for building without polluting the source dir)
$ cd build
$ ../configure --target=arm-none-linux-gnueabi --host=arm-none-linux-gnueabi --enable-optimized (the target/host value may change depending on the toolchain that you have chosen.)
$ make

Now, you just need to wait for a couple of minutes :-) Next step is NFS-mounting the LLVM dir in your Raspberry Pi.

NFS-Mounting

Now, LLVM is cross-compiled and we are able to use it and perform some tests on Raspberry Pi. I prefer NFS as partition mounting solution. An important information is that you have to have the same path on both environments (desktop and remote). It means that if you are building LLVM on /home/user/llvm desktop folder, you have to mount it on /home/user/llvm remote folder (in most of cases, you have to create the user on your Raspberry environment - just use the command).  Firstly, set up your desktop machine:
  • Install NFS packages
    @desktop$ sudo apt-get install portmap nfs-kernel-server nfs-common
  • Edit /etc/exports and add the following line
    /home/user/llvm 192.168.1.0/24(rw,sync,no_subtree_check)
  • Restart NFS system
    @desktop$ sudo service nfs-kernel-server restart
  • Change access permission of your shared folder (in this case, I use the same folder)
    @desktop$ chmod -R 777 /home/user/llvm

Configuration on Raspberry Pi is also straightforward:

  • Install NFS packages (most of Raspberry distributions contain NFS installed, so maybe this step is not necessary)
    @remote$ sudo apt-get install portmap nfs-kernel-server nfs-common
  • Create the host folder
    @remote$ mkdir llvm
    @remote$ chmod 777 llvm
  • Mount the NFS-shared folder
    @remote$ sudo mount 192.168.1.4:/home/user/llvm /home/user/llvm
Now it is done and you can use your cross-compiled LLVM on your Raspberry.

Testing

Let´s now focus on LLVM tests and how we can run them on ARM platforms. In this case, I am running tests under /home/user/llvm/tests folder.

  • CD to test dir
    @remote$ cd llvm/build/test
  • Run tests
    @remote:/home/user/llvm/build/test$ make 

After some minutes, LLVM test suite runs all integration tests and provides a report with results.

@remote:/home/user/llvm/build/test$ make
...
PASS: LLVM :: ExecutionEngine/MCJIT/test-shift.ll (59 of 59)
Testing Time: 11.65s
********************
Unexpected Passing Tests (1):
    LLVM :: ExecutionEngine/MCJIT/test-data-align-remote.ll
********************
Failing Tests (7):
    LLVM :: ExecutionEngine/MCJIT/2003-01-04-ArgumentBug.ll
    LLVM :: ExecutionEngine/MCJIT/2005-12-02-TailCallBug.ll
    LLVM :: ExecutionEngine/MCJIT/fpbitcast.ll
    LLVM :: ExecutionEngine/MCJIT/pr13727.ll
    LLVM :: ExecutionEngine/MCJIT/test-call.ll
    LLVM :: ExecutionEngine/MCJIT/test-common-symbols.ll
    LLVM :: ExecutionEngine/MCJIT/test-ret.ll
  Expected Passes    : 45
  Expected Failures  : 6
  Unexpected Passes  : 1
  Unexpected Failures: 7
UPDATE: Raspberry Pi the ARM target platform for LLVM project at this moment. If you want to contribute to LLVM project by supporting ARM features, check a PandaBoard development kit.