How to cross compile python

Steps to Cross-Compile Python 2.7.8 (this is based on MIPS, but with some tweaks, should work for any other platform as well).

Cross-compiling python for MIPS was challenging for me for two reasons:

1) The lack of cross-compilation help available online for python
2) The lack of option in python to exclude the stuff that you don't want to be bundled.

It took me a week to get the basic stuff working, so, decided to write this up - so that, its useful for someone looking for answers, as I was a while ago.

Here is what I did to cross-compile python 2.7.8 for MIPS.

1) Download the tarbll from https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz

2) Untar it under a directory (say) Python-2.7.8-build,

tar -zxvf Python-2.7.8.tgz

3) Now, we need the below patches to be applied on the Python-2.7.8-build directory

(cd Python-2.7.8-build; \
patch -p1 < /home/proj/python-006-cross-compile-getaddrinfo.patch; \
patch -p1 < /home/proj/python-002-setup.py-ssl-zlib-path-fix.patch; \
patch -p1 < /home/proj/python-001-cross-compile-fix-test-grammar.patch; \
);



3) These two switches need to be set in config.site.

echo "ac_cv_file__dev_ptmx=no" >> Python-2.7.8-build/config.site

echo "ac_cv_file__dev_ptc=no" >> Python-2.7.8-build/config.site

4) Generate host part of Python and Pgen

cd Python-2.7.8-build; \
rm -rf config.cache; \
./configure; \
make python; \
mv python hostpython; \
mv Parser/pgen hostpgen; \
make distclean; \

5) Now, we start cross-compiling

Three switches are key for configure:

host-dev : gcc -dumpmachine

host-dev was 'x86_64-pc-linux-gnu' in my case, so stripped off the stuff thats after first hyphen, suffixed the hostname on which the build is run. For example:

x86_64-pc-server1-linux-gnu

build - x86_64-server1-linux-gnu

host - mips-linux

target - x86_64-server1-linux-gnu

Configuring part:

(cd Python-2.7.8-build; \; rm -rf config.cache; \
./configure \
--enable-ipv6 \
--build=x86_64-server1-linux-gnu \
--host=mips-linux \
--target=x86_64-server1-linux-gnu \
--prefix=/python \
--sysconfdir=/etc/python2.7 \
CONFIG_SITE=config.site; \
);

6) Cross-compilation:

(cd Python-2.7.8-build; \
make -j1 HOSTPYTHON=./hostpython \
HOSTPGEN=hostpgen \
BLDSHARED="/home/project/tc/bin/mips-linux-uclibc-gcc -shared" \
CROSS_COMPILE_TARGET=yes \
CROSS_COMPILE=/home/project/tc//bin/mips-linux-uclibc- \
HOSTARCH=mips-linux \
BUILDARCH=x86_64-pc-linux-gnu \
);

At the end of the above step, you are likely to see these errors:

Python build finished, but the necessary bits to build these modules were not found:

_bsddb _curses _curses_panel
_sqlite3 _tkinter bsddb185
dbm dl gdbm
imageop linuxaudiodev nis
ossaudiodev readline sunaudiodev

To find the necessary bits, look in setup.py in detect_modules() for the module's name.

This is most likely not an issue unless you want these pieces cross-built. In which case, you may have to tinker with the setup.py

7) Install : Note the -i (to ignore errors)

( cd Python-2.7.8-build; mkdir -p ${TARGET_ROOT}/usr/local; \

make -j1 -i install HOSTPYTHON=./hostpython \
BLDSHARED="/home/project/tc/bin/mips-linux-uclibc-gcc -shared" \
CC=/home/project/tc/bin/mips-linux-uclibc-gcc \
CXX=/home/project/tc/bin/mips-linux-uclibc-g++ \
AR=/home/project/tc/bin/mips-linux-uclibc-ar \
RANLIB=/home/project/tc/bin//mips-linux-uclibc-ranlib \
CROSS_COMPILE_TARGET=yes \
CROSS_COMPILE=/home/project/tc/bin/mips-linux-uclibc- \
LDFLAGS="-L${TARGET_ROOT}/lib -L${TARGET_ROOT}/usr/lib -L${TARGET_ROOT}/usr/local/lib" \
prefix=${TARGET_ROOT}/usr/local \
);

You will see the below errors:

Python build finished, but the necessary bits to build these modules were not found:

_bsddb _curses _curses_panel
_sqlite3 _tkinter bsddb185
dbm dl gdbm
imageop linuxaudiodev nis
ossaudiodev readline sunaudiodev

To find the necessary bits, look in setup.py in detect_modules() for the module's name.

which, is OK - unless you need those modules. If you need those, you might have to tinker more with the setup.py script.
You should see most of the shared libraries under on the target root directory

/usr/local/lib/python2.7/lib-dynload

After this point, any number of packages like boto, pexpect, pcrypto etc., can be added.
You are good to go if you reach this far without lot of trouble.