Debian on the Intenso Memory 2 Move
These notes refer in particular to the 1TB "Black" model:
Basic operating modes:
As described in the manual (mirrored
here for convenience), the device has at least two operating modes. When connected via USB to a computer in the initial "power off" state, it behaves as a simple USB-to-SATA bridge, making the whole hard-disk externally accessible.
When indipendently switchted on, the internal embedded
SoC Linux system takes over, and after a relatively fast booting sequence it will make a series of services available through the WiFi / Ethernet ports, while using any later USB connection only as a power source for topping up the internal battery of about 3000 mAh.
The SoC is rated for a typical power load of about 3W, to which the hard-disk consumption must be added - see below. If the chemistry would be a standard 3.7 V one, we would have 3.7*3=11.1 Wh of energy, or little more than 2 hours of continuous, standalone use once the top disk load of about 2 W is also accounted for.
Notice that while the hard-disk is properly spinned down after 3 minutes of uninterrupted idle, the default kernel appears to be compiled without any power management support, see below. Longer running time than the advertised 5 hours may be achievable.
Hardware description and default working modes:
The internal hard-disk seems to be a Seagate / Samsung part:
# hdparm -I /dev/sdb:
[...]
Model Number: ST1000LM024 HN-M101MBB
[...]
... which should have the following specifications:
- Form factor: 2.5", 9.5mm
- Rotation speed: 5400rpm
- Cache: 8MB
- Power consumption: 2.20W (load), 0.70W (idle)
- Loudness: 26dB(load), 24dB (idle)
- SMART feature set: supported.
There have been reports of possible replacement with other disks, but even larger capacities than 1 TiB, despite possibly working, are normally thicker and don't fully fit within the case. Some rough ideas about performances:
- When driven in standalone mode, from the embedded SoC Linux system:
# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 114 MB in 2.03 seconds = 56.21 MB/sec
Timing buffered disk reads: 28 MB in 3.10 seconds = 9.04 MB/sec
- When driven from the USB bridge mode, from an external PC:
[...]
The internal SoC is very flexible, if somehow resources -limited.
The devices uses a
Ralink RT5350 MIPS24 KEc (cat /proc/cpuinfo: MIPS 24 Kc V4.12, corresponding to gcc -march=24kec switch, see below) with 32MiB of RAM, in a
little-endian layout. As remarked
here, this particular CPU variant lacks a hardware floating point unit (FPU), which has to be emulated in software. This is fine for the current deployment as
portable NAS unit, or for the other common usage in routers etc. More demanding mathematical tasks, such as video compression in IP cameras etc., are based on variant with FPU included.
Some device -specific details are also tabulated on
https://wikidevi.com/wiki/Intenso_Memory_2_Move_1TB_White.
Default services in the stock firmware ver. 1.1 are based on
ulibc (the system will also re-create, if needed, and use an extra 64MiB of swap as a rather inefficient ".vst/swapfile" on the first VFAT/NTFS partition /dev/sda1 of the internal hard-disk) and include:
- minidlna: DLNA (via UPnP) multimedia files serving, for instance to clients such as Android WonderShare player or Apple IOS analogus;
- smbd: SMB protocol support for filesystem access from Microsoft Windows / Apple OSX / Linux;
- lighttpd: HTTP protocol support for configuration and/or filesystem access from standard browsers
- ...
These notes report on the installation of a complete, if slightly out-of-date, Debian MIPSEL distribution
"Lenny". This makes readily available for installation and use all packages listed i.e.
here.
Compiling and running standalone code:
It is possible to develop and cross-compile standalone programs for the default embedded Linux system, working i.e. on a standard x86 Debian PC and just copying the resulting MIPS executables on the device as generic files.
This requires availability of a cross-compilation toolchain. A suitable one for Jessie/Sid may be installed by adding to /etc/apt/sources.list the lines:
# INTENSO MIPS
deb http://www.emdebian.org/debian/ squeeze main
Then simply:
apt-get install gcc-4.3-mipsel-linux-gnu
...will pull in all required dependencies.
A nice example of what can be compiled and run this way is the CLI "
Tetris" game by Victor Nilsson.
Installing a full, ready-to-use Debian release:
Better capabilities can be made accessible through installation of a complete Debian environment.
"Lenny" i.e. Debian release 5.0 is the latest release that can run with the stock kernel 2.6.21. This allows in particular easiest and safest deployment on top of the existing firmware, as the whole system can work in a "
chroot" jail with no risk of bricking the device.
Gaining root:
The stock firmware ver. 1.1 (mirrored
here for convenience, MD5sum: 8acc9d9bbb8d18d5da2ad6b6a20a043d )
contains a backdoor. To my eyes this is by the way very useful, but an extremely dangerous data securety issue too.
In practice the system will boot in standalone mode with a telnetd daemon running on default port 23, and offering root login with:
username: root
password: 20080826
The instructions below mitigate the problem by switching off such daemon, and offering instead SSH access with a chosen password instead. This must however be repeated at every reboot, as by default the system starts in insecure mode.
Further notes below document a different attempt to address the issue by rebuilding the embedded Linux firmware. One should consider the security implication of the classic "
Trusting trust", and more recent discussions such as
this post by Bruce Schneier.
Choosing a suitable Debian version:
After logging in as root through the stock firmware telnet backdoor, the system reveals itself as:
- Processor brand and featurs:
# cat /proc/cpuinfo
system type : Ralink SoC
processor : 0
cpu model : MIPS 24K V4.12
BogoMIPS : 239.61
wait instruction : yes
microsecond timers : yes
tlb_entries : 32
extra interrupt vector : yes
hardware watchpoint : yes
ASEs implemented : mips16 dsp
VCED exceptions : not available
VCEI exceptions : not available
- The physical RAM is:
# free
free
total used free shared buffers cached
Mem: 28008 26028 1980 0 2016 13384
- The kernel version in use is:
# uname -a
Linux M2M 2.6.21 #256 Thu Apr 25 09:35:37 CST 2013 mips GNU/Linux
- The default booting arguments were:
# cat /proc/cmdline
console=ttyS1,57600n8 root=/dev/mtdblock8 rootfstype=squashfs quiet
(a serial console should in other words be available for debugging, if interfacing with the appropriate pins on the mainboard);
- The default flash memory layout reads:
# cat /proc/mtd
dev: size erasesize name
mtd0: 00800000 00010000 "ALL"
mtd1: 00030000 00010000 "Bootloader"
mtd2: 00010000 00010000 "Config"
mtd3: 00010000 00010000 "Factory"
mtd4: 00180000 00010000 "Kernel_RootFS"
mtd5: 00010000 00010000 "params"
mtd6: 00010000 00010000 "user_backup"
mtd7: 00010000 00010000 "user"
mtd8: 00600000 00010000 "Rootfs
All the above points to the conclusion that a viable option for full Linux environment installation will be Debian 5.0.10 "Lenny", in the little endian "MIPSEL" architecture.
A later, more up-to-date distribution release will not be compatible with the stock firmware kernel.
Debian root filesystem option 1: As a loopback image within the default hard-disk partion scheme:
Advantages:
- Simplest deployment;
- Safer against unmounting;
Disadvantages:
- Fairly inefficient I/O, which makes many operations painful to use on the rather resources -starved SoC;
In view of the above considerations, a different installation scheme based on proper hard-disk re-partitioning and ext2 formatting (outlined below) is preferred. In either cases, the first steps of passing through a loopback image are shared, and detailed in the instructions below:
- Prepare and mount an image filesystem to work on:
# dd if=/dev/zero of=/tmp/debianfs bs=1M count=2000
(something smaller than 2GiB may also be sufficient);
# mkfs.ext2 /tmp/debianfs
mount /tmp/debianfs /tmp/db -o loop
(modprobe loop may be required);
- Fill the image filesystem with a basic Debian "Lenny" installation:
# apt-get install debootstrap
# debootstrap --foreign --verbose --arch=mipsel lenny /tmp/db/ http://archive.debian.org/debian-archive/debian/
(apt-get install debootstrap may be required);
- Transfer the image filesystem to the device:
# umount /tmp/db ; mount -t cifs //10.10.10.254/WiFiDisk1_Volume1/ /mnt -o username=admin
(or just switch the disk off and connect & mount it as USB mass storage, to perform a faster copy);
- Log on the stock firmware Linux system through the backdoor:
# telnet 10.10.10.254
(default IP assignment for WiFi connection),
username: root
password: 20080826
- Mount the empty filesystem image to complete the Debian base installation, automatically downloading all the needed "base" packages from the network:
# mount data/UsbDisk1/Volume1/debianfs /mnt/ -o loop
(on a single line:)
# DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C chroot /mnt /debootstrap/debootstrap --second-stage
DEBIAN_FRONTEND=noninteractive
(on a single line:)
# DEBCONF_NONINTERACTIVE_SEEN=true LC_ALL=C LANGUAGE=C LANG=C chroot /mnt dpkg --configure -a
- Set up the Debian system network and virtual filesystems: verify that its /etc/resolv.conf points to a proper nameserver, i.e. contains something like:
# cat /mnt/etc/resolv.conf
nameserver 192.168.43.1
...and that these mountpoints are present and properly populated:
# mount none /mnt/proc -t proc
# mount -t sysfs sysfs /mnt/sys/
# mkdir /mnt/dev/pts
# mount -t devpts devpts /mnt/dev/pts/
# mount -t usbfs usbfs /mnt/proc/bus/usb/
Further useful devices may be creted with:
# mknod /dev/sda b 8 0
# mknod /dev/sda1 b 8 1
# mknod /dev/sda2 b 8 2
... and the default VFAT partition may be made visible to the Debian system too with:
# mount /dev/sda1 /mnt/mnt
- Set up the correct time zone:
# dpkg-reconfigure tzdata
The command:
# ntpdate-debian
... may later be used for precise NTP clock syncronization.
- Install any other needed Debian package as usual. At the very least, a SSH server is advised, such as dropbear:
# apt-get install dropbear
- Verify SSH is working:
# netstat -tan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:8200 *:* LISTEN
tcp 0 0 *:netbios-ssn *:* LISTEN
tcp 0 0 *:www *:* LISTEN
tcp 0 0 *:81 *:* LISTEN
tcp 0 0 10.10.10.254:5880 *:* LISTEN
tcp 0 0 192.168.43.246:5880 *:* LISTEN
tcp 0 0 localhost:6010 *:* LISTEN
tcp 0 0 *:microsoft-ds *:* LISTEN
tcp 0 0 10.10.10.254:www 10.10.10.2:56894 ESTABLISHED
tcp 0 0 localhost:81 localhost:3122 TIME_WAIT
tcp6 0 0 [::]:ssh [::]:* LISTEN
tcp6 0 0 10.10.10.254:ssh 10.10.10.2:35305 ESTABLISHED
- close the stock firmware backdoor:
# killall telnetd
(the backdoor will be automatically re-opened upon rebooting, until the "rootfs" default initrd squashfs flashed on the NAND memory is customized as per instructions below)
Debian root filesystem option 2: as a standalone ext2 partition within a re-partitioned hard-disk
The stock kernel has been compiled with no more modern filesystem capabilities than "ext2".
Advantages:
- faster;
- slightly safer from accidental erasure when using the system as a dumb USB external disk;
This latter method is preferred, and better described below.
Repartition:
- The stock firmware Linux system is set up to assume that a (ex)VFAT or NTFS partition should be present as /dev/sda1, and be used for all the default standalone device services.
- Create a swap partition. Suggested size is just 64MiB, i.e. twice as big as the physical RAM; for better performances it is best to set it up as close as possible to hard disk cylinder 0, i.e. as /dev/sda2 just after the resized /dev/sda1;
- Create a ext2 partition for the new Debian MIPSEL installation. Suggested size is about 3GBi, as /dev/sda3;
- Create a ext2 partition with the rest of the unallocated space, i.e. as /dev/sda4.
This step can be carried out via i.e. the Gnome Partition Editor "
http://gparted.org/liveusb.php", available as standard Linux application or a standalone, bootable USB image.
Install the base system:
Same as for the method above. At the end of it, simply copy everything to the desired ext2 partition, such as /dev/sda3 in the scheme above (make sure not to unnecessarily copy bare devices themselves from /dev !).
Prepare an automatic Debian launcher script
The following "~/bin/autolog.sh" may be used to semi-automatically boot the Debian system after each device reboot. It assumes the "telnet" client and "
expect" interpreter being available, and works through the default firmware ver. 1.1 backdoor:
#!/usr/bin/expect
# Script to deploy the Debian chroot and close the default telnet backdoor.
# Access the backdoor:
spawn "telnet" "10.10.10.254"
expect "M2M login:"
send "root\r"
expect "Password:"
send "20080826\r"
# expect "BusyBox"
expect "#"
# We are in. Switch from the VFAT -hosted swapfile to a swap partition (faster):
send "swapon /dev/sda2\r"
expect "#"
send "swapoff /data/UsbDisk1/Volume1/.vst/swapfile\r"
expect "#"
# Mount the Debian and leftover storage partition. Sync mode is used, even if slower (the SoC offers limited performances anyway, we are limited by it rather than the I/O subsystem) to afford a margin of safety against sudden switching off.
send "mount /dev/sda3 /mnt/ -o sync\r"
send "mount none /mnt/proc -t proc\r"
expect "#"
# Populate the virtual filesystem:
send "mount -t sysfs sysfs /mnt/sys/\r"
expect "#"
# send "mkdir /mnt/dev/pts\r"
send "mount -t devpts devpts /mnt/dev/pts/\r"
expect "#"
send "mount -t usbfs usbfs /mnt/proc/bus/usb/\r"
expect "#"
# Start the chroot. WARNING: no standard services are initialized, this saves resources but breaks some Debian available services assumptions.
send "chroot /mnt /bin/bash\r"
expect "#"
# Before mounting the leftover partition, the Debian fsck.ext2 tool is used to check it for errors in case it has not been properly unmounted before:
send "fsck.ext2 -p -v -C0 /dev/sda4\r"
expect "#"
send "mount /dev/sda4 /home/ -o sync\r"
expect "#"
# Mount everything else may have been configured:
send "mount -a\r"
expect "#"
# Launch the SSH server: either Dropbear (fewer resources) or OpenSSH:
send "/etc/init.d/dropbear start\r"
expect "Starting Dropbear SSH server: dropbear."
# send "/etc/init.d/ssh start\r"
# expect "Starting OpenBSD Secure Shell server: sshd."
# Close this telnet backdoor, leaving only SSH (and other stock firmware services such as UPnP i.e. minidlna, SAMBA i.e. smbd etc.) access:
send "killall telnetd\r"
# interact
Prepare a custom firmware:
Temporary notes, high risk of bricking the device. Proceed with caution! No kernel upgrade has been tried yet, only userspace modifications.
Analysis of a stock firmware image and the flashing process:
The following notes apply to the stock firmware ver. 1.1:
$ unzip 1389341356.zip gives a license, and the actual firmware image fw-M2M-1.100.000
$ file fw-M2M-1.100.000 returns:
fw-M2M-1.100.000: POSIX shell script executable (binary data)
$
binwalk fw-M2M-1.100.000 returns:
DECIMAL HEX DESCRIPTION
-------------------------------------------------------------------------------------------------------
4326 0x10E6 gzip compressed data, was "initrdup", from Unix, last modified: Mon Jul 1 09:44:07 2013, max compression
1539822 0x177EEE LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 65536 bytes
[...]
In practice, the first 170 ASCII text lines are a shell script that instruct on how to unpack (and flash) the rest. Following these, we begin with:
tail -n +171 fw-M2M-1.100.000 > upfs.gz
which gives:
$ file upfs.gz
upfs.gz: gzip compressed data, was "initrdup", last modified: Mon Jul 1 09:44:07 2013, max compression, from Unix
$ gunzip upfs.gz
gives a upfs file, which is a standard ext2 filesystem image:
$ file upfs
upfs: Linux rev 1.0 ext2 filesystem data, UUID=48ee846c-1fa2-4ad2-a3d0-692446b63b1f
This can be mounted as loopback device for further analysis:
# mount upfs initdup/ -o loop
initdup# ls -l
total 15
drwxr-xr-x 2 root root 2048 Mar 21 2013 bin
drwxr-xr-x 3 root root 1024 Jul 15 2012 boot
drwxr-xr-x 3 root root 1024 Jul 15 2012 config
drwxr-xr-x 5 root root 1024 Jul 15 2012 dev
drwxr-xr-x 2 root root 1024 Jun 9 2013 etc
drwxr-xr-x 2 root root 1024 Jul 1 2013 firmware
drwxr-xr-x 5 root root 2048 Jul 15 2012 lib
drwxr-xr-x 3 root root 1024 Jul 15 2012 mnt
drwxr-xr-x 2 root root 1024 Jul 15 2012 proc
drwxr-xr-x 2 root root 1024 Jul 15 2012 sys
-rwxr-xr-x 1 root root 1151 Nov 21 2012 update.sh
drwxr-xr-x 5 root root 1024 Jul 15 2012 var
The "update.sh" script does the flashing of the
MTD NAND, as directed by its numerical argument. In particular, it takes versioning informations, the kernel binary, its rootfs and the
U-Boot bootloader from the "firmware" subdirectory:
# cat firmware/firmware.conf
NEWFILE=M2M
NEWVER=1.100.000
NEWBUILD=7
# file firmware/kernel
firmware/kernel: u-boot legacy uImage, Linux Kernel Image, Linux/MIPS, OS Kernel Image (lzma), 1439867 bytes, Thu Apr 25 03:35:52 2013, Load Address: 0x80000000, Entry Point: 0x8043F000, Header CRC: 0x6A7044ED, Data CRC: 0x652C3C28
The first difficulty appears in interpreting the firmware/rootfs file. At first its
magic is not recognized:
# file firmware/rootfs
firmware/rootfs: data
but binwalk reveals:
# binwalk firmware/rootfs
DECIMAL HEX DESCRIPTION
-------------------------------------------------------------------------------------------------------
0 0x0 Squashfs filesystem, little endian, non-standard signature, version 3.0, size: 5167458 bytes, 926 inodes, blocksize: 65536 bytes, created: Mon Jul 1 09:44:02 2013
As shown by
hexedit, the file begins with "shsq" rather than the usual "hsqs", thus the "non-standard signature" statement. This denotes a
LZMA compression scheme with SWAPPED fields, which is not yet supported by Debian "unstable" unsquashfs-tools ver. 1:4.0-8 :
# unsquashfs -h
[...]
Decompressors available:
gzip
lzo
xz
To unpack this, a later version of the squashfs tools may be downloaded from SourceForge via
GIT (apt-get install git may be needed):
$ git clone git://git.code.sf.net/p/squashfs/code squashfs-code
It must be compiled with the new compression scheme enabled, but also a patch to support this particular deployment.
- Download a patch from: http://sourceforge.net/p/squashfs/patches/20/, and apply it:
$ git apply --stat ~/Downloads/0001-unsquashfs-add-support-for-LZMA-magics.patch
...will show:
$ cd .. ; git apply --check ~/Downloads/0001-unsquashfs-add-support-for-LZMA-magics.patch
squashfs-tools/squashfs_fs.h | 6 ++++++
squashfs-tools/unsquashfs.c | 24 ++++++++++++++++++------
2 files changed, 24 insertions(+), 6 deletions(-)
... should show no output (i.e. no errors or conflicts, the patch may be applied cleanly);
- Actually apply the patch; either via git, or by hand (removing all lines before:
--- a/squashfs-tools/squashfs_fs.h
+++ b/squashfs-tools/squashfs_fs.h , and issuing a simple:
patch -p1 < 0001-unsquashfs-add-support-for-LZMA-magics.patch )
- Edit the squashfs-tools/Makefile and make sure that it reads (apt-get install liblzma-dev may be needed):
[...]
########### Building XZ support #############
#
# LZMA2 compression.
#
# XZ Utils liblzma (http://tukaani.org/xz/) is supported
#
# To build using XZ Utils liblzma - install the library and uncomment
# the XZ_SUPPORT line below.
#
XZ_SUPPORT = 1
[...]
and further below:
[...]
########### Building LZMA support #############
#
# LZMA1 compression.
#
# LZMA1 compression is deprecated, and the newer and better XZ (LZMA2)
# compression should be used in preference.
#
# Both XZ Utils liblzma (http://tukaani.org/xz/) and LZMA SDK
# (http://www.7-zip.org/sdk.html) are supported
#
# To build using XZ Utils liblzma - install the library and uncomment
# the LZMA_XZ_SUPPORT line below.
#
# To build using the LZMA SDK (4.65 used in development, other versions may
# work) - download and unpack it, uncomment and set LZMA_DIR to unpacked source,
# and uncomment the LZMA_SUPPORT line below.
#
LZMA_XZ_SUPPORT = 1
#LZMA_SUPPORT = 1
[...]
- Alternatively, an archive of the patched & configured snapshot of the tools is made available here. Simply unpack and issue a:
make
Now the root filesystem may be unpacked and studied in full detail:
# unsquashfs -li /tmp/initdup/firmware/rootfs
For instance, the default passwords for the "admin" user, and the "root" backdoor may be brute-forced via
John the Ripper (apt-get install john may be needed):
# john etc/passwd
Created directory: /root/.john
Loaded 2 password hashes with 2 different salts (md5crypt [MD5 32/64 X2])
Press 'q' or Ctrl-C to abort, almost any other key for status
00000 (admin)
This will be the default configuration password for the stock firmware HTTP interface.
Some additional information for this firmware release can be found in the etc/versioninfo file:
[...]
uboot=5350/bootloader-mt5350-32M-2013-04-25-09-25.img
kernel=5350/kernel-rt5350-32M-2013-04-25-09-31
... which would point to a
Ralink 5350 part running at 360MHz, and confirming availability of 32MiB RAM.
Customizing a stock firmware image:
When rootfs modifications are satisfactory, the new filesystem can be re-squashed via:
mksquashfs . ../new_rootfs -comp lzma
The mksquashfs compiled above, or the default Debian "Jessie" ver. 4.0, will however not be suitable for re-compressing the image before flashing. Besides compressor support for other algorithms such as LZMA, the squashfs binary format has been changed before vanilla kernel inclusion, and is not backwards compatible with the patched support of stock firmware kernel ver. 2.6.21, which will not mount our new new_rootfs file with error message:
SQUASHFS error: Major/Minor mismatch, trying to mount newer 4.0 filesystem
SQUASHFS error: Please update your kernel
Upgrading kernel at the same time as the root filesystem on an embedded system is risky, we may lose all access to the device if something goes wrong.
We can however simply produce a vintage format of our modified rootfs, using obsolete versions of the mksquashfs as available from i.e.
the Debian archive. After unpacking the original firmware rootfs through the patched, latest unsquashfs v4.0, it doesn't really matter if the new, modified rootfs will be compressed via gzip rather than the more space efficient LZMA method.
The mksquashfs utility compiled from squashfs_3.1r2.orig.tar.gz works:
# mksquashfs squashfs-root/ new_rootfs_v3.1_format
...gives a file in the older ver.3.0 format:
$ binwalk new_rootfs_v3.0_format
DECIMAL HEX DESCRIPTION
-------------------------------------------------------------------------------------------------------
0 0x0 Squashfs filesystem, little endian, version 3.0, size: 6043973 bytes, 920 inodes, blocksize: 65536 bytes, created: Fri May 30 17:14:40 2014
...which the stock firmware kernel ver. 2.6.21 can mount without problems, for instance as loopback filesystem just for testing before flashing:
# mount /mnt/new_rootfs_v3.0_format /opt/ -o loop
M2M:/# ls /opt/
bin boot data dev etc etc_ro home lib media mnt opt proc sbin sys tmp usr var www
The standard magic "hsqs" header will now denote in particular reliance on the older, less efficient compression algorithm.
When all customizations are satisfactory, the "new_rootfs" can replace the original "firmware/rootfs":
$ mv new_rootfs_v3.0_format firmware/rootfs
The whole pre-flashing filesystem can then be unmounted:
# umount initdup
...and re-compressed via gzip:
# gzip initdup
As a last step a self-extracting firmware file has to be created. In particular, the modified ext2 initdup.gz payload will have a different CRCSUM, as computed by the standard 'cksum' utility after some file massaging by sed.
Following the original firmware file scripted lines, the checksum is computed as:
$ sed '1,3d' fw-M2M-1.100.000 |cksum|sed -e 's/ /Z/' -e 's/ /Z/'|cut -dZ -f1
3696346624
... which matches the expected value as listed within the self-extracting archive:
$ head fw-M2M-1.100.000
#!/bin/sh
# constant
CRCSUM=3696346624
[...]
The new self-extracting firmware can be assembled as follows:
- Extract the unpacking and flashing script lines:
$ head -n 170 fw-M2M-1.100.000 > self_extracting_fw_header.sh
- Prepare a temporary firmware (complete payload, but still invalid CRCSUM) with:
$ cp self_extracting_fw_header.sh self_extracting_fw.sh
$ cat upfs.gz >> self_extracting_fw.sh
- Compute the valid CRCSUM via:
$ sed '1,3d' self_extracting_fw.sh |cksum|sed -e 's/ /Z/' -e 's/ /Z/'|cut -dZ -f1
...and replace the resulting value inside the new self_extracting_fw.sh file.
Normally, more than a single iteration can be required: the CRCSUM field is also part of the file content, for which the CRC is calculated, hence it changes as the field is updated. Such CRC calculation procedure is however a sort of lossy compression scheme. As such, modification of a few bytes, like the CRCSUM=... constant, will not change the outcome dramatically. Recalculation a second time is typically sufficient; just repeat this step 3) until the check value is stable and matches the file content.
The customized firmware is now ready for flashing, simply using the default HTTP interface and internal tools. Be aware that this will also reset the WiFi network configuration options.
Building a complete alternative:
A custom embedded Linux image may be assembled ("backed", in Android jargon...) via the
BuildRoot system. This allows a great amount of customization, as it provides a set of rules and Makefiles to download, configure and cross-compile only the needed package and specifically supporting the device hardware, for a more efficient usage of the available SoC resources.
Apparently, Ralink also provides a customized BuildRoot SDK. A mirror copy of ver. 4.0.1.0 (including a more recent, but patched for a later board, Linux kernel 2.6.36 besides an older, patched ver. 2.6.21) is available for download for instance
here.
The SDK above still needs an external cross-compilation toolchain. The oldish kernel version provided looks in particular not compatible with later GCC developments, such as ver. 4.4 included in the above MIPS(el) toolchain.
It would fail with error:
CHK include/linux/version.h
CHK include/linux/utsrelease.h
CC arch/mips/kernel/asm-offsets.s
arch/mips/kernel/asm-offsets.c: In function 'output_mm_defines':
arch/mips/kernel/asm-offsets.c:233: error: invalid 'asm': invalid use of '%X'
arch/mips/kernel/asm-offsets.c:234: error: invalid 'asm': invalid use of '%X'
arch/mips/kernel/asm-offsets.c:235: error: invalid 'asm': invalid use of '%X'
/tmp/rt5350_GPL_source/kernel/linux-2.6.21.x/./Kbuild:42: recipe for target 'arch/mips/kernel/asm-offsets.s' failed
make[1]: *** [arch/mips/kernel/asm-offsets.s] Error 1
Makefile:863: recipe for target 'prepare0' failed
make: *** [prepare0] Error 2
It seems the SDK refers to a specific GCC version, gcc-3.4.2. This is available from a study of another Ralink -based product, the
Coolcam NIP-09 NIP-02, also based on the RT5350 (in the F = FPU variant). And also, by the way, including a telnet backdoor! (username: root, password: 123456) and is mirrored
here for convenience.
Instructions on how to use the SDK and toolchain are available from
here, and have been also mirrored
here for convenience. In practice, the content of buildroot-gcc342.tar.bz2 should be placed within /opt, and one may issue:
$ export MANPATH=$MANPATH:/opt/buildroot-gcc342/man
... to furthermore have access to the included manpages.
Another, slightly more up-to-date version of the SDK (ver. 4.1.1.0) has been also made available online, and is mirrored for convenience
here (including compiler toolchain etc.).
Further packages which may require installation are, for instance:
# apt-get install flex bison
... and so on, as per list of dependencies.
LZMA compression:
- Beside that toolchain, a (patched up) standalone LZMA compressor is also needed, if one wants to follow the default SDK build process. It seems slightly different from this version, and should be provided as part of the RT288X_SDK for instance. The utility "lzma_alone" must be compiled and placed in /opt/buildroot-gcc342/bin/
- Alternatively, it seems possible to switch to the older, less space efficient but far more established gzip compression scheme. The crucial stage where the rootfs is compressed, as part of the kernel building process, is controlled by the scrip linux-2.6.21.x/scripts/gen_initramfs_list.sh, which can be modified not to use the (vendor -patched) lzma_alone compressor by changing interpretation of option use_lzma to "n":
[...]
"-l")
use_lzma="n"
;;
[...]
To use the SDK, unpack it and enter the "source" subdirectory. Issue a:
$ make menuconfig
... command (apt-get install libncurses5-dev may be needed) and press Enter on the "Select the Product you wish to target", then select "Ralink Products", and "RT5350". Then select "Default configuration file" and choose "4M/32M(AP+NAS), and exit repeteadly.
The item "Kernel/Library/Defaults Selection" allows to change what cross-compiler chain to use, which parts of the BuildRoot distribution will be included in the firmware, which libc will these linked with, and which Linux kernel should drive the whole.
Even if no parameters will at first be changed, it is crucial to let BuildRoot scripted process take care of setting up the right cross-compiler toolchain in all subdirectory. This can be achieved by selecting for customization all the fields:
- Customize Kernel Settings
- Customize Vendor/User Settings
- Customize Busybox Settings
- Customize uClibc Settings
- Customize uClibc++ Settings
... so that each one sub-menu will be brought up at least once, and updated with the right parameters from the main menu upon saving.
Once happy with all selection, save and exit the main menuconfig interface, and issue a simple:
# make
The build will take a few minutes but succeed without errors. The (safer) default BuildRoot behaviour has been modified in the SDK version, so as to try and create device files too under /dev in the rootfs. This will fail if the user is not root.
The outcome is stored in the images/ subdirectory, and attempts are made to also copy it as a /tftboot file in the host root filesystem (!). In particular, even after swiching off usage of the lzma_alone utility, a images/zImage.lzma file gets created and may be decompressed via:
$ unlzma ralink_sdk-master/source/images/zImage.lzma
Again binwalk provides better insight on its content:
$ binwalk zImage
$ binwalk zImage
DECIMAL HEX DESCRIPTION
-------------------------------------------------------------------------------------------------------
752777 0xB7C89 LZMA compressed data, properties: 0x88, dictionary size: 1048576 bytes, uncompressed size: 4608 bytes
940089 0xE5839 LZMA compressed data, properties: 0x38, dictionary size: 1048576 bytes, uncompressed size: 4160 bytes
950753 0xE81E1 LZMA compressed data, properties: 0x30, dictionary size: 524288 bytes, uncompressed size: 992 bytes
2261044 0x228034 Linux kernel version "2.6.21 (root@t420) (gcc version 3.4.2) #2 Sat May 31 21:56:45 C 3.4.2) #2 Sat May 31 21:56:45 CEST 2014CEST 2014"
2267836 0x229ABC LZMA compressed data, properties: 0x02, dictionary size: 16777216 bytes, uncompressed size: 50331648 bytes
2424580 0x24FF04 ASCII cpio archive (SVR4 with no CRC), file name: "sh entry"
2875460 0x2BE044 LZMA compressed data, properties: 0x02, dictionary size: 16777216 bytes, uncompressed size: 512 bytes
2875541 0x2BE095 LZMA compressed data, properties: 0x02, dictionary size: 131072 bytes, uncompressed size: 33620481 bytes
2887043 0x2C0D83 LZMA compressed data, properties: 0x02, dictionary size: 65536 bytes, uncompressed size: 32 bytes
2887067 0x2C0D9B LZMA compressed data, properties: 0x02, dictionary size: 65536 bytes, uncompressed size: 128 bytes
2887259 0x2C0E5B LZMA compressed data, properties: 0x02, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887283 0x2C0E73 LZMA compressed data, properties: 0x02, dictionary size: 65536 bytes, uncompressed size: 128 bytes
2887307 0x2C0E8B LZMA compressed data, properties: 0x31, dictionary size: 65536 bytes, uncompressed size: 32 bytes
2887331 0x2C0EA3 LZMA compressed data, properties: 0x30, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887355 0x2C0EBB LZMA compressed data, properties: 0x31, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887379 0x2C0ED3 LZMA compressed data, properties: 0x30, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887427 0x2C0F03 LZMA compressed data, properties: 0x30, dictionary size: 65536 bytes, uncompressed size: 128 bytes
2887451 0x2C0F1B LZMA compressed data, properties: 0x40, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887475 0x2C0F33 LZMA compressed data, properties: 0x30, dictionary size: 65536 bytes, uncompressed size: 64 bytes
2887499 0x2C0F4B LZMA compressed data, properties: 0x40, dictionary size: 65536 bytes, uncompressed size: 128 bytes
2887523 0x2C0F63 LZMA compressed data, properties: 0x40, dictionary size: 65536 bytes, uncompressed size: 256 bytes
2892036 0x2C2104 LZMA compressed data, properties: 0x01, dictionary size: 131072 bytes, uncompressed size: 2 bytes
2892136 0x2C2168 LZMA compressed data, properties: 0x09, dictionary size: 131072 bytes, uncompressed size: 2 bytes
2892156 0x2C217C LZMA compressed data, properties: 0x0A, dictionary size: 131072 bytes, uncompressed size: 2 bytes
2894087 0x2C2907 LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2894107 0x2C291B LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2894127 0x2C292F LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2894147 0x2C2943 LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2894167 0x2C2957 LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2894187 0x2C296B LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 896 bytes
2905944 0x2C5758 LZMA compressed data, properties: 0x01, dictionary size: 16777216 bytes, uncompressed size: 16777473 bytes
3084288 0x2F1000 gzip compressed data, from Unix, last modified: Sat May 31 21:56:39 2014, max compression
The crucial information is at the end. As a result of the gen_initramfs_list.sh script modification, the rootfs archive has been appended at the end of the file in gzip format. It can be extracted for verification by skipping all preceeding parts of the file:
$ dd if=zImage of=rootfs_recomp.gz seek=3084288 bs=1
and simply:
$ gunzip rootfs_recomp.gz
... to reveal a standard CPIO archive:
$ file rootfs_recomp
rootfs_recomp: ASCII cpio archive (SVR4 with no CRC)
As per standard Linux kernel documentation, this can be opened via:
$ cpio -i -d -H newc -F rootfs_recomp --no-absolute-filenames
(again, attempts at creating special /dev files will fail, without root permission)
In other words, we should be able to check that this SDK -recompiled CPIO archive of a rootfs matches the stock firmware "firmware/rootfs" ext2 image, which is itself normally shipped within the initrdup.gz payload of a ready-to-flash self-extracting firmware file.
This allows to upgrade the userspace parts of the embedded Linux environment, without changing yet the default kernel. It is possible in particular to close the default telnet backdoor; add other packages such as i.e. like the
Snort NIDS to better monitor the network; and provide i.e. a out-of-the-box dropbear SSH server for better local access security.
One problem with the old GCC toolchain that Ralink SDK would use by default is lack of support for better CPU optimization. Contrary to the more up-to-date compiler that even Debian "Lenny" ships, GCC ver. 3.4.2 supports at most a:
-mips32r2 -O3
... optimized build target, thereby not really exploiting more advanced capabilities of the SoC
MIPS_24KEc processor.
Another alternative worthy considering would be plain adoption of an embedded Linux distribution such as
OpenWRT, see for instance their
useful explanation of the flash memory layout.
Support for the base mainboard was added with
this commit, and compatibility with the hardware should only improve with time, as it is officially adopted for further deployments of Ralink RT5350 "router on a chip" such as
this one.
Extra safety before flashing firmware from a completely different codebase would suggest first gaining access to the "
Das U-Boot" bootloader serial console (and before that, the Linux getty login which appears running in the standard firmware), by opening the device as illustrated
here, and connecting a TTL <-> 5V level converting serial cable
to the appropriate pins. One suitable adapter should be for instance the very common, USB-ready
CP2102 module, or clones.
[...continues...]
Cool tricks:
Access through SSH:
- SSH client for various Windows versions: try PuTTY;
- Konqueror "FISH" sshfs -lookalike access: just use fish://root@10.10.10.254/ address in the address bar;
- Automatic, incremental backup (Apple "Time Capsule" -alike functionality) via Back In Time, relying on the rsync protocol to reduce network traffic, SSH for data encryption during the transfer, and ext2 hard-links (as opposed to the cruder VFAT capabilities) to save disk space by only storing changes.
"Back In Time" and rsync in particular would also work, with the stock firmware and through the SAMBA interface, making the exported VFAT partition accessible on Linux via a CIFS mountpoint. This introduces however extra overhead.
Non-interactive, safe SSH access may be configured with
public key authentication.
Non-interactive
CIFS access from a Linux system to the default /dev/sda1 partition made available through the SMB daemon may be set up with the "/etc/fstab" line:
//10.10.10.254/WiFiDisk1_Volume1/ /mnt/M2M_Backup cifs credentials=/home/[USERNAME]/.smbcredentials,users,iocharset=utf8,sec=ntlm,noauto 0 0
... which points to the /home/[USERNAME]/.smbcredentials file (with Samba crentials in clear text!):
username=admin
password=[password chosen through the unit HTTP interface]
Package recompilation with enhanced CPU support:
Even without switching to a fully custom firmware via i.e. BuildRoot above, some critical Debian packages may easily be recompiled on the unit itself with optimization options better geared to the available hardware. Options:
-O3 -march=24kec -mdsp -mips32r2
... show some speedup.
It is sufficient to include the appropriate APT source line in /etc/apt/sources.list:
[...]
deb-src http://archive.debian.org/debian-archive/debian/ lenny main contrib non-free
[...]
... getting sources (and build dependencies) via:
apt-get source [packagename]
The package compilation flags may be changed within its debian/rules file, and a customized version prepared via:
dpkg-buildpackage -b
therefore installing the resulting pacakgename.deb via:
dpkg -i packagename.deb
...as usual. Since the "Lenny" distribution suggested above is in "Archived" status, there will not be later package revisions from the central repository to overcome local preferences.
Alternatively, the more complete apt-build solution may also be used.
Even with these optimization, the choking point in network backups appears to be the SSH server. The embedded CPU is just not powerful enough to handle on-the-fly strong encryption at the speed otherwise supported by the WiFi network, or the local hard-disk.
Using "Back in Time" CLI interface shows:
$ backintime --benchmark-cipher 20
Back In Time
Version: 1.0.34
Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.
create random data file
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 1.61118 s, 13.0 MB/s
- 3des-cbc: 20MB 585.1KB/s 00:35
20MB 640.0KB/s 00:32
- aes128-cbc: 20MB 1.1MB/s 00:19
20MB 1.1MB/s 00:18
- aes256-cbc: 20MB 1.1MB/s 00:18
20MB 1.1MB/s 00:19
- blowfish-cbc: 20MB 1.1MB/s 00:18
20MB 1.1MB/s 00:18
... while the plain, unencrypted rsync protocol could be used by launching the utility in daemon mode with:
$ rsync --daemon --no-detach --port=2000 -vvv --config=rsyncd.conf
and benchmarked from the client with:
- Highest compression, no SSH:
$ time rsync -vvv -az --compress-level=9 /tmp/re rsync://m2m:2000/pub
sent 20,983,554 bytes received 34 bytes 1,678,687.04 bytes/sec
total size is 20,971,520 speedup is 1.00
[sender] _exit_cleanup(code=0, file=main.c, line=1183): about to call exit(0)
real 0m11.875s
- No compression, no SSH:
$ time rsync -vvv -a /tmp/re rsync://m2m:2000/pub
sent 20,976,730 bytes received 34 bytes 1,824,066.43 bytes/sec
total size is 20,971,520 speedup is 1.00
[sender] _exit_cleanup(code=0, file=main.c, line=1183): about to call exit(0)
real 0m11.107s
- No compression, SSH (twice as slow!):
$ time rsync -vvv -e ssh /tmp/re m2m:/tmp/testrsync
sent 20,976,707 bytes received 437 bytes 975,681.12 bytes/sec
total size is 20,971,520 speedup is 1.00
[sender] _exit_cleanup(code=0, file=main.c, line=1183): about to call exit(0)
real 0m21.642s
[continues...]
