The Clarity Ensemble phone is an Android-based captioning land-line phone. The newest model has an 8" touchscreen. Older model has 7" touchscreen. It comes with an app that runs at startup and keeps you from gaining access to the Android home screen or any other Android apps or settings. While booting up you momentarily see the time and can pull down to touch on Settings and bring up the regular Android settings but very soon as the boot process continues the splash screen and later the ThorB app will take over the screen.
In order to telnet to the device, you first need to start telnetd running on the Ensemble. This can be done by configuring your computer to appear to the Ensemble to be the update server. I directly connected the phone to a laptop Ethernet port. On the laptop, I installed a DHCP server, a DNS server, and a web server. I am running Windows and I used "DHCP Server for Windows" version 2.5.1, ApateDNS, and WWebserver with PHP 5.4.45. I set the laptop to a static IP of 8.8.4.4 since Wireshark revealed that the Ensemble was using that as the DNS server. I set ApateDNS server to return 8.8.4.4 as the IP address for all queries.
In my htdocs folder, I created a directory called thorbfota and inside that a directory called purple_prod. Inside purple_prod I created three files, download_file.php, query_site.php, and query_versions.php.
Code:
<?php
//download_file.php
ignore_user_abort(true);
set_time_limit(0);
//Replace with actual path to your files
$path = "C:/Users/User/Documents/ClarityEnsembleFiles/";
$dl_file = preg_replace("([^\w\s\d\-_~,;:\[\]\(\).]|[\.]{2,})", '', $_GET['filename']);
$dl_file = filter_var($dl_file, FILTER_SANITIZE_URL);
$fullPath = $path.$dl_file;
if ($fd = fopen ($fullPath, "r")) {
$fsize = filesize($fullPath);
$path_parts = pathinfo($fullPath);
$ext = strtolower($path_parts["extension"]);
switch ($ext) {
case "pdf":
header("Content-type: application/pdf");
header("Content-Disposition: attachment; filename=\"".$path_parts["basename"]."\"");
break;
case "bin":
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=\"".$path_parts["basename"]."\"");
break;
case "zip":
header("Content-type: application/zip");
header("Content-Disposition: attachment; filename=\"".$path_parts["basename"]."\"");
break;
case "apk":
header("Content-type: application/vnd.android.package-archive");
header("Content-Disposition: attachment; filename=\"".$path_parts["basename"]."\"");
break;
//Add more headers for other content types here
default;
header("Content-type: application/octet-stream");
header("Content-Disposition: filename=\"".$path_parts["basename"]."\"");
break;
}
header("Content-length: $fsize");
header("Cache-control: private");
while(!feof($fd)) {
$buffer = fread($fd, 2048);
echo $buffer;
}
}
fclose ($fd);
exit;
Code:
<?php
//query_site.php
//This forum would not allow me to post links since this is my first post.
//Feel free to move the "h" below right up against the "ttp..."
echo "h" . "ttp://clarityengineering.us/thorbfota/purple_prod/";
?>
Code:
<?php
//query_versions.php
//Replace with actual path to your files
$path = "C:/Users/User/Documents/ClarityEnsembleFiles/";
//Replace file versions with your current version numbers
//To cause phone to update a file, use a number larger that the current version
$file = "ThorB.apk";
$file_ver = "2.63";
$fullPath = $path.$file;
echo $file . "=" . $file_ver . "," . strtoupper(md5_file($fullPath)) . "|\r";
$file = "thorb-ota.zip";
$file_ver = "20150305.182516";
$fullPath = $path.$file;
echo $file . "=" . $file_ver . "," . strtoupper(md5_file($fullPath)) . "|\r";
$file = "dcx.bin";
$file_ver = "b033";
$fullPath = $path.$file;
echo $file . "=" . $file_ver . "," . strtoupper(md5_file($fullPath)) . "|\r";
$file = "eep.bin";
$file_ver = "be25";
$fullPath = $path.$file;
echo $file . "=" . $file_ver . "," . strtoupper(md5_file($fullPath)) . "|\n";
echo "survey=0,0|";
?>
I found that eep.bin was actually just a shell script that is downloaded to the device and run as root. I put my update files in "C:\Users\User\Documents\ClarityEnsembleFiles" but you can put them anywhere you like, just make sure to update the php files above to reflect their location. So far I have only used eep.bin but to keep my php script happy I also created placeholder files, dcx.bin, thorb-ota.zip, and ThorB.apk and placed them with eep.bin in my ClarityEnsembleFiles folder. Below is my eep.bin that starts telnet and simulates pressing the Home button. Just touch "Home Sample" when the "Complete action using" window pops up on the Ensemble. The semicolon at the end of the line avoids having the carriage return kill the command. Alternatively, you could run dos2unix on the eep.bin file and not need the semicolon at the line end.
Code:
#eep.bin
telnetd -l /system/bin/sh;am start -a android.intent.action.MAIN -c android.intent.category.HOME;
Every time you change the eep.bin file and want to run it on the phone make sure to close the Software upgrade screen and touch "Check now" button and then "Upgrade" button.
To install apps on the phone, first download the apk file to the phone with wget and then run "pm install -r YourApp.apk".
I have not found a physical Home, Back or Menu button on the phone so one of the first things you may want to install is a software solution for those. I installed "To Home" and it didn't work when configured with the root option for "Floating Buttons". It works fine when configured with the non-root option for "Floating Buttons". I have not tried any of the several other soft button apps available.
There is a 14 pin connector on the underside of the phone that presumably is used in the factory to connect to a dock for programming. I have not investigated the function of any of the pins but I suspect USB is there as well as possibly serial port(s) and maybe JTAG.
Before connecting the phone to the internet, you probably will want to either disable/uninstall the ThorB.apk app or create a firewall on the phone or on your router to keep it from being able to automatically update and from being able to report back to it's maker.
Besides being available for purchase, the phone is also available from ClearCaptions at no charge if you provide them with a 3rd party certification of being hard of hearing.
As far as using the phone, "Federal law prohibits anyone but registered users with hearing loss from using this device with the captions on." So if your hearing it fine, make sure to turn captions off or don't turn them on.
Telnet is great but I wanted a more secure connection to the phone so I set up an Android cross-compiler and compiled the latest version of dropbear (dropbear-2016.73). I don't have a 64-bit computer so in order to use the latest version of the Android toolchain, I had to boot into Windows and install Cygwin.
Thanks to serasihay for patches to an earlier version of dropbear. I adapted them to work with the latest version of dropbear. The patch can be found by searching dropbear-2016.73-android-20160427.patch on pastebin. Most of the warnings generated during compile were from pre-patched dropbear code and can be viewed on pastebin by searching for "Compile warnings for compiling dropbear-2016.73.android"
After setting up the toolchain, dropbear can be compiled with the following commands:
Code:
tar jxf dropbear-2016.73.tar.bz2
cd dropbear-2016.73
patch -p1 < /path/to/patch/dropbear-2016.73-android-20160426.patch
./configure --build=x86-windows --host=arm-linux-androideabi --disable-zlib --disable-largefile --disable-loginfunc --disable-shadow --disable-utmp --disable-utmpx --disable-wtmp --disable-wtmpx --disable-pututline --disable-pututxline --disable-lastlog
make MULTI=1 SCPPROGRESS=1 PROGRAMS="dropbear dropbearkey scp dbclient"
arm-linux-androideabi-strip.exe dropbearmulti
This generates a single binary file, dropbearmulti which you will want to copy to the phone to /system/xbin/dropbearmulti. Next, you will want to create symbolic links like this:
Code:
cd /system/xbin
ln -s dropbearmulti dbclient
ln -s dropbearmulti dropbear
ln -s dropbearmulti dropbearkey
ln -s dropbearmulti scp
I should probably redo the patch to enable the -R option to create the host keys but for now you can do it with:
Code:
mkdir /etc/dropbear
dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key
dropbearkey -t ecdsa -f /etc/dropbear/dropbear_ecdsa_host_key
dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key
To start dropbear every time the phone boots, I put my startup command in /system/etc/install-recovery.sh since it is called by init.rc. I would have put it straight in init.rc but init.rc is recreated from boot.img every boot and I didn't feel like getting into changing boot.img yet. Just make sure to make install-recovery.sh executable. The following line is what I use to start dropbear:
Code:
dropbear -A -N root -R /data/.ssh/authorized_keys -U 0 -G 0
Next you will need to copy your public key(s) into /data/.ssh/authorized_keys. You should now be able to ssh to your Clarity Ensemble phone. You can also use scp to copy files to and from the phone. If you use Putty pscp to transfer files, make sure to use the -scp option to force SCP protocol. If not, you will get the error "/usr/libexec/sftp-server: not found" since pscp tries to use sftp which is not installed on the phone.
So can you post a video or pics of what the device screen looks like now? can you actually use the device as a tablet?
Native ARM/static Linux binaries
(for all ARMv7+ compatible platforms)
Open-source Linux binaries that are either not available on Android (e.g. in Termux)
or make sense to be statically compiled (e.g. to run in TWRP/recovery for data recovery).
These are root tools and might damage your device severely. Use at your own risk. I take no responsibility whatsoever. If in doubt don't use them.
Minimum CPU: ARMv7/vfpv3-d16. Compiled against musl-libc/Android Kernel 3.4. Binaries are static, bionic/libc independent and should run on Android, TWRP, emulator or any other compatible ARM device. Musl is patched (info)(info2)(patch file: patch -p0 -u -b -i musl-android-smp.patch) to iterate CPU cores by /proc/stat instead of _SC_NPROCESSORS_CONF/sched_getaffinity to prevent false detection due to ARM cpu core powersaving (permanently turning cores on/off). This should report CPU cores more reliably to multithreading apps.
Example instructions how to build EncFS can be found here.
Some Cryptsetup compile recipes are here.
Changelog:
20190923 - f2fs-tools added
20190915 - dislocker, ntfs-3g, mount.exfat-fuse added
20190910 - VeraCrypt v1.24-b5 added
20191215 - musl smp patch added
20191224 - hstr v2.2.0 updated
20191225 - Testdisk, PhotoRec v7.2-wip-dec2019 updated
20200103 - tar v1.32 updated (with selinux, acl, xattr support)
20200513 - Cryptsetup v2.3.2 added
20200518 - fscrypt 0.2.7, strace56(aarch64) added
20200525 - p7zip v17.01 added
20200603 - parted v3.3 added
20200606 - fxz v1.1.0alpha added
20201212 - ddrescue v1.25 added
20201212 - Cryptsetup v2.3.4 updated
20210113 - f2fs-tools updated to v1.14.0
20210125 - Several tools compiled by @Borovets. See 'Misc' tools.
20210413 - Cryptsetup v2.3.5 updated
20210916 - Cryptsetup v2.4.1 updated. Thx to @misterhsp.
20211108 - rsync v3.2.3 updated
20211118 - Cryptsetup v2.4.2 updated. Thx to @misterhsp.
20220103 - mmc-utils added
20220106 - More tools from @Borovets. See spoiler.
Spoiler
bash-5.1.16-[1]-[2022.01.05].tar.gz
openssl3-3.0.1-[2021.12.14]-static.tar.gz
tree-2.0.0-[2021.12.23]-static.tar.gz
e2fsprogs-1.46.5-[2021.12.31]-static.tar.gz
openssl-1.1.1-m-[2021.12.15]-static.tar.gz
libsqlite-3.37.1-[2021.12.30]-static.tar.gz
ldns-host-1.7.1-[2021.12.30]-static.tar.gz
bootimg-info-2.0-[2021.12.18]-static.tar.gz
bc-5.2.1-[2021.12.29]-static.tar.gz
openssl3-tool-3.0.1-[2021.12.14]-static.tar.gz
openssl-tool-1.1.1-m-[2021.12.15]-static.tar.gz
sqlite-3.37.1-[2021.12.30]-static.tar.gz
stunnel-5.61-[2021.12.17]-static.tar.gz
toybox-0.8.6-borovets-295-applets-[2021.12.30]-static.tar.gz
unrar-6.10-beta-3-[2021.12.11]-static.tar.gz
zstd-1.5.1-[2021.12.22]-static.tar.gz
20220107 - parted v3.4 updated
20220113 - cryptsetup v2.4.3 updated. Thx to @misterhsp.
20220114 - gptfdisk v1.0.8 added
20220212 - tar v1.34 updated
20220622 - gptfdisk v1.0.9 (armv7) added
20220724 - dialog v1.3 added
20220728 - f2fs tools v1.15.0 updated
20220730 - cryptsetup v2.5.0 updated. Thx to @misterhsp.
20220806 - 7z-zstd v22.01 added. Thx to @xenosaur
20220910 - rsync v3.2.6 updated
20220913 - htop v3.2.1 added
20220913 - gocryptfs v2.3 updated. Thx to @misterhsp
20220922 - veracrypt v1.25.9 updated
20220924 - fdisk v2.38.1 and file v5.43 added
20221129 - cryptsetup v2.6.0 updated. Thx to @misterhsp
20221213 - f2fs tools v1.15.0 fixed (uuid.h missing)
20230215 - cryptsetup v2.6.1 updated. Thx to @misterhsp
20230307 - gocryptfs v2.3.1. Thx to @misterhsp
Data recovery tools:
- PhotoRec 7.2 - PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from hard disks, CD-ROMs, and lost pictures (thus the Photo Recovery name) from digital camera memory. PhotoRec ignores the file system and goes after the underlying data, so it will still work even if your media's file system has been severely damaged or reformatted.
- Testdisk 7.2 - Recover lost partitions and partition tables. For external sdcards. Never use it on internal mmc unless you know what you're doing.
- ext4magic 0.3.2 (with supplementary gnu date binary that can handle relative time like 'date -d "-20minutes" +%s')
- fidentity - A little utility sharing PhotoRec signature database. It identifies the type of data contained in a file and reports the extension as seen by PhotoRec.
- debugfs - Might be helpful on ext2 systems or other stuff.
- strace 4.20 - For debugging. Mainly to catch syslog messages (as Android has no traditional /dev/log buffer).
- strace 5.6 - For aarch64.
- ddrescue v1.25 - Data recovery tool for block devices with errors.
Compression tools:
p7zip v17.01 (fork) - (Download) A new p7zip fork with additional codecs and improvements
pixz - Parallel, indexed xz compressor
xz - Multicore aware version of xz/lzma (use --thread=0)
tar v1.32 - Tar provides the ability to create tar archives, as well as various other kinds of manipulation. Download below. More builds from @mirfatif here.
fxz - (Download) FXZ Utils is a fork of XZ Utils. It adds a multi-threaded radix match finder and optimized encoder.
Misc:
- hexcurse v1.60.0 - Hexcurse is a curses-base hex editing utility that can open, edit, and save files, editing both the hexadecimal and decimal values. 'ncurses' ui layout depends on TERM env variable. Change temporary with eg. 'TERM=xterm-256color hexcurse <file>'. See /system/etc/terminfo for possible terminals (xterm-256color, linux..).
- nethogs v0.8.5 - ncurse/nettop-like per-app separated speedmeter and traffic counter supporting high refresh rate. Try 'nethogs -d0' (speedmeter) or 'nethogs -v1' (traffic counter).
- rsync v3.2.3 - rsync is an open source utility that provides fast incremental file transfer. (--with-rsyncd-conf=/data/etc/rsyncd.conf)
- smbnetfs v0.6.1 - SMBNetFS is a Linux/FreeBSD filesystem that allow you to use samba/microsoft network in the same manner as the network neighborhood in Microsoft Windows. More info see below.
- progress v0.14 - Linux tool to show progress for cp, mv, dd, ... (formerly known as cv). Download here.
- archivemount (20180801) - A fuse filesystem for mounting archives in formats supported by libarchive. Download here.
- squashfuse v0.1.103 - FUSE filesystem to mount squashfs archives Download here.
- FuseISO - FuseISO is a FUSE module to mount ISO filesystem images (.iso, .nrg, .bin, .mdf and .img files). It currently support plain ISO9660 Level 1 and 2, Rock Ridge, Joliet, and zisofs. Download here.
- HSTR v2.2.0 - HSTR (HiSToRy) is a command line utility that brings improved Bash/zsh command completion from the history. It aims to make completion easier and more efficient than Ctrl-r. (If history is empty try setting HISTFILE in /system/etc/bash/bashrc e.g. export HISTFILE=/data/.bash_history).
- GNU screen, tmux - Thanks to @mirfatif.
- dislocker, ntfs-3g, mount.exfat-fuse - Thanks to @mirfatif.
- f2fs-tools - Thanks to @mirfatif. Update: v1.14.0 here.
- parted v3.3 - GNU Parted (the name being the conjunction of the two words PARTition and EDitor) is a free partition editor, used for creating and deleting partitions. Note: It might be useful to partition external sdcards (e.g. to limit adoptable storage). I do not recommend to use it on internal memory. It might brick your phone.
- Several tools compiled by @Borovets
Spoiler: Borovets tools
Borovets tools 2021.01.25
arptables-0.0.5-[2021.01.17]-static.zip
autoflushtest-1.0-[2021.01.14]-static.zip
btrfs-compsize-1.3-[build-2]-[2020.12.27].zip
btyacc-3.0-[2021.01.18]-static.zip
c-blosc-1.21.1-development-[2020.12.22].zip
c-blosc2-2.0.0-beta-6-development-[2020.04.21].zip
cabextract-1.9.1-[2021.01.08]-static.zip
compsize-1.3-[2021.01.07]-static.zip
convert-color-space-0.1-[2021.01.18]-static.zip
cpustat-0.02.13-[2021.01.13]-static.zip
doxygen-1.9.2-[2021.01.17]-static.zip
ed-1.17-[2021.01.11]-static.zip
hello-2.10-[2021.01.08]-static.zip
htop-3.0.5-[2021.01.13]-static.zip
ipcalc-ng-1.0.0-[2020.12.28]-static.zip
iw-5.9-[2021.01.08]-static.zip
libsqlite-3.34.1-[2021.01.20].zip
libtar-1.2.20-[2021.01.16]-static.zip
m5-1.0-[2020.12.31]-static.zip
sqlite-3.34.1-[2021.01.20]-static.zip
Borovets tools 2021.01.27
lcab-1.0-beta-12-[2021.01.17].zip
memdump-1.01-[2021.01.25].zip
memdumper-0.4-[2021.01.25].zip
memtester-4.5.0-[2021.01.09].zip
tcpdump-4.99.0-[libcap-1.9.1]-[2021.01.05].zip
wget2-1.99.2-[2020.12.12].zip
wolfssl-4.5.0-[2020.12.12].zip
xfsprogs-5.10.0-[2021.01.01].zip
Crypttools:
(These crypttools are mostly frontend tools for the main backend that resides in the kernel. If your kernel hasn't been configured accordingly at compile time you might not be able to use all features.)
Cryptsetup v2.3.5 - (Download) Cryptsetup is an utility used to conveniently setup disk encryption based on DMCrypt kernel module. These include plain dm-crypt volumes, LUKS volumes, loop-AES and TrueCrypt (including VeraCrypt extension) format.
eCryptfs-utils v111 - Frontend tools for the enterprise cryptographic filesystem for Linux. That's what Android/Google use for encryption. It's file-based (no container) and mounting can be automated by Termux widget. Needs shared libraries but is still portable. See notes below.
EncFS v1.9.5 - EncFS provides an encrypted filesystem in user-space. It runs in userspace, using the FUSE library for the filesystem interface.
gocryptfs - An encrypted overlay filesystem written in Go. Download here. Thanks to @mirfatif.
VeraCrypt - VeraCrypt is a free open source disk encryption software. Download here. Thanks to @mirfatif.
fscrypt 0.2.7 - (Download) fscrypt is a high-level tool for the management of Linux filesystem encryption. Needs at least kernel 4.1.
Crypttools info:
Cryptsetup:
General Notes:
- Features like TrueCrypt, VeraCrypt and LUKS2 need 'userspace crypto api' enabled in kernel. Most Android kernels are probably not configured for that and you have to recompile your kernel or contact your kernel maintainer. For kernel 3.4 you need this:
Code:
CONFIG_CRYPTO_USER=y
CONFIG_CRYPTO_USER_API=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CRYPTO_USER_API_SKCIPHER=y
- If 'cryptsetup benchmark' is incomplete and says 'userspace crypto api not available' you might be affected. You can still use LUKS1 though. A full benchmark looks like this:
Code:
# cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1 249186 iterations per second for 256-bit key
PBKDF2-sha256 327680 iterations per second for 256-bit key
PBKDF2-sha512 58829 iterations per second for 256-bit key
PBKDF2-ripemd160 227555 iterations per second for 256-bit key
PBKDF2-whirlpool 33539 iterations per second for 256-bit key
argon2i 4 iterations, 208288 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
argon2id 4 iterations, 207817 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 77.8 MiB/s 88.4 MiB/s
serpent-cbc 128b N/A N/A
twofish-cbc 128b 58.5 MiB/s 61.9 MiB/s
aes-cbc 256b 61.5 MiB/s 68.4 MiB/s
serpent-cbc 256b N/A N/A
twofish-cbc 256b 58.5 MiB/s 61.8 MiB/s
aes-xts 256b 95.1 MiB/s 86.9 MiB/s
serpent-xts 256b N/A N/A
twofish-xts 256b 60.0 MiB/s 61.8 MiB/s
aes-xts 512b 74.1 MiB/s 67.2 MiB/s
serpent-xts 512b N/A N/A
twofish-xts 512b 60.3 MiB/s 62.0 MiB/s
LUKS:
Code:
** 10MB test image (luks.img) **
dd if=/dev/zero of=luks.img bs=1M count 10M
cryptsetup luksFormat luks.img
cryptsetup open luks.img myluks
mke2fs -t ext4 /dev/mapper/myluks
mkdir luks
mount /dev/mapper/myluks luks
** luks folder is ready here **
umount luks
cryptsetup close myluks
- If standard luksFormat cipher (aes-xts-plain64) doesn't work (not supported by your kernel) you can try one of the more compatible ciphers:
Code:
cryptsetup luksFormat -c cbc-essiv:sha256 luks.img myluks
cryptsetup luksFormat -c aes-plain luks.img myluks
- For LUKS2 (experimental) use:
Code:
cryptsetup luksFormat --type luks2 luks.img
- Use "cryptsetup -v --debug" for more verbose output (debugging). In case of errors.
Veracrypt:
Code:
cryptsetup open --type tcrypt --veracrypt veracrypt.tc myvera
cryptsetup status myvera
mkdir /data/myvera
mount /dev/mapper/myvera /data/myvera
umount /data/myvera
cryptsetup close myvera
- Use container from desktop system (created with real Veracrypt)
- "veracrypt.tc" is the veracrypt container name
- "myvera" is an arbitrary name (handle)
- Use "cryptsetup -v --debug" for more verbose output (debugging). In case of errors.
eCryptfs-utils:
General Notes:
These tools are not built statically as they explicitly rely on 'dlopen' (plugin system). Instead they are compiled with relative rpaths (./libs). That means dependencies (libraries in subfolders) must be present in the binaries folder and you have to be in the binaries folder itself (with 'cd') before invoking any binary. By this the binaries are still portable (system independent) as long as the subfolders are present. I've put the files into a tar.gz archive so permissions should be set +x already. Extract the archive into /data/local/bin for 'Example' below.
Code:
mkdir -p /data/local/bin
cd /data/local/bin
tar xf crypttools.armv7.20180204.tar.gz
cd ecryptfs
./ecryptfs-stat --help
More info: ArchLinux Wiki
Example:
Tested on /sdcard based on FUSE filesystem. sdcardfs untested. Might need selinux permissive.
We create a folder /sdcard/pics that can be enabled (files present) or disabled (no files present) by a click on a widget button (Termux script) and entering our password. The encryption is done on a per-file basis. The actual files are stored encrypted in /sdcard/efs/pics.
- You might need SuperSU or Magisk Superuser for 'su -mm'. That makes sure that all apps can see the mounted folder (mount namespace separation).
- Busybox needed
- Install Termux and Termux:Widget from F-Droid or Playstore
- Start it and enter:
Code:
pkg upgrade
pkg install tsu
exit
- Create script /data/data/com.termux/files/home/.shortcuts/efs-pics.sh and make sure permissions(700) and owner (take from parent folder) are correct.
Code:
#!/system/xbin/bash
su -mm -c "/system/xbin/bash -c /data/local/scripts/$(basename "$0")"
- Create script /data/local/scripts/efs-pics.sh (770/root):
Code:
#!/system/xbin/bash
set -e
PATH=$PATH:/data/data/com.termux/files/usr/bin
# Necessary because rpaths are relative
cd /data/local/bin/ecryptfs
# /data/myefskey contains the salted key.
# Don't forget to make a backup.
# Without it encrypted data is lost.
function enter_passphrase {
read -p "Enter passphrase: " passphrase
sig=$(printf "%s" "$passphrase" | ./ecryptfs-insert-wrapped-passphrase-into-keyring /data/myefskey -) || exit
sig=$(echo $sig | cut -d "[" -f2 | cut -d "]" -f1)
}
CPATH1="/data/media/0/efs/pics"
CPATH2="/data/media/0/pics"
if ! mountpoint -q ${CPATH2}; then
enter_passphrase
echo ""
mount -t ecryptfs -o ecryptfs_sig=$sig,ecryptfs_fnek_sig=$sig,ecryptfs_cipher=aes,ecryptfs_key_bytes=16 ${CPATH1} ${CPATH2} || (echo "$(basename "$0") mount failed!"; exit)
./keyctl clear @u
echo "$(basename "$0") mount successful! :)"
else
umount ${CPATH2} || (echo "$(basename "$0") umount error $? :("; exit)
echo "$(basename "$0") umount successful :)"
fi
# uncomment to force-close Termux window
# killall com.termux
- If your rom uses encryption already (/data/data) beware the './keyctl clear @u' command. It might flush *all* keys in the kernel including the Android encryption one (i'm not sure). This might lead to unpredicted behavior. Either comment it out (then your once injected key remains in the kernel keystore and someone could simply remount your folder without passphrase) or make yourself familiar with the keyctl command and handle it yourself. My phone is not encrypted so i cannot help here.
- Create random keyfile (/data/myefskey) and wrap it with passphrase. This might need 1-2 minutes depending on your devices entropy pool (/dev/random). Backup this key (/data/myefskey). Without it your encrypted data is lost. And don't forget the trailing '-' (minus) at the end of the line, it's important.
Code:
cd /data/local/bin/ecryptfs
read -p "Enter passphrase: " passphrase; printf "%s\n%s" $(busybox od -x -N 100 --width=30 /dev/random | head -n 1 | busybox sed "s/^0000000//" | busybox sed "s/[[:space:]]*//g") "${passphrase}" | ./ecryptfs-wrap-passphrase /data/myefskey -
- Create folders:
Code:
mkdir -p /sdcard/efs/pics /sdcard/pics
- Create Widget (Termux) and select 'efs-pics.sh'.
- Start it and enter your passphrase (you used above). If everything goes well (it will tell you) you can place files into /sdcard/pics and scrambled files should come up in /sdcard/efs/pics. Never write into /sdcard/efs/pics directly.
- Activate widget again. /sdcard/pics should get emptied.
- Optional: You can set /data/media/0/efs/pics to 700/root so no one can access/see the encrypted data.
SMBNetFS info:
Note: The library paths are relative. You need to be in the folder (with 'cd') to spawn the executable (./smbnetfs). You can extract the archive wherever you want though as far as the file/folder structure remains intact.
Example:
Code:
mount -o remount,rw /
mkdir -p /data/local/bin /mnt/cifs
mount -o remount,ro /
tar xf smbnetfs.tar.gz -C /data/local/bin
cd /data/local/bin/smbnetfs
export HOME=/data/local/bin/smbnetfs/home
* enter your smb credentials into smbnetfs/home/.smb/smbnetfs.auth (eg. auth "192.168.1.2" "${user}" "${pass}")
./smbnetfs /mnt/cifs
cd /mnt/cifs/192.168.1.2/${share}
I think it usually should list the samba environment in /mnt/cifs but i'm not sure which prerequisites are necessary for that (edit: maybe it needs real workgroup/hostname instead of IPs). If nothing comes up this should work:
The folder 192.168.1.2/${share} is unreachable by Androids folder picker (unless you can enter the path manually). So either pre-create the folder structure beforehand (mkdir -p /mnt/cifs/192.168.1.2/${share}) and add/register the folder to your app by folder picker (eg. MXPlayer) and then overmount that with the actual ${share}. Or bindmount the folder:
Code:
mount --bind /mnt/cifs/192.168.1.2/${share} /mnt/cifs2
Edit: Another option is to let smbnetfs create a static link (actually a symbolic link) to the share in the mountpoint root (/mnt/cifs). Its not as robust as the bindmount though. MXPlayer doesn't find any files (even though the folder picker shows the folders properly).
Code:
echo "link myfiles 192.168.1.2/${share}" > /data/local/bin/smbnetfs/home/.smb/smbnetfs.host
chmod 700 /data/local/bin/smbnetfs/home/.smb/smbnetfs.host
I've noticed that MXPlayer shows the samba folders just for a glimpse of a second. But if you enter one of the local folders and then go back all samba folders are there. Not sure why this is happening or maybe its just my system.
Edit2: Not yet tested but.. check the permissions. Its possible that SMBNetFS mounts with 755 or something. That's inaccessible for Android apps. Try this:
Code:
./smbnetfs -o umask=000,noatime,noexec,nodev,nosuid /mnt/cifs
Samba 4.8.3 configuration:
Code:
_idmap_modules=idmap_rid,idmap_hash,idmap_tdb2
_pdb_modules=pdb_tdbsam,pdb_smbpasswd,pdb_wbc_sam,pdb_samba4
_auth_modules=auth_unix,auth_wbc,auth_server,auth_netlogond,auth_script,auth_samba4
waf configure --prefix=/tmp/myout \
-C \
--sysconfdir=./conf/etc/samba \
--with-configdir=./conf/etc/samba \
--localstatedir=./conf/var \
--libexecdir=./conf/usr/lib \
--enable-fhs \
--with-lockdir=./conf/var/cache/samba \
--with-piddir=./conf/run/samba \
--with-logfilebase=./conf/var/log/samba \
--without-pam \
--without-systemd \
--without-ads \
--with-shared-modules=$_idmap_modules,$_pdb_modules,$_auth_modules \
--disable-cups \
--without-gettext \
--bundled-libraries=NONE,com_err,ldb,uid_wrapper,resolv_wrapper,socket_wrapper,nss_wrapper,ntdb,roken,wind,hx509,asn1,heimbase,hcrypto,krb5,gssapi,heimntlm,hdb,kdc,cmocka,talloc,tdb,pytdb,ldb,pyldb,tevent,pytevent \
--disable-rpath-install \
--disable-python --without-ad-dc --without-acl-support --without-ldap \
--hostcc=/usr/bin/gcc \
--cross-compile --cross-execute='qemu-arm -L /media/devpart/qemu/root'
waf build -j4
waf install
Compression tools added.
Next are crypttools (ecryptfs-utils, cryptsetup).
DualJoe said:
Compression tools added.
Next are crypttools (ecryptfs-utils, cryptsetup).
Click to expand...
Click to collapse
Please add ecryptfs-simple
xyne.archlinux.ca/projects/ecryptfs-simple
Thanks.
Ecryptfs-simple is not POSIX compliant. It relies on an argv interface (to parse command-line parameters) that is a GNU extension that musl doesn't support.
The original eCryptFS is simple enough anyway (and its the upstream project). I will provide a quickstart example and a one button GUI controlled solution (Termux widget) to handle it.
Please to add gifsicle,
http://github.com/kohler/gifsicle
Thanks.
I only have gifsicle. The other ones are too complex for my setup atm.
DualJoe said:
I only have gifsicle. The other ones are too complex for my setup atm.
Click to expand...
Click to collapse
Thank you very much.
Please help me again to build giflossy (fork of gifsicle).
I really need it to compress (--lossy=N) the Gif file to be smaller.
https://github.com/kornelski/giflossy
Thanks.
Do you use them directly on your phone for web postings or something? What's your use case to not prefer a desktop system for this?
DualJoe said:
Do you use them directly on your phone for web postings or something? What's your use case to not prefer a desktop system for this?
Click to expand...
Click to collapse
I use it directly on the phone, for learning purposes.
Using it on the phone is so handy that it can be easily used anywhere.
Thanks.
Please help me again to build lbzip2
http://lbzip2.org/
Thanks.
Here it is.
DualJoe said:
Compression tools added.
Next are crypttools (ecryptfs-utils, cryptsetup).
Click to expand...
Click to collapse
When will Crypttools be released.
I've waited for the major update of cryptsetup. Its out now indeed. I should get it up this week.
Crypttools and quickstart tutorials added.
Mountpoint is not writable (eCryptfs)
DualJoe said:
Crypttools and quickstart tutorials added.
Click to expand...
Click to collapse
Can't write to mountpoint.
# touch /sdcard/pics/test
touch: /sdcard/pics/test: Permission denied
# cp file /sdcard/pics
cp: can't create '/sdcard/pics/file': Permission denied
buengeut said:
touch: /sdcard/pics/test: Permission denied
Click to expand...
Click to collapse
What are your permissions?
Code:
# stat /data/media/0/pics
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
# stat /data/media/0/efs
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
# stat /data/media/0/efs/pics
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
How does your mount look like?
Code:
# mount |grep pics
/data/media/0/efs/pics on /data/media/0/pics type ecryptfs (rw,relatime,ecryptfs_fnek_sig=56b1f3c519fb3412,ecryptfs_sig=56b1f3c519fb3412,ecryptfs_cipher=aes,ecryptfs_key_bytes=16)
Is /sdcard linked?
Code:
# ls -l /sdcard
lrwxrwxrwx 1 root root 21 May 10 1973 /sdcard -> /storage/self/primary
What Android version and kernel do you have?
DualJoe said:
What are your permissions?
Code:
# stat /data/media/0/pics
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
# stat /data/media/0/efs
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
# stat /data/media/0/efs/pics
Access: (775/drwxrwxr-x) Uid: (1023/media_rw) Gid: (1023/media_rw)
How does your mount look like?
Code:
# mount |grep pics
/data/media/0/efs/pics on /data/media/0/pics type ecryptfs (rw,relatime,ecryptfs_fnek_sig=56b1f3c519fb3412,ecryptfs_sig=56b1f3c519fb3412,ecryptfs_cipher=aes,ecryptfs_key_bytes=16)
Is /sdcard linked?
Code:
# ls -l /sdcard
lrwxrwxrwx 1 root root 21 May 10 1973 /sdcard -> /storage/self/primary
What Android version and kernel do you have?
Click to expand...
Click to collapse
Android 6.0 kernel 3.18.14
/sdcard is symlink to /mnt/sdcard, i changed /sdcard to /mnt/sdcard
Code:
# mount -t ecryptfs
/mnt/sdcard/efs/pics on /mnt/sdcard/pics type ecryptfs (rw,relatime,ecryptfs_fnek_sig=1b77138d91206e66,ecryptfs_sig=1b77138d91206e66,ecryptfs_cipher=aes,ecryptfs_key_bytes=16)
Code:
# stat /mnt/sdcard/pics
Access: (775/drwxrwxr-x) Uid: (1000/ system) Gid: (1015/sdcard_rw)
# stat /mnt/sdcard/efs
Access: (775/drwxrwxr-x) Uid: (1000/ system) Gid: (1015/sdcard_rw)
# stat /mnt/sdcard/efs/pics
Access: (775/drwxrwxr-x) Uid: (1000/ system) Gid: (1015/sdcard_rw)
Code:
# touch /mnt/sdcard/pics/test
touch: /mnt/sdcard/pics/test: Permission denied
What about the permissions of /data/media/0 folders? That's the most important part.
If your sdcard is not at /data/media/0 you probably don't have a multiuser environment (older phone?) and /mnt/sdcard is probably a real partition. This is early Kitkat partition layout (/sdcard and /data have separate partitions). On later systems both are on /data partition and /sdcard is abstracted by a FUSE file system that would automatically set the proper permissions whenever you write something to it (even as root).
In case you are on an old layout you would need to set proper permissions to /sdcard/pics and /sdcard/efs yourself. Just take a look at the other folders with 'ls -l /mnt/sdcard' and set accordingly. You would also need to change /data/media/0 to /mnt/sdcard in the script.
What do you get with this?
Code:
# mount |grep sdcard
# mount |grep storage
What phone is it? Kernel 3.18 doesn't sound all too old.
Edit: Another theory is your internal sdcard is scardfs or something. If so, it might break "stacking" folders (mount over). Try to use /data/pics and /data/efs/pics as a test.
It works in Permissive mode (setenforce 0)
I need Busybox with SELinux-enabled and use it to set it to Permissive mode
Code:
# busybox getenforce
Enforcing
# busybox setenforce 0
# busybox getenforce
Permissive
And then execute the efs-pics.sh and test it
Code:
# cp file /mnt/sdcard/pics ; echo $?
[b]0[/b]
# ls /mnt/sdcard/pics
[b]file[/b]
Horreee.... it Works.
It's surprising that a bunch of people are interested in this article. But I have to say that some processes I mentioned in this article just happened to work. I don't necessarily understand why they work, which also means they probably cannot be generalized to any devices. Google is making security patches to every Android releases, which break some procedures in this article, including the vdc command. For anyone who wants to recover data from an encrypted device, I'm afraid you might have to do your own research such as reading the AOSP source code, because I havn't been following the changes in AOSP since this article was written, and sorry I cannot provide useful information. Finally I hope this article is helpful in some way and good luck.
-----------------------------------------------------------------------------------------------------------
About a year ago I encountered such a problem: https://forum.xda-developers.com/mate-9/help/mate-9-how-to-decrypt-fbe-encrypted-t3735545
To summarize, I flashed a newer ROM with the file encryption enabled, while I already had my phone decrypted (userdata was not encrypted). I forced rebooted my phone when it was booting and encrypting my files. As a consequence, I couldn't unlock my phone or access the encrypted files.
This guide is about how to retrieve these encrypted files.
Requirements:
Device: Huawei Mate 9 MHA-AL00
ROM: EMUI 5.0.1 B233
(This guide might also work on other Huawei devices or other EMUI 5.)
Please make sure that your device is "decrypted" (i.e. boot without "fileencryptioninline" option), rooted, has busybox installed and avaliable for using.
A complete userdata partition image that you need to decrypt. Usually you can make this image in TWRP using dd command.
(In this guide, you are supposed to dump the encrypted userdata partition and flash another usable system)
A terminal APP (like juicessh) to execute commands. Alternatively, you can use adb shell.
Backup your current data, just in case.
A linux system (I'm using ubuntu 1804 as example).
(hopefully) you are familiar with some linux commands.
Notes:
If you encounter such situation (I described at the beginning), you can probably try to fix the system first (for example, make a backup and delete /data/system/gatekeeper.password.key and /data/system/gatekeeper.pattern.key to disable lock screen password).
Encryption option:
There is a mount option called "fileencryptioninline" in fstab.hi3660 in rootfs of origin kernel, which could be recognized by init so that it can control whether to perform a file based encryption (FBE). Removing this option (or changing to "encryptable"?) can disable FBE (before userdata partition is encrypted).
How does FBE encrypt files:
Google has developed fscrypt in linux kernel to implement FBE. To use fscrypt, a key with description "fscrypt:xxxxx" should be added to kernel, where "xxxxx" is a 8 byte value in HEX format. This value is unique and used to identify encryption policy. f2fs can retrieve this key. If we want to encrypt some files, we use ioctl to set an encryption policy (which is the 8 byte value) to an empty directory. The files copied to it will be encrypted by the cooresponding key. If we access an encrypted file, f2fs will get its encryption policy and find the key that matches this policy. This key will be used to decrypt the file. Multiple keys and policies are allowed.
In order to protect the fscrypt keys (I described above), FBE uses keymaster to encrypt and store them to userdata partition. A set of encrypted keys usually consists these files: "encrypted_key" "keymaster_key_blob" "secdiscardable" "stretching" "version". Keymaster is able to use them and communicate with a hardware based Trusty TEE to obtain the real key for fscrypt. The decryption is related to hardware so only the device which creates these keys are able to decrypt them.
FBE has at least 3 sets of keys. (assumed that you are user 0):
global device key (global DE): stores in /data/unencrypted/key.
Policy: all directories in /data other than "lost+found", "system_ce", "system_de", "misc_ce", "misc_de", "media", "data", "user", "user_de".
device key (DE): stores in /data/misc/vold/user_keys/de/0.
Policy: usually the directories ended with "_de".
candidate key (CE): stores in /data/misc/vold/user_keys/cd/0/current.
Policy: usually the directories ended with "_ce" and /data/data, /data/media/0.
Please note that CE and DE keys should have already been encrypted by Global DE key
vold (looks like volume daemon)
vold is the volume manager of android (and it runs as a daemon). It can be controlled by vdc (volume daemon control?). In the source code of vold there is a command listener which defines the avaliable commands of vdc. vold controls the key management of FBE.
Steps:
Create a keyring called "e4crypt".
Unfortunately, android does not have a tool to manipulate linux key-management facility. To add this keyring, I'm using "add_key" system call.
A sample program and sample operations:
Code:
#include <stdio.h>
#include <unistd.h>
#include <linux/keyctl.h>
#include <sys/syscall.h>
int main() {
int ret = syscall(__NR_add_key, "keyring", "e4crypt", NULL, 0, KEY_SPEC_USER_SESSION_KEYRING);
if (ret != -1) {
printf("Successfully created keyring \"e4crypt\"\n");
}
else {
perror("add_key");
}
return 0;
}
Compile this code and run it on your phone. You can use android NDK to compile it. (Actually I think any arm/arm64 toolchain for linux will work). I don't want to download a very large NDK so I'm using gcc-aarch64-linux-gnu.
On ubuntu 1804:
(Use "sudo apt install gcc-aarch64-linux-gnu" to install this toolchain)
Code:
[email protected]:~/add_keyring$ cat add_keyring.c
#include <stdio.h>
#include <linux/keyctl.h>
#include <sys/syscall.h>
int main() {
int ret = syscall(__NR_add_key, "keyring", "e4crypt", NULL, 0, KEY_SPEC_USER_SESSION_KEYRING);
if (ret != -1) {
printf("Successfully created keyring \"e4crypt\"\n");
}
else {
perror("add_key");
}
return 0;
}
[email protected]:~/add_keyring$ aarch64-linux-gnu-gcc -static add_keyring.c -o add_keyring
[email protected]:~/add_keyring$ ls
add_keyring add_keyring.c
On your phone (adb shell):
Assume that we've already placed the "add_keyring" executable binary in /data/add_keyring
Code:
HWMHA:/ $ su
HWMHA:/ # cd /data
HWMHA:/data # ls -l add_keyring
-rw-r--r-- 1 root root 546888 2018-12-29 23:31 add_keyring
HWMHA:/data # chmod +x add_keyring
HWMHA:/ # cat /proc/keys
032c30ec I--Q--- 1 perm 1f3f0000 0 65534 keyring _uid_ses.0: 1
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
HWMHA:/data # ./add_keyring
Successfully created keyring "e4crypt"
HWMHA:/data # cat /proc/keys
032c30ec I--Q--- 1 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: empty
HWMHA:/data #
This step is completed if you see a keyring called "e4crypt".
Mount the partition image to your device.
You can copy the image file to a USB storage device and use otg so that you can access the partition image on your phone (but it seems to be unstable). In this guide I'll mount a samba share which contains that image on my phone.
My operation logs:
On ubuntu:
I've created a samba share called "image" which points to the directory containing that partition. The partition image file is called "sdd46". The IP address of this computer is 192.168.1.120
On your phone:
Prepare the partition image (sdd46):
Code:
HWMHA:/ $
HWMHA:/ $ su
HWMHA:/ # mkdir /computer
mkdir: '/computer': Read-only file system
HWMHA:/ # busybox mount -o remount,rw /
HWMHA:/ # mkdir /computer
HWMHA:/ # busybox mount -t cifs -o nolock,username=nobody '\\192.168.1.120\image' /computer
HWMHA:/ # ls /computer
sdd46
Mount this image (the "force_no_inline_enc" option is required):
Code:
HWMHA:/ # mkdir /decrypt_data
HWMHA:/ # busybox mount -t f2fs -o ro,force_no_inline_enc /computer/sdd46 /decrypt_data
Check if your image is successfully mounted:
Code:
HWMHA:/ # ls /decrypt_data/
adb camera fusion_daemon_rpipe inv_ipld_wpipe mediadrm ramdump system update
anr cota fusion_daemon_wpipe ioloader misc resource-cache system_ce user
apkpush cust gps ivp misc_ce samba system_de user_de
app cust_ver.bin hcs keyie misc_de sec_storage_data t vsftpd
app-asec custom.bin hisi_logs libnfc-nxp.conf nfc security takess vsftpd.conf
app-ephemeral dalvik-cache hw_init light nvram share takess.sh
app-lib daniuc.dex hwzd_logs local offlinelogs skin themes
app-private data img log ota ss timetest
app_acc drm inotify lost+found pppd_via su.img tmp
backup encrypted_flag inputie lp product.bin suhide.img tombstones
bootchart fpie inv_ipld_rpipe media property supersu unencrypted
HWMHA:/ # cd /decrypt_data/
HWMHA:/decrypt_data # cd misc
misc/ misc_ce/ misc_de/
HWMHA:/decrypt_data # cd misc
HWMHA:/decrypt_data/misc # ls vold
yVsKT2+BrPIOKcQdVYyetC
You can see an encrypted directory in /decrypt_data/misc/vold, which stores the CE and DE keys. If you can't find this directory, it might not be encrypted and should located in /decrypt_data/unencrypted/data/misc/vold.
Install Global DE key:
You need to copy global DE key to /data/unencrypted and execute:
Code:
vdc --wait cryptfs enablefilecrypto
My operation logs:
Copy Global DE key to /data/unencrypted:
Code:
HWMHA:/ $ su
HWMHA:/ # cd /data
HWMHA:/data # mkdir unencrypted
HWMHA:/data # cd unencrypted
HWMHA:/data/unencrypted # ls
HWMHA:/data/unencrypted # cp -nr /decrypt_data/unencrypted/key ./
HWMHA:/data/unencrypted # ls
key
HWMHA:/data/unencrypted # ls key
encrypted_key keymaster_key_blob secdiscardable stretching version
encrypted_key.backup keymaster_key_blob.backup secdiscardable.backup stretching.backup version.backup
then, install this key:
Code:
HWMHA:/data/unencrypted # cat /proc/keys
032c30ec I--Q--- 1 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: empty
HWMHA:/data/unencrypted # vdc --wait cryptfs enablefilecrypto
200 3966 1
HWMHA:/data/unencrypted #
HWMHA:/data/unencrypted # cat /proc/keys
032c30ec I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: 1
258344d4 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:773e9f60adca3172: 72
You can see a new key "fscrypt:773e9f60adca3172" is added to kernel.
Check if you can access CE and DE keys and copy them to /data/misc/vold
My operation logs:
Check if you can access CE and DE keys:
Code:
HWMHA:/data/unencrypted # cd /decrypt_data/misc
HWMHA:/decrypt_data/misc # ls vold
user_keys
Copy CE and DE keys to the right location:
Code:
HWMHA:/decrypt_data/misc # cd /data/misc/vold
HWMHA:/data/misc/vold # ls
bench
HWMHA:/data/misc/vold # cp -nr /decrypt_data/misc/vold/user_keys .
HWMHA:/data/misc/vold # ls
bench user_keys
HWMHA:/data/misc/vold # cd user_keys
HWMHA:/data/misc/vold/user_keys # ls
ce de
HWMHA:/data/misc/vold/user_keys # cd de/0/
HWMHA:/data/misc/vold/user_keys/de/0 # cat version
1HWMHA:/data/misc/vold/user_keys/de/0 #
HWMHA:/data/misc/vold/user_keys/de/0 #
Install DE key
Just set ro.crypto.type to "file" and execute this command:
Code:
vdc --wait cryptfs init_user0
My operation logs:
Before installing DE key you will see some ecrypted files protected by it.
Code:
HWMHA:/decrypt_data/user_de/0 # ls
++gBT,VFvFeD,vgVoSVpUqeDNoC Wno64AdMq3Wde+F8LqWYvWiAFaFIiU810wX84B
+ExRFZKrTrX5PAZWjgzJKV26in24FxSt Ws+aoxf5sborLpV0EZLhvA
+Lyki0vu0dbWrX5PvAq3g932ONE WyHV8MQblZaCmdNpO6WPQSN1TgQoGxzw3mn4vB
+T95aXkKGnakajMwgSxcblTh0+8Vp3RI X0WDXQ5BsVDV6u45CJ9etzjba9JkWeQG
..............
Check the keys in kernel:
Code:
HWMHA:/decrypt_data # cat /proc/keys
032c30ec I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: 1
258344d4 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:773e9f60adca3172: 72
You'll need to set a property before installing the key.
Code:
HWMHA:/decrypt_data # getprop ro.crypto.type
HWMHA:/decrypt_data # setprop ro.crypto.type file
HWMHA:/decrypt_data # getprop ro.crypto.type
file
HWMHA:/decrypt_data # vdc --wait cryptfs init_user0
200 10711 Command succeeded
HWMHA:/decrypt_data # cat /proc/keys
032c30ec I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: 2
258344d4 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:773e9f60adca3172: 72
3c670371 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:521acd13c187513c: 72
HWMHA:/decrypt_data #
Check whether you can access the files protected by DE key.
Code:
HWMHA:/decrypt_data #
HWMHA:/decrypt_data # cd user_de/0
HWMHA:/decrypt_data/user_de/0 # ls
abcmeasurecorp.com.measureit com.huawei.bluetooth
androdns.android.leetdreams.ch.androdns com.huawei.ca
android com.huawei.camera
androidhwext com.huawei.compass
..............
Install CE key.
Just execute this command:
Code:
vdc --wait cryptfs unlock_user_key 0 0 "" ""
The last two arguments are empty strings.
My operation logs:
Before installing CE key you'll find some encrypted files protected by it.
Code:
HWMHA:/decrypt_data # cd media/0
HWMHA:/decrypt_data/media/0 # ls
0M8msgkIuhwegkVYqu2zvC OK0B0zzWFSQ5pDHwSlAIvA aNrURou98klfwIaGnFAdPA rHzJIvFcgtIcIz,WOjZrRD w59yxPZvec,eu9HMMdDpuB
7VDq++zOwS5xaV35TuZbmB WSWzdKdAYAC2Vc1jOs6tqA jz,xyRZMpSLq2ghtL158yA rokqUTbYC7eMhGrghh0CSB
8rMqWow5AXxsZqHqbZyN9C XyLh+kAVQ5ZWXlWrc7wc5D pjgHBo3uPcxDi13euKN4PB tZkWYvxkrEufTMZ47f89cD
Install the key:
Code:
HWMHA:/decrypt_data/user_de/0 #
HWMHA:/decrypt_data/user_de/0 # vdc --wait cryptfs unlock_user_key 0 0 "" ""
200 11848 Command succeeded
HWMHA:/decrypt_data/user_de/0 #
HWMHA:/decrypt_data/user_de/0 #
HWMHA:/decrypt_data/user_de/0 # cat /proc/keys
032c30ec I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid_ses.0: 2
0d64f2db I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
1db74fa6 I--Q--- 1 perm 3f010000 0 0 keyring e4crypt: 3
258344d4 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:773e9f60adca3172: 72
30baee27 I--Q--- 1 perm 3d010000 0 0 logon fscrypt:e1294ea7636feee7: 72
3c670371 I--Q--- 407 perm 3d010000 0 0 logon fscrypt:521acd13c187513c: 72
You can see a new key "fscrypt:521acd13c187513c" is added to kernel.
Check whether you can access the files protected by CE key.
Code:
HWMHA:/decrypt_data # cd media/0
HWMHA:/decrypt_data/media/0 # ls
Alarms Android DCIM Download Movies Music Notifications Pictures Podcasts Ringtones backups baidu huawei
Nou you should be able to access the encrypted files.
Clean up:
After you have backuped up the files you wish to retrieve, please delete the keys you copied to /data and reboot your phone. Don't change any security settings (like lock screen password) before rebooting.
Unmount partitions:
Code:
busybox umount /decrypt_data
busybox umount /computer
References and useful links:
Offical FBE doc:
https://source.android.com/security/encryption/file-based
Some FBE source code analysis:
https://blog.csdn.net/myfriend0/article/details/77094890 (Chinese)
https://github.com/novelinux/android/wiki/Android-FBE (Chinese)
http://hooltech.com/android-p-fbe.html (Chinese)
Hardware-backed Keystore:
https://source.android.com/security/keystore/index.html
Trusty TEE:
https://source.android.com/security/trusty/index.html
Something about Huawei's Trustzone:
https://github.com/OpenKirin/Documentation/blob/master/04-Trustzone.md
fscrypt:
https://www.kernel.org/doc/html/v4.15/filesystems/fscrypt.html
Make sure you always backup your data before performing any flashing/upgrading, especially when you are using a non-official ROM. I have spent a lot of time reading posts and analyzing source code. Luckily I succeeded. This was a lesson telling me the importance of backup.
Hi!
First of all, this is a truly amazing guide. The work done is incredible.
I am in a similar situation, although the details are different: my phone is a Oneplus 5, and it got bricked on Android Pie (stock OOS 9). I have an image of the userdata partition (all other partitions as well actually), it is FBE encoded, I used a PIN which I know. Some questions that you might be able to help with:
1) My userdata was encrypted on a stock ROM, bootloader locked (no root). Your guide obviously requires rooting. Can that even work? Would the phone have access to the necessary TEE functionality?
2) The first significant difference I run into is that vold is missing in the locations you suggested. All the directory names in /misc are encrypted. Any ideas?
3) Also, I do not see support for the "force_no_inline_enc" in busybox (or on Ubuntu). Could not find any documentation either. Could you explain what it does? Or provide some reference? The image mounts successfully without it in my case.
Thanks for this incredible guide. However my case is somehow different, I'm wondering whether you could give me some suggestion, thanks!
My phone got bricked after flashing a new ROM, thus I erased the /system and /data, expect my internal storage. After that, I can't decrypt my internal storage any more. The command 'twrp decrypt XXXXX' does not work, too.
So here I know my pin but can't decrypt, follow your guide, I can see /data/unencrypted folder, but /data/misc is not there, what can I do? Thanks.
Besides, running vdc always gives 'Segmentation fault', which I have no idea.
amk43 said:
Hi!
First of all, this is a truly amazing guide. The work done is incredible.
I am in a similar situation, although the details are different: my phone is a Oneplus 5, and it got bricked on Android Pie (stock OOS 9). I have an image of the userdata partition (all other partitions as well actually), it is FBE encoded, I used a PIN which I know. Some questions that you might be able to help with:
1) My userdata was encrypted on a stock ROM, bootloader locked (no root). Your guide obviously requires rooting. Can that even work? Would the phone have access to the necessary TEE functionality?
2) The first significant difference I run into is that vold is missing in the locations you suggested. All the directory names in /misc are encrypted. Any ideas?
3) Also, I do not see support for the "force_no_inline_enc" in busybox (or on Ubuntu). Could not find any documentation either. Could you explain what it does? Or provide some reference? The image mounts successfully without it in my case.
Click to expand...
Click to collapse
Hi @amk43,
1. I'm not sure whether this method will work, because the implementations of TEE are different. But I've known that some version of TWRP for Snapdragon 835 devices supports decrypting data partition (i.e. it allows users to enter password/pattern inside TWRP and then users can access the encrypted files). Based on this, I think the decryption is possible. I would suggest you to have a try using my steps or try to work with such kind of TWRP (https://forum.xda-developers.com/oneplus-5/development/recovery-twrp-3-2-3-pie-encryption-t3837342)
2. Have you installed "global device key" before seeing the encrypted directory names in /misc ? The encryption policy might be applied to the entire /misc so it looks different to my example. Another possible reason is, Android 9.0 has introduced Metadata encryption, which makes things more complicated. (https://source.android.com/security/encryption/metadata) I'm afraid the mechanism might have changed, since it would store the encryption key in another partition called "metadata". Check whether this partition exists first. If your device has enabled Metadata encryption, additional steps will be required.
3. I double checked the AOSP source code and didn't find this option. I think this option is introduced by HUAWEI and not available in other OS. Actually this option is inside this file: (https://github.com/Ante0/MHA-NG_EMUI5.0_opensource/blob/master/kernel/fs/f2fs/super.c) line 126.
So you probably don't need to use "force_no_inline_enc".
Finally, good luck with your files.
lkytal said:
Thanks for this incredible guide. However my case is somehow different, I'm wondering whether you could give me some suggestion, thanks!
My phone got bricked after flashing a new ROM, thus I erased the /system and /data, expect my internal storage. After that, I can't decrypt my internal storage any more. The command 'twrp decrypt XXXXX' does not work, too.
So here I know my pin but can't decrypt, follow your guide, I can see /data/unencrypted folder, but /data/misc is not there, what can I do? Thanks.
Besides, running vdc always gives 'Segmentation fault', which I have no idea.
Click to expand...
Click to collapse
Hi, @lkytal,
If you can't find /data/misc , it means you've lost your "CE key", which is used to decrypt internal storage (/data/media/0). Unfortunately, I think there is no way to get it back, unless you can recover deleted files, which is difficult and almost impossible.
I also have no idea why vdc crashed with 'Segmentation fault'.
hi,my system hadn't vdc,can i used compiled vdc ?
cofface said:
hi,my system hadn't vdc,can i used compiled vdc ?
Click to expand...
Click to collapse
Hi, I think it's a bit weird that your system does not have vdc, since vdc is a basic component of android. Basically vdc only communicates with a running vold process through a socket (some vdc commands will be directly sent to vold). The way of communication might vary in different versions/implementations (refer to its source code). I'm not sure whether it is going to work if you compile vdc from source code. You might have to deal with strange issues when compiling or running it.
Amazing Guide.mine same as your question,i have full root access now.but it is android 9,i cant find Global DE key /data/unencrypted/key ,any idea？
bl4ckluna said:
Amazing Guide.mine same as your question,i have full root access now.but it is android 9,i cant find Global DE key /data/unencrypted/key ,any idea？
Click to expand...
Click to collapse
Hi, bl4ckluna
I suspect the location of keys has changed. I have a quick check of the vold source code of Android 9.0, it seems to change a lot. I can no longer find "unencrypted keys". Instead, a "systemwide_volume_key" (which locates in /data/msic/vold/volume_keys) presents. Besides, Android 9 has introduced metadata encryption, which makes it much more complex. I have no idea how the encryption works before reading all the vold source code. You can probably check the source code here:
https://android.googlesource.com/platform/system/vold/+/refs/tags/android-9.0.0_r45/Ext4Crypt.cpp
https://android.googlesource.com/platform/system/vold/+/refs/tags/android-9.0.0_r45/KeyStorage.cpp
https://android.googlesource.com/platform/system/vold/+/refs/tags/android-9.0.0_r45
Wow. Nice. Excellent information. I was searching in google and didn't find any useful info.
amazing！Where can I contact you? I want to share some interesting things with you! Please contact me as soon as possible! Best wishes!
Have you considered making an automated tool for this? It would help many people.
Does somebody update this script also for Android 10 ?
Hi, and thank you for the effort put into this. Despite software having changed since then I'm wondering if you could provide any kind of insight with my problem.
I'm on a Galaxy s10e (exynos) with LineageOS+MicroG (Android 12), which I broke while trying to install a magisk module. The device is in a bootloop ever since. I've tried many things which didn't work, o now I'm just trying to recover the data which is encrypted. That's how I landed here. I can access the device via adb while it's booting, and that's how I've been interacting with it. Following your guide I dumped the /data partition on an external SDCard, mounted it in the device and looked around. The thing is, everything is decrypted besided /data/media/0. So I'm stuck at stage 3 of the process.
When I try to run `vdc --wait cryptfs unlock_user_key 0 0 "" ""`, it fails with
Code:
ProcessState D 12-27 12:31:36 32158 32158 Binder ioctl to enable oneway spam detection failed: Invalid argument
vdc V 12-27 12:31:36 32158 32158 vdc.cpp:66] Waited 0ms for vold
vdc E 12-27 12:31:36 32158 32158 vdc.cpp:216] Raw commands are no longer supported
From my understanding, manually decrypting doesn't work anymore. The data is all there, and from my understanding so are all the keys. It should be able to be recovered, right?
I'm stuck and don't know what else to try. Has anyone got any insight?
Ștefan Radu said:
Hi, and thank you for the effort put into this. Despite software having changed since then I'm wondering if you could provide any kind of insight with my problem.
I'm on a Galaxy s10e (exynos) with LineageOS+MicroG (Android 12), which I broke while trying to install a magisk module. The device is in a bootloop ever since. I've tried many things which didn't work, o now I'm just trying to recover the data which is encrypted. That's how I landed here. I can access the device via adb while it's booting, and that's how I've been interacting with it. Following your guide I dumped the /data partition on an external SDCard, mounted it in the device and looked around. The thing is, everything is decrypted besided /data/media/0. So I'm stuck at stage 3 of the process.
When I try to run `vdc --wait cryptfs unlock_user_key 0 0 "" ""`, it fails with
Code:
ProcessState D 12-27 12:31:36 32158 32158 Binder ioctl to enable oneway spam detection failed: Invalid argument
vdc V 12-27 12:31:36 32158 32158 vdc.cpp:66] Waited 0ms for vold
vdc E 12-27 12:31:36 32158 32158 vdc.cpp:216] Raw commands are no longer supported
From my understanding, manually decrypting doesn't work anymore. The data is all there, and from my understanding so are all the keys. It should be able to be recovered, right?
I'm stuck and don't know what else to try. Has anyone got any insight?
Click to expand...
Click to collapse
Hi Ștefan,Sorry I also have no idea for this. Many vdc commands were deprecated since a few Android versions ago. I was tracking the source code of lock screen and end up found vdc, which did work in Andorid 7. But Google seemed to make many changes and I'm not sure how it works now. (please also read the top of this article which I just updated)
If your device got a bootloop because of a magisk module, you can also try the following things:
1. try adb shell during the bootloop and quickly remove the module that causes the issue (usually under /data/adb/*)
2. try a TWRP which supports data decryption through a lock screen passwrod and remove the module. Or you can guess which encrypted folder is for the module/magisk and remove it. (You can remove files even if they are encrypted)
3. unpack and (un)patch the kernel (initrd) so that magisk stops working or not loading modules.
Thanks for taking the effort to write this down! I am trying to open a userdata backup from my bootlooping OnePlus 8 Pro.
Following your steps, I get this error
Code:
255|OnePlus8Pro:/ # busybox mount -t f2fs -o ro,force_no_inline_enc /storage/406E82FF6E82ED4A/userdata_after_crash_for_testing.img /data/computer
mount: can't setup loop device: No such file or directory
Though both the file (the image) and the directory (/data/computer) exist. Any idea why this basic task does not work?
By the way, on the phone these settings are used to mount the userdata partition:
Code:
/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,reserve_root=32768,resgid=1065,fsync_mode=nobarrier,inlinecrypt latemount,wait,resize,check,formattable,fileencryption=ice,wrappedkey,quota,reservedsize=128M,sysfs_path=/sys/devices/platform/soc/1d84000.ufshc,checkpoint=fs
HaTeNL said:
Thanks for taking the effort to write this down! I am trying to open a userdata backup from my bootlooping OnePlus 8 Pro.
Following your steps, I get this error
255|OnePlus8Pro:/ # busybox mount -t f2fs -o ro,force_no_inline_enc /storage/406E82FF6E82ED4A/userdata_after_crash_for_testing.img /data/computer
mount: can't setup loop device: No such file or directory
Though both the file (the image) and the directory (/data/computer) exist. Any idea why this basic task does not work?
By the way, on the phone these settings are used to mount the userdata partition:
/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,reserve_root=32768,resgid=1065,fsync_mode=nobarrier,inlinecrypt latemount,wait,resize,check,formattable,fileencryption=ice,wrappedkey,quota,reservedsize=128M,sysfs_path=/sys/devices/platform/soc/1d84000.ufshc,checkpoint=fs
Click to expand...
Click to collapse
Hi,
Looks like the loop device wasn't set up correctly. In order to mount an image file, a loop device has to be set up to simulate a block device for the file. Can you try setting up it manually, for example,
Bash:
losetup -f --show /storage/406E82FF6E82ED4A/userdata_after_crash_for_testing.img
and see what device is printed out, then try to mount the device.
use another busybox or toybox losetup with -s flag. also for mounting f2fs ro you should add disable_roll_forward to mount flags (noload for ext4) required for mounting dirty file systems.
Thanks both! I see I already have toybox installed, and I also tested toybox-ext (for Magisk), but unfortunately I get errors again.
I tried
Code:
losetup -f --show /storage/406E82FF6E82ED4A/userdata_after_crash_for_testing.img
with output "/dev/block/loop31"
So now I do the following and get an error again.
Code:
toybox-ext mount -t f2fs -o ro,force_no_inline_enc,disable_roll_forward /dev/block/loop31 /data/testdata
mount: '/dev/block/loop31'->'/data/testdata': Invalid argument
With dmesg I see
Code:
[ 7572.232295] (2)[24714:toybox-ext][20230212_19:42:16.465650]@2 F2FS-fs (loop31): Magic Mismatch, valid(0xf2f52010) - read(0x6970be9f)
[ 7572.232312] (2)[24714:toybox-ext][20230212_19:42:16.465670]@2 F2FS-fs (loop31): Can't find valid F2FS filesystem in 2th superblock
Maybe because the f2fs filesystem is encrypted? Do I need other mount options?
To be clear, what I used to backup the userdata was dd
file system is corrupt. try fsck but get a copy first.
your device uses metadata encryption, this guide is not for you. any further discussion please in new thread, it's off-topic here.