Trying to reach even non-computer people

This is a long post. Most of it is instructions for modifying Android apps for your own purposes. In the first portion, in which I talk about motivations, I will attempt to make it interesting even for non-computer people:

• I was surprised how easily and well java sources can be recovered from any Android app
• It’s easy to customize apps. You can easily change the pictures and the sound clips.
• Banner ads in kids apps are surprisingly easy to remove
• Some commentary on kids apps in general

The second part is a how to:

• How to install the needed tools
• How to use the installed tools to extract java and related sources from an app package (apk),
• How to make a change, and then how to put it all back together to yield an installable app.

Android apps for little kids

As a parent, I’ve looked around for some good games for my two kids, aged 3 and 6. 1 There are some good ones out there. For example, all of the apps produced by Lego are excellent2. We look for educational ones. Endless Alphabet is a good one.3 Wonster Words.4 Beck and Bo is excellent…. There are some companies out there making great games for kids.
A step down from these, but still pretty good are games like FireFightersFireRescue. It’s a fun app and it appeals to little boys who are enamored with fire fighter related stuff. 5 The downside is that it’s got a big banner along to bottom for ads, and for adult users, that’s not a problem. This game involves moving a firetruck ladder to rescue people stuck in the building. It’s easy for little fingers to get dangerously close to the ads area. Even my 6 year old daughter doesn’t have perfect tablet swiping fingers6. So my son often clicks ads; ads that he has no chance of being interested in. Looking at the screenshot below, I don’t think he’s looking for a free ebook from resources.office.com.

Banner ads don’t really make sense for kids games

App developers need to make a living. As device users, we’ve voted that we don’t want to pay for anything. So instead of the apps being the product, we users have become the product; developers sell their users to advertisers. Ads is the main game in town. 7
Still, it doesn’t seem that ads make sense for little kids. Surely Google has thought about this and I imagine they’re in a bind. They want ad revenue, but they also don’t want the perception that they’re not friendly to kids. They have to know that some of their ads are served to an entirely inappropriate audience.
Anyway, my son enjoys playing the game. As parents, my wife and I don’t enjoy helping him get back to it after he clicks the ad. So I went to the playstore to find a paid version of the firefighter game without ads.
They don’t offer an ad free one. Bummer.

Looking for adblocking led to the rabbithole

Kids games like the firefighter game are pretty common, otherwise, I’d have just moved onto the next game. As a computer guy, I figured I’d look for a way to make these games more playable. As happens so often, I was led down a rabbithole.
I started by trying an adblocker. That didn’t work. The ads were just replaced with offers about their other apps, however, in the process, I stumbled on the youtube video below. They make it look so easy. More important, it made me curious about how apps are put together.8

In the end, I was successfully able to remove ads from the firefighter app. How I did this, is next in this post.

Finally the instructions

I am writing these instructions based on running in a fresh virtualbox install of ubuntu 16.04. The only prior thing I’ve installed is emacs24. I mention it because maybe other stuff gets installed with it. I’ve split this section into four parts:

• installation of the needed tools
• setting up your android device
• using the tools to unpackage and decompile
• selective recompile and repackage

Installing the tools

The instructions later in this post will want an env var APK_TOOLS that points to an area of installed tools.
In these install instructions, I’m attempting to enable you to simply cut/paste the commands. Things will be installed in groups. The main exception to this is Oracle’s java and Google sdk manager. The reason for this, is that the installers insist on you typing y to agree to their terms and conditions.

Java 1.8

Oracle doesn’t make it convenient to install java on ubuntu. So we get it from an alternate place. I’m usually wary of such alternates, but I found a couple sites that directed me here. In particular, this askubuntu answer:
http://askubuntu.com/questions/464755/how-to-install-openjdk-8-on-14-04-lts
The installer will ask you to accept some license conditions.
[code]
sudo add-apt-repository ppa:webupd8team/java -y
sudo apt-get update
sudo apt-get -y install oracle-java8-installer
[/code]

android buildtools

At the end of the process to put an app back on your device, you will need to sign it. If you have android studion ide installed, the signer comes with the buildtools package which will probably end up in your home dir $HOME/Android/Sdk/build-tools/25.0.2/apksigner. This is the way to go if you think you might want to do android development.
Here I’m going to describe a less heavy handed way. First we need the sdkmanager https://developer.android.com/studio/index.html#downloads. We’ll then use the sdkmanager to install buildtools.
[code]
cd $APK_TOOLS
mkdir android_tools
cd android_tools
wget https://dl.google.com/android/repository/tools_r25.2.3-linux.zip
unzip tools_r25.2.3-linux.zip
# tools/bin/sdkmanager –list
# –> build-tools;25.0.2 | 25.0.2 | Android SDK Build-Tools 25.0.2
mkdir sdks
tools/bin/sdkmanager –sdk_root=sdks ‘build-tools;25.0.2’
[/code]

32 bit support, apktools, dex2jar, luyten, jdgui, adb, zipalign

adb, in this context, is used to copy apk files from/to your device. apk files are what you’re downloading from the playstore when installing apps.
Apktool is used to package/unpackage apk files. It can be installed using the normal Ubuntu package system, but that version gives me errors. Instead, these instructions download apktool directly from the website.
Compiled Java code is normally stored in jar files. In android apk files, the are in dex files. dex2java is used to convert between the two.
Luyten and JDGui are two java decompilers. They seem pretty good. My main gripe with them is that they both insist on you using the GUI.
zipalign is part of the apk signing process. signing is important for security reasons. We don’t want people accidentally installing fake Chase or Bank of America apps. The android OS requires signing before it will allow you to install an app.
From a small comment in the apktool instal instructions: https://ibotpeaches.github.io/Apktool/install

Make sure you have the 32bit libraries (ia32-libs) downloaded and installed by your linux package manager, if you are on a 64bit unix system.
(This helps provide support for the 32bit native binary aapt, which is required by apktool)

To fulfill this requirement, we’ll follow these instructions: https://blog.teststation.org/ubuntu/2016/05/12/installing-32-bit-software-on-ubuntu-16.04/
Because sudo asks for your password, I find it useful to do “sudo ls” right before cut/pasting these.
[code]
# 32 bit stuff
sudo dpkg –add-architecture i386
sudo apt-get update
sudo apt-get -y install libc6:i386 libstdc++6:i386
sudo apt-get -y install zlib1g:i386
# adb git zipalign. (I think I don’t really need git anymore)
sudo apt -y install adb git zipalign
# Apktool
# APK_TOOLS is a directory where you want these to be installed.
cd $APK_TOOLS
mkdir Apktool
cd Apktool
wget https://bitbucket.org/iBotPeaches/apktool/downloads/apktool_2.2.2.jar
ln -s apktool_2.2.2.jar apktool.jar
wget https://raw.githubusercontent.com/iBotPeaches/Apktool/master/scripts/linux/apktool
chmod ugo+x apktool
# This package works in windows and linux. Since I use linux, I want all of the sh files. Let’s make them executable.
cd $APK_TOOLS
mkdir dex2jar
cd dex2jar
wget https://downloads.sourceforge.net/project/dex2jar/dex2jar-2.0.zip
unzip dex2jar-2.0.zip
chmod ugo+x dex2jar-2.0/*.sh
# luyten
cd $APK_TOOLS
mkdir luyten
cd luyten
wget https://github.com/deathmarine/Luyten/releases/download/v0.5.0/luyten-0.5.0.jar
# jdgui
cd $APK_TOOLS
mkdir jdgui
cd jdgui
wget https://github.com/java-decompiler/jd-gui/releases/download/v1.4.0/jd-gui-1.4.0.jar
[/code]

generating a keystore for apk signing

Before you can install a new apk file on an android device, it has to be signed. To sign, you need a signature. Because keystore generation is a one time thing, I’m including it here in the instructions. Let’s generate that now with the keytool command. My system has keytool without doing anything extra. I imagine adb added it for me.
It will ask for a password followed 6 questions. I just use the default of unknown. I’m not trying to put these apps on the playstore. I just want different versions on my device. Finally, it’ll ask for confirmation that everything’s correct:
Is CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown correct?
[no]: yes
[code]
mkdir ${APK_TOOLS}/keystore
keytool -genkey -v -keystore ${APK_TOOLS}/keystore/my-release-key.jks -keyalg RSA -keysize 2048 -validity 10000 -alias my-alias
[/code]

building apktool yourself (optional)

In case you want to build apktool yourself from latest code. https://ibotpeaches.github.io/Apktool/install/
[code]
cd $APK_TOOLS
git clone https://github.com/iBotPeaches/Apktool.git
./gradlew build fatJar
# get wrapper script
wget https://raw.githubusercontent.com/iBotPeaches/Apktool/master/scripts/linux/apktool
chmod ugo+x apktool
ln -s ./brut.apktool/apktool-cli/build/libs/apktool-cli.jar apktool.jar
[/code]
 

Developer mode and unknown apk sources

On your phone you’ll need to activate developer mode.

• Settings->about device
• Tap on ‘build number’ 7 times and a new menu will appear on the top
  menu.
• Tell your phone it’s ok for a computer to try to talk to it via adb:
  Settings->developer options->Android debugging
• Tell your phone it’s ok to install apks from unknown sources. You’ll need this later, after you’ve modified the game. You’ll be the unknown source
  Security -> unknown sources to on

At this point, connect your device to your computer with its usb cable. Let’s check that you can connect to it. When you run the next command, your phone will ask if it’s ok for your specific computer to connect. You’ll want to say yes and that it should remember.
[code]adb shell[/code]

Retreive, unpack and decompile

Again, I’m attempting to give bigger blocks of cut/pasteable commands. Here’s an overview of what we need to do:

• use adb to find and retrieve the original app package (this needs to be it’s own step)
• use apktool to unpackage it.
• use dex2jar to… convert the dex containing the compiled into a jar file (still compiled)
• use luyten or jdgui to get java sources.9

Some interesting things to look at

After you unpackage an apk with apktool, you’ll have a directory with something like an android project in it. Take a look around. In particular, you’ll find an assets directory. The assets directory contains all of the pictures and sound files of the game. Feel free to customize the app. Change character pictures. Put in some gangsta rap. Make sure the file names stay the same. When we rebuild from that area, any assets changes will come along for the ride.
In the instructions below, I use a environment variable to hold a base name for the stuff we’re processing. In this example it’s
[code]
export APK_NAME=firefightersFireRescue
[/code]

Getting the package location

This step is not really automatable unless you’ll trying to get everything.10
[code]adb shell pm list packages[/code]
Will give you a bunch of lines, including one like this:
[code]package:com.bestopgames.firefightersFireRescue[/code]
Now you want to find out where the apk for that app is on your device
[code]adb shell pm path com.bestopgames.firefightersFireRescue[/code]
gives me:
[code]package:/data/app/com.bestopgames.firefightersFireRescue-1/base.apk[/code]
Now that we know where it is, copy from the device to your local unix disk
[code]
cd <some path where you’ll be doing this experiment>
export APK_NAME=firefightersFireRescue
mkdir apk
cd apk
adb pull /data/app/com.bestopgames.firefightersFireRescue-1/base.apk ${APK_NAME}.apk
cd ..
[/code]

The other steps for unpackage to decompile

[code]
mkdir unpack
${APK_TOOLS}/Apktool/apktool d -s apk/${APK_NAME}.apk -o unpack/${APK_NAME}
${APK_TOOLS}/dex2jar/dex2jar-2.0/d2j-dex2jar.sh unpack/$APK_NAME/classes.dex -o dex2jar/${APK_NAME}.jar
java -jar ${APK_TOOLS}/luyten/luyten-0.5.0.jar dex2jar/firefightersFireRescue.jar
[/code]
Additional notes:

• One note about the -s flag to apktool. This flag is also called the –no-src flag if you don’t give this flag, you won’t get the classes.dex file which you’ll need in the next step.
• luyten doesn’t have a command line interface beyond telling it what jar to read. To save the javas you’ll need to use the gui.

Modify java, repackage, sign and upload.

In this section, I explain the steps I follow to get new java into an android app. Similar to the install section, I’ll have a cut/pasteable snippet at the end of this one.
Using both the luyten and jdgui decompilation tools I did not get a set of javas that just compiled. Trying to do this did not work:
[code]
javac `find . -name "*.java"
[/code]
In both cases, I get syntax errors, though not the same errors. Perhaps files from the two could be combined to get a clean full compile. For the purposes of what I’m showing here, you don’t need a clean compile. More on that later.

Turning off ads in the code

Here’s where things get really interesting (I think). Most games are pretty genericly written. They use a limited number of game engines. They use a limited number of in app advertising platforms. Poking around the firefighter game, I find two libraries in particular.
The first of these is cocos2dx: http://www.cocos2d-x.org/ Inside of it, I found the AISActivity class, which has the method “hideAd()”. That told me there’s a way to turn off ads with a switch. hmm. When I look some more, I find that the class ais.constants:Config.class has this:
[code]
package com.ais.constants;
public class Config {
public static void init() {
org.cocos2dx.lib.AISCommon.enableAdmob = true;
org.cocos2dx.lib.AISCommon.enableInterstitial = true;
org.cocos2dx.lib.AISCommon.enableInApp = false;
org.cocos2dx.lib.AISCommon.enableLocalNotification = false;
}
}
[/code]
Can it really be this easy? Now the trick is changing this file and recompiling. But how?
Again, if I do this:
[code]javac `find . -name "*.java"`[/code]
I get a ton of errors; recompiling everything would be a pain. Can I recompile just this one class?
[code]javac com/ais/constants/Config.java[/code]
Doing that, yields a bunch of android related errors. missing symbols. I tried a couple things. I downloaded the cocos2dx library, but there the problem is which version? I need to compile against something. Then I realized, I have a jar file!
So, here’s what you do. Change the two falses to trues and save it, along with any other files you want to change. You only need to compile modified files. For the commands below, place them under a “newjava” directory. It’s important to retain the intermediate paths to the files. In this example, that’s the com/ais/constants part.11
[code]
mkdir -p newjava/com/ais/con<code>tants/
cp <modified Config.java> newjava/com/ais/constants/
cd newjava
javac -cp ../dex2jar/${APK_NAME}.jar `find . -name "*java"`
jar uvf ../dex2jar/${APK_NAME}.jar `find . -name "*class"`
[/code]
You can use the following line to verify that you didn’t duplicate the file
[code]jar tf ../dex2jar/firefighter.jar | grep ‘/Config'[/code]
Here’s an important note. Unlike other compilers that I’ve worked with, java really pays attention to your directory structure. When running the javac and jar commands, it’s important to run the command from the directory that contains the com directory.

repackaging complete

Ok, now you have a new java file and I’ve showed how it can be easy to recompile it. There are several steps to get to something you can install.
apksigner has a nice feature that you can put your signing password into an environment varible instead of embedding into a script. In the commands below, I’m using SIGNPASS as the env var.

• compile java to class files
• add class files to jar
• convert jar to dex
• rebuild apk file
• zipalign the apk file
• sign the apk file
• copy it to your device

[code]
export APK_NAME=firefightersFireRescue
export APK_TOOLS=../../tools/
# this assumes you’ve already dont the commented keytool step below.
# export SIGNPASS=YOUR_SIGNING_PASSWORD
cd newjava
javac -cp ../dex2jar/${APK_NAME}.jar `find . -name "*java"`
jar uvf ../dex2jar/${APK_NAME}.jar `find . -name "*java"`
cd ..
${APK_TOOLS}/dex2jar/dex2jar-2.0/d2j-jar2dex.sh -f dex2jar/${APK_NAME}.jar -o unpack/${APK_NAME}/classes.dex
mkdir rebuilt
${APK_TOOLS}/Apktool/apktool build unpack/${APK_NAME} -o rebuilt/${APK_NAME}_rebuilt.apk
zipalign -v -p 4 rebuilt/${APK_NAME}_rebuilt.apk rebuilt/${APK_NAME}_rebuilt_aligned.apk
${APK_TOOLS}/android_tools/sdks/build-tools/25.0.2/apksigner sign –ks-pass env:SIGNPASS –key-pass env:SIGNPASS –ks ${APK_TOOLS}/keystore/my-release-key.jks –out rebuilt/${APK_NAME}_rebuilt_signed.apk rebuilt/${APK_NAME}_rebuilt_aligned.apk
# now push to your device
adb push rebuilt/${APK_NAME}_rebuilt_signed.apk /storage/self/primary/Download
[/code]

Extra stuff that may be helpful

If you get an error like this one,
[code]
Exception in thread "main" brut.androlib.AndrolibException: brut.androlib.AndrolibException: brut.common.BrutException: could not exec: [/tmp/brut_util_Jar_5394410189585563704.tmp, p, –forced-package-id, 127, –min-sdk-ver
[/code]
it’s because of a small comment in the apktool install instructions:
https://ibotpeaches.github.io/Apktool/install/

Make sure you have the 32bit libraries (ia32-libs) downloaded and installed by your linux package manager, if you are on a 64bit unix system.
(This helps provide support for the 32bit native binary aapt, which is required by apktool)

To fulfill this requirement, make sure you’ve done the 32 bit stuff in the install section above.
To solve errors like this one:
[code]
W: /tmp/brut_util_Jar_3065852416515877270.tmp: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
Exception in thread "main" brut.androlib.AndrolibException: brut.androlib.AndrolibException:
[/code]
You need this
[code]
sudo apt-get install zlib1g:i386
[/code]

Install it on your device

Now you just need to install it. Using the adb push command from your xterm, it’ll be in your downloads fold. On your phone, navigate to it in the file manager and click it. If you get a popup about unknown apks, you want to change the setting in settings->developer options->allow unknow sources. Make sure you copy over the signed apk otherwise you’ll get “an unknown error occurred”
[code]
adb push firefighter_noads_signed.apk /storage/self/primary/Download
[/code]

Please comment

This post took a lot of time to put together. It took more time than I really should have spent on it. The only way for me to justify it, is to know that others have benefited. If people respond positively, I’ll do other posts when I go down future rabbit holes.