Thursday, October 12, 2017

Implementing supervised Android and iOS Monkey tests

I assume that any QA engineer who works in mobile project has implemented in one or another way monkey tests for their applications. Today I'm going to explain how to implement in about 1 hour your own supervised monkey (or pseudo-monkey) tests for Android or iOS apps.

In this article I'll use the following testing frameworks - Espresso together with UIAutomator for Android and XCTest for iOS.

Usually monkey tests are implemented using 3rd party's libraries or scripts which can bring us some disadvantages:
  • Monkey tests are not the part of the project codebase and not controlled by Espresso or XCTest test frameworks
  • Is usually a 3rd party library with it's own issues and need for maintenance
  • Hard to fetch and process test results

Having monkey tests in native testing frameworks like Espresso for Android of XCTest for iOS brings us the following advantages:
  • Monkey tests are the part of the UI tests codebase. Fully owned and controlled by you
  • Possibility to use other tests in combination with Monkey (for example use UI test to login and afterwards start Monkey tests)
  • Easy to fetch and process test results, using existing reporting infrastructure
  • Monkey tests can be supervised. Meaning that in case they leave the application for some reason during execution we are able to identify this and launch application back
  • Different UI events or gestures can be implemented in case of need

Below are the requirements to our supervised monkey tests:
  1. Tests should operate only on application under test UI not interacting with other applications or system navigation elements.
  2. It should be possible to identify if monkey tests left the application under test and launch it again.
  3. They should have a wide range of different UI actions or gestures.
  4. Test results should be easy collectable after test execution.
And finally the implementation examples:
  • iOS and XCTest - all the description in comments. One thing to note - iOS device vertical and horizontal window dimensions are in range from 0 to 1:
import Foundation
import XCTest

class MonkeyTests: MonkeyHelper {
    
    func testMonkey() {
        var numberOfMonkeySteps: Int = 500

        //below values are taken experimentally. Modify if you wish.
        var tapBack: Int = 9
        var tapCrossTopRightButton: Int = 13
        var tapTabBarIcon: Int = 20
        var swipeFromLeftToRight: Int = 18

        /* Here use UI tests to login or navigate 
        to needed place in the application under tests */

        //Each iteration do some action and tap randomly. 
        //Can be replace by switch block. All methods below in MonkeyHelper class
        for i in 1...numberOfMonkeySteps {
            
            if i % tapBack == 0 {
                tapBackButton()
            }
            if i % tapCrossTopRightButton == 0 {
                tapCrossButton()
            }
            if i % tapTabBarIcon == 0 {
                tapRandomTabBarIcon()
            }
            if i % swipeFromLeftToRight == 0 {
                swipeFromLeftEdgeToRight()
            }

            //tap random coordinate
            let windowCoordinate = 
                app.coordinate(withNormalizedOffset: generateRandomVector())
            windowCoordinate.tap()
        }
    }
}
import Foundation
import XCTest

/* MonkeyHelper class can extend base UI test class with setUp() to prepare test environment 
and tearDown() to take screenshot on failure */
class MonkeyHelper: BaseTestClass {
    
    let app = XCUIApplication()

    func tapBackButton() {
        app.coordinate(withNormalizedOffset: CGVector(dx: 0.05, dy: 0.05)).tap()
    }
    
    func tapCrossButton() {
        app.coordinate(withNormalizedOffset: CGVector(dx: 0.95, dy: 0.05)).tap()
    }
    
    func swipeFromLeftEdgeToRight() {
        let leftEdge = 
            app.coordinate(withNormalizedOffset: CGVector(dx: 0, dy: 0.5))
        let toCoordinate = 
            app.coordinate(withNormalizedOffset: CGVector(dx: 0.6, dy: 0.5))
        leftEdge.press(forDuration: 0, thenDragTo: toCoordinate)
    }
    
    func swipeFromRightEdgeToLeft() {
        let rightEdge = 
            app.coordinate(withNormalizedOffset: CGVector(dx: 1, dy: 0.5))
        let toCoordinate = 
            app.coordinate(withNormalizedOffset: CGVector(dx: 0.4, dy: 0.5))
        rightEdge.press(forDuration: 0, thenDragTo: toCoordinate)
    }
    
    func tapRandomTabBarIcon () {
        let x = CGFloat(Float(arc4random()) / Float(UINT32_MAX))
        let y = CGFloat(0.98)
        app.coordinate(withNormalizedOffset: CGVector(dx: x, dy: y)).tap()
    }
    
    func generateRandomVector () -> CGVector {
        let x = CGFloat(Float(arc4random()) / Float(UINT32_MAX))
        let y = CGFloat(Float(arc4random()) / Float(UINT32_MAX))
        return CGVector(dx: x, dy: y)
    }
}

  • Android and Espresso. A bit more code but similar approach:
@RunWith(AndroidJUnit4.class)
@RequiresApi(api = Build.VERSION_CODES.KITKAT)
public class MonkeyTest {

    @Rule
    public ActivityTestRule mActivityRule = new ActivityTestRule<>(
            MainActivity.class);

    /* use TestWatcher() rule to take an action on failure */
    @Rule
    public TestRule watcher = new TestWatcher() {
        @Override
        protected void failed(Throwable e, Description description) {
            //do whatever you want here
            //maybe take screenshot 
        }
    };

    @Test
    public void executeMonkey() {
        MonkeyHelper monkeyHelper = new MonkeyHelper();
        monkeyHelper.letMonkeyPlay();
    }
}
public class MonkeyHelper {

    private static final int NUMBER_OF_STEPS = 10000;
    private static final int NOTIFICATION_BAR_HEIGHT = 100;
    private static final int SYS_BUTTONS_HEIGHT = 200;
    private static final int OPEN_MENU = 5;
    private static final int DRAG_RANDOMLY = 7;
    private static final int PRESS_BACK = 13;
    private static final int MENU_X = 70;
    private static final int MENU_Y = 100;
    private final int yCoordinateOverSystemButtons;
    private final int yCoordinateBelowNotificationBar;
    private final UiDevice mDevice;
    private int mWidth = 0;

    public MonkeyHelper() {
        mDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());
        mWidth = mDevice.getDisplayWidth();
        //we should not click on system buttons and on notification/tool bar
        yCoordinateOverSystemButtons = mDevice.getDisplayHeight() - SYS_BUTTONS_HEIGHT;
        yCoordinateBelowNotificationBar = NOTIFICATION_BAR_HEIGHT
    }

    public void letMonkeyPlay() {

        for (int i = 0; i <= NUMBER_OF_STEPS; i++) {
            int randomX = getRandomX();
            int randomY = getRandomY();

            if (checkCurrentVisibleAppIsYourApp()) {
             //relaunch the app if went out for some reason
            }
            if (i % DRAG_RANDOMLY == 0) {
                dragRandomly(randomX, randomY);
            }
            if (i % OPEN_MENU == 0) {
                mDevice.click(MENU_X, MENU_Y);
            }
            if (i % PRESS_BACK == 0) {
              pressBack();
            }
            mDevice.click(randomX, randomY);
        }
    }

    private void pressBack() {
     /* sometimes needed a guard to not press back 
     if you are in the main activity */
        mDevice.pressBack();
    }

    private void checkCurrentVisibleAppIsYourApp() {
        if (!mDevice.getCurrentPackageName().equals("your_package_name")) {
            //relaunch the app under tests
        }
    }

    private void dragRandomly(final int endX, final int endY) {
        mDevice.drag(getRandomX(), getRandomY(), endX, endY, 10);
    }

    private int getRandomY() {
        /* generate randomY value between yCoordinateBelowNotificationBar 
        and yCoordinateOverSystemButtons giving you possibility 
        to implement it on your own */
        return randomY;
    }

    private int getRandomX() {
        return (int) (Math.random() * mWidth);
    }
}
As you can see our requirements are met. Monkey gestures are easily extendable, test results reporting and screenshots functionality may be taken from existing UI tests.

This supervised Android and iOS Monkey tests approach is robust and flexible.
Please leave a comment if you see ways to improve them. Thanks.

Friday, January 13, 2017

Preparing android emulator for UI test automation.

This post is about setting up android emulator for UI test automation. Properly configured emulator is the basis for reliable tests. Hundreds or thousands of professionally written test cases is great but if they become flaky because of the environment they are running on, their value reduces a lot.

I will give you a couple of advices I'm following in my test automation projects. In general we will go through below topics:
  1. Managing emulator system animations
  2. Controlling soft keyboard appearance
  3. Changing emulator system locale
Tweaking first and second points will reduce to minimum flakiness in our automated tests which can be caused by emulator.

For those who are lazy to read the whole article at the bottom of the post I shared youtube video where I describe the same points with one more additional hint on top :)

1. There are three types of system animation we may control:
  • window animation scale
  • transition animation scale
  • animator duration scale

Emulator system animations can be controlled manually from inside the "Settings -> Developer options" as shown on screen shot below 



or by executing shell commands as following:
adb shell settings put global window_animation_scale 0.0

adb shell settings put global transition_animation_scale 0.0
adb shell settings put global animator_duration_scale 0.0
2. Running UI tests with soft keyboard is the tricky thing. Based on my experience UI tests are failing from time to time due to clicking on the keyboard instead of UI element. That is why you may want to disable it completely.

Again you can do it manually by switching "Show virtual keyboard" toggle inside "Languages & input -> Physical keyboard" settings section



or by sending below commands to emulator shell:
adb shell settings put secure show_ime_with_hard_keyboard 0
3. Nowadays most of the applications support more than one language. This fact forces us to test our apps with multiple languages.

During manual testing this is achievable by just changing system language from Settings but the same hardly doable from automated tests. Fortunately android emulator has preinstalled "Custom locale" application which can be used to change system language by simply sending specific intent with extra language parameter to it as below:
adb shell am broadcast -a com.android.intent.action.SET_LOCALE --es com.android.intent.extra.LOCALE EN
More information you can find in the following video: