A black technology for mobile testing based on dynamic plug-in system

A mobile testing black technology based on dynamic plug-in system

Baidu MTC is the industry's leading mobile application testing service platform, which reduces the costs faced by developers in mobile application testing. Provide solutions to , technology and efficiency issues. At the same time, industry-leading Baidu technology is shared, and the authors come from Baidu employees and industry leaders.

Author of this article: hyxbiao & tony xin
Author Email: hyxbiao@gmail.com&&jiankangxin@gmail.com

Background

Mobile APP plug-in is a platform The product is a powerful tool for solving system limitations (65535), module decoupling, and multi-team collaboration. Its biggest feature is the dynamic delivery of modules, which brings obvious benefits to the product. However, in Baidu, this system has brought new ideas to mobile terminal testing technology

Mobile terminal online problem location Several scenarios:

Scenario 1: Cloud users report that a certain function is unavailable. RD guesses several possible triggering reasons. The client management log information collected online is incomplete and cannot be fully confirmed. question. Stuck in an endless loop, online users continue to expose problems, and offline users cannot reappear stably, and problems cannot be located in a timely manner. How to break?
Scenario 2: Managing user behavior through client-side pre-embedding methods often leads to the problem of incomplete management, and locating online problems often requires these behavior logs to provide a good basis for problem locating. Steps to reproduce. How to obtain all user and client interaction logs through technical means without coding?
Scenario 3: Many good offline tools can be turned into SDKs, bringing a lot of positive benefits to online problem discovery and location. However, because the testing capability itself will affect the performance of the integrated app, Or the development team's schedule and other reasons cannot be integrated, and a large number of online problems cannot be fully exposed. How to solve this problem gracefully?
Scenario 4: Baidu’s online client small traffic experiment shows that online problems are actually a normally distributed random event, and TOP problems often only require a small amount of sampling Users can be recalled without affecting all users

Several scenarios for mobile offline testing:

Scenario 1: Client testing is simple but complex. A simple skill tree of a client tester may include the ability to analyze these problems: ANR, crash, lag, memory leak, memory, CPU usage, power analysis, startup speed analysis, etc. skills. And often, some QA personnel are not full stack. And the use of these tools itself is a large collection of tools. How to enable client testers, or non-professional testers, to have all client problem analysis capabilities with just a few clicks without any background
Scenario 2: Same as the online problem locating just mentioned, a large number of excellent offline functions cannot be applied to all users, because they themselves have the ability to hurt the enemy by eight hundred and themselves by a thousand. . How to recall as many problems as possible while minimizing the loss of user experience?

A new way of locating online problems on the mobile terminal:

Based on the plug-in system, make a test plug-in to incorporate all the offline testing capabilities we find useful , while integrating well-known good frameworks in the industry, such as dexposed, leakcanary, etc., and some systems are open but the main version uses good things such as traceviewer, Choreographer, ActivityLifecycleCallback, etc. that will affect performance. All are downloaded to the cloud plug-in, and distributed to the mobile phones of specific target users through the dynamic module low-traffic system that has been built in the cloud, continuously exposing problems

NICE JOB! !

How the cloud plug-in works:
The cloud plug-in itself is just a plug-in for the host. What can really unleash its testing capabilities is the large number of test scenarios built offline and the dynamics of the plug-in itself. Loading mechanism so that our test scenario can be effective online. When it comes to this, we have to stir up old rice in a new pot:

Parental delegation mechanism: Under the class loading mechanism of Java, the child classloader can look up the loading content of the parent classloader, thereby providing cloud plug-in It provides prerequisites for dynamically searching various host class information (multi-process plug-in system... please ignore me)
dexclassloader: can load any JAR in the file system (including dex files) ), zip, apk files
patchclassloader: can only load apk files of data/app/, often used in multi-dex split projects
dexfile: can load dynamic files and extract the class information inside the file. This is something that dexclassloader does not have , the corresponding class cannot be compiled into the plug-in package. In this way, after the scene plug-in is loaded, it can call back the host's log system to collect information
After the above, the scene plug-in packaged in the JAR or APK in the picture below can be The cloud plug-in is dynamically loaded, and at the same time, it reads and collects various classes of the host, local space, and host-related information in the system. As for hooks, reflection, code injection, exception capture, instrumentation, etc., these are just a means

Example:

Note: Although the following case is about the problem locating scenario of cloud plug-in, it is not just cloud plug-in. We will open source this part of the technology in the form of SDK later. Therefore, apps that integrate this SDK can also do this, but without the plug-in system, their own security requires integrated developers to pay attention to their own security. Of course, you don’t have to pay attention. On a rooted phone, your app itself has been completely exposed to hackers (BLESS…)

Text: Taking fluency as an example, let’s see how to build it very quickly The case of the cloud debugging plug-in

Fluency: It can be understood as the speed of the android system drawing UI. In theory, the human eye will feel that the program is smooth only after receiving 60 frames of images within 1 second. At the beginning of the Android system, fluency has always been the target of criticism. It was not until the Android 4.1 system that there was a marked project Buffer, and three major elements were introduced, VSYNC (vertical synchronization), Triple Buffer and Choreographer. Among them, Choreographer is the goal of our discussion today. It is the coordinator of the entire mechanism, and all loopers share a Choreographer object

Choreographer opens a FrameCallback thing to the outside world. Every time the system draws, it will call back the doFrame function. Through this function, you can Calculate the number of times the current page is drawn within 1 second. But here comes the problem:
A black technology for mobile testing based on dynamic plug-in system_PHP tutorial Looking at the picture above, the meaning is that although this product is good, it is recommended that you developers should not use it. . . How to play this? I originally wanted to put it on a fluency monitor. . At this point you will definitely think that we have a cloud debugging plug-in.

The construction is very simple, as follows, you only need to copy this code to any android project, and then package it. Note that NutXError is only used as a compilation dependency. If there is a plug-in system, this code can be loaded directly And ran

public class     NutFrameMonitor extends BaseCase {  private static final String TAG = "NutFrameMonitor";  @Override  public void invoke(Context context) {    int sdkVersion = 0;    try {      sdkVersion = Build.VERSION.SDK_INT;    } catch (Exception e) {      // TODO     }     if (sdkVersion >= 16) {       try {         Choreographer.getInstance().postFrameCallback(NutFrameCallback.callback);       } catch (Exception e) {         // TODO       }     }   }   private static class NutFrameCallback implements Choreographer.FrameCallback {     static final NutFrameCallback callback = new NutFrameCallback();     private long mLastFrameTimeNanos = 0;     private long mFrameIntervalNanos = (long)(500000000) - 1;     @Override     public void doFrame(long frameTimeNanos) {       if (mLastFrameTimeNanos != 0) {         final long jitterNanos = frameTimeNanos - mLastFrameTimeNanos;         if (jitterNanos > mFrameIntervalNanos) {           NutXError error = new NutXError(TAG);           error.setDetailMessage("frame Choreographer waste more than 500ms");           error.dump();         }       }       mLastFrameTimeNanos = frameTimeNanos;       Choreographer.getInstance().postFrameCallback(NutFrameCallback.callback);     }   } }

Copy after login

Result

The following screenshot is a fluency problem discovered when the operation client is running (note that the monitoring case is dynamically loaded through the cloud plug-in) . At the same time, you can also see that after monitoring the drawing problem, each interface where the user stayed was also drawn. Then you can follow this trace to reproduce the results offline~
Isn’t it cool~~~

A black technology for mobile testing based on dynamic plug-in system_PHP tutorial

Postscript

As above, this concludes the technical discussion on Baidu’s mobile testing. However, there are actually many things that have not been mentioned, such as what automated cases have been built and how compatible this mechanism is. Moreover, sharp-eyed students will also find that the system itself may have compatibility issues. How Do a good job in problem control and ensure that the cloud debugging plug-in (actually we call it Nut Cloud) is recycled effectively and in a timely manner. In fact, we are confident enough to try this thing online. On the factory line, we have a complete set of dynamic module small traffic system. The cloud plug-in itself is actually used as a dynamic module. When problems occur online, our cloud plug-in will play its value

For more information sharing, please pay attention to "Baidu MTC Academy" http://mtc.baidu.com/academy/article

that developers face in mobile application testing