Validating Digital Toolkits: Insights from Mobile Test Lab Deployments

Abstract– The impact of mobile computing has transformed the way we connect and consume information since it allows for seamless access to data. Mobile apps lie at the heart of this revolution, offering breakthrough interventions in healthcare, education, enterprise and personal productivity. Application development or porting Enterprise solutions to various mobile devices, across the global business domains has been growing in leaps & bounds. All these users of mobile devices, like iPhone, iPad, Android phones, Windows Phone, or  even eReaders like the Amazon Kindle, create a never-ending demand for more and more unique and useful mobile applications. Since, this arena of mobile applications is vast with users wanting mobile applications to be simple and fast, just one nagging bug or usability issue can spoil the entire experience.

Introduction

Above the user experience the security of the data shared over mobile devices through the native apps also pose a major concern. This directly might affect the business potential because, with so much competition in this new space, users not having an excellent experience with your application, they might switch to a rival product/vendor/solution provider faster than you can actually say and make amends.

The world mobile application Vendors’ scenario is turning such that, one simply cannot afford to go to market with an application that might have a critical bug or usability issue or any lapses in security. Things become more difficult as there is no previously existing comprehensive guide on how to test particular complexities of mobile device application. So, a proper, system based and tested enough process and methodology need to be in place to cater to cross platform, device centric, individual mobile application testing and publishing results.

A successful approach is to divide the testing efforts for mobile apps into main concerning areas. This simplifies the overall testing process and gives us good result oriented categorization. They are as follows:

1. Security Testing
2. Extensive Device Testing ( on various OS )
3. General Interface Testing
4. External Factors Testing
5. Stress Testing
6. Code Overviews ( wherever required )
7. Ethical Hacking

Functional testing has been excluded from specific mention in the approach effort because testing is not unique to Mobile device testing which we are discussing. It can be carried out in the same way; one would perform on any other kind of application.

Another point to be taken care of is, in all mobile devices testing exercise is diligent usage of emulators versus devices. Although, initial testing starts with emulators but final test reports cannot be based upon them. It has to involve a real device to do the same. Finally, testing strategies are not generic in nature. They should depend largely upon the types of applications, the software requirements etc. Creation of test plans, formulating test reports, publishing release notes etc. can be specific and application/device dependent.

Key Strategies

Security Testing :

Today’s popular mobile platforms are all designed with security; however these provisions could not sufficiently protect the application assets in a device.

Major threats are:

• Web based and Network based attacks
• Malware
• Social engineering attacks
• Resource abuse attacks
• Data Loss
• Data Integrity threats

Mobile Device Security Models

• Traditional access controls
• Application provenance:  Author identification stamping with tamper resistant
• Data Encryption
• App Isolation: Provide limitation of application’s ability to access sensitive data or system on device
• Permissions based access control
• Application Reverse Engineering Prevention
• Memory and Data Leaks prevention
• Cache preventions

General Interface Testing:

The first area to explore in test plans is the user interface. Look and feel and overall UI behavior is crucial for a mobile device based application.

Few checkpoints are:

• Comparing the feature UI of the application with native apps.
• Overall color scheme/theme of the device
• Icon based evaluations
• Menus – what they contain and nature of their invoking
• Device centric responsiveness
• Orientation & Resolutions
• Touch screen matrix – Long, Short, Single, Multiple etc.
• Buttons matrix – Size, Position, Effectiveness etc.
• Different input method interactions
• Soft Keyboard, Hard Keys & Shortcuts process flow analysis

External Factors Testing:

Number of factors catering to the external influence to an application itself, but inherent to the mobile device, needs to be tested where the application runs.

Few checkpoints are:

• Network connections
• SD card interactions
• Phone calls and other interruptions, if any
• Device options like sound profiles, connectivity, screen orientation etc.

Stress Testing:

Mobile devices have much more memory and CPU constraints than desktop applications. Since, traffic environment is much higher, this exercise is a must to find out exceptions, hangs, and deadlocks that may go unnoticed during functional and user interface testing.

Few checkpoints are:

• Loading as much as data as possible
• Perform repeated functions that utilize large data within the application
• Perform functions in varying speeds
• Application execution to be run for a longer interval of time i.e. functionality as well as idle wise respectively
• Random checks and usage of screen taps & keystrokes
• Have multiple applications running and switch between them keeping the testing application in the nodal point
• Reporting of bugs which do not occur every time

Code Overviews:

The test report should also contain time to time suggestions highlighting the vulnerabilities of the code and possible suggestions. 

Ethical Hacking:

Intentional Ethical Hack testing needs to be performed to highlight the vulnerabilities.

Process Brief

It is very important to outline the process which the testing team should follow during all the processes in Device testing. These processes will have active participation of the developer’s pool which will ultimately provide an advantage to the development pool by optimizing development standards.

Few Checkpoints:

• Developer Hand – holding of the test procedures
• Awareness to the developers towards minimal pre-test bug occurrences
• Sensitizing the overall testing process to be able to make the developer pool get acclimatized to an error free environment

Reporting:

• The resources should be in regular touch with the project coordinator.
• They should be available on a daily basis, during their work hours.
• If needed they should also be made available for conference calls, video conferencing or face to face meetings, as per pre-scheduled time lines.
• The Program Manager, to whom the resources will be reporting to, should be coordinating with point of contact regarding all meeting schedules and work updates

Conclusion

Dividing the testing efforts for mobile apps into the suggested concerning areas simplifies the overall testing process. Testing effort should be designed in a 2 to 4-weeks sprint, for handling spikes. Companies with a considerable team size in Mobile Development should have no issues in deploying additional resources as and when the demand arises. Companies while offering the scale-on-demand service, the quality team needs to be well accustomed to handling spikes.

About Me: Dr. Ranita Ganguly is a Certified Software Testing professional with a demonstrated history of working in the IT industry for more than 11 years in Mobile and web-based applications. I have solid understanding of QA methodology, and software development life cycle. Skilled in creating test cases, traceability matrix, test scenarios using specifications and requirements. 

Company: 3D Technologies LLC

Designation: Quality Consultant

URL: https://www.3dtechnos.com/