Automated functional testing of mobile applications: a systematic mapping study

Context Testing is a critical and costly activity in the life cycle of a mobile application, due to the growing request of new applications and to the rapid evolution of mobile devices and frameworks. Testing automation may represent an effective solution to improve the quality of mobile applications and to reduce testing costs. Objective We have performed a systematic mapping study to find, analyze, and classify papers in the scientific literature that are related to the automation of functional testing of mobile applications with the aim to provide a classification scheme useful for researchers and practitioners to have a clear view of the state of the art and to easily find existing solutions to their issues. Method We have conducted the study on the basis of a set of 18 research questions. Search queries have been formulated and applied to 7 search engines and the resulting papers have been filtered by considering sets of inclusion and exclusion criteria. The selected papers have been systematically classified and, in addition, a bibliometric analysis has been performed. Results A systematic map including 131 papers has been obtained and is publicly available. The papers have been classified on the basis of the supported testing activities, the characteristics of the techniques and tools they present, and the evaluation methodologies adopted to validate them. The bibliometric analysis has allowed the identification of the most active researchers, the most attractive venues, and the most influential papers. Conclusions The analysis of the systematic mapping has allowed the identification of some research trends and gaps in this field of study. For example, we have observed a strong prevalence of Android-based approaches, a lack of contributions from industry, and the absence of specific venues and journals focused on mobile testing automation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Rent this article via DeepDyve

Similar content being viewed by others

Problems and Solutions in Mobile Application Testing

Mobile Application Testing in Industrial Contexts: An Exploratory Multiple Case-Study

Mobile Application Verification: A Systematic Mapping Study

Explore related subjects

Notes

References

• Adamsen, C.Q., Mezzetti, G., Moller, A. (2015). Systematic execution of android test suites in adverse conditions. In Proceedings of the 2015 International Symposium on Software Testing and , ISSTA 2015. https://doi.org/10.1145/2771783.2771786 (pp. 83–93). New York: ACM.
• Ahmad, A., Li, K., Feng, C., Asim, S.M., Yousif, A., Ge, S. (2018). An empirical study of investigating mobile applications development challenges. IEEE Access, 6, 17:711–17:728. https://doi.org/10.1109/ACCESS.2018.2818724. ArticleGoogle Scholar
• Akanksha Ashok Magare, M.D.L. (2016). Automated gui testing for android application. Imperial Journal of Interdisciplinary Research, 2(8), 884–888. Google Scholar
• Amalfitano, D., Fasolino, A.R., Tramontana, P. (2011). A gui crawling-based technique for android mobile application testing. In ICST Workshops (pp. 252–261).
• Amalfitano, D., Fasolino, A., Tramontana, P., De Carmine, S., Imparato, G. (2012a). A toolset for gui testing of android applications (pp. 650–653). https://doi.org/10.1109/ICSM.2012.6405345, cited By 16.
• Amalfitano, D., Fasolino, A.R., Carmine, S.D., Memon, A., Tramontana, P. (2012b). Using gui ripping for automated testing of android applications. In ASE ’12: Proceedings of the 27th IEEE international conference on Automated software engineering. DC: IEEE Computer Society Washington.
• Amalfitano, D, Fasolino, A, Tramontana, P, Amatucci, N. (2013a). Considering context events in event-based testing of mobile applications. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 126–133). https://doi.org/10.1109/ICSTW.2013.22.
• Amalfitano, D., Fasolino, A., Tramontana, P., Robbins, B. (2013b). Testing android mobile applications: Challenges, strategies, and approaches. Advances in Computers, 89, 1–52. https://doi.org/10.1016/B978-0-12-408094-2.00001-1 .
• Amalfitano, D, Amatucci, N, Fasolino, A, Tramontana, P. (2015a). Agrippin: A novel search based testing technique for android applications (pp. 5–12), https://doi.org/10.1145/2804345.2804348, cited By 1.
• Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P. (2015b). A conceptual framework for the comparison of fully automated gui testing techniques. In Sixth International Workshop on Testing Techniques for Event BasED Software (TESTBEDS) (pp. 2015).
• Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P., Kowalczyk, E., Memon, A. (2015c). Exploiting the saturation effect in automatic random testing of android applications. In The Proceedings of the 2nd ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft) (pp. 2015).
• Amalfitano, D., Fasolino, A., Tramontana, P., Ta, B., Memon, A. (2015d). Mobiguitar: Automated model-based testing of mobile apps. IEEE Software, 32(5), 53–59. https://doi.org/10.1109/MS.2014.55, cited By 29.
• Amalfitano, D., Amatucci, N., Memon, A., Tramontana, P., Fasolino, A. (2017). A general framework for comparing automatic testing techniques of android mobile apps. Journal of Systems and Software, 125, 322–343. https://doi.org/10.1016/j.jss.2016.12.017, cited By 0. ArticleGoogle Scholar
• Amatucci, N. (2016). Automated gui testing techniques for android applications. PhD thesis, Ph.D. course in Information Technology and Electrical Engineering.
• Anand, S., Naik, M., Yang, H., Harrold, M. (2012). Automated concolic testing of smartphone apps. No GIT-CERCS-12-02.
• Anbunathan, R., & Basu, A. (2015). A recursive crawler algorithm to detect crash in android application (pp. 256–267), https://doi.org/10.1109/ICCIC.2014.7238518, cited By 0.
• Anbunathan, R., & Basu, A. (2016a). Automatic test generation from uml sequence diagrams for android mobiles. International Journal of Applied Engineering Research, 11(7), 4961–4970. cited By 0.
• Anbunathan, R., & Basu, A. (2016b). Data driven architecture based automated test generation for android mobile. https://doi.org/10.1109/ICCIC.2015.7435772, cited By 0.
• Arnatovich, Y.L., Ngo, M.N., Kuan, T.H.B., Soh, C. (2016). Achieving high code coverage in android ui testing via automated widget exercising. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC) (pp. 193–200), https://doi.org/10.1109/APSEC.2016.036.
• Arzensek, B., & Hericko, M. (2014). Criteria for selecting mobile application testing tools (vol. 1266, pp. 1–8), cited By 0.
• Azim, T., & Neamtiu, I. (2013). Targeted and depth-first exploration for systematic testing of android apps. SIGPLAN Not, 48(10), 641–660. https://doi.org/10.1145/2544173.2509549. ArticleGoogle Scholar
• Baek, Y.M., & Bae, D.H. (2016). Automated model-based android gui testing using multi-level gui comparison criteria (pp. 238–249), https://doi.org/10.1145/2970276.2970313, cited By 0.
• Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S. (2015). The oracle problem in software testing: a survey. IEEE Transactions on Software Engineering, 41(5), 507–525. https://doi.org/10.1109/TSE.2014.2372785. ArticleGoogle Scholar
• Basili, V.R., Caldiera, G., Rombach, H.D. (1994). The goal question metric approach. In Encyclopedia of Software Engineering. Wiley.
• Behrouz, R.J., Sadeghi, A., Garcia, J., Malek, S., Ammann, P. (2015). Ecodroid: An approach for energy-based ranking of android apps. In 2015 IEEE/ACM 4th International Workshop on Green and Sustainable Software (pp. 8–14). https://doi.org/10.1109/GREENS.2015.9.
• Bielik, P., Raychev, V., Vechev, M. (2015). Scalable race detection for android applications (vol. 25-30-Oct-2015, pp. 332–348). https://doi.org/10.1145/2814270.2814303, cited By 5.
• Canfora, G., Mercaldo, F., Visaggio, C.A., DAngelo, M., Furno, A., Manganelli, C. (2013). A case study of automating user experience-oriented performance testing on smartphones. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation (pp. 66–69). https://doi.org/10.1109/ICST.2013.16.
• Choi, W., Necula, G., Sen, K. (2013). Guided gui testing of android apps with minimal restart and approximate learning. In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages and applications (pp. 623-640). ACM.
• Choudhary, S.R., Gorla, A., Orso, A. (2015). Automated test input generation for android: Are we there yet? (e). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) (pp. 429–440). https://doi.org/10.1109/ASE.2015.89.
• Coelho, T., Lima, B., Faria, J.P. (2016). Mt4a: a no-programming test automation framework for android applications. In Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation (pp. 59–65), A-TEST 2016. New York: ACM. https://doi.org/10.1145/2994291.2994300.
• Corral, L., Sillitti, A., Succi, G. (2015). Software assurance practices for mobile applications. Computing, 97(10), 1001–1022. https://doi.org/10.1007/s00607-014-0395-8. ArticleMathSciNetGoogle Scholar
• Costa, P., Paiva, A., Nabuco, M. (2014). Pattern based gui testing for mobile applications (pp 66–74). https://doi.org/10.1109/QUATIC.2014.16, cited By 6.
• Crispin, L., & Gregory, J. (2009). Agile Testing: A Practical Guide for Testers and Agile Teams, 1st edn. Addison-Wesley Professional.
• de Sousa Santos, I., de Castro Andrade, R.M., Rocha, L.S., Matalonga, S., de Oliveira, K.M., Travassos, G.H. (2017). Test case design for context-aware applications: Are we there yet? Information and Software Technology (pp. –). https://doi.org/10.1016/j.infsof.2017.03.008.
• De Cleva Farto, G., & Endo, A. (2015). Evaluating the model-based testing approach in the context of mobile applications. Electronic Notes in Theoretical Computer Science, 314, 3–21. https://doi.org/10.1016/j.entcs.2015.05.002, cited By 2. ArticleGoogle Scholar
• Delamaro, M., Vincenzi, A., Maldonado, J. (2006). A strategy to perform coverage testing of mobile applications. In Proceedings of the 2006 international workshop on Automation of software test (pp. 118–124). ACM, .
• Dev, R., Jaaskelainen, A., Katara, M. (2012). Model-based gui testing. case smartphone camera and messaging development. Advances in Computers, 85, 65–122. https://doi.org/10.1016/B978-0-12-396526-4.00002-3, cited By 2. ArticleGoogle Scholar
• Do, Q., Yang, G., Che, M., Hui, D., Ridgeway, J. (2016). Regression test selection for android applications (pp. 27–28). https://doi.org/10.1145/2897073.2897127, cited By 0.
• Dubinsky, Y., & Abadi, A. (2013). Challenges and research questions for testing in mobile development: Report on a mobile testing activity. In Proceedings of the 2013 ACM Workshop on Mobile Development Lifecycle, MobileDeLi ’13 (pp. 37–38). New York: ACM. https://doi.org/10.1145/2542128.2542140.
• Dutia, S.N., Oh, T.H., Oh, Y.H. (2015). Developing automated input generator for android mobile device to evaluate malware behavior. In Proceedings of the 4th Annual ACM Conference on Research in Information Technology, RIIT ’15 (pp. 43–43). New York: ACM. https://doi.org/10.1145/2808062.2808065.
• Fazzini, M., Freitas, E.N.D.A., Choudhary, S.R., Orso, A. (2017). Barista: A technique for recording, encoding, and running platform independent android tests. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) (pp. 149–160). https://doi.org/10.1109/ICST.2017.21.
• Gao, J., Bai, X., Tsai, W.T., Uehara, T. (2014). Mobile application testing: a tutorial. Computer, 47(2), 46–55. https://doi.org/10.1109/MC.2013.445. ArticleGoogle Scholar
• Garousi, V., Mesbah, A., Betin-Can, A., Mirshokraie, S. (2013). A systematic mapping study of web application testing. Information and Software Technology, 55(8), 1374–1396. https://doi.org/10.1016/j.infsof.2013.02.006. ArticleGoogle Scholar
• Garousi, V., & Mantyla, M.V. (2016). When and what to automate in software testing? a multi-vocal literature review. Information and Software Technology, 76, 92–117. https://doi.org/10.1016/j.infsof.2016.04.015. ArticleGoogle Scholar
• Gomez, L., Neamtiu, I., Azim, T., Millstein, T. (2013). Reran: Timing- and touch-sensitive record and replay for android. In Proceedings of the 2013 International Conference on Software Engineering, IEEE Press, Piscataway, NJ, USA, ICSE ’13 (pp. 72–81).
• Gomez, M. (2015). Debugging of Mobile Apps in the Wild Guided by the Wisdom of the Crowd. In 2nd International Conference on Mobile Software Engineering and Systems - ACM Student Reasearch Competition. Florence.
• Gomez, M., Rouvoy, R., Adams, B., Seinturier, L. (2016). Reproducing context-sensitive crashes of mobile apps using crowdsourced monitoring (pp. 88–99). https://doi.org/10.1145/2897073.2897088, cited By 1.
• Griebe, T., & Gruhn, V. (2014). A model-based approach to test automation for context-aware mobile applications. In Proceedings of the 29th Annual ACM Symposium on Applied Computing, SAC ’14 (pp. 420–427). New York:ACM. https://doi.org/10.1145/2554850.2554942.
• Griebe, T., Hesenius, M., Gruhn, V. (2015). Towards automated ui-tests for sensor-based mobile applications. Communications in Computer and Information Science, 532, 3–17. https://doi.org/10.1007/978-3-319-22689-7_1, cited By 0. ArticleGoogle Scholar
• Griebe, T., Hesenius, M., Gesthuesen, M., Gruhn, V. (2016). Test Automation for Speech-Based Applications. In Fujita, H., & Papadopoulos, G. A. (Eds.) New Trends in Software Methodologies, Tools and Techniques, Frontiers in Artificial Intelligence and Applications. https://doi.org/10.3233/978-1-61499-674-3-85, 15th International Conference on New Trends in Intelligent Software Methodology Tools, and Techniques (SoMeT), CYPRUS, SEP 13-15, 2016, (Vol. 286 pp. 85–100).
• Gudmundsson, V., Lindvall, M., Aceto, L., Bergthorsson, J., Ganesan, D. (2016). Model-based testing of mobile systems - an empirical study on quizup android app (vol. 208, pp. 16–30). https://doi.org/10.4204/EPTCS.208.2https://doi.org/10.4204/EPTCS.208.2, cited By 0.
• Hao, S., Liu, B., Nath, S., Halfond, W.G., Govindan, R. (2014). Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’14 (pp. 204–217). New York: ACM. https://doi.org/10.1145/2594368.2594390.
• Harrison, R., Flood, D., Duce, D. (2013). Usability of mobile applications: literature review and rationale for a new usability model. Journal of Interaction Science, 1(1), 1. https://doi.org/10.1186/2194-0827-1-1. ArticleGoogle Scholar
• Hesenius, M., Griebe, T., Gries, S., Gruhn, V. (2014). Automating ui tests for mobile applications with formal gesture descriptions (pp. 213–222). https://doi.org/10.1145/2628363.2628391, cited By 4.
• Holl, K., & Elberzhager, F. (2016). Quality assurance of mobile applications: A systematic mapping study. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia, MUM ’16 (pp 101–113). New York:ACM. https://doi.org/10.1145/3012709.3012718.
• Hsiao, C.H., Pereira, C., Yu, J., Pokam, G., Narayanasamy, S., Chen, P., Kong, Z., Flinn, J. (2014). Race detection for event-driven mobile applications. ACM SIGPLAN Notices, 49(6), 326–336. https://doi.org/10.1145/2594291.2594330, cited By 3. ArticleGoogle Scholar
• Hu, C., & Neamtiu, I. (2011a). Automating gui testing for android applications. In Proceedings of the 6th International Workshop on Automation of Software Test (pp. 77–83).
• Hu, C., & Neamtiu, I. (2011b). A gui bug finding framework for android applications (pp. 1490–1491). https://doi.org/10.1145/1982185.1982504, cited By 6.
• Hu, G., Yuan, X., Tang, Y., Yang, J. (2014). Efficiently, effectively detecting mobile app bugs with appdoctor. In Proceedings of the Ninth European Conference on Computer Systems, EuroSys ’14 (pp. 18:1–18:15). New York: ACM. https://doi.org/10.1145/2592798.2592813.
• Hu, Y., & Neamtiu, I. (2016). Fuzzy and cross-app replay for smartphone apps (pp. 50–56), https://doi.org/10.1145/2896921.2896925, cited By 0.
• Hu, Y., Azim, T., Neamtiu, I. (2015). Versatile yet lightweight record-and-replay for android (vol. 25-30-Oct-2015, pp. 349–366), https://doi.org/10.1145/2814270.2814320, cited By 4.
• Hu, Y., Neamtiu, I., Alavi, A. (2016). Automatically verifying and reproducing event-based races in android apps. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016 (pp. 377–388). New York: ACM. https://doi.org/10.1145/2931037.2931069.
• Imparato, G. (2015). A combined technique of gui ripping and input perturbation testing for android apps (vol. 2, pp. 760–762). https://doi.org/10.1109/ICSE.2015.241, cited By 0.
• ISO 29119 Software Testing Standard. (2013). Software and systems engineering software testing part 1: concepts and definitions. ISO/IEC/IEEE, 29119-1 (E), 1–64. https://doi.org/10.1109/IEEESTD.2013.6588537. Google Scholar
• Jaaskelainen, A., Takala, T., Katara, M. (2012). Model-Based Gui Testing of Android Applications, Addison-Wesley Professional (Pearson Education) (pp. 253–275).
• Jamrozik, K., & Zeller, A. (2016). Droid mate: A robust and extensible test generator for android (pp. 293–294). https://doi.org/10.1145/2897073.2897716, cited By 0.
• Jensen, C.S., Prasad, M.R., Moller, A. (2013). Automated testing with targeted event sequence generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, ISSTA (pp. 67–77). New York: ACM. https://doi.org/10.1145/2483760.2483777.
• Jha, A., Lee, S., Lee, W. (2015). Modeling and test case generation of inter-component communication in android (pp. 113–116). https://doi.org/10.1109/MobileSoft.2015.24, cited By 2.
• Jiang, B., Long, X., Gao, X. (2007). Mobiletest: A tool supporting automatic black box test for software on smart mobile devices. https://doi.org/10.1109/AST.2007.9, cited By 1.
• Jiang, B., Chen, P., Chan, W.K., Zhang, X. (2016). To what extent is stress testing of android tv applications automated in industrial environments? IEEE Transactions on Reliability, 65(3), 1223–1239. https://doi.org/10.1109/TR.2015.2481601. ArticleGoogle Scholar
• Jiang, B., Zhang, Y., Chan, W.K., Zhang, Z. (2017). Which factor impacts gui traversal-based test case generation technique most? a controlled experiment on android applications. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS) (pp. 21–31). https://doi.org/10.1109/QRS.2017.12.
• Joorabchi, M., Ali, M., Mesbah, A. (2016). Detecting inconsistencies in multi-platform mobile apps (pp. 450–460). https://doi.org/10.1109/ISSRE.2015.7381838, cited By 0.
• Kaasila, J., Ferreira, D., Kostakos, V., Ojala, T. (2012). Testdroid: Automated remote ui testing on android. https://doi.org/10.1145/2406367.2406402, cited By 5.
• Kirubakaran, B., & Karthikeyani, V. (2013). Mobile application testing 2014; challenges and solution approach through automation. In 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering (PRIME) (pp. 79–84). https://doi.org/10.1109/ICPRIME.2013.6496451.
• Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE Technical Report EBSE-2007-01, Software Engineering Group School of Computer Science and Mathematics. Keele University and Department of Computer Science, University of Durham.
• Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S. (2009). Systematic literature reviews in software engineering - a systematic literature review. Information and Software Technology, 51(1), 7–15. https://doi.org/10.1016/j.infsof.2008.09.009. ArticleGoogle Scholar
• Kochhar, P.S., Thung, F., Nagappan, N., Zimmermann, T., Lo, D. (2015). Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST) (pp. 1–10). https://doi.org/10.1109/ICST.2015.7102609.
• Li, A., Qin, Z., Chen, M., Liu, J. (2014a). Adautomation: An activity diagram based automated gui testing framework for smartphone applications (pp. 68–77). https://doi.org/10.1109/SERE.2014.20, cited By 1.
• Li, L., Bissyande, T., Papadakis, M., Rasthofer, S., Bartel, A., Octeau, D., Klein, J., Le Traon, Y. (2016a). Static analysis of android apps: A systematic literature review. Technical report, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg, Fraunhofer SIT Technische Universitat Darmstadt. Germany: University of Wisconsin and Pennsylvania State University.
• Li, Q., Jiang, Y., Gu, T., Xu, C., Ma, J., Ma, X., Lu, J. (2016b). Effectively manifesting concurrency bugs in android apps. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC) (pp. 209–216). https://doi.org/10.1109/APSEC.2016.038.
• Li, X., Jiang, Y., Liu, Y., Xu, C., Ma, X., Lu, J. (2014b). User guided automation for testing mobile apps. In Software engineering conference (APSEC), 2014 21st asia-pacific (Vol. 1, pp. 27–34). https://doi.org/10.1109/APSEC.2014.13.
• Li, Y., Yang, Z., Guo, Y., Chen, X. (2017). Droidbot: A lightweight ui-guided test input generator for android. In Proceedings of the 39th International Conference on Software Engineering Companion, ICSE-C ’17 (pp. 23–26). Piscataway: IEEE Press. https://doi.org/10.1109/ICSE-C.2017.8.
• Liang, C.J.M., Lane, N.D., Brouwers, N., Zhang, L., Karlsson, B.F., Liu, H., Liu, Y., Tang, J., Shan, X., Chandra, R., Zhao, F. (2014). Caiipa: Automated large-scale mobile app testing through contextual fuzzing. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (pp. 519–530). New York: ACM. https://doi.org/10.1145/2639108.2639131.
• Lin, Y.D., Rojas, J., Chu, E.H., Lai, Y.C. (2014). On the accuracy, efficiency, and reusability of automated test oracles for android devices. Software Engineering. IEEE Transactions on, 40(10), 957–970. https://doi.org/10.1109/TSE.2014.2331982. Google Scholar
• Linares-Vasquez, M. (2015a). Enabling testing of android apps (Vol. 2, pp. 763–765). https://doi.org/10.1109/ICSE.2015.242, cited By 1.
• Linares-Vasquez, M., White, M., Bernal-Cardenas, C., Moran, K., Poshyvanyk, D. (2015b). Mining android app usages for generating actionable gui-based execution scenarios (vol. 2015-August, pp. 111–122). https://doi.org/10.1109/MSR.2015.18, cited By 2.
• Liu, Z., Gao, X., Long, X. (2010a). Adaptive random testing of mobile application. In 2010 2nd international conference on Computer engineering and technology (ICCET) (Vol. 2, pp. V2–297). IEEE .
• Liu, Z., Liu, B., Gao, X. (2010b). Test automation on mobile device (pp. 1–7). https://doi.org/10.1145/1808266.1808267, cited By 0.
• Liu, Y., & Xu, C. (2013). Veridroid: Automating android application verification. https://doi.org/10.1145/2541534.2541594, cited By 0.
• Liu, C.H., Lu, C.Y., Cheng, S.J., Chang, K.Y., Hsiao, Y.C., Chu, W.M. (2014a). Capture-replay testing for android applications. In 2014 International Symposium on Computer, Consumer and Control (IS3C) (pp. 1129–1132). https://doi.org/10.1109/IS3C.2014.293.
• Liu, Y., Lu, Y., Li, Y. (2014b). An android-based approach for automatic unit test (Vol. 2014), https://doi.org/10.1049/cp.2014.1290, cited By 0.
• Liu, Y., Xu, C., Cheung, S.C., Yang, W. (2014c). Checkerdroid : Automated quality assurance for smartphone applications. Int J Software and Informatics, 8, 21–41.
• Liu, C.H., Chen, S.L., Chen, H.K. (2015). Robotdroid-a keyword-driven testing tool for android applications. Frontiers in Artificial Intelligence and Applications, 274, 1865–1874. https://doi.org/10.3233/978-1-61499-484-8-1865 , cited By 0. Google Scholar
• Liu, P., Zhang, X., Pistoia, M., Zheng, Y., Marques, M., Zeng, L. (2017). Automatic text input generation for mobile testing. In Proceedings of the 39th International Conference on Software Engineering, ICSE ’17 (pp 643–653). Piscataway: IEEE Press. https://doi.org/10.1109/ICSE.2017.65.
• Lu, L., Hong, Y., Huang, Y., Su, K., Yan, Y. (2012). Activity page based functional test automation for android application (pp. 37–40). https://doi.org/10.1109/WCSE.2012.15, cited By 3.
• Ma, X., Wang, N., Xie, P., Zhou, J., Zhang, X., Fang, C. (2016). An automated testing platform for mobile applications (pp. 159–162). https://doi.org/10.1109/QRS-C.2016.25, cited By 0.
• Machado, P., Campos, J., Abreu, R. (2013). Mzoltar: Automatic debugging of android applications (pp 9–16). https://doi.org/10.1145/2501553.2501556, cited By 3.
• Machiry, A., Tahiliani, R., Naik, M. (2013). Dynodroid: an input generation system for android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013 (pp. 224–234). New York: ACM. https://doi.org/10.1145/2491411.2491450.
• Mahmood, R., Mirzaei, N., Malek, S. (2014). Evodroid: Segmented evolutionary testing of android apps. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014 (pp. 599–609). New York: ACM. https://doi.org/10.1145/2635868.2635896.
• Maiya, P., Kanade, A., Majumdar, R. (2014). Race detection for android applications. SIGPLAN Not, 49(6), 316–325. https://doi.org/10.1145/2666356.2594311. ArticleGoogle Scholar
• Majeed, S., & Ryu, M. (2016). Model-based replay testing for event-driven software (vol. 04-08-April-2016, pp. 1527–1533). https://doi.org/10.1145/2851613.2851794, cited By 0.
• Maji, A., Arshad, F., Bagchi, S., Rellermeyer, J. (2012). An empirical study of the robustness of inter-component communication in android. In 2012 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) (pp. 1–12). https://doi.org/10.1109/DSN.2012.6263963.
• Mao, K., Harman, M., Jia, Y. (2016). Sapienz: Multi-objective automated testing for android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016 (pp. 94–105), New York: ACM. https://doi.org/10.1145/2931037.2931054.
• Matalonga, S., Rodrigues, F., Travassos, G.H. (2017). Characterizing testing methods for context-aware software systems: Results from a quasi-systematic literature review. Journal of Systems and Software, 131(Supplement C), 1–21. https://doi.org/10.1016/j.jss.2017.05.048. ArticleGoogle Scholar
• Mayan, A.J., Menezes, J.R., Bruce, J. (2015). Developing a mobile based automated testing tool for windows phone 8. Modern Applied Science, 9, 91–98. ArticleGoogle Scholar
• Mendez-Porras, A., Nieto Hidalgo, M., Garcia-Chamizo, J., Jenkins, M., Porras, A. (2015a). A top-down design approach for an automated testing framework. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9454, 37–49. https://doi.org/10.1007/978-3-319-26401-1_4, cited By 0.
• Mendez-Porras, A., Quesada-Lopez, C., Jenkins, M. (2015b). Automated testing of mobile applications: a systematic map and review. In Araujo, J., Condori-Fernandez, N., Goulão, M., Matalonga, S., Bencomo, N., Oliveira, T., de la Vara, J. L., Brito, I. S., Antonelli, L., Pimentel, E., Miranda, J. J., Kalinowski, M., Pastor, Ó., Olsina, L., Guizzardi, R., España, S., Cuadros-Vargas, E. (Eds.) XVIII Ibero-american conference on software engineering, URP,SPC,UCSP (pp. 195–208). Lima-Peru: UCSP.
• Meng, Z., Jiang, Y., Xu, C. (2015). Facilitating reusable and scalable automated testing and analysis for android apps. In Proceedings of the 7th Asia-Pacific Symposium on Internetware, ACM, New York, NY, USA, Internetware ’15 (pp. 166–175). https://doi.org/10.1145/2875913.2875937.
• Mirzaei, N., Malek, S., Pasareanu, C.S., Esfahani, N., Mahmood, R. (2012). Testing android apps through symbolic execution. SIGSOFT Softw Eng Notes, 37(6), 1–5. https://doi.org/10.1145/2382756.2382798. ArticleGoogle Scholar
• Mirzaei, H., & Heydarnoori, A. (2015). Exception fault localization in android applications (pp 156–157). https://doi.org/10.1109/MobileSoft.2015.42, cited By 0.
• Mirzaei, N., Bagheri, H., Mahmood, R., Malek, S. (2016a). Sig-droid: Automated system input generation for android applications (pp. 461–471). https://doi.org/10.1109/ISSRE.2015.7381839, cited By 0.
• Mirzaei, N., Garcia, J., Bagheri, H., Sadeghi, A., Malek, S. (2016b). Reducing combinatorics in gui testing of android applications (Vol. 14-22-May-2016, pp. 559–570). https://doi.org/10.1145/2884781.2884853, cited By 0.
• Moran K, Linares-Vasquez, M., Bernal-Cardenas, C., Vendome, C., Poshyvanyk, D. (2016). Automatically discovering, reporting and reproducing android application crashes. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) (pp. 33–44). https://doi.org/10.1109/ICST.2016.34.
• Moran, K., Linares-Vasquez, M., Bernal-Cardenas, C., Vendome, C., Poshyvanyk, D. (2017). Crashscope: A practical tool for automated testing of android applications. In Proceedings of the 39th International Conference on Software Engineering Companion, ICSE-C ’17 (pp. 15–18). Piscataway: IEEE Press. https://doi.org/10.1109/ICSE-C.2017.16.
• Morgado, I., & Paiva, A. (2016a). Impact of execution modes on finding android failures (Vol. 83, pp. 284–291). https://doi.org/10.1016/j.procs.2016.04.127, cited By 0.
• Morgado, I., & Paiva, A. (2016b). Testing approach for mobile applications through reverse engineering of ui patterns (pp 42–49). https://doi.org/10.1109/ASEW.2015.11, cited By 0.
• Muccini, H., Di Francesco, A., Esposito, P. (2012). Software testing of mobile applications: Challenges and future research directions. In 2012 7th International Workshop on Automation of Software Test (AST) (pp. 29–35). https://doi.org/10.1109/IWAST.2012.6228987.
• Nagowah, L., & Sowamber, G. (2012). A novel approach of automation testing on mobile devices (Vol. 2, pp. 924–930). https://doi.org/10.1109/ICCISci.2012.6297158, cited By 3.
• Nguyen, C., Marchetto, A., Tonella, P. (2012). Combining model-based and combinatorial testing for effective test case generation (pp. 100–110). https://doi.org/10.1145/04000800.2336765, cited By 25.
• Packevicius, S., Usaniov, A., Stanskis, S., Bareisa, E. (2015). The Testing Method Based on Image Analysis for Automated Detection of UI Defects Intended for Mobile Applications, (pp. 560–576). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-24770-0_48. Google Scholar
• Paulovsky, F., Pavese, E., Garbervetsky, D. (2017). High-coverage testing of navigation models in android applications. In Proceedings of the 12th International Workshop on Automation of Software Testing, AST ’17 (pp. pp 52–58). Piscataway: IEEE Press. https://doi.org/10.1109/AST.2017.6.
• Petersen, K., Feldt, R., Mujtaba, S., Mattsson, M. (2008). Systematic mapping studies in software engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, EASE’08 (pp. 68–77). Swinton: British Computer Society.
• Petersen, K., Vakkalanka, S., Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: an update. Information and Software Technology, 64, 1–18. https://doi.org/10.1016/j.infsof.2015.03.007. ArticleGoogle Scholar
• Prathibhan, C.M., Malini, A., Venkatesh, N., Sundarakantham, K. (2014). An automated testing framework for testing android mobile applications in the cloud. In 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies (pp. 1216–1219). https://doi.org/10.1109/ICACCCT.2014.7019292.
• Puspika, B., Hendradjaya, B., Danar Sunindyo, W. (2015). Towards an automated test sequence generation for mobile application using colored petri net (pp. 445–449). https://doi.org/10.1109/ICEEI.2015.7352542, cited By 0.
• Qin, Y., Xu, C., Yu, P., Lu, J. (2016a). Sit: Sampling-based interactive testing for self-adaptive apps. Journal of Systems and Software, 120, 70–88. https://doi.org/10.1016/j.jss.2016.07.002, cited By 0.
• Qin, Z., Tang, Y., Novak, E., Li, Q. (2016b). Mobiplay: A remote execution based record-and-replay tool for mobile applications (Vol. 14-22-May-2016, pp. 571–582). https://doi.org/10.1145/2884781.2884854, cited By 1.
• Raut, P., & Tomar, S. (2014). Android mobile automation framework. International Journal of Multidisciplinary Approach and Studies. 1–12.
• Ravindranath, L., Nath, S., Padhye, J., Balakrishnan, H. (2014). Automatic and scalable fault detection for mobile applications. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’14, pp. 190–203. ACM. https://doi.org/10.1145/2594368.2594377.
• Reddy, K., Babu Rajesh, V., Pareek, H., Patil, M. (2016). Dynaldroid: A system for automated dynamic analysis of android applications (pp. 124–129). https://doi.org/10.1109/RAECE.2015.7510239, cited By 0.
• Saad, N., & Awang Abu Bakar, N. (2014). Automated testing tools for mobile applications. https://doi.org/10.1109/ICT4M.2014.7020665, cited By 0.
• Sadeh, B. (2011). A study on the evaluation of unit testing for android systems. International Journal of New Computer Architectures and their Applications (IJNCAA), 4(1).
• Sahinoglu, M., Incki, K., Aktas, M.S. (2015). Mobile Application Verification: A Systematic Mapping Study, (pp. 147–163). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-21413-9_11. Google Scholar
• Salihu, I.A., & Ibrahim, R. (2016). Systematic exploration of android apps’ events for automated testing. In Proceedings of the 14th International Conference on Advances in Mobile Computing and Multi Media, MoMM ’16 (pp. 50–54). New York: ACM. https://doi.org/10.1145/3007120.3011072.
• Salva, S., & Laurencot, P. (2015). Model inference and automatic testing of mobile applications. In International Journal On Advances in Software, Vol. 8. Iaria.
• San Miguel, J.L., & Takada, S. (2016). Gui and usage model-based test case generation for android applications with change analysis. In Proceedings of the 1st International Workshop on Mobile Development, Mobile! 2016 (pp. 43–44), New York: ACM. https://doi.org/10.1145/3001854.3001865.
• Shabaan, M.M., Hamza, H.S., Omar, Y.M.K. (2017). Effects of fsm minimization techniques on number of test paths in mobile applications mbt. In 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA) (pp. 297–302). https://doi.org/10.1109/SERA.2017.7965741.
• Shan, Z., Azim, T., Neamtiu, I. (2016). Finding resume and restart errors in android applications. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2016 (pp. 864-880). New York: ACM. https://doi.org/10.1145/2983990.2984011.
• She, S., Sivapalan, S., Warren, I. (2009). Hermes: A Tool for testing mobile device applications. In Software engineering conference, 2009. ASWEC’09 (pp. 121–130). Australian: IEEE.
• Silva, D.B., Endo, A.T., Eler, M.M., Durelli, V.H.S. (2016). An analysis of automated tests for mobile android applications. In 2016 XLII Latin American Computing Conference (CLEI) (pp. 1–9). https://doi.org/10.1109/CLEI.2016.7833334.
• Song, K., Han, A.R., Jeong, S., Cha, S. (2015). Generating various contexts from permissions for testing android applications (Vol. 2015-January, pp. 87–92). https://doi.org/10.18293/SEKE2015-118, cited By 0.
• Su, T. (2016). Fsmdroid: Guided gui testing of android apps. In Proceedings of the 38th International Conference on Software Engineering Companion, ICSE ’16 (pp. 689–691). New York: ACM. https://doi.org/10.1145/2889160.2891043.
• Sun, C., Zhang, Z., Jiang, B., Chan, W.K. (2016). Facilitating monkey test by detecting operable regions in rendered gui of mobile game apps. In 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS) (pp. 298–306). https://doi.org/10.1109/QRS.2016.41.
• Takala, T., Katara, M., Harty, J. (2011). Experiences of system-level model-based gui testing of an android application. In Proceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST ’11 (pp. 377–386). Washington: IEEE Computer Society. https://doi.org/10.1109/ICST.2011.11.
• Tang, H., Wu, G., Wei, J., Zhong, H. (2016). Generating test cases to expose concurrency bugs in android applications (pp. 648–653). https://doi.org/10.1145/2970276.2970320, cited By 0.
• Tao, C., & Gao, J. (2016). On building test automation system for mobile applications using gui ripping (vol 2016-January, pp. 480–485). https://doi.org/10.18293/SEKE2016-168, cited By 0.
• van der Merwe, H., van der Merwe, B., Visser, W. (2012). Verifying android applications using java pathfinder. SIGSOFT Softw Eng Notes, 37(6), 1–5. https://doi.org/10.1145/2382756.2382797.
• Vilkomir, S., & Amstutz, B. (2014). Using combinatorial approaches for testing mobile applications. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (pp. 78–83). https://doi.org/10.1109/ICSTW.2014.9.
• Vilkomir, S., Marszalkowski, K., Perry, C., Mahendrakar, S. (2015). Effectiveness of multi-device testing mobile applications. In 2015 2nd ACM International Conference on Mobile Software Engineering and Systems (pp. 44–47). https://doi.org/10.1109/MobileSoft.2015.12.
• Wang, P., Liang, B., You, W., Li, J., Shi, W. (2014). Automatic android gui traversal with high coverage. In Proceedings of the 2014 Fourth International Conference on Communication Systems and Network Technologies, CSNT ’14 (pp. 1161–1166). Washington: IEEE Computer Society. https://doi.org/10.1109/CSNT.2014.236.
• Wen, H.L., Lin, C.H., Hsieh, T.H., Yang, C.Z. (2015). Pats: a parallel gui testing framework for android applications. In 2015 IEEE 39Th annual computer software and applications conference (Vol. 2. pp. 210–215). https://doi.org/10.1109/COMPSAC.2015.80.
• White, M., Linares-Vasquez, M., Johnson, P., Bernal-Cardenas, C., Poshyvanyk, D. (2015). Generating reproducible and replayable bug reports from android application crashes. In 2015 IEEE 23rd International Conference on Program Comprehension (ICPC) (pp. 48–59). https://doi.org/10.1109/ICPC.2015.14.
• Wu, X., Jiang, Y., Xu, C., Cao, C., Ma, X., Lu, J. (2016). Testing android apps via guided gesture event generation. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC) (pp. 201–208). https://doi.org/10.1109/APSEC.2016.037.
• Yang, W., Prasad, M.R., Xie, T. (2013). A grey-box approach for automated gui-model generation of mobile applications. In Fundamental Approaches to Software Engineering (pp. 250–265). Springer.
• Ye, H., Cheng, S., Zhang, L., Jiang, F. (2013). Droidfuzzer: Fuzzing the android apps with intent-filter tag. In Proceedings of International Conference on Advances in Mobile Computing and Multimedia, MoMM ’13 (pp. 68:68–68:74). New York: ACM. https://doi.org/10.1145/2536853.2536881.
• Yeh, C.C., Lu, H.L., Chen, C.Y., Khor, K.K., Huang, S.K. (2014). Craxdroid: Automatic android system testing by selective symbolic execution (pp. 140–148). https://doi.org/10.1109/SERE-C.2014.32, cited By 3.
• Yoo, H., & Lee, Y. (2017). An automatic mobile app. testing method with user event scenario. In 2017 18th IEEE International Conference on Mobile Data Management (MDM) (pp. 394–396). https://doi.org/10.1109/MDM.2017.71.
• Yu, S., & Takada, S. (2016). Mobile application test case generation focusing on external events. In Proceedings of the 1st International Workshop on Mobile Development, Mobile! 2016 (pp. 41-42). New York: ACM. https://doi.org/10.1145/3001854.3001864.
• Zaeem, R.N., Prasad, M.R., Khurshid, S. (2014). Automated generation of oracles for testing user-interaction features of mobile apps. In Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation, ICST ’14 (pp. 183–192). Washington: IEEE Computer Society. https://doi.org/10.1109/ICST.2014.31.
• Zein, S., Salleh, N., Grundy, J. (2016). A systematic mapping study of mobile application testing techniques. Journal of Systems and Software, 117, 334–356. https://doi.org/10.1016/j.jss.2016.03.065. ArticleGoogle Scholar
• Zeng, X., Li, D., Zheng, W., Xia, F., Deng, Y., Lam, W., Yang, W., Xie, T. (2016). Automated test input generation for android: Are we really there yet in an industrial case?. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016 (pp. 987–992). New York: ACM. https://doi.org/10.1145/2950290.2983958.
• Zhang, S., & Pi, B. (2015a). Mobile functional test on taas environment. In 2015 IEEE Symposium on Service-Oriented System Engineering (pp. 315–320). https://doi.org/10.1109/SOSE.2015.27.
• Zhang, T., Gao, J., Cheng, J., Uehara, T. (2015a). Compatibility testing service for mobile applications. In 2015 IEEE Symposium on Service-Oriented System Engineering (pp. 179–186). https://doi.org/10.1109/SOSE.2015.35.
• Zhang, A., He, Y., Jiang, Y. (2016). Crashfuzzer: Detecting input processing related crash bugs in android applications. In 2016 IEEE 35th International Performance Computing and Communications Conference (IPCCC) (pp. 1–8). https://doi.org/10.1109/PCCC.2016.7820625.
• Zhauniarovich, Y., Philippov, A., Gadyatskaya, O., Crispo, B., Massacci, F. (2015). Towards black box testing of android apps. In 2015 10th International Conference on Availability, Reliability and Security (ARES) (pp. 501–510). https://doi.org/10.1109/ARES.2015.70.
• Zheng, H., Li, D., Liang, B., Zeng, X., Zheng, W., Deng, Y., Lam, W., Yang, W., Xie, T. (2017). Automated test input generation for android: Towards getting there in an industrial case. In Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP ’17 (pp. 253–262). Piscataway: IEEE Press. https://doi.org/10.1109/ICSE-SEIP.2017.32.
• Zhu, H., Ye, X., Zhang, X., Shen, K. (2015). A context-aware approach for dynamic gui testing of android applications (vol 2, pp. 248–253). https://doi.org/10.1109/COMPSAC.2015.77, cited By 0.
• Zun, D., Qi, T., Chen, L. (2016). Research on automated testing framework for multi-platform mobile applications. In 2016 4th International Conference on Cloud Computing and Intelligence Systems (CCIS) (pp. 82–87). https://doi.org/10.1109/CCIS.2016.7790229.

Author information

Authors and Affiliations

1. Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy Porfirio Tramontana, Domenico Amalfitano, Nicola Amatucci & Anna Rita Fasolino

1. Porfirio Tramontana

Corresponding author

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

About this article

Cite this article

Tramontana, P., Amalfitano, D., Amatucci, N. et al. Automated functional testing of mobile applications: a systematic mapping study. Software Qual J 27, 149–201 (2019). https://doi.org/10.1007/s11219-018-9418-6

• Published : 24 October 2018
• Issue Date : 15 March 2019
• DOI : https://doi.org/10.1007/s11219-018-9418-6

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

• Mobile applications
• Testing automation
• Functional testing
• Systematic mapping