Let’s talk a bit about the testing matter. You know why? Because we try to be as transparent about our work process as possible. Of course we care about the quality of our services, but don’t expect us to give empty promises. In this post we are going to speak about what is mobile app testing and what we actually mean when guarantee crashes and bugs minimization.
To start with, I’d like to make it clear that if your product goes through hands of a QA team, it is on the right path! Jumping over the whole process of testing, let’s land at the after release phase. It may happen that you or the users have found a bug and even wrote a review mentioning that. Why on Earth could that happen? Sounds like debunking myths about QA, huh? After all, nothing is perfect, and a word-combination «bugs minimization» doesn’t mean «bugs exclusion». To head for testing rules, let’s figure what sort of bugs we actually come across.
Bugs' infernal majesty
There is absolutely no mobile apps without bugs. In fact, bugs are more like bacteria, than like those pretty insects in the picture. They are ubiquitous and everlasting with varying levels of inconvenience to the user of the program.
We decided to illustrate the types of bugs on the example of a well-known Skype in order to remind you the basics of the bug hunting studies.
Blocker bug literary kills your app. Example: a user can’t login, send a message or make a call.
Critical bug is some sort of an artificial limb that can’t function as the real one, but you can still walk with it. Example: there is a group chat which doesn’t work when you try to send a message, but you can send this message to each member of the group chat separately.
Major bug shoots off some parts of the app’s body. For example, settings cannot be changed, volume doesn’t react to your attempts to turn it up or down, a user can’t upload or change a profile picture, search by contacts doesn’t work, etc.
Minor is a minor breakage which you can ignore if you aren’t a perfectionist. Example: some obvious issue with UI, like when you can’t increase or decrease the text entry field.
Trivial bug is something to be ashamed of and hope no one will notice. It can originate from a third party service or a library and has no effect on the overall quality of the product. For example, typo (apart from those in the name of the app or on the main pages).
The idea of a bug comes up when the program doesn’t act as you expect it to, not mentioning some obvious troubles that anyone may notice while simply pushing the buttons.
So what exactly do you expect from your product? And most importantly how can we learn about your app’s expected behavior?
What to expect from mobile app testing
So, why to test your mobile app? The job of a mobile app tester is to find the maximum number of critical bugs in the minimal time or confirm that your app fulfills its call of duty. It’s a big mistake to believe that the goal of the testing procedure is to prove the absence of bugs. On the contrary, our target here is the detection of bugs. Probably only a superQAman can test the program and guarantee no bugs whatsoever.
In order to detect bugs that survived after a professional surgery we would advise launching a beta testing and employ real users of the app for the role of QA. Apha + Beta testing and not vice versa is the right path to high-quality result.
Even though the work of QA team cannot guarantee a sterile app, it prevents the situation of the app crash and other serious troubles that I would never wish your users to reveal.
Our mobile application testing process starts with a test plan, where we thoroughly describe what exactly there is to test (login, push notifications, sharing to social networks and so on), what types of testing we use, testing environment and the testing procedure itself. Be sure we will send you a document with bugs and description of the found problems. Check them out but please return as we still need to expand on the idea of this article.
The QA team hands in the app to the customer after:
- All the test cases on all the devices and operating systems have been passed.
- All the testing resources (people, time) have been employed.
- All the bugs found during the process of testing have been fixed.
NOTE: Beware of the products that have not undergone the testing procedure performed by professionals and never wish your app to follow in their erroneous steps!
Where does the expected behavior come from?
Usually together with the wish to build a product, a customer also explains how this product should function. Some could do it verbally but you should know that we would ask to document the program behavior in the form of requirements specification, wireframes or user cases before the coding phase starts. (By the way, you can always make these guys write everything by themselves and, afterward, seal their technical composition with your client’s approval).
Some may say: «Come on, sometimes there are obvious bugs clear for a human brain without a need to look into a doc of any kind!» That’s true that crashes or inconsistency with a common sense can be found without a spec. But this style of defects detection isn’t great for the apps with complex internal logic and can never guarantee that the customer will get exactly what they expect.
But yeah, we know, writing a spec is a long and difficult task that busy entrepreneurs would prefer to avoid. Also a spec is more applicable to Waterfall development model, where there is always a separate testing phase near the completion of an implementation phase.
So in case you don’t have a spec with clearly described program behavior for every part of the app’s functional, we have something to offer.
User stories & acceptance tests
What we can do and do it well is provide you with a list of user stories describing the main functional blocks. For example, «as a user I can sign up to the app using my email and password», or «as a user I can share my photos to social networks».
The details of this functionality, such as complexity of the password a user enters to register, how the app will behave in case the user’s email is invalid, which social networks user pictures will be shared to, whether it is possible to add a comment to the picture, what can the comment contain, etc., are specified in the acceptance tests.
User stories and acceptance tests are much faster to write than a spec. If you choose Agile (Scrum) methodology for your product development which is based on short sprints (iterations), the user stories and acceptance tests are getting prepared little by little for each next sprint, not for the whole project.
This approach helps to significantly save time and to work out all the details of a particular function. In order to get acceptance tests into gear we would only need communication with the customer.
You can read more about iOS and Android app testing and quality assurance at Yalantis in our in-depth article.