Imagine that a Project Manager has approached you with a question “how is the testing going?”. We will let you know how and what to answer in this article.
An inexperienced tester will dive into numbers straight away, and his answer will sound like this: “Why, everything’s cool. I’ve completed 234 test cases out of 500. 16 automated tests out of 100 have gone down”. This here is a bad answer. Dry numbers with no context whatsoever do not reflect the state of the product, thus, useless. They do not help the manager decide what to do next and how the team should proceed.
An experienced tester has to provide his team with useful information that will help assess risks correctly and set up the priorities.
How to present information?
At Fora Soft, we present the useful information in this order:
Explain the state of the product. What serious problems we’ve encountered, why they are serious, and how they can affect our customers. This information helps the team understand what to deal with first
Explain how the testing is going. What still needs testing, what has already been tested, what we will not test and why is that. It’s important to mention how the testing was being done, what environment was used and why. This data is mandatory to assess the risk of receiving problems in the untested product areas and correct the testing plan, should the need arise
Explain why we test the way we do it. Why the tests we’ve chosen are more effective than those we haven’t. When time and resources are limited, it’s crucial that we choose the right testing areas and sort out priorities
Let the team know about the problems we’ve encountered during testing. Namely, what makes testing harder, what may cause us to miss bugs, what could help us make testing faster and simpler. If your team knows about your problems, they can help you
The main task the tester has is finding problems that put the product’s value in danger and reporting on those problems to the project manager and the team. Providing this information in a timely manner allows creating a high-quality product without missing deadlines.
To catch any problem that endangers the product’s value quickly, we use test plans and test strategies. Stay tuned to find out how the plan and strategy are created!
Do you want to learn more about our processes and how we do things? Do not hesitate to contact us via our contact form! It is right here 🙂
According to Businessofapps, mobile apps become more and more popular. They offer more functions that are in need of testing. This is a crucial process in development. No one wants to use a lousy app that keeps crashing or working bad, right? Also, what if there’s a payment function in an app? Working with a product that can let you down when you send money somewhere isn’t a good idea.
To improve the quality of an app, we need to test it. It’s believed that the developer’s eye eventually starts swimming and they can miss some problems. So, we have testers for that. To increase the speed of testing and implement a system in it, testers automate the process.
Let’s take a look at testing automation using iOS apps as an example. Nowadays, mobile apps have way more functionality than before, so testing takes more time, too. And don’t forget that there are many kinds of iOS devices out there, which increases the time spent on testing even further! To guarantee that an app works correctly on all devices, many tests need to be done, which makes the process longer and more expensive. Testing automation is out there to deal with it because testers won’t need to check some functions manually. It’s enough to write a script and edit it from time to time. Sounds good, doesn’t it? Nothing is perfect, though, so let’s turn to the pros and cons of this method:
Concurrent testing on multiple devices
Faster testing process
No human factor. Sometimes, bugs might appear that are difficult to catch. Even if a tester is able to catch those, they can not always understand what the cause was. Automated testing may help with the precision of understanding
Testing transparency. When the testers swap, old scripts stay, and the testing process will continue as intended. Regression testing stays the same, too. If you need to change something, or if a new tester wants to check the application logic, the script will work as documentation. This is one of the main advantages
When iOS updates, you have to wait for the automated testing tools to update, too
The tester needs to have special knowledge about automated testing scripts. The company needs to teach employees to do that or hire more expensive ones
The auto testing tools have pros and cons about them. It’s difficult to find a perfect tool, and oftentimes you need to sacrifice convenience in working with a tool or with its possibilities.
When you need automated testing
Do you even need it? The answer is “yes”, if:
Your app has too many functions and features, and you are going to support it while adding new things along the way. Why do you need auto testing then? New functionality can conflict with the old one. For example, you’ve introduced a chat and calls broke. To find out the problem, the tester has to test the whole app manually. It takes a lot of time, and the tester is at risk of missing something else! Auto testing helps avoid that problem, as the tester won’t have to test everything after introducing each feature. Whatever stayed the same will be checked automatically, as long as you launch them and then collect the data. It helps reduce the testing time and the development costs
You are going to adapt the app for each new iOS version and take advantage of new system possibilities. Every iOS update can break something within the app. Even if you never planned to update the app in the near future, it might be that you have to. In that case, automated testing will help with that. After that, you’ll understand what it was that broke and be able to solve the problem. Obviously, you wouldn’t add automated testing with this sole purpose, but they will help a great deal if they are already implemented
There are testers on your team that possess some knowledge of automated testing. They at least have to know some popular programming language if you go for, say, Appium. Or, they have to know Swift if your choice is XCTest / Earl Gray / KIF. Testers also need to know all the possible testing methods and needed tools. If your employees only know how to manually test apps and have no knowledge of programming languages whatsoever, you will either have to teach them or hire new ones
However, writing automated tests is programming, although you’re not writing new functions for your app but rather a program that goes through your product and checks it. It is expensive. It won’t be worth it to add automated tests if:
The app is small. It doesn’t have lots of functions and it is easy to test it manually. With that being said, you are not planning on adding new functions on a constant basis
The app is supposed to be developed and distributed within a short period of time, such as those for the World Cup-2018 or the Olympics-2014
The app changes frequently. The functionality is unstable. For instance, a startup that looks for its client and keeps changing the main features
Tools for automated testing in iOS
After finding out the main advantages and disadvantages of this approach, let’s take a look at the tools.
Apple developed a fully native tool that is out there only for testing iOS apps. Since it is native, external dependencies won’t be there. You develop tests in Swift or Objective-C, which helps developers and testers interact more effectively. However, developing in those languages isn’t that simple. It might be that testers will turn to developers to ask for help far too often, which will make work a bit chaotic.
There is a test recorder, too. It records real actions with an app and creates a test out of them, but using it is actually quite hard. It isn’t very accurate and it’s best to use it as an assisting tool while developing main tests in Swift or Objective-C. XCUITest/XCTest also works in a separate stream, doesn’t read the state of an app. Therefore, delays in updating the data may lead to an impossibility of seeing requested elements.
The Framework by Google. It requires tests in Objective-C or Swift. The framework synchronizes requests, UI, and streams – that’s the advantage of it. However, EarlGray isn’t very popular because you can only test iOS apps with it. It isn’t very different from XCUITest, yet it is not native, so testers would rather use XCUITest.
KIF is a framework that has to be added to the project to use it. Objective-C or Swift are the testing languages. Its realism is its main competitive edge. KIF can simulate interaction with a user, therefore it’s very good for UI testing.
You see the iOS-only tools above but when mobile development is in question, oftentimes the developers go for both iOS and Android apps. So no wonder there are cross-platform tools for automated testing.
Appium is the most popular tool nowadays. It allows testing apps regardless of the platform, type, and system version. Writing tests for each platform is possible using unified API, without transforming an app into a special, network-compatible kind. Appium doesn’t require adding to the app source code. It’s working as a separate tool. Let’s take a look at its advantages:
A big number of languages for tests: Java, C#, Python, Ruby. It means that Appium doesn’t only work with Objective-C or Swift, so all testers will be able to create tests
An app doesn’t need re-compiling or changing it for automation’s sake. It’s important because the test source code and the app source code aren’t in the same project, and they are developed separately. These two don’t depend on each other, so one can avoid many problems. For example, if somebody wrote the tests incorrectly and they don’t compile, it won’t affect the app in general
It is cross-platform. The testers can develop tests for iOS and Android in the same environment, in the same language. They can even re-use the code. It saves time and money
Wide functionality. You can launch and stop the app, check the visibility of elements on the screen, and use gestures. Simulator and real devices work with Appium
Appium has some disadvantages, too. It is essentially a superstructure over native iOS and Android drivers. The tests can break more often due to the mistakes in the superstructure code. It’s important to notice here that Appium is very popular, develops quickly, so arising problems will be likely solved in the future.
Automated tests gain popularity in mobile development. They have advantages and disadvantages. Introducing automated tests is worth it when the benefit outweighs the costs. It is not magic, it’s still development, but you’re not developing new functionality but the ways to test the old. If an app has many functions and you keep updating it, but the majority of functions stay the same with each version, take a look at automated testing. After spending money once, you will save more later on.
Testers won’t have to test everything manually with each update. You only changed the teacher’s profile, but all courses, login, payment, booking, admin dashboard have to work the way they did? The automated tests will have to work for it then. If they didn’t, it means that your small update broke something, now you have to figure out what. If the tests went through – be happy, because the tester will only manually check the teacher’s profile instead of the whole functionality.
We took a look at the tools that enable automated testing. It’s worth saying that we didn’t talk about all the tools out there. There are still paid, unpopular solutions or solutions that are not supported by the developer anymore. Whenever you choose a tool, consider the application’s peculiarities, check if there will be an Android version of the app. Also, consider your tester team! 🙂
We at Fora Soft also use automated testing for some of our projects and find success with it. Do you want to learn more about this topic? Message us using the Contact us form!
When testing mobile apps, newbies QA frequently forget to check the app with an unstable Internet connection. But in many cases this is critical: connection speed directly influences user experience and workability of the main functions. It is especially true for applications where geolocation and mobile Internet are heavily in use. For example, video chats, messengers, and other multimedia products we specialize in.
In this article, we’ll show how to simulate slow Internet connection on a test device with no hassle.
Network Link Conditioner
Let’s start with a standard utility Network Link Conditioner for iOS apps testing. It lets the QA adjust the Internet connection as they need.
To switch on this function on iPhone, you need a MacOS device:
iOS lets us choose one of pre-installed presets of connection quality – or create our own preset.
For our own preset these settings are available:
Here we see that Apple took care of testing apps with different levels of connection quality and gave us almost all the needed settings.
Having got acquainted with Network Link Conditioner for iOS, we’ve been sure such a feature would be on Android too. God, how much we’ve been mistaken.
Network Link Conditioner on Android
It appeared to be impossible to emulate a slow or unstable connection on a real Android with the help of standard tools. Therefore, I faced 2 paths: download some apps from Google Play that emulate slow connection, or use a specifically precise adjustment of the Internet connection access point.
Apps didn’t work out for me ☹ All the apps that give this function require Root access, and this breaks the concept of testing in real-world conditions.
So, having left the Root access as the last resort, I decided to look closer at path #2 — adjustment of the access point.
In the past, when I was a student, mobile internet traffic was ending up quickly (and we needed to read, watch something while on the lesson), and we used an iPhone as an access point. The idea came to mind: to mix the student experience and recently gathered knowledge.
Using Network Link Conditioner and an access point made of macOS or iOS doesn’t require any extra knowledge and are easy to adjust. Exactly what’s needed if we want to save time.
So, to emulate bad connection on Android we need and Android device and… an iPhone with mobile internet Developer Tools switched on.
Make iPhone the access point (Settings > Mobile Data > Personal Hotspot)
Adjust connection with Network Link Conditioner
Connect to the access point with the Android device
Ready. You’re awesome 🙂
Cloud device farms
You don’t have to own a device farm to test an app on a wide range of mobile devices. It’s also quite inconvenient if you or your employees mostly work remotely. We started using cloud device farms, and let us tell you — it’s a big gamechanger. You should try it for better mobile testing.
Some farms (e.g. Browserstack or LambdaTest) allow connection throttling — artificial connection slowdown — for testing on mobile devices.
It might require a paid subscription to throttle the connection. For Browserstack the individual plan price starts from $29.
Android emulator testing
Another way to simulate slow connection is to use an emulator. It’s software for a PC that copies another OS. Android emulator imitates the performance of a real smartphone, so you can test your mobile app on it if you don’t have an actual device.
Testing on emulators could never compare to testing on real devices but it’ll do for the first development stages.
With Android studio emulators you can set the limit based on the cellular type (LTE, EDGE, etc.). Each type has its own speed limitations (in kbps):
3. When creating / editing the emulator pick the cellular type:
Tap Show advanced settings
Select speed in the Network section
4. Run the emulator
5. Launch your app in the emulator
6. Turn off the WiFi on the emulator for it to use cellular connection settings
Changing cellular type in the device settings
You can pick a certain cellular type on an actual Android or iOS device. To do that you’ll also need a SIM-card with Internet access.
How to switch to another network on an Android device?
Go to Settings
Then Network & Internet
Go to SIM settings
Tap Preferred network type
Works for Android 12. Section names and placings may differ depending on the device and OS version.
How to switch to another network on an iOS device?
Go to Settings
Then to Mobile Data
Open Mobile Data Options
Tap Voice & Data
Works for iOS 15.
Bandwidth throttling in web debugging tool Charles
If cloud farms are expensive, launching an emulator is complicated, and you don’t have any SIM, you can use the network throttling function in Charles.
Basically, you can use Charles to keep track of the mobile devices traffic when you carry out functional testing or debugging. But you can also use it to simulate slow internet connection for testing.
How to imitate slow connection with Charles?
Setup your host machine with Charles installed as a proxy for the device and the app you’re testing. Learn more about it from the Charles documentation.
When in Charles go to Proxy > Throttle settings
Tap Enable Throttling
Select the speed limitation preset or set it up manually
Save the settings
Click Proxy > Start throttling
Throttling the speed in the router settings
You can also go for a radical way and throttle the speed in your router settings. It will definitely work if you test your app at home, on a real device, and you have access to the router admin panel. Keep in mind that having such an option depends on the router.
How to throttle the connection on a router?
Go to the router settings. To do that, enter the special URL or the router IP in the address bar of your browser (192.168.1.1 or 192.168.0.1)
Enter login and password. You can find them on the router itself (usually both are “admin” by default)
Open the Bandwidth control section
Don’t forget to cancel the limitation once you’re done 🙂
There’re many ways to test an app and fake bad internet connection. The most convenient one, at least for us, is Network Link Conditioner. But you’re free to choose the best for yourself.
Tested everything, it all worked out? Don’t forget to report on it 🙂