Software Quality Standards—How and Why We Applied ISO 25010

When I first heard about ISO/IEC 25010 in 2017, I was still taking my first baby steps in the world of software quality assurance. The last time I heard it mentioned, meanwhile, was at Quality Excites, the nationwide conference on software quality held last June in Gliwice. So, it could be said that it’s an important topic, given that we’re still talking about it two years later. And back when it was released, the QA community—made up of detail-oriented, standard-seeking individuals, who consider quality to be of first-rate importance—greeted it with great interest.

In general, international ISO standards help us evaluate software. Not at all satisfied with such a superficial definition, one day I started wondering exactly what sort of impact do international standards like these have on my daily work, and what benefits they offered to both my company and the clients we service.

Even though ISO/IEC 25010 may seem repulsive at first glance (as formal standards are wont to be), it kept popping up time and again in my daily work, and my appreciation for it grew quickly as soon as I realized how useful and valuable it can be, especially once it's explained using more real-life examples.

I often hear the same question being asked with regard to this particular standard: “Is this different than the International Software Testing Qualifications Board (ISTQB) and how exactly does it differ from its predecessor, ISO 9126?” So first, let me quickly explain the difference between the standards and then we’ll move on to the key advantages of ISO/IEC 25010.

The Differences Between ISO 25010, ISO 9126, and ISTQB

The previous standard for software quality measurement was ISO/IEC 9126. It categorized software quality into six characteristics (factors), which were further broken down into subcharacteristics (criteria).

ISO 25010, however, introduced two additional factors, therefore the difference between the two lies mainly in how they categorize and define those characteristics of software quality requirements that we call non-functional.

So, in short, we move from this—

ISO/IEC 9126 categorization of software quality requirements. Source: Journal Of Object Technology

—to this:

ISO/IEC 25010 categorization of software quality requirements Source: ISO20500.com

The new characteristics include security and compatibility, and they now seem to be more logically located.

ISTQB, on the other hand, is an internationally accepted software testing certification and it is generally believed that securing the certification can improve the quality of testing being performed. The certificate naturally proves the high quality of software testing, but it’s not exactly a tool you will get to use much in your daily life. The ISO standard is a much more precise reflection of reality and offers a much better description of what QA really is.

 

OK, so what is QA, then?

What is Quality and How We Look at The Metrics

Quality may be defined as the non-inferiority or superiority of something; a measure of that thing being suitable for its intended purpose (fitness for purpose) while satisfying customer expectations. Simply put, the quality of an application basically boils down to the way it’s working.

One additional precept that informs our work is: “Anything can be measured, but that does not mean that everything should be measured.”

A quality app must meet the following requirements:

• must work without crashing or producing errors,
• must be stable,
• all of its functionalities must work, too,
• must make all necessary calculations,
• must show accurate data,
• all users must have appropriate permissions.

As you will see in a minute, those requirements are all included in ISO 25010.

What Does ISO/IEC 25010 Include?

As I already mentioned, the standard categorizes app functionalities and lists all aspects of the app that must be verified before the app may be released. Now, let me quickly explain what those official ISO terms mean in more simple terms.

1. Functional Suitability

—i.e. what does the app do? In particular:

1. 1. Functional completeness—is the app in line with the specification? Does it have the functions it was supposed to have?
   2. Functional correctness—does it provide the correct results?
   3. Functional appropriateness —does it fulfill its function?

2. Performance efficiency

—does the app use an optimal amount of resources?

1. Time behavior—are the response and processing times or throughput rates reasonable?
2. Resource utilization—are the amounts and types of resources used reasonable?
3. Capacity—are the maximum limits reasonable? 

If the app is supposed to be performant or handle larger amounts of data, we can propose more server power or suggest other solutions based on available funds. It’s all down to resource optimization. 

3. Compatibility

—can the app work cross-platform or share data with other products, systems or components?

1. 1. Co-existence—can the app share a common environment and resources with other products?
   2. Interoperability—can it exchange information and use the information that has been exchanged?

4. Usability

—can specific users use the app in specific conditions?

As you know, every app is different and has different users. In the case of Merck DORA—an app we developed for the African healthcare market—we had to adjust the UI for users of all ages and make it run on all kinds of phones, including feature phones.

1. Appropriateness recognizability—can users can recognize whether the app is appropriate for their needs?
2. Learnability—is it easy to learn how the app works?
3. Operability—is the app easy to operate and control?
4. User error protection—does the app protect users against making errors?
5. User interface aesthetics—is the user interface pleasing to the eye? (watch out—that’s a very subjective issue!)
6. Accessibility—can the app be used by people of all characteristics and capabilities?

The last factor is particularly important, as we should keep in mind all sorts of prospective users that might end up using our app. For this purpose, I’d recommend checking out the Axe accessibility browser tool—a handy extension for verifying apps e.g. against color-blindness. 

5. Reliability

—an extremely important issue. Here, we look closer at:

1. 1. 1. Maturity—how stable is the app during everyday use?
      2. Availability—can the users use the app when they need to? Remember, some apps really need to work under specific conditions. E.g., when we were building an app for farriers, it was really important for the app to work offline, because many pastures may be out of the mobile broadband reach.
      3. Fault tolerance—can the app work even when there are some hardware or software faults?
      4. Recoverability—in the event of an interruption or a failure, can the app recover the data affected directly and re-establish the system? For a bank or any other business dealing with large amounts of data, recoverability is of prime importance.

6. Security

—does the app protect information and data? For EU countries this is additionally connected with GDPR rules, which we need to be particularly aware of.

1. Confidentiality—is data accessible only to authorized people?
2. Integrity—does the app prevent unauthorized access to, or modification of, computer programs or data? 
3. Non-repudiation—does the app collect information whether specific actions or events have taken place? 
4. Accountability—can the actions of an entity can be traced back to that particular entity?
5. Authenticity—can you prove the identity of a subject or resource?

To help apps comply with the above requirements, make sure that the provided databases are secure and compliant with OAuth 2.0 standards.

7. Maintainability

—will it be possible for the app to be modified or improved in the future, or will it adapt to changes in the environment?

1. 1. Modularity—if an app is built with components, does changing one component impact other components? (Which makes any changes to the app easier and faster.)
   2. Reusability—can an asset be used in more than one system, or in building other assets? Again, this might be extremely time-saving when changing or expanding the app.
   3. Analyzability—is it easy to analyze any activities in the app that need to be taken into account? (Again, do not overanalyze. Look at what is important.)
   4. Modifiability—is the app easy to modify without harming present product quality?
   5. Testability—can the app be tested, also automatically?

Maintainability should be taken into account at the planning stage of the app development cycle. Therefore, the best practice is to engage a QA team member at the very beginning of the process. You see, if you take a QA on board to join a set of developers, you will be able to foresee any possible future requirements and save yourself a lot of effort (and money) down the line. Also, instead of fixing things on the go, you will be able to build a well-planned, robust app.

8. Portability

—can the software be used in various environments?

1. 1. 1. 1. 1. Adaptability—can the app be adapted for different or evolving hardware, software or other operational or usage environments?
            2. Installability—is key for mobile apps—can they be successfully installed and/or uninstalled in a specified environment?
            3. Replaceability—can the app replace another software product for the same purpose in the same environment?

How ISO 25010 Standard Helps Us Measure Quality

Software development is a process. And we need to have full control of this process in order to arrive at satisfactory results—i.e. a working piece of software that meets our requirements and quality goals—and do so within a reasonable timeframe and budget.

As I already pointed out above, at Monterail we care deeply about the core functions of the software we make. They need to work, period. And to check whether they work correctly, we test them in and out, trying to take a fresh approach to the task of evaluation every single time. The problem is, however, that you miss some things if you repeat the same procedures every day.

So, in order to systematize the work, we use Jira and file all the steps we take, and all the tasks and bugs related to a given app. But to make sure I didn’t miss anything, after all that is said and done, I always open the ISO 25010 Standard link and go through all of the columns, asking myself questions about the elements on the list. It’s sort of a QA of the QA process.

Obviously, not all of the items on the list are applicable or important, for example—installability is crucial for mobile apps, but not for the QA process.

In these cases, I look at the ISO/IEC 25010 standard and run a quality check of the application following those eight quality factors.

Is ISO 25010 Standard Good For Everyone?

Each project is different, so you cannot exactly treat the list as a ready-made plan of action. First, think about what is important for the client and the user. And remember to think about it from the very beginning of your work with the client.

Every organization benefits from “best practices” and predictability. Process standardization and automation of testing saves us considerable time and money, and helps protect ourselves against common bugs. But predicting obstacles and errors cannot be added to the product after the product itself is already built.

It’s good to take a QA team member with you to workshops with the client and make sure the client understands the role QA has to play in the development process. A QA specialist will immediately realize the possible challenges and may propose solutions to tackle them at a very early stage.

And if you’re already in the process of creating your software, it’s good to have this ISO standard at the back of your head, even if you’re a frontend QA and already have a list of core criteria to perform a FE QA.

ISO 25010 is a great framework to define software metrics important for a particular project. It is not a comprehensive, detailed map, but rather a guide you can use, depending on the circumstances. Every development project has different priorities and metrics, and this standard allows enough leeway to work with all of them.