|
A practical strategy for developing functional, reliable, and secure .NET code
.NET languages are becoming increasingly popular for driving the application logic for business-critical SOA and Web applications. In these contexts, functional errors are simply not acceptable, and reliability, security, and performance problems can have serious repercussions. Yet, few development teams have the resources to ensure that their code is free of implementation errors, let alone also worry about reliability, security, and performance. Whether or not your team has a satisfactory strategy for functional testing, you're taking several significant risks if you haven't yet implemented a comprehensive team-wide quality-management strategy:
- New code might cause the application to become unstable, produce unexpected results, or even crash when the application is used in a way that you didn't anticipate (and didn't test for).
- New code might open the only door that an attacker needs to manipulate the system and/or access privileged information.
- The functionality that you worked so hard to design, implement, and verify might be broken when other team members add and modify code.
This paper explains a simple four-step strategy that has been proven to make .NET code more reliable, more secure, and easier to maintain - as well as less likely to experience functionality problems:
- As you write code, comply with development rules for improving code functionality, security, performance, and maintainability.
- Immediately after each piece of code is completed or modified, use unit-level reliability testing (exercising each function/method as thoroughly as possible and checking for unexpected exceptions) to verify that it's reliable and secure.
- Immediately after each piece of code is completed or modified, use unit-level functional testing to verify that it's implemented correctly and functions properly.
- Use regression testing to ensure that each piece of code continues to operate correctly as the code base evolves.
All four steps can be automated to promote a consistent implementation and let your team reap the potential benefits without disrupting your development efforts or adding overhead to your already hectic schedule. Moreover, automating these practices lets you concentrate on deeper design/logical issues during code review.
It's important to note that the strategy and practices discussed here were designed for team-wide application. Unless they're applied consistently across an entire development team, they won't significantly improve the software that the team is building. Having a development team inconsistently apply software development standards and best practices as it implements code is like having a team of electricians wire a new building's electrical system with multiple voltages and incompatible outlets. In both cases, the team members' work will interact to form a single system. Consequently, any hazards, problems, or even quirks introduced by one "free spirit" team member who ignores the applicable guidelines and best practices can make the entire system unsafe, unreliable, or difficult to maintain and upgrade.
1. Comply with development rules for improving code functionality, security, performance, and maintainability
The first step in improving the quality, reliability, and security of your code is to comply with applicable development rules as you write it. Many developers think that complying with development rules involves just beautifying code. However, there is actually a wealth of available .NET development rules that have been proven to improve code robustness, security, performance, and maintainability. In addition, each team's experienced developers have typically developed their own (often informal) rules that codify the application-specific lessons they've learned over the course of the project.
Why is it important?
The key benefits of complying with applicable development rules are:
- It cuts development time and cost by reducing the number of problems that need to be identified, diagnosed, and corrected later in the process.
Complying with meaningful development rules prevents serious functionality, security, and performance problems. Each defect that is prevented by complying with development rules means one less defect that the team has to identify, diagnose, correct, and recheck later in the development process (when it's exponentially more time-consuming, difficult, and costly to do so).
Or, if testing doesn't expose every defect, each prevented defect could mean one less defect that will impact the released/deployed application. On average, one defect is introduced for each 10 lines of code according to A. Ricadela writing "The State of Software" a few years ago, and over half of a project's defects can be prevented by complying with development rules according to what R.G. Dromey said in "Software Quality - Prevention Versus Cure." Do the math for a typical program with millions of lines of code and it's clear that preventing errors with development rules can save a significant amount of resources. And considering that it takes only three to four defects per 1,000 lines of code to affect the application's reliability, according to Ricadela, it's clear that ignoring defects is not an option.
- It makes code easier to understand, maintain, and reuse
Different developers naturally write code in different styles. Code with stylistic quirks and undocumented assumptions probably makes perfect sense to the developer as he's writing it, but may confuse other developers who later modify or reuse that code - or even the same developer, when his original intentions are no longer fresh in his mind. When all team members write code in a standard manner, it's easier for each developer to read and understand the code. This not only prevents the introduction of errors during modification and reuse but improves developer productivity and reduces the learning curve for new team members.
What's required to do it?
a. Decide which development rules to comply with.
First, review industry standard .NET development rules and decide which ones would prevent your project's most serious or common defects. The rules defined by Microsoft's .NET Framework Design Guidelines and the rules implemented by automated .NET static analysis tools offer a convenient place to start. If needed, you can supplement these rules with the ones listed in books and articles by .NET experts. As you're deciding which rules to comply with, aim for quality over quantity to ensure that the team members get the greatest benefit for the least work. The point of having the team comply with development rules is to help the team write better code faster. Checking compliance with too many insignificant rules could defeat that purpose.
Next, consider practices and conventions that are unique to your organization, team, and project (for instance, an informal list of lessons learned from past experience). Do your most experienced team developers have an informal list of lessons learned from past experience? Have you encountered a specific bug that can be abstracted into a rule so that the bug never occurs in your code stream again? Are there explicit rules for formatting or naming conventions that your team is expected to comply with?
b. Configure all team tools to check the designated rules consistently.
To reap fully the potential benefits of complying with development rules, the entire development team must check the designated set of rules consistently. Consistency is required because even a slight variation in tool settings among team members could allow non-compliant code to enter the team's shared code base. If the team has carefully selected a set of meaningful development rules to comply with, just one overlooked rule violation could cause serious problems.
For instance, assume a developer checks in code that doesn't comply with rules for closing external resources. If your application keeps temporary files open until it exits, normal testing - which can last a few minutes or run overnight - won't detect any problems. However, when the deployed application runs for a month, you can end up with enough temporary files to overflow your file system, and then your application will crash.
c. Check new/modified code before adding it to source control.
Study after study has shown that the earlier a problem is found, the faster, easier, and cheaper it is to fix. That's why the best time to check whether code complies with development rules is as soon as it's written or updated. If you check whether each piece of code complies with the designated development rules immediately, while the code is still fresh in your mind, you can then quickly resolve any problems found and add it to the source control with increased confidence.
d. Check all new/modified code in the team's shared code base on a nightly basis.
Even if all team members intend to check and correct code before adding it to the source control, code with problems might occasionally slip into the team's shared code base. To maintain the integrity of the team's shared code base, you schedule your testing tool to automatically check the team's code base at a scheduled time each night.
e. Hold weekly reviews for bug root cause analysis and prevention.
Hold weekly meetings to analyze the root cause of the various bugs (defects that were reported by testers, customers, etc. - not violations of development rules) that were fixed during that week. The best time to do root cause analysis on a bug is when it's still fresh in your mind. After the root cause analysis, try to identify a set of rules that will prevent the same bugs from reoccurring then add these rules to the set of development rules you check.
2. Use reliability testing to verify that each piece of code is reliable and secure The next step toward reliable and secure code is to do unit-level reliability testing (also known as white-box testing or construction testing). In .NET, this involves exercising each function/method as thoroughly as possible and checking for unexpected exceptions.
Why is it important?
If your unit testing only checks whether the unit functions as expected, you can't predict what could happen when untested paths are taken by well-meaning users exercising the application in unanticipated ways - or taken by attackers trying to gain control of your application or access to privileged data. It's hardly practical to try to identify and verify every possible user path and input. However, it's critical to identify the possible paths and inputs that could cause unexpected exceptions because:
- Unexpected exceptions can cause application crashes and other serious runtime problems.
If unexpected exceptions surface in the field, they could cause instability, unexpected results, or crashes. Many development teams have had trouble with applications crashing for unknown reasons. Once these teams started identifying and correcting the unexpected exceptions that they previously overlooked, their applications stopped crashing.
- Unexpected exceptions can open the door to security attacks.
Many developers don't realize that unexpected exceptions can also create significant security vulnerabilities. For instance, an exception in login code could allow an attacker to bypass the login procedure completely.
What's required to do it?
a. Design, implement, and execute reliability test cases.
To identify unexpected exceptions, you test each method with a large number and range of potential inputs then check whether exceptions are thrown.
b. Review and address all reported exceptions.
All exceptions exposed by the tests should be reviewed and addressed before proceeding. Each method should be able to handle any valid input without throwing an exception. If code should not throw an exception for a given input, the code should be corrected now before you (or a team member) unwittingly introduce additional errors by adding code that builds on or interacts with the problematic code. If the exception is expected or if the test inputs are not expected/permissible, document those requirements in the code. When other developers working with the code know exactly how the code is supposed to behave, they'll be less likely to introduce errors.
c. Check all new or modified code in the team's shared code base on a nightly basis.
Same reason and procedure as in step 1.d.
3. Use functional testing to verify that each piece of code is implemented correctly and operates properly
Next, extend your reliability test cases to verify each unit's functionality. The goal of unit-level functional testing is to verify that each unit is implemented according to the specification before that unit is added to the team's shared code base.
Why is it important?
The key benefit of verifying functionality at the unit level is that it lets you identify and correct functionality problems as soon as they are introduced, reducing the number of problems that need to be identified, diagnosed, and corrected later in the process. Finding and fixing a unit-level functional error immediately after coding is easier, faster, and anywhere from 10 to a hundred times less costly than finding and fixing that same error later in the development process. When you perform functional testing at the unit level, you can quickly identify simple functionality problems, such as a "++" in a pre-fix notation substituted for a "++" in post-fix, because you are verifying the unit directly.
If the same problem entered the shared code base and became part of a multimillion-line application, it might surface only as strange behavior during application testing. Here, finding the problem's cause would be like searching for a needle in a haystack. Even worse, the problem might not be exposed during application testing and remain in the released/deployed application.
What's required to do it?
a. Add and execute more functional test cases as needed to verify the specification fully.
After you've worked through the exceptions reported for the automatically generated test, check if the code you've written actually functions as expected. Functional unit tests are meant to do just that. Without regard to internal function behavior, this means specifying function inputs and checking if the output is as expected. Such tests should be created based on the class API specification, or class use cases.
4. Use regression testing to ensure that each piece of code continues to operate correctly as the code base evolves
Finally, collect all of the project's existing test cases to create an automated regression test suite that verifies whether each unit continues to function as expected when the code base grows and evolves.
Why is it important?
The key benefit of doing unit-level regression testing is to ensure that code additions and changes don't break the verified functionality. In the rush to accommodate last-minute requests, developers often unknowingly change or break previously verified functionality. Moreover, previously verified functionality is often impacted by the code modifications made during routine maintenance. In fact, according to R.B. Grady in Software Process Improvement, studies have shown that, on average, 25% of software defects are introduced while developers are changing and fixing existing code during maintenance.
Why are code additions and modifications so problematic? With complex software, even a seemingly innocuous change in one part of the application can impact other functionality. However, these changes are difficult to detect without a thorough unit-level regression test suite. Application testing might catch obvious problems that affect the application's interface, but subtle internal problems could easily go unnoticed.
What's required to do it?
a. Configure the regression test to run nightly in the background.
The regression test should check the team's shared code base by executing all applicable functional and reliability test cases. To ensure that regression testing is done regularly and unobtrusively, schedule your unit-testing tool to automatically run the complete test suite at a scheduled time each night.
b. Respond to test findings daily.
The purpose of regression testing is to expose all changes as soon as they are introduced so that an appropriate response can be taken immediately. Each test case failure (a test case that doesn't produce the baseline outcome expected for a set of baseline input[s]) indicates a change in the code's behavior. This change may be intentional or unintentional. When code functionality changes intentionally - as a result of a feature request, specification change, etc. - test cases related to that behavior are expected to fail because the new expected outcomes will be different than the ones recorded in the baseline. However, very often, other test cases will also fail unexpectedly. If so, this reveals a complex functional problem caused by the code modifications. If no unexpected failures are identified, you know that the modifications didn't break the existing functionality.
The appropriate response to a test case failure depends on whether the change was expected. If the new outcome is now the correct outcome, the expected test case outcome is updated, and it becomes a part of the baseline. If not, the code is corrected.
c. Update the regression test suite as needed.
If the regression test suite isn't kept current, it might report annoying false positives or overlook true errors. When code functionality changes intentionally, some of the existing test cases may have to be updated to reflect new expected outcomes, reflect removed or renamed classes/methods, and so on. When new code is added, new test cases will have to be written for that code.
Conclusion
This article explained a four-step strategy that has been proven to make .NET code more reliable, more secure, easier to maintain, and less likely to experience functional problems. Other desirable side effects of using this strategy consistently across a .NET development team include:
- Developers spend less time finding and fixing errors, which means less "crunch time" at the end of the project and more time for more challenging and interesting tasks, such as developing and implementing new technologies.
- The QA team no longer has to chase implementation-level errors and has more time to dedicate to higher-level verification.
- Releases are more likely to occur on time, on budget, and with the expected functionality.
Source :dotnet.sys-cont
|
