Changes between Version 2 and Version 3 of TestingBuilds
- Timestamp:
- 3/22/2012 8:56:53 PM (11 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
TestingBuilds
v2 v3 1 = Testing Eraser Builds = 2 3 '''What we mean by user testing''' 1 = User Testing of Eraser Builds = 2 == What we mean by user testing == 4 3 For Eraser, this essentially means using a target build in a user environment (i.e separately from the development environment) to confirm that: 5 4 … … 8 7 9 8 Inevitably, this begs some questions, but I think it is good enough for working purposes. 10 ''' 11 How we test''' 12 For anything to do with the UI, it is simply a matter of running the program and observing behaviour. The only external program we will need is Windows Explorer, for context menu erasures and to checl that erased files disappear from the list. For testing the engine, we need tools to attempt file recovery and also to check that identifiable parts of erased files do not remain on the disk suface. For these tasks, I intend to use Recuva and HXD respectively. 13 ''' 14 The hierarchy of user testing''' 9 == How we test == 10 For anything to do with the UI, it is simply a matter of running the program and observing behaviour. The only external program we will need is Windows Explorer, for context menu erasures and to checl that erased files disappear from the list. For testing the engine, we need tools to attempt file recovery and also to check that identifiable parts of erased files do not remain on the disk suface. For these tasks, I intend to use Recuva and HxD respectively. 11 12 == The hierarchy of user testing == 15 13 We cannot and should not test everything on every build. So we need to agree what kind of testing is needed for each particular build. In order of increasing complexity and effort required, the levels of testing are: 16 14 17 * ''No testing'' Given the number of individual builds and the way the development process works, this is the default case; the user should only be asked to test things which need user verification; also, it may make sense to test a number of different changes on a single build which incorporates them all;15 * ''No testing''; given the number of individual builds and the way the development process works, this is the default case; the user should only be asked to test things which need user verification; also, it may make sense to test a number of different changes on a single build which incorporates them all; 18 16 * ''Bug fix testing'' This involves implementing an agreed schedule of activity either to isolate and describe the user experience of possible bugs, or to verify that fixes are working for the user; it is concerned only with the specifics of a bug; 19 17 * ''Regression testing'' This is like bug fix testing, but is used where a bug has already been fixed and there is a possibility that it might have been re-introduced as the result of a subsequent code change; … … 21 19 * ''Erasure effectiveness testing'' This most complex and time-consuming level of user testing should only be required when significant changes are made to the erasing engine or when a build is proposed for issue as a stable release; following the erasure of a defined data set, an attempt is made to recover the data using a trusted file recovery program (Recuva in my case), and this procedure may be supplemented by an attempt to use a sector editor (in my case HxD) to find known and unusual contents of erased files on the disk surface. 22 20 23 Comment 24 Bug fix and feature testing can be done quickly; the main concern will be to ensure that we design and implement these tests appropriately, as each test will be more or less bespoke. Each erasure effectiveness will require hours and in some cases days of running, so we need working assumptions as to what is the minimum testing needed to provide assurance of effectiveness. The modular design of Eraser helps us here, and I think it is reasonable to say that21 == Comment == 22 Bug fix, regression and feature testing can be done quickly; the main concern will be to ensure that we design and implement these tests appropriately (particularly in the case of regression testing), as each test will be more or less bespoke. Each erasure effectiveness will require hours and in some cases days of running, so we need working assumptions as to what is the minimum testing needed to provide assurance of effectiveness. The modular design of Eraser helps us here, and I think it is reasonable to say that 25 23 * because there has never been a recorded case (to my knowledge) where erasing has worked with one method but not with another, we can test effectiveness with only one method and assume that the results apply to other methods also; 26 24 * similarly, if erasing initiated in one procedure (context menu, drag and drop or schedule) produces certain results in a test, those results can be assumed to apply also to the other procedures. 27 25 28 These assumptions would need to be reconsidered if changes are made to the erasing method or to the engine. For example, once the eraser engine is implemented as a service, we shall need to devise a specific test procedure to assure ourselves that the changes have not compromised the reliability of the old version. 26 These assumptions would need to be reconsidered if changes are made to the erasing method or to the engine. For example, once the Eraser engine is implemented as a service, we shall need to devise a specific test procedure to assure ourselves that the changes have not compromised the reliability of the old version. 27 29 28 30 29 This is a first version of the working assumptions about user testing, and is subject to amendment.
