Let's pretend you work in QA
Mar. 2nd, 2005 03:21 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dtmOkay, so let's say that your job includes testing a new way of doing daily job FOO, and in order to test it, you're running the new FOO procedure on a test box nightly and comparing the results against the results of FOO running in production.
Now, suppose that after a month of this, every day matches up except for two days that show wildly different results.
Now, in reporting this to a developer who has to pour through detailed dumps of data files in order to figure out what happened, would you also report that those two days were the only two days you had to re-run the test FOO process because you'd typoed a password halfway through the test process, or would you leave that as a surprise for the developer to discover on his own?
In actuality, this isn't quite the scenario I just finished tracing through. In fact, the test FOO job had been restarted on three different days, but on one of those days the tester had properly restored the database before restarting the job. Also, I don't know the exact causes for failure - it could be a typo'ed password, could be a disk ran out of space, could be anything. The lack of notice about these restarts, though, is completely accurate.
Now, suppose that after a month of this, every day matches up except for two days that show wildly different results.
Now, in reporting this to a developer who has to pour through detailed dumps of data files in order to figure out what happened, would you also report that those two days were the only two days you had to re-run the test FOO process because you'd typoed a password halfway through the test process, or would you leave that as a surprise for the developer to discover on his own?
In actuality, this isn't quite the scenario I just finished tracing through. In fact, the test FOO job had been restarted on three different days, but on one of those days the tester had properly restored the database before restarting the job. Also, I don't know the exact causes for failure - it could be a typo'ed password, could be a disk ran out of space, could be anything. The lack of notice about these restarts, though, is completely accurate.