A Silly Mistake Can Lead to a Big Mistake

Writing code and testing software are both challenging tasks in Software Development. It is never easy to make cent percent synchronization between these two. It is challenging for a developer to write almost flaw less code similarly it is challenging for a tester to cover almost cent percent of testing path.

Very often we are in a situation where we are forced to or we have to overlook some minor analyses and we leave some very minor flaws in code or in testing.  Because of the ignorance of the analysis apparently we think that those minor flaws won’t affect the whole system in its major functionalities. But in reality a very minor flaw in the system can lead it be an unusable system.

Few days back I was asked to develop a tool to modify an SQL file which were generated form a replicated database and had to execute the modified SQLs to a pgSQL database. The original file size was 70-80MB and total number of lines were about 600K.

The main task was to filter out some specific SQL blocks from the file then writing those modifications into two set of files. One set of files was a single file that contained the whole modified SQLs and another set of files was with multiple files that contained the modified SQLs as several chunks.

Since number of lines were huge so it was difficult for me to test all those line manually, but I tried my level best and found it to be according to the requirement. I opened those files manually and seemed to be OK. I executed the second set of files (the chunk SQL blocks) and these were successfully executed to the desired database. BUT when I executed the first set which contained the whole modified SQLs then it showed me syntax error.

I started to analyze the file and found that very few of the very last lines were not present in it. It was also very interesting that the skipped lines were started from a middle of a line, I was sure about my logic that either it will write a complete line or skip it completely, fraction of line is impossible.

So at first glance I supposed it to be a memory related problem and started to minimize the original file size but found the same problem every time  I minimized the files. That really shocked me as the other chunk files even bigger than my problematic was OK. Then I went to the source code and found that every StreamReader or StreamWriter I used for reading/writing files were properly closed except the problematic one. I modified the source code and then executed the tool which then solved the problem.

As we know an opened stream made a file inaccessible but in my case I was able to use it and that made me think that closing the stream wasn’t big a issue apparently.  But that silly mistake made my application a useless one. That is why every developer and tester should be aware about his silly mistake, they need to increase their analysis even after a very very minor case, they have to increase their foreseeable power.

Software Testing in Proper Test Environment is a Great Challenge in SDLC

No doubt software testing is a great challenge in SDLC. To get a clear picture of your testing application you must have proper preparation in proper time and you have to execute all those preparations in proper time and proper places. On the other hand all the plans, hard works regarding software testing will be in vain if testing is not done in proper Test Environment.

In my professional experience I had found some bugs that were bugs because of not testing the application in proper test environment. Similarly I have missed some bugs that were because of not testing the application in proper test environment.

Proper test environment is not an easy term to understand. A proper test environment may relate hundred of tasks to do before testing any application. Here are some of  them:

> Application is built from proper SVN revision?

> Application is installed/deployed in proper way?

> In which OS the application is running on? What version is it?

> In which Browser it is running? Version?

> It is running where? Location, time, timezone, daylight saving?

> and so on………………..

Very recent I was missing a bug because of improper test environment. A desktop application runs mainly in USA, it communicates with server DB through web services.  The application is used by many users from several places of USA. Recently two date fields are added in it. The DB columns for these fields are of type Date, so it doesn’t consider Time/Zone related information.

In the mean time client complained us that many times they saw different dates than they provided for those fields. So I started to investigate the problem, executed different scenarios but couldn’t reproduce the bug. I tested the application both in local and live environment. The dates are selected from a calendar control and these are properly saved in DB server. Since the data type was Date so I thought there won’t be any TimeZone issues.

Just for curiosity I set different TimeZonse in my application running machine and server running machine and then started code debugging. From web service the data were returning to client side through a DataSet. When the data are assigned into a DataTable in client side then I wondered to see the return values in different environments. Here are the environments and return values:

Environment #1:
a)      App in Local PC (Time: 14:50, Time Zone GMT +6)
b)      Webservice in Live PC( Time: 03:50, Time Zone GMT-5)
c)      Original Start Date=12/05/2010 and End 12/11/2010)
d)      Now the DataTable have Start Date=12/05/2010 11:00:00 AM, End Date=12/11/2010 11:00:00 AM.
e)      So Start Date and End  Date are showing as 12/05/2010 and 12/11/2010.

Environment #2:
a)      App in Local PC(Time: 15:01, Time Zone GMT -5)
b)      Webservice in Live PC( Time: 04:01, Time Zone GMT -5)
c)      Original Start Date=12/05/2010 and End 12/11/2010)
d)      Now the DataTable have Start Date=12/05/2010 12:00:00 AM, End=12/11/2010 12:00:00 AM.
e)      So Start Date and End are showing as 12/05/2010 and 12/11/2010.

Environment #3:
a)      App in Local PC(Time: 15:01, Time Zone GMT +13)
b)      Webservice in Live PC( Time: 04:01, Time Zone GMT -5)
c       Original Start Date=12/05/2010 and End 12/11/2010)
d)      Now the DataTable have Start Date=12/05/2010 06:00:00 PM, End=12/11/2010 06:00:00 PM.
e)      So Start Date and End Date are whowing as  12/05/2010 and 12/11/2010.

Environment #4:
a)      App in Local PC(Time: 04:10, Time Zone GMT -5)
b)      Webservice in Local Newtwork( Time: 04:01, Time Zone GMT +6)
c)      Original Start Date=12/05/2010 and End Date 12/11/2010)
d)      Now the DataTable have Start Date=12/04/2010 01:00:00 PM, End=12/10/2010 01:00:00 PM.
e)      So Start Date and End Date are showing as 12/04/2010 and 12/10/2010 respectively. And this was the bug in different TimeZone.

Later I knew that whenever Date/Time related information are passed/retrieved to/from web service through DataSet/DataTable then it automatically adjusts time with TimeZone in SOAP XML [see here].

I couldn’t reproduce the bug if I had all things right except the different TimeZone. So this is how it is important to test application in proper environment.

Really Software Testing is Not Inevitable and Never Complete

It is not new or I need not to prove it again that Testing is not Inevitable and Never Complete. But last month I had an experience from where again I have concluded that really Testing is not Inevitable and Never Complete. I’m sharing the experience here.

It was the day when we planned to ship a pilot release. According to our plan we were testing the product and on the final day and final moment we were not getting any bug from the product. Our SQA manager asked me whether there were any findings or we would ship the product. I told him that the product seemed to be stable, and we were ready for the release. I requested few minutes from him just for an overview of the product.

In the last moment I found a bug that was not a bug to me until a new feature was implemented in this new release. I reported the bug in the issue tracker and manually informed our SQA manager. Our SQA manager took it and instantly went to the Project Manager room. While the bug was fixing then I found another bug and before reporting it to the issue tracker I went to the Project Manager room and notify it to both SQA manager and PM.

Within a very short time the bugs were fixed and again we were testing the product. After finding those bugs some exploratory thinking came to my mind that I could find more bugs in the same area. Alas! I was true! I found another new bug that wasn’t a bug to me until I reported those bugs and started thinking about those bugs. In that day we found another three/four bugs in the same area. So a total of 6/7 bugs were found and that was within a very short time. The moment was really very exciting. We found a bug and PM fixed it and again we found a new bug and PM fixed it…. Once a time our PM told us that really it was interesting. It was like ACM Programming contest.

Form this experience, I found how Software Testing is challenging and how unpredictable is it! While a minute ago I was thinking there was no bug in the product there we fond so many bugs within very short time.  The software testing seems me like Unpredictable Test Cricket. Look at the score of 2nd Innings of 2nd Test Match 2010 between Bangladesh and India. Once a time Bangladesh were 290 for 3 and the innings came last to 312 for 10.  7 wickets gone for only 22 runs. Can you believe it?

That is why all we say Really Software Testing in Not Inevitable and Never Complete.

Do You Have a Hit List?

In software development life cycle this is a common scenario that the tester gets less time than the anticipation. And when he goes to execute test runs then he finds a very big gap between time and test cases. He blames the shortage time.

But devising a technique he can overcome much of the ‘time limitation’ problem. If he has a list with some key points that does not require writing down in test cases and without altering the keys he can apply much of the points in any situation and in any type of application, then of course it will save time to find/identify bugs. I call this list as a Hit List.

Basically the Hit List will have two types of contents. One is Context dependent key points and another is context independent key points.

Context dependent key points: Basically these key points remain unchanged throughout the application, points are identified through testing experiences on the application.  Example: Fixed format of Price, Acc No, Title, frequent occurrence of critical bug,…, etc.

If your are keen to grab these type of points then you can easily apply these to your testing software very quickly.

Context independent key points: These points that do not depend on your testing software. Example: data type conversion/overflow error, rounding/truncating problem of number/string/character.. data types, entering special characters to input fields, unplug network connection, read/write file accessing error, …,etc. You can identify same type of more points for your Hit List.

When you have identified the key points then you can apply these to your testing software/application very quickly. Depending on how fast you have access on the points, it will save much of your testing time.

Now the question where you will save your Hit List? The best position is to keep it in your mind. If the List grow enough or you are confused recalling them in time, then you can save these to writing documents like notepad, wordpad,..,etc.

On July 28, 2009 I had a presentation on it. You can find more from here(http://www.sqabd.com/lt_present.php?id=lt5).

Good luck