Why and how to write reproducible test cases

We all have been in situations where the application we were developing or maintaining did not do what we expect it to do. This happens all the time. No big deal you think, but after hours of trying to figure out the problem, you still are stuck.

In my experience it helped to stand back a bit and trying to look at the problem from a different angle. There are different ways to do this. One is to try to talk to someone else and to try to explain the problem. I  sometimes see the problem during explaining the problem, sometimes when I try to explain what I have tried to resolve the problem. The interesting part is the the person you talk to does not even have to be a programmer.

In my other post I tried to give some rules on how to ask questions on OTN, so I assume you have done anything in this direction and still don’t have a working solution. Now, there are problems which are not solved after this. This is the point where you should think about writing a reproducible test case. The remaining post is about how to write such a test case in a way you get the most out of it and make it easy for others to help you solve the problem.

A test case should fulfill some requirements:

  1. it must reproduce the problem
  2. it must be as easy as possible in doing this
  3. it should be easy to use by other users which don’t work in your environment
  4. it should use a data model which is known by other users (without the need to study table definitions, triggers, …)
  5. if a known data model can’t be used the model must be described in great detail
  6. if possible it should not be necessary to change  or insert data
  7. if data has to be changed or inserted you should include scripts or steps to get the original data back
  8. it should not take longer than 15-20 minute to setup everything

You might think that it will cost a lot of your time to create such a test case. You are right, it will cost you some time, but this time is well spend:

  1. chance is that you’ll find the problem and it’s solution while creating the test case. Then you are done and can be proud of yourself.
  2. creating the test case forces you to get a clear picture of the problem. This helps to describe the problem to others
  3. providing a test case to the community increases your chance that somebody find the problem and helps you fixing it drastically
  4. If nobody can help you and you ask support.oracle.com for help, they will ask you for a reproducible test case too. Normally the work  faster if you can provide one and have a good description of the problem.

After you now know the reasons and requirements for a test case let’s talk about how to create one. I always use the HE DB schema provided with every Oracle DB (or you can get it from Oracle Sample Model and Scripts). Almost every developer has access to an running HR DB instance. Depending on the problem and the relation to the original data I choose only the minimum number of tables needed to create the test case.

Lets say the problem in your app happens in a data entry form which consists of a master-detail-detail data. Then I look at the HR model and use Regions, Countries, Locations as data model for the test case. Depending on the data you can choose different tables, e.g. Departments, Employees, Jobs.

Once the data model (or the tables) are known, I create a simple ‘ADF Fusion Web Application’ which will create a workspace with two projects. The model project I create by using the ‘Business Components from Table…’ option you get by right clicking on the model project and then selection ‘New’. As the whole process is to long to describe I made screenshots from every dialog . The sample application is created with JDev version 12.2.1.2.0 so you might not exactly see the same images.

Creating the workspace:

Creating the model project:

Checking the model project:

After this the basic workspace for the test case is finished and can be used to add the problem specific parts. Here you start with the business logic and add whatever you need to show hte problem in the UI.

If your problem is related to cascading selectOneChoice in a form, it’s a good idea to add the configuration to the model project and test this using the application module tester. Once the business logic is running you continue with setting up the UI. Here you don’t have to design a pretty looking UI. Make it as simple as you can. Concentrate on showing the problem.

Here are some specific things you should take into account:

  1. if your problem needs a fancy design, e.g. you have a layout problem or a skin problem, then you sure add all needed resources to reproduce the problem.
  2. if your problem needs specific data in the DB, you should create DDL scripts to create the tables and to insert data into the tables. Avoid to add table space names and other storage information in hte scripts as other users likely don’t have those table spaces. A description of the data model is mandatory!
  3. if your case needs third party libraries you must provide the exact version of the libraries and hte location where we can download them! Smaller libraries you should add to the test case zip file.
  4. Write a detailed description on how to start the application (which page to run) and what to do to get to the problem.
  5. Format your code! I hate reading unformatted code. It costs time to interpret things which I would like to spend on solving the problem, not by trying to figure out which part of code I’m looking at.
  6. Comment your code! The better the code is commented the easier we can understand what you have tried to do. In a perfect world I wold like to understand the code by just reading the comments of the methods without having to go into the details of the implementation. Once the problem area is clear I start looking into the implementation. One thing I do is to read the comment of a method and checking it against it’s implementation.

Finally you zip everything together and make the zip file available to the public. Make sure that you delete unnecessary files and folders from the workspace (like .data and .classes). These folders are huge and are recreated automatically when you compile the application. Pay special attention to library files (jars) which are sometimes huge. As you are given us the version and location where to download them (and instructions where to put them!) they can be omitted. You can use Google Drive or Dropbox which are well known to other developers. I you use an unknown file host service you risk that nobody downloads the test case.

One final word of advice

We are all trying to help but have our normal work to do too. Creating a test case, even after you are asked to provide one, doesn’t mean that you get help from other users. There is no service level agreement (SLA) attached!

If you need urgent support ask Oracle Support or help. The test case will help you there too as support don’T need weeks to understand and reproduce the problem.

 

 

 

 

Advertisements

One thought on “Why and how to write reproducible test cases

  1. Pingback: JDev12c: Searching an af:tree | JDev & ADF Goodies

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s