Visual Studio Webtest Logo

I use Visual Studio Team Testers Edition everyday at work and one of the things that seemed really strange to me was that Microsoft had missed so many tricks when they decided to release it.

Maybe they just decided to build in reasons for users to upgrade,  knowing that there will always be another version of VSTS around the corner.

One of the things which is sorely missing is the ability to verify HTML source code against a datasource. Sure you can load a data source and then verify what is returned on the page when that row of the data-source is called however there is no functionality to check that the value of the row that you have submitted is returned somewhere in the HTML source (known as the response in VSTS).

So as per usual when you need something doing then, do it yourself, and hence I wrote a small piece of code that looks at the source code and then checks the value of the datasource row to make sure that the item you are submitting is returned in the HTML response.

All the data is dynamic from your source so you can’t just hard code validation rules, as there could be thousands of rows.

For an example take a Google search test. Imagine I want to check that my datasource of 10,000 records is retuned on every response.

So if I search for “System Testers” as a parameter value

“Parameter Name = q” and

“Parameter Value = Google DataSource.Directory_03062010#csv.SearchParams”

In the above QueryString Parameter you can see

the Datasource is called  “Google DataSource”

the Actual FileName is “Directory_03062010″ (and it’s a CSV file)

the Column name is SearchParams

I’ve commented my code to make it easier to understand however if you need any help just ask.

Code Below.

/End Code.

The above script is in Visual Basic.Net and it should work for you and if you need any help in how to install and run it then either send an email or reply in the comments.

One last thing to mention is that the values from the data source are loaded in a fake Parameter Name (Just make one up) which will be ignored by the Website however it will show as a Context Name in the Visual Studio Web Test, so we are really just comparing two Context Names

Good Luck

Martin H

Testing Different Browsers

This is a document I originally wrote a few years back.

It’s a living document meaning that it’s constantly being updated and it’s never finished.

I’ve removed all references to the company I work for and have instead listed in bold text the places where you should insert your company name or your companies website name or the stats gathering tool which you use.

It should all hopefully be all rather self explanatory however if you need help in fleshing it out a bit just send me a mail or reply in the comments and I’ll be glad to help you.

I hope you find it useful.

Browser Support Testing Standards

Author(s)

Martin Hall

Contents

Introduction,  – Test Items, – Not To Be Tested, – Approach, – Test Applications And Resources, – Resources, – Communication, – Defect Logging, – Pass/Fail Criteria, – Suspend/Resume Criteria, – Test Deliverables

Revision History

1. Introduction

1.2.1 As testers we accept that the nature of the internet medium is such that web pages cannot be produced in such a way as to be uniformly rendered in all browsers, so as to provide a consistent experience for all users. We accept that small variations in this experience are acceptable within the ‘Levels of Support’. (See appendix 1 for details)

1.2.2. Web browsers will be assigned a ‘Level of Support’ by the insert your company name here Test Manager, in the Browser Support Table. (See appendix 2 for details of how these levels of support are decided).

1.2.3 Web browser support levels will be amended on a regular basis depending on the statistics that come from the Insert the name of your statistics system here (Webtrends for example).

2. ‘Levels of Support’ definitions

2.1. Support Level 1 – supported web browser – Support definition

2.1.1. All content and functionality MUST work.

2.1.2. Variations to presentation of content MUST be minimised.

2.1.3. Where CSS layout is used, the CSS MUST be rendered by supported web browsers, so that a fully-styled version of the page is presented to the user.

2.1.4. Variations between browsers are inevitable. In these situations, when deciding which browsers should offer a better experience than other browsers (i.e. which would be the closest to the original design), you MUST base your decision on which outcome would maximise the ‘Objectives of Web Browser Support’.

2.1.5. Pages SHOULD be developed to maximise the user experience for users of the web browser with the highest proportion of users (IE7 Windows PC at this time) UNLESS this would greatly compromise the ‘Objectives of Web Browser Support’.

2.2. Support Level 2 – partially-supported web browser – Support definition:

2.2.1. All core content MUST be readable and usable

2.2.2. Navigation functionality MUST work.

2.2.3. Any degradation to (client-side) application functionality SHOULD be graceful degradation.

2.2.4. Any degradation to presentation SHOULD NOT obscure content.

2.2.5. Where CSS layout is used, you MAY choose provide a non-styled version of the page to partially-supported browser.

2.3. Support Level 3 – unsupported web browser – Support definition:

2.3.1. No support or testing necessary.

2.3.2. All web browsers not specifically listed in the support table are unsupported.

2.3.3 There is no need to code or make any alterations for unsupported browsers.

IE = Internet Explorer,

FF = Firefox,

Chrome = Google chrome,

Safari = Apple Safari,

Opera = Opera

Appendix

Appendix 1 – Background

Although we are a UK based and UK targeted company Web pages are available to viewers across the internet, anywhere in the world, using a wide variety of web browsers. These web browsers are developed by a number of different organisations with a variety of motives (some commercial), each of which interpret the source code used to produce web pages slightly differently. As a result, any single web page will not be uniformly rendered by all web browsers, to produce a consistent user experience as intended by the producer.

While this has improved over time, standardisation of web browsers is still to be achieved. The quest for standardising source code and the manner in which web browsers interpret it, has been led by the W3C (World Wide Web Consortium), an organisation whose stated role is to try to “lead the Web to its full potential”. The support amongst the community of web developers for standardisation has come to be known as the ‘Web Standards’ movement.

This insert your company name here standards document exists so as to produce web pages that can be viewed (as consistently as can reasonably be achieved) by the greatest number of people possible.

HTML and CSS for separation of meaning from presentation

Because of the lack of web standards in the past, the only way to provide a consistent user experience was to use only the small range of source code features which worked across all web browsers. Specifically, the inconsistent rendering of CSS meant that CSS was not used much. HTML, which was more consistent, was used for both meaning and presentation; although its intended purpose was semantic rather than presentational.

With the advance of web standards, this has changed. HTML should thus be used to mark-up web content, to give it semantic meaning; while CSS should be used to define the presentation of content.

Not only is this how HTML and CSS are intended to be used (as outlined by the W3C), but it brings a wealth of advantages that considerably improve the potential to improve public value at insert your company name here; especially in terms of reducing costs and improving usability, e.g. accessibility. The use of HTML and CSS in this manner is often referred to as ‘standards compliant code’.

Web browsers employ varying degrees of compliance to W3C recommendations (web standards) and consequently insert UI Developers name often has to employ hacks and work-arounds* to meet the above Browsers Support Levels so that our content can be viewed by the widest possible audience, while providing the optimum user experience. However, these hacks and work-arounds cost time and money, for supporting older, less standards-compliant browsers.

This insert company name standard defines which web browsers are supported by the insert company name, based on these and other contributory factors, to optimise value for users of the insert company website name website.

• A hack is something that is not valid
• a work-around is valid, but requires additional code

Appendix 2 – Objectives of Web Browser Support

When considering which ‘Level of Support’ a web browser should be assigned, the following are some of the issues that are considered.

1. Firstly we should check our insert statistics tools name stats to see the browsers that our current customers use and make sure we are proving content that all of our current users can view. Any browser used by more than 3% of our customers is added to the SHOULD TEST list and any browser which is used by more than 5% of our customers is added to the MUST TEST list.

• Web standards are good for the web and meeting these standards offers value for money. Meaning that we will not have to recode a page because of incompatibilities.
• Insert company name customers use a variety of web browsers. Our aim should be to provide the best possible experience to the largest number of people.
• All of our customers are considered to be valuable.
• Some people use accessibility tools (Elderly or Disabled) – these people benefit greatly from web pages that are standards compliant.

1. Insert company name should not, where possible, provide a service that gives a competitive advantage to any particular product or service. (E.g. we should not add to any page ‘works best in x browser’).
2. Insert company website name represents insert company name. The companies image is considered to be of importance and, therefore, as a main communication point with our potential customers we should be displaying content that works well and has a nice look and feel to as many who view as possible.

Martin Hall

The Testing Jigsaw

A couple of years ago I was asked to present a talk at a testing conference.

I did a whole presentation that I will post up to the blog in a week or two. The one part of the presentation that seemed to grab everyone’s attention was a story I told about the Developer / Testing Jigsaw.

I think the reason it was so widely accepted was because its given in the form of a story, and we all like a story right?

So if your all sitting comfortably then I’ll begin.

There was once a small boy and his father sitting down together one Sunday afternoon.  The father said to his son I have a small surprise for you. I’ve made you a little jigsaw puzzle, its of a tree with grass and sky as the background. Wow, said the little boy as his face lit up, can we do the jigsaw now please?.

Of course said the dad.

So the dad tipped out all of the pieces onto the mat and begun attempting to put the jigsaw together. The son asked his dad if he could help by looking at the picture on the box and advising his father on where he thought the pieces were meant to go.

However his father said “no need – I created this puzzle so I know exactly how it should go together”.

The father struggled on for another 3 hours and wasn’t really any closer to getting the puzzle finished. He then got in a mood and said that he gives up and some pieces must have got lost.

The son then took over and compared each piece to the box making sure that he was putting the pieces where they were meant to go.  He did the smart thing first by putting all of the corner pieces where he thought they should go.  He looked at the box once more and thought to himself that blue is the sky, the green is the grass and the brown is the tree.  So he separated the coloured pieces in 3 piles

He eventually finished the puzzle in about 45 minutes.

There is an obvious key to what is happening above in this  story.

The Son is the Tester

The box is the Functionality Specification Document.

The Dad is the Developer.

The jigsaw is the piece of software that has been developed

Yes developers can test code, in fact I encourage it (peer reviews of other developers code and Unit tests),  however they should not be the sole testers of code, especially if the code is written by them.  Going down that route is a recipe for disaster.

The son who had never seen the jigsaw before managed to finish it in a faster time than his father and also in a more methodical manner.

Testing is a mindset. It’s an art that I, and many others spend every day attempting to perfect.  Developers spend most of their day writing code, (also testing their code – Unit tests etc.)

I’m all for test driven development, however lets not forget that with specialisation comes speed and efficiency savings.

MySQL Logo

This little piece of MySQL Code I wrote a month or so ago.

I had a defect and was trying to provide some decent feedback to the developer  on Test-Track (our defect logging system).

I knew that the defect was in a stored procedure and to attempt to run profiler in Mysql can be a bit of a pain so I figured that I’d work out a way to search in every Stroed Proc for the bit of code causing the error.

I knew that the error was caused by the peice of code calling “Videos”.

I looked around on the net but I couldn’t find anything that suited.  I did find one other piece of sample code but I found that that only search inside the first 256 chars of the Stored Procedure and I needed something that would search inside the whole of the procedure no matter how long it was.

Once again it’s a nice and simple piece of code.

– Author : Martin Hall
– Date 09th April 2010
– Search for Text inside a MySQL Stored Proc
– MYSQL Version
– See the Like Query for an example of Use.

SELECT * FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_DEFINITION LIKE “%video%”
Order by Routine_Name;

Hope you find it useful.

any issues then let me know.

Martin H

mysql logo

You may remember that about three weeks ago I posted a handy script to search for the names of Tables and Columns in a Microsoft SQL database.

Well now today it’s the turn of MySql.

Just a nice and simple script as before however this time there are two seperate scripts one for tables and one for columns names.

– Author : Martin Hall
– Date 09th April 2010
– Search Table or Column for text
– MYSQL Version
– Top query for Table Names and bottom query for column names

SELECT * FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME like ‘%users%’
Order by table_name asc;

SELECT * FROM INFORMATION_SCHEMA.COLUMNS
WHERE Column_NAME like ‘%video%’
Order by column_name asc;

Enjoy.

Martin H

To Do lists

I’ve been using TADA lists for about 3 months now and I love them. They just do exactly what you would expect.

You set up a list and you can move the priorities around depending on how your plans throughout the day change.

I have lists created for

In Test

In Staging

In Planning (soon to be handed to test)

Handed Back to Developement for defect correction.

Let me make clear that this is not meant to replace any defect management system you may have in place. Its far to basic for that.

However if like me your daily priorities are constantly changing and you need to maybe make other team members aware of what things are being worked on in which environments then I have found this as a quick and easy fix.

It can of course be used for many other things like shopping lists. I’ve taken my mobile phone shopping before and when my wife gets home she jumps on the PC and just updates my TADAlist with any items that she needs that I may have missed off.

I then get an email on my mobile phone and bingo I now have an updated list to walk around the shop with. Yes she could ring me but I’d have to write it down. I just find this just easier.

Maybe on that note in the future I should make a post about the perils of using technology just because its there.

Anyway enough of the digressing I hope you find the tool as useful as I and the people I work with have.

Martin H

Software is Rarely Good, Cheap and Fast

This post will be about the topic I like to call the Universal Constants of testing.

What I find really important in my job is to get the business to buy into these constants and once achieved I find that everyone is happier with the end product when it is delivered.

Look at the triangle on the left to get an idea of what I mean by the Universal Constants.

I’ll also give some examples as I find that an example always to explain a new concept.

The basics of the Constant is that

• You can have it Fast and Cheap however it won’t be Good
• You can have it Fast and Good however it won’t be Cheap
• You can have it Good and Cheap however it won’t be Fast.

There is nothing wrong with delivering decent software on time, that’s we we all do everyday hopefully.
what I’m talking about is that one exception to the rule when someone wants something pushed out there quickly, they unfortunately may have a lack of understanding that you are already working to your full capacity.

Once everyone has bought into this way of thinking then you’ll find that they have a more realistic hat on when they are talking about the expected outcomes of a project/product.

There is nothing wrong with a Business Analyst wanting something delivered quicker than usual because they have given at they promise to a customer without checking with you, they will also be wanting it to be “right” on the first release.  However for this to happen there will be a cost to the business which means that it won’t be cheap.

You can see where I’m going with this and you can work out the other scenarios in your head for the other two points in the triangle.

It not only a Constant for testing but also for Resources , Scope and Project Schedules.

• The Project can be completed with that many People and within that Deadline however you’ll have to allow us to change the Scope of the Requirement.
• The Project can be completed with many People and with the Requirement you have submitted however you’ll have to allow us to change the Deadline.
• The Project can be completed with that Requirement and the Deadline you require however you’ll have to all us to change the number of Staff required to work on the project.

Code

In this post I want to give people a nice heads up to some of the tools I use in my daily role.

These are bookmarklets which is just another word for bookmarks which contain javascript.

I use these with firefox although IE and Opera should also fine fine for them also. In Firefox just add them to the bookmark toolbar and you’ll have them at your fingertips.

Zap Cookies! This will clear out any stored cookies for the current page/site

Edit Cookies! This will allow you to edit and stored cookies for the current page/site

View Cookies! This does exactly what it states, it allows you to view and stored cookies for the current page/site

Edit Page

Allows you to edit any page you use this on. All changes are temporary of course and only visible to you.  (will you ever trust a web page screen shot again?). Not yet sure how this fits into the testing arena, however I though I would include it as someone may make decent use out of it.

Find Redirects! This should list any redirects for the current page, however its currently a tiny bit hit and miss and is does not work 100% of the time, it should however suffice for now and I’ll most likely have to rewrite this at some point in the near future.

remove redirects Lets see what happens if we now remove those redirects we just found using the above Bookmarklet.

Wikipedia lookup This allows you to select any text on a page and once clicked it will lookup that text on Wikipedia

Yahoo site search This allows you to select any text on a page and once clicked it will search on Yahoo for more links from that domain with the same text.

MSN IP Search Firstly I should thank Robert Hansen (RSnake) for this one. Once clicked it will carry out an IP search which can help you detect a wider network for your testing.

numbered list One of my favourites this one. It allows you to make a nice numbered list of all parameters on the page which contain numbers.

show hiddens This and Zap Cookies are my most used Bookmarklets. this one will display all hidden fields on a webpage and also allow you to edit them.

remove maxlength This will remove all the max lenghts from all input fields (think buffer overflows and code boundary issues)

undisable Who says you can’t click that button . This Bookmarklet will enable any disabled objects on the page.

up This will take you up one directory level in the site structure

top This will take you to the top of the domain.

decrement If your URL ends in a number it will reduce it by one every click

increment As above but the opposite

check images This will check the current page for broken images.

view variables This will list all variable types found on the page. This is more for Developers than testers however its still a useful one to have.

view scripts Like above however it will list all scripts what can be called on the current page.

zap images This should clear all of the images from the page. Works about 98% of the time. This script may need a little tweaking if I ever get the time.

full urls as link text Very useful if you want to see where a link is pointing to.

Enjoy

Martin H

These are great especially if you are having a conference call with other team members and want to show them an idea visually.

http://www.dabbleboard.com/draw

Neither of the above require a loging to use or share.