Tuesday, March 22, 2011

Watin on Crack

Here is an example of how we can create some simple and elegant Watin test code:
namespace Website.WebTests.Users
{
    public class AddNewUserTests : BaseTests
    {
        private static AddNewUserPage CurrentPage(IeBrowser browser)
        {
            return browser
                    .GoToPage<LoginPage>(page => page.Login())
                    .GoToPage<AdminPage>()
                    .GoToPage<AddNewUserPage>();            
        }
 
        [Test]
        public void AddNewUserPage_AddUser()
        {
            using (var browser = new IeBrowser())
            {
                var firstName = RandomString(15);
                var lastName = RandomNumber(10);
 
                browser
 
                // Arrange
                .GoToPage(CurrentPage)
 
                // Act
                .AddNewUser(new UserModel { FirstName = firstName, LastName = lastName })
 
                // Assert 
                .AssertPageHasText(page => page.Contains("Name: " + firstName));
            }      
        }              
    }
}

Easy to read, clean and maintainable...

Main points to note:
1) Each test includes a static method called CurrentPage, this has the steps required to navigate from the home page to the currently tested page, as per a user navigating to that page, including any actions that might need to occur along the way (eg page.Login)
2) Simple fluent interface for navigating pages and for setting up our Arrange, Act, Assert scenarios!

It may be a case of love it or hate it, however if you think as I do, that the above implementation is easy to read and understand then read on to see how it is done...

INTRODUCTION

As part of a project to create a new set of Watin tests for an existing Web Application, I decided to try to introduce some new functionality to achieve the following goals:

1) Speed up the development of the Watin web tests
2) Use the new Watin Page class to enable better reuse of code
3) Add extra helper methods to reduce issues with timing
4) Use a fluent interface to increase readability

In the past we were writing Watin tests as one long method with a bunch of actions in a row and then some kind of check or assert at the end. The problem with this approach is that there usually ended up being a lot of repeated code, for example when two tests have to navigate through the same page, then the various actions on the page could end up in two places.

The new (ok it's been around a couple of years) way to segregate your Watin code is to create a Watin Page class for each of your web pages. Thus you might have a login page, a search page, register page, confirmation page, etc. If all the pages include some common master page then those pages could derive from a base page with common elements. Each page would then have a bunch of properties, one property per UI element that you want to use, and then a number of actions that can apply to that page.

The beautiful part of the Watin Page class is that we simply create properties for each UI element, and add an attribute above each one specifying how we identify it on the page, for example IdRegex = "UserName", would look for an element which has Id="UserName". You could also do Id="UserName", and there are almost hundreds of variations. As you use the properties to retrieve or set data on the web page the Watin framework automatically finds the correct element using the [FindBy()] attribute, where you have passed in the method of identification.

Example of a LoginPage class:
namespace WebApplication.WebTests.Pages
{
    public class LoginPage : BasePage<LoginPage>
    {
        private const string UserName = "admin";
        private const string Password = "#test1";
 
        [FindBy(IdRegex = "UserName")]
        public TextField UsernameTextField;
 
        [FindBy(IdRegex = "Password")]
        public TextField PasswordTextField;
 
        [FindBy(ValueRegex = "Log on")]
        public Button LoginButton;
 
        /// <summary>
        /// Logs the user in
        /// </summary>
        /// <returns></returns>
        public LoginPage Login()
        {
            UsernameTextField.Value = UserName;
            PasswordTextField.Value = Password;
 
            LoginButton.Click();
 
            return this;
        }
    }
}

The above page class encapsulates the logic that is specific to our Login page. Namely three UI elements, the username and password text fields and the login button. I have also included an action that is applicable to this page, ie logging in, and if one were to call the Login method, then the Watin test would log us in.

Each page in our web site has a matching page, similar to the above one, and I structure them in our WebTests project using a similar folder hierarchy as per the websites folder hierarchy.

The first thing to note about these Page class implementations is that they all derive from BasePage<T>. This allows us to reuse some common functionality across all pages, for example the navigation buttons along the top which occur on all pages, the log off link that we have on all pages, and a number of useful Assert helper methods, some useful 'Wait' methods to deal with timing issues and some methods that start with 'GoTo' which we use for navigation (to be explained later).

The second thing to note about the methods in these Page classes, is that all methods return a reference (return this) to themselves. This is how we enable our fluent interface, each page action can be appended on to the last page action, and in the base class there are actions that allow us to navigate between pages as well so the entire Watin test could be one long set of chained methods!

THE BASE PAGE : BasePage<T>

The idea behind having a generic base class is that even the BasePage actions can return the Type of the derived class. If the BasePage wasn't generic, then we would need to specify which Type to return in our fluent base methods, either that or just return a reference to the BasePage, which would mean that in the test itself we would need to do a cast or else miss out on being able to access derived class methods.

Being able to define base class methods like this:
        public T Logoff()
        {
            LogoffLink.Click();
            return (T)this;
        }    
Means that every derived BasePage class automatically has a fluent method that does a Logoff and also returns a reference to the derived class, automatically allowing us to chain the next derived fluent method without having to interrupt our fluency to add some unsightly casts. 

THE BROWSER CLASS

Watin tests all start with the creation of a browser instance. For IE, which is what I will be using, the class is also called IE. I then derive a class from this, called the IeBrowser class. This will be our starting point for tests. Instantiating this IeBrowser class automatically opens a copy of IE on your desktop. The IeBrowser class has a number of GoTo methods that get our navigation started.

Here is the constructor:
        public IeBrowser()
        {
            AutoClose = true;
            Settings.AutoMoveMousePointerToTopLeft = false;     
            
            ClearCache();
            ClearCookies();
        }

Here are our Browser GoTo navigation methods:

        public T GoToPage<T>() where T : BasePage<T>, new()
        {
            GoTo(BaseUrl);
 
            return Page<HomePage>().GoToPage<T>();
        }

The first method is our initial GoToPage method. It's a generic method so we just pass the Type of the page we want to navigate to. It starts by taking us to an absolute URL (the only hard coded url within the project) which is the home page of the website. Then the page that you have specified needs to be one that is accessible from the HomePage, this ensures that the test follows the normal flow of user navigation.

        public T GoToPage<T>(Func<IeBrowser, T> stepsToGetToThisPage) where T : BasePage<T>, new()
        {
            var page = stepsToGetToThisPage(this);
 
            page.WaitUntilComplete();
 
            return page;
        }  

In addition to the first GoToPage method, we have a second more elaborate version which allows us to pass an anonymous function returning the page type T, and accepting an IeBrowser instance. This anonymous function then can contain a chain of GoToPages, ie to quickly allow us to navigate to a certain page. Each page also can also contain some custom WaitUntilComplete functionality so that we only return from the method when we know that the page has finished loading.

BASE PAGE GOTO METHODS

The BasePage class also contains similar GoToPage methods, and here is where the real magic takes place. To keep things as simple as possible, when we create our new derived Page classes, we check to see what navigation links are present. For example on our HomePage I have:

    public abstract class BasePage<T> : Page where T : BasePage<T>
    {          
        [LinkedPage(typeof(RegisterPage))]
        [FindBy(IdRegex = "RegisterLink")]
        public Link RegisterLink;
 
        [LinkedPage(typeof(ResultsPage))]
        [FindBy(IdRegex = "ResultsLink")]
        public Link ResultsLink;
 
        [LinkedPage(typeof(AdminPage))]
        [FindBy(IdRegex = "AdminLink")]
        public Link AdminLink;
 
        [LinkedPage(typeof(LoginPage))]
        [FindBy(Id = "logonLink")]
        public Link LogonLink;    

The Page starts with our Watin user UI elements as per normal, but any UI elements that are actually navigation links include a new LinkedPage attribute. This signifies to our framework that the particular link is also a conduit that allows us to navigate to the specified Page type.  The above links are common across all pages, that is why they are in the base class, however our derived page classes will also contain similar links and LinkedPage attributes where necessary.

The BasePage GoToPage method looks like this:

        /// <summary>
        /// Goes to page of the specified type by clicking on the  matching link and then returning the correct page type
        /// </summary>
        /// <typeparam name="TPage"></typeparam>
        /// <returns></returns>
        public virtual TPage GoToPage<TPage>() where TPage : BasePage<TPage>, new()
        {
            // click on the link which will take us to matching page (see LinkedPage attribute on Link properties above)
            LinkedPageAttribute.GetLinkedElement<TPage, Link>(this).Click();
 
            // create required page
            var basePage = Document.Page<TPage>();
 
            // allows us to add some custom wait functionality to delay processing until we are happy that page has loaded
            basePage.WaitUntilComplete();
 
            return basePage;
        }

Thus on the current page, when we call GoToPage<RegisterPage> for example, the call to GetLinkedElement will look for a type 'Link' which is of type 'TPage' and call the Click method on it. Note we could pass in other Types, eg Button if we so wish. The Page is then created using the Watin Document.Page<T> method and we wait for the page to complete. These GoToPages are then chained together and as long as each page has links to the next page the whole set of chained actions will complete, and to the user watching the test in action it will just appear as if a user was navigating through the site.

I have also added another version of the above GoToPage method which also allows us to pass an action to perform on that page:

        /// <summary>
        /// Version of GoToPage as per above, which calls GoToPage, but which also takes an action to perform on that page
        /// as a parameter and then performs that action on the page
        /// </summary>
        public virtual TPage GoToPage<TPage>(Func<TPage, TPage> actionToCall) where TPage : BasePage<TPage>, new()
        {
            // go to the required page as per usual
            var page = GoToPage<TPage>();
 
            // call the required action on the page
            return actionToCall(page);
        }

BASE PAGE TIMING METHODS

Timing is a big issue with web testing. The issue of course, is that each web page can take a different time to render, and different parts of each page can render differently, some with the page load, some using ajax etc. Therefore I have added a virtual method that can be overridden in each Page derivation so that we can specific custom Wait functionality. Here is the method in the BasePage:

        /// <summary>
        /// Override this method if you have a specific element or text you want to
        /// wait to be loaded before continuing with processing
        /// </summary>
        public virtual T WaitUntilComplete()
        {
            return (T)this;            
        }

For example a derived page may display a data grid and then a button at the end to add a new item. The button is only displayed after the grid is drawn and this could take several seconds. We don't want to operate on the page until the grid is complete, so we can put a check on whether the add button exists, eg:

        /// <summary>
        /// Wait until the add button is displayed before continuing
        /// any tests that use this page.
        /// </summary>
        public override GridPage WaitUntilComplete()
        {
            AddButton.WaitUntilExists();
 
            return this;
        }

There is also a useful BasePage generic extension method for waiting for text:

        public static T WaitForText<T>(this BasePage page, string text) where T : BasePage, new()
        {
            page.Document.WaitUntilContainsText(text);
 
            return (T)page;
        } 

BASE PAGE ASSERT HELPER METHODS

To be able to perform fluent asserts, I have added a number of useful Assert helper methods to our BasePage class.

Here is a quick overview:

1) Check to see that the page contains some text. The WaitUntilContainsText makes sure that we rather time out waiting for the text than looking for the text too early. The end result is no timing issues.


        public T AssertPageHasText(string text)
        {
            Document.WaitUntilContainsText(text);
            Assert.IsTrue(Document.Text.Contains(text));
            return (T)this;
        }

2) Assert text exists by using our own custom logic.

        public T AssertPageHasText(Func<string,bool> doesPageContainText)
        {
            Document.WaitUntil(() => doesPageContainText(Document.Text));
            Assert.IsTrue(doesPageContainText(Document.Text));
            return (T)this;
        }

Usage example for above:

        // Assert (can either be a successful addition / or if Id is already used then we get a validation message) 
        .AssertPageHasText(page => page.Contains("Summary: " + fullName) || page.Contains("The Id is already in use"));

3) The next lot of asserts are self explanatory

          public T AssertPageMissingText(string text)
        {
            Assert.IsFalse(Document.Text.Contains(text));
            return (T)this;
        }
 
        public T AssertPageHasLink(string text)
        {
            Assert.IsNotNull(Document.Link(link => link.Text.Contains(text)));
            return (T)this;
        }
 
        public T AssertPageMissingLink(string text)
        {
            Assert.IsTrue(Document.Links.Filter(link => link.Text.Contains(text)).Count == 0);
            return (T)this;
        }
 
        public T AssertTextFieldHasText(string text)
        {
            Assert.IsNotNull(Document.TextField(textfield => textfield.Text.Contains(text)));
            return (T)this;
        }

4) Assert CatchAll, for anything else that I haven't thought of, we can use this method. We can essentially
put any assert or other anonymous method that returns a bool.
        /// <summary>
        /// Allows us to wrap any assert statement into our fluent interface
        /// </summary>
        public T AssertThat(Action assert)
        {
            assert();
            return (T)this;
        }
Usage of this assert would be something like this:
        .AssertThat(() => Assert.AreEqual(newYear, currentYear));

TEST CLASSES

The test classes are also kept in a similar folder hierarchy as per the pages, but under the heading Tests.

Each test page has a static method called CurrentPage, this has the sequence of navigation steps required to get to the current page. Then each test that follows essentially calls the GoToPage method on the IeBrowser class passing in that function. That has the effect of navigating us to the desired page, whereupon we can do the Act and Assert parts of the test. The idea of having this CurrentPage method is that it can be reused by all the test methods within the Test class. Each one will re-navigate from the home page back to the current page.

There will usually be a number of test methods in the test page class. For example one to see the page exists and that navigation to the page works. Then one to add an item if possible, one to search for example, one to delete if possible, others as necessary, etc.

Here is a random example of a test page:

namespace Website.WebTests.Admin
{
    public class UserLevelTests : BaseTests
    {
        private static UserLevelsPage CurrentPage(IeBrowser browser)
        {
            return browser
                      .GoToPage<LoginPage>(page => page.Login())
                      .GoToPage<AdminPage>()
                      .GoToPage<UserLevelsPage>();
        }
 
        [Test]
        public void UserLevels_Exist()
        {
            using (var browser = new IeBrowser())
            {
                browser
                    
                // Arrange
                .GoToPage(CurrentPage)
 
                // Assert
                .GetResultsTable()
                .AssertTableHasRows();
            }
        }
 
        [Test]
        public void UserLevels_AddNewValidLevel_CanBeAdded()
        {
            using (var browser = new IeBrowser())
            {
                var userLevel = Guid.NewGuid().ToString();
 
                browser
 
                // Arrange
                .GoToPage(CurrentPage)
 
                // Act
                .AddUserLevel(new UserLevel { UserLevelName = userLevel, Comments = "Comments", IsActive = true })
 
                // Assert
                .AssertPageHasText(userLevel);
            }
        }
 
        [Test]
        public void UserLevels_CanBeEdited()
        {
            using (var browser = new IeBrowser())
            {
                var comments = Guid.NewGuid().ToString();
 
                browser
                    
                // Arrange
                .GoToPage(CurrentPage)
                   
                // Act
                .EditUserLevelComment(1, comments)
 
                // Assert
                .AssertPageHasText(comments);
            }
        }
    }
}

To get an idea of what the action is doing on the UserLevelsPage, here is the AddUserLevel action:

        public UserLevelsPage AddUserLevel(UserLevel userLevel)
        {
            AddUserLevelButton.Click();
 
            LevelNameTextField.Value = userLevel.UserLevelName;
            LevelCommentsTextField.Value = userLevel.Comments;
            ActiveCheckBox.Checked = userLevel.IsActive;
 
            AddUserLevelOkButton.Click();
 
            return this;
        }

I tend to reuse existing domain entities or view classes for passing data into the Page classes.

Sometime it is also useful to pull out information from the page class so we can use this data in our Asserts, for example:

        [Test]
        public void UserDetails_RemoveUserFee_TotalSpentDecreases()
        {
            using (var browser = new IeBrowser())
            {
                decimal oldUserFeesSpent, newUserFeesSpent, feeAmount;
 
                browser
 
                // Arrange
                .GoToPage(CurrentPage)
                .GetUserFeesSpent(out oldUserFeesSpent)
 
                // Act
                    .RemoveLastUserFee(out feeAmount)
               .GetUserFeesSpent(out newUserFeesSpent)
 
                // Assert
                    .AssertThat(() => Assert.AreEqual(oldUserFeesSpent - feeAmount, newUserFeesSpent));
            }
        }  

To keep the fluent interface intact I had to use 'out's to return information retrieved from the page. This way this information can be acted upon within the test, for example in the Assert.

A more elaborate example with more steps:

        [Test]
        public void AddNewUser_AdminUser()
        {
            using (var browser = new IeBrowser())
            {
                browser
 
                // Arrange
                .GoToPage(CurrentPage)
 
                // Act
                .SelectApplication(UserTypeEnum.Admin)
                .EnterCaptcha()
                .AcceptTermsAndConditions()
                .AddPersonalDetails(AdminApplication)
                .AddAddressDetails(AdminApplication)
                .Next(2)
                .AddObjective(AdminApplication)
                .AddAttachment(FileToUploadPath)
                .SubmitApplication()
 
                // Assert
                .AssertPageHasText("User application complete");
            }
        }

I haven't quite managed to integrate a test which pops up a new browser window which also needs to be included in the test, this is the best I can do:

        [Test]
        public void UserBiographyPage_DefaultSearch_ReturnsResults()
        {
            using (var browser = new IeBrowser())
            {
                browser
 
                // Arrange
                .GoToPage(CurrentPage)
 
                // Act
                .SearchUsers(new UserRequest { CategoryIds = new List<int> { 2 }, YearFrom = 2010, YearTo = 2011 });
 
                // Assert
                using (var poppedUpBrowser = browser.GetPopupBrowser("Website/Results/ViewUserReport"))
                {
                    poppedUpBrowser.Page<UserDetailsPage>()
 
                    .WaitUntilComplete()
                    .GetResultsTable()        
                    .AssertTableHasRows();
                }
            }
        } 
One last point that was kindly raised by a reader was a concern about having to re-navigate to the CurrentPage for each test in the module. Sometimes this is unnecessary and if you know that it is ok to run your tests one after the other on the same page without causing undue effects then we can gain a time advantage by not having to re-navigate, ie calling the GoToPage method in the Test.

The way I suggest to handle this situation is to create the Browser instance and then do the GoToPage navigation in the Setup method for the test module. This way it happens once, and each test then skips the GoToPage and instead we would call a new method (yet to be added) maybe called GetPage<T> where we can return the page of that type and start working directly on that.

SOME OTHER FUNKY STUFF

Sometimes you can have a wierd situation where you have to operate on a UI element which hasn't yet been created. This can happen for example when some AJAX on some condition then reveals a new TextField or some other such control which we then need to populate.

Therefore I have created a generic extension method to the Watin Document class which takes a Func (with some wait logic) and returns the specified type. This extension is called 'WaitUntilExists', here is an example of its use:

            AddEmailButton.Click();
 
          Func<TextField> getNewEmailTextField = () =>
          {   var row = EmailsTable.GetLastRow();
              var textField = row.TextFields.FirstThatIsNotHidden();
              return row.TextFields.FirstThatIsNotHidden().Value.IsNullOrEmpty() ? textField : null;
          };
 
          // do above step until we get text field back
             var emailAddressField = Document.WaitUntilExists(getNewEmailTextField);
 
            // populate last email address field
            emailAddressField.Value = emailDetails.EmailAddress;

Clicking the AddEmailButton button will pop in a new row into our list of email addresses. The WaitUntilExists keeps polling the Func<T> that is passed in, until it returns a valid T (rather than null), which happens when it exists and then returns this.

The extension method looks like this:

        /// <summary>
        /// Useful extension method that returns an element that you need in the case when the element
        /// might take some time to render due to AJAX etc. Simply pass in a method that returns that
        /// variable or null if it doesn't exist. This generic extension method than invokes the function
        /// until it gets a non null element out which it passes back. We also put a timeout in so we
        /// don't end up hanging if the element is never found. 100 attempts = 20 seconds.
        /// </summary>
        public static T WaitUntilExists<T>(this Document document, Func<T> existsCheck) where T : Element
        {
            var attempts = 100;  
            T element;
 
            while ((element = existsCheck()) == null)
            {
                Thread.Sleep(200);
 
                if (--attempts == 0) break;
            }
 
            return element;
        }

There is also another extension method for more simple situations where we just need to wait for a True check, eg:

            .....
 
            SaveButton.Click();
 
            // wait until save button is re-enabled again (means save has gone through)
            Document.WaitUntil(() => SaveButton.Enabled);
 
            return this;
        }

Where the extension method is:

        /// <summary>
        /// Waits the until the specified func returns true.
        /// </summary>
        public static void WaitUntil(this Document document, Func<bool> trueCheck)
        {
            var attempts = 100;
 
            while (trueCheck() == false)
            {
                Thread.Sleep(200);
 
                if (--attempts == 0) break;
            }
        }

LINKEDATTRIBUTE FUNCTIONALITY               

This is the attribute which allows us to specify which page a given UI element will navigate to. Simply pass in the Type of the Page that the specified element will go to when clicked. Our code for this attribute class looks something like this:

namespace Website.WebTests.Attributes
{
    public class LinkedPageAttribute : Attribute
    {
        public Type PageType { getset; }
 
        public LinkedPageAttribute(Type pageType)
        {
            PageType = pageType;
        }
 
        /// <summary>
        /// Gets the element of the specified type on the page passed in, which matches
        /// the provided page type, based on the LinkedPage attribute that is applied to the element.
        /// </summary>
        public static TElement GetLinkedElement<TPage,TElement>(Page page) where TElement : class
        {
            var fieldInfos = page.GetType().GetFields();
 
            foreach (var fieldInfo in fieldInfos)
            {
                var attributes = fieldInfo.GetCustomAttributes(typeof(LinkedPageAttribute), true);
                if (attributes.Length == 0) continue;
 
                var linkedPageAttribute = (from a in attributes where a.GetType() == typeof(LinkedPageAttribute) select a).FirstOrDefault();
                if (linkedPageAttribute == null || !(linkedPageAttribute is LinkedPageAttribute)) continue;
 
                if((linkedPageAttribute as LinkedPageAttribute).PageType == typeof(TPage))
                {
                    return fieldInfo.GetValue(page) as TElement;
                }
            }
 
            // you are trying to navigate to a page which cannot be reached from the current page, check to see you have a link with LinkedPage attribute
            throw new ArgumentException("You don't have a link to this page {0} from this page {1}".FormatWith(typeof (TPage), page.GetType()));
        }
    }
}

The meat of this class is in GetLinkedElement. Here we iterate over all the properties in the Page looking for any that are decorated with the LinkedPage attribute. We find the on the for the given page type and return that as a TElement type for clicking on...

Hopefully this has been useful and/or interesting, please post any comments or criticisms below :-)

Monday, February 21, 2011

Roll your own RFH2 header for WebSphere MQ

Do you have an application that is sending messages to an IBM WebSphere Message Queue? And the receiving application is JMS based (Java Message Service) while the sending application is a traditional non-JMS (ie native) WebSphere MQ application.

In this case you may run into the same problem that I encountered. Specifically that the JMS application requires the presence of an MQRFH2 header. This header generally just contains some superfluous information, but apparently it is a requirement for JMS applications. It may be possible to configure the JMS application to ignore this header, however I don’t have any information on that.

I spent several days researching online and testing various methods and generally trying everything out until I had something that now works. Below I will give you the most important facts.

 
Structure of the RFH Header

This article is very useful at explaining at how the MQ message is structured in the case of JMS applications:


It also explains the format of the MQRFH2 header.  The format consists of a fixed part (of 36 bytes length) which has a set of bytes to indicate the total length, the encoding used, character sets etc. See the article above for the full breakdown of this fixed part.

Example of Fixed Part of the Header
StrucId (MQCHAR4)     Structure identifier.  Must be MQRFH_STRUC_ID (value: "RFH ") (initial value).
Version (MQLONG)     Structure version number.     Must be MQRFH_VERSION_2 (value: 2) (initial value).
StrucLength (MQLONG)     Total length of MQRFH2, including the NameValueData fields.
Encoding (MQLONG)    Data encoding.
CodedCharSetId (MQLONG)    Coded character set identifier.
Format (MQCHAR8)    Format name.
Flags (MQLONG)    Flags.     MQRFH_NO_FLAGS =0. No flags set.
NameValueCCSID (MQLONG)

The second part of the header is the Variable part which is generally made up from a few XML folders which you can fill with some user data (usr section) or jms specific data (jms section)


Example of Variable Part of the Header

<mcd><Msd>jms_text</Msd></mcd><jms><Dst>queue:///APPS/BOB/TEST</Dst><Tms>1297042644307</Tms><Dlv>2</Dlv></jms><usr><msg_id>445566</msg_id><receiver_id>TestReceiver</receiver_id></usr>

 
Sample Code Creating JMS Headers

Here are some examples of code that I had a look at in my investigations on how to create this header. The first is an example showing some Java code which creates a message (this code uses the JMS library so the header is auto generated). The second example is some straight C code which uses the native interface to inject the header manually.

 
Java  Sample Code

The guys we were working with, who had developed the JMS application which was picking up our messages and which required the JMS header, sent us some sample code to show how they were creating messages with headers. Note this is Java code which uses JMS libraries. There is no actual code to create the header, this is done automatically for you just by using the JMS APIs.

 
C Sample Code
There used to be a lot of samples for MQ on the IBM site, however  because they are so old they have been taken off and now are no longer available. However someone has kindly downloaded everything and placed it on their site:


The most interesting one from our point of view is:  Jsmqput.c  This one is a C source code file (very tricky to read if you aren’t used to C or C++...) but it does show the general mechanism behind putting together a message with an RFH header. One thing I noticed on this one is that they pad each folder within the variable part of the message to a multiple of four. In the end my version only pads the entire variable part, not each section as well. The other interesting thing that I picked up from that sample is that each section within the variable part of the header is preceded with a length, this wasn’t obvious from other documentation that I had read.

An example of the code for generating the header:


 These samples where useful from a reference point of view, however I still had to come up with a way to do the same thing in .NET which is the environment that I am using.


Possible Solutions

Option A – The Sensible Way – Use the IBM XMS Interface

This option is probably the way to go, however you do need to upgrade to the version 7 MQ Client libraries (if you haven’t already) and then install the IBM Message Service Client library (XMS). This then adds some new DLLs which you can reference from your code (specifically IBM.XMS.dll) and this provides a wrapper for the JMS API which can be used in a .NET environment.

Here is where you can download this library from:



There are also sample programs which come with this to illustrate its use, for example here is some code:





You can see that there is no need to do anything specific to create the header, simple just set the variables you want passed in the header itself.

Unfortunately I was restricted from upgrading our system to version 7, we were still running version 6 of the client libraries, therefore I had to move on to the next best option...

Option B – The Mad Way – Roll your own MQRFH2 Header

If you are using version 6 of the MQ Client libraries, then you are limited to this option, which is to handcraft the header yourself and insert it into the message as you send the message. This is the approach that I have used, and it was difficult to get it working, but it means no need to upgrade anything. You could use this option for any version of the client libraries.



First of all I assume that you have some existing code to send messages into the queue, for example something like this:


If you look towards the end of the above screenshot you can see where we actually write the message body to the MQMessage before it is sent to the Queue.

I added another line before this WriteString where I send the MQMessage to a new RfhHeaderProvider class (injected into the above class using our dependency injection framework being StructureMap).

Then the code looks more like this:



The header provider class is where the crux of the logic sits. We load the RFH header details from a config file, this includes a switch to turn it on/off and the xml for the three folders that we include in the variable part of the message. Have a look at the code below:


   public class RfhHeaderProvider : IHeaderProvider
    {
        #region Fields

        private readonly IRfhConfigurationProvider _rfhHeaderDetailsConfiguration;

        #endregion Fields

        #region Constructors
     
        /// <summary>
        /// Initializes a new instance of the <see cref="RfhHeaderProvider"/> class.
        /// </summary>
        /// <param name="rfhHeaderDetailsConfiguration">The RFH header details configuration.</param>
        public RfhHeaderProvider(IRfhConfigurationProvider rfhHeaderDetailsConfiguration)
        {
            _rfhHeaderDetailsConfiguration = rfhHeaderDetailsConfiguration;
        }

        #endregion Constructors
   
        /// <summary>
        /// Injects an optional RFH header, the settings for this are loaded via the config file.
        /// This header is usually only used by JMS applications, ie Java Websphere MQ applications using the JMS apis.
        /// since the retrieving part of the system is written by HP using JMS they would like this header so their end
        /// can process the messge without modifying their code
        /// INFO ON RFH Header
        /// http://middleware.its.state.nc.us/middleware/Documentation/en_US/htm/csqzaw09/csqzaw0937.htm
        /// There is an alternative to handcrafting this header, and that is by using the following library:
        /// http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24011756&loc=en_US&cs=utf-8&lang=en.
        /// However this requires us to install this package and the v7 MQ client libraries.
        /// The structure of the header is also defined here:
        /// http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/index.jsp?topic=/com.ibm.etools.mft.doc/aq06930_.htm
        /// If you want to modify this structure and examine whether the message is still valid then I recommend RfhUtils.exe
        /// This is a tool that can be run on the server to browse messages on the MQ to view their data and RFH header properties:
        /// http://sites.google.com/site/ibmmiddleware/rfhutils
        /// </summary>
        /// <param name="mqMsg">The message.</param>
        public void WriteHeader(MQMessage mqMsg)
        {
            // THE HEADER IS NOT ENABLED, THEREFORE NOTHING TO DO
            if (_rfhHeaderDetailsConfiguration.HeaderDetails.IsEnabled == falsereturn;

            var encoding = new System.Text.UTF8Encoding();

            // OLD HARD CODED STRINGS
            //var mcdHeader = @"<mcd><Msd>jms_text</Msd></mcd>";
            //var jmsHeader = @"<jms><Dst>queue:///APPS/OOT/REQUEST</Dst><Tms>1297042644307</Tms><Dlv>2</Dlv></jms>";
            //var usrHeader = @"<usr><msg_id>1122</msg_id><receiver_id>TestReceiver</receiver_id></usr>";

            // NEW DYNAMIC CONFIG LOADED STRINGS
            var mcdHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.McdHeader;
            var jmsHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.JmsHeader;
            var usrHeader = _rfhHeaderDetailsConfiguration.HeaderDetails.UsrHeader;

            var rfhHeader = mcdHeader + jmsHeader + usrHeader;

            while (encoding.GetByteCount(rfhHeader) % 4 != 0)
            {   rfhHeader = rfhHeader + " ";                        // add space here for tracking total length (don't actually do anything with this one)
                usrHeader = usrHeader + " ";                        // last item in name value section gets the extra spaces
            }
            var rfhHeaderLen = encoding.GetByteCount(rfhHeader);

            Int32 iStrucLength = MQC.MQRFH_STRUC_LENGTH_FIXED_2 + rfhHeaderLen + 12; // 3 header lengths * 4 bytes

            // RFH HEADER 
            mqMsg.Format = MQC.MQFMT_RF_HEADER_2;   // Msg Format 
            mqMsg.WriteBytes(MQC.MQRFH_STRUC_ID);   // Structure identifier   
            mqMsg.WriteInt4(MQC.MQRFH_VERSION_2);   // Structure version number
            mqMsg.WriteInt4(iStrucLength);          // Total length of MQRFH2 including NameValueData
            mqMsg.WriteInt4(MQC.MQENC_NATIVE);      // Numeric encoding of data that follows NameValueData   
            mqMsg.WriteInt4(MQC.MQCCSI_DEFAULT);    // Character set identifier of data that follows NameValueData
            mqMsg.WriteBytes(MQC.MQFMT_STRING);     // Format name of data that follows NameValueData
            mqMsg.WriteInt4(MQC.MQRFH_NO_FLAGS);    // Flags
            mqMsg.WriteInt4(1208);                  // Character set identifier of NameValueData, 1208 = UTF-8

            // Optional NameValueData folders and their content must occur in the sequence (length, data)

            mqMsg.WriteInt4(encoding.GetByteCount(mcdHeader));
            mqMsg.WriteBytes(mcdHeader);

            mqMsg.WriteInt4(encoding.GetByteCount(jmsHeader));
            mqMsg.WriteBytes(jmsHeader);

            mqMsg.WriteInt4(encoding.GetByteCount(usrHeader));
            mqMsg.WriteBytes(usrHeader);         
        }
    }

 
The first part of the WriteHeader method pads out the variable part of the header to a multiple of four. Note that I can make this code a bit more efficient (ie no need to append spaces to the rfhHeader variable since I am not using it anyway, could keep a count there instead), however I have left it as is since I now want to write something up about this code before I start optimising it further. Once the variable part has been padded, we need to work out the structure length. This is quite critical to get the length right otherwise you will get an error message when you try to retrieve the message from the queue.

The structure length is made up from the length of the fixed part of the header, plus the length of the padded variable part of the header, plus 4 (bytes) for each header length we insert before each folder.

The second part of the method writes the fixed part of the header, this includes things like the encoding, character sets used, formats etc.  I haven’t changed it too much from the default, and you should probably find that using the same settings as mine will work fine for you as well.

The last part of the method writes the variable part of the header. Each xml folder (I am using three, ie <mcd>, <jms> and <usr>) needs to be pre-pended with its length.

Once you have implemented the code as per above, you will also be able to write your own RFH header into your MQ message for use by JMS applications!

 
Tools for Analysing and Debugging the RFH Header

There is a useful (although quite hard to understand) tool out there for pulling messages of a WebSphere MQ. This is the RfhUtils.exe application which is available from here:


There is a useful article on how to use it at this location:


The best way to start this thing is to write a few lines into a cmd file as per the instructions. (ie where you set the required paths and pass in the IP address etc).

Once you have started the application it will look something like this:


The options that I used most frequently were “Browse Q” this will show you any messages in the queue, and specifically if there are any errors or not. If there aren’t any errors the message will be loaded and you will be able to see the data breakdown (ie data size) of the body versus the RFH header. You can then also switch to the RFH tab to see the RFH header details, and also to the jms and usr tabs to see the content of these xml folders.

There is an option called “Purge Q” which is useful if you want to clear the messages before you put another one, or you could hit “Start Browse” which steps you through all the messages if you have more than one and want to step through them all.

Hope that helps and let me know if I have missed anything?