.Net Discoveries

VB.Net and GroupWise Soap Pt. 4 - HTML Messages

Prologue

In my last few posts, I talked about accessing GroupWise via the SOAP services that they provide. In the last post, specifically we looked at sending and creating messages. I left with the promise that if I could get draft HTML messages to work that I'd post an article about it. Here it is.

Problem

Our problem, continued from the last post, was to figure out how to login and create a draft message in the user's Work in Progress folder that is a template of something the company uses currently in hard copy. In this article, we'll leverage all the stuff we've done up to this point and add the functionality to create an HTML draft message.

Solution

We will be building on the application that we started last time. If you haven't created the login procedures from Part 1, the folder listing subroutines from Part 2, and the creating/sending messages subroutines from Part 3, I suggest you do it now, we will need each of those pieces and will just be adding additional functionality.

When I started this post, this first line actually read, "We actually have very little to do do make this work." That's not correct, we have a bunch of stuff to do to get it to work. In addition to adding HTML messages to our functionality, we'll add functionality so that we can embed images into our HTML and they would display in our message. This added a decent amount of code to the project.

I ran into a lot of trouble with getting images to display and had a lot of discourse via the forum with Novell people about how to get it done. Finally my troubles came down to 2 things: 1) The image wasn't attaching as it should, it was only 10% of what it should be, and 2) you have to open the draft message in the mode it was "created" or intended to be in. #1 I will address a little later in this post when it makes more sense to do so. #2 however took me quite a long time to figure out, (actually I got a decent amount of help from an outside consultant who replied on the post). Basically, if you're default compose view is set to plain text, when you open a draft HTML item, it WON'T be opened as HTML. If you want to save an HTML message (or create one in our instance) as a draft and then reopen it as an HTML message, you need to set your default compose view to HTML. You can do this under Tools->Options->Environment->Views(tab), and select HTML as the 'Default Compose View & Font'. I learned this the hard way. One would think that the message itself would have enough identifying information in it that the program can figure out which to do... Nope. Or one would figure that if it opened plain text, that switching to HTML view would restore your original HTML message... Nope. Neither way displays your images and the other expanded features of HTML. Ok, enough complaining about that for now, lets get started.

First, lets modify our form. We only need to add one control, a checkbox named chkHtml with text of 'HTML Message'. Using this, we can create an HTML message (using HTML code) or a plain text message using the same textbox and createMessage code.

We have a number of things to create in the code so that this will work, and then we'll finally go back and modify our CreateMessage subroutine to include our HTML stuff.

If we look in the Novell documentation for creating an HTML message, we find "HTML messages are returned as the first attachment and are named text.htm. Any embedded images or associated files are also downloaded as attachments. These HTML associated files are all marked hidden." So, to create an HTML message we need to add an attachment with our HTML content (rather than create a message body... ok, that's not strictly true, it's recommended to enter a text version of your HTML as the message body in addition to your attachment). That means, we'll just add an attachment to our message body IF we check the chkHTML checkbox.

Further reading in the Novell documentation reveals: "Any embedded images or associated files are also downloaded as attachments. The HTML message body and accompanying files are grouped together. The text.htm file is first, followed by the other associated files. Each of the accompanying attachments has a contendId element." What this means is that we need to create an attachment for our HTML code, this should be the first attachment and it should be named text.htm. Also, we should create an attachment for each image that should be embedded in the message, and they should follow in order. Now, this only applies for images that are embedded INTO the email. This doesn't apply to images that are linked from outside sources (their src is a web address instead of a file path). In our example we'll only use images that are embedded into the email, we won't be processing image files that are linked from outside sources (in fact our code will probably break a linked image if we do it that way, more on that later).

Ok, now with that all said, we should have a basic understanding of what we need to create, so let's get started. Right off the bat, we want to create a new function that will return our HTML message and embedded images as an attachment array (HTML message first followed by the images). Let's create a function to do this and define it as follows:

Private Function ConvertHtmlMessageToAttachmentArray _
   (ByVal sMessageToConvert As String) As GWWS.AttachmentItemInfo()

End Function

We are going break up all our pieces and return them as a GroupWise message AttachmentItemInfo array. Since we don't know right off the bat how many images we are going to add to our attachment array, we'll take the easy way out and rather than define an array, we'll create a generic list(of) our AttachmentItemInfo and just keep adding them. Then we can have our list output an array. So, we need to add some imports statements at the top of our code, so let's add them all now and we'll get to each in time. Add the following to the top of your code:

Imports System.Collections.Generic
Imports System.Text.RegularExpressions
Imports System.IO

The important one for our list(of) operations is the Collections.Generic. If you haven't played with list(of) yet, try it out it is VERY cool stuff. Anyhow, we're ready to begin defining some variables for our code. Let's start with our list(of) definition, add the following just inside your function declaration:

Dim theAttachments As New List(Of GWWS.AttachmentItemInfo)

This defines a list of AttachmentItemInfo objects for us, we'll use it later. If you remember we need to add our HTML message attachment first, and then our images as attachments. Now we COULD create our HTML attachment first, but we need to rename our image attachments and update the references to them in our HTML code. If we add our HTML attachment first, then the HTML attachment will have incorrect code and won't display our images as we would like. So what we'll do is add our images first, and THEN create our HTML attachment and insert it at the beginning of the list (LOVE GENERIC LISTS!).

To find all the instances of our images, we'll use the REGEX object and create a regular expression to find our image tags, extract the SRC and replace it with our updated src. (This is where the RegularExpressions import statement comes in). Now, not being an expert on regular expressions, it gets to be a little bit confusing getting everything just right, but I managed to get a regular expression that can find <img > and return everything from < to >. So we'll do is create two regex objects, and instantiate them with the correct regular expression to return what we need. Add the following just below our last variable declaration:

Dim regImage As New Regex("<img[^>]*>")
Dim regSrc As New Regex("src=" & Chr(34) & "[^>]*" & Chr(34))
Dim matchesImages As MatchCollection = regImage.Matches _
      (sMessageToConvert)
Dim iCount as Integer = 0 'Counter variable

First we define a regex object to find all our image tags, and then we define one to find the src within the tag, the regex for the image tag is defined like this:

<img - Match the characters "<img" literally
[^>] - Match a single character NOT in the list (our list consists
          of only the > character)
* - Match the proceeding between zero and unlimited times (meaning
     match our [^>] block 0 to unlimited times)
> - Match ">" literally

So basically, we find "<img" followed by anything except ">" followed by ">". In the next to last line, we request our Regex object to find matches of this sequence and store them in a matches collection. We'll iterate through the collection later and use our other Regex object, it finds the src. Basically, the regSrc object is defined like this:

src= - Match "src=" literally
Chr(34) - adds the double quote (") to the src= statement, so we're actually finding src=" literally
[^>] - Match a single character NOT in the list (our list consists
          of only the > character)
Chr(34) - adds the double quote to the END of our statement

So basically, we're looking for src=" literally (including the double quotes, we had to use the Chr(34) code because the way strings are handled we couldn't just type the double quotes), followed by anything EXCEPT a > followed by " (double quotes). This returns the entire src="xyz" string. (Confused  yet?) (As an FYI, we will be replacing the content of the src attribute. If you have an src that links to an external website, it will be replaces. You'll need to add a check to differentiate between embedded images and linked images. I haven't done so.)

And finally we setup a counter variable, iCount. Ok, so we've setup our regex objects, and we've populated a collection with the strings that match our <img > tags. Now we want to cycle through the collection and do two things 1) create an attachment with our image, and 2) replace the src attribute with an updated src that reflects that the image is embedded in the email. So first thing, let's setup a For Each loop to loop through our image tags collection. Add the following:

For Each m As Match In matchesImages

Next

Now we want to extract the src portion, and use it to make an attachment out of the image. So what we'll do is create a couple of variables here inside the for each loop to do that. Add to your For Each loop, the following:

Dim matchSrc As Match = regSrc.Match(m.Value)
Dim sSrc As String = matchSrc.Value.Replace _
     ("src=", "").Replace(Chr(34), "")
Dim theAttachment As GWWS.AttachmentItemInfo = _
     GetAttachment(sSrc, True)

What we're doing is creating a regex match object and matching the src phrase in our first img object. If you remember, this will return the entire src="xyz" phrase. We want just the file name for now, so the second variable, sSrc removes the src= and the double quotes from both ends, so we should just have a file path to an image and nothing more. Finally, we need to retrieve the image using the filename and make it an attachment. So we pass its filepath to the GetAttachment function we are about to create. This returns an AttachmentItemInfo object that we will use in a minute. There are some other things we need to do before our attachment is finished, but first we need the file to be an attachment. So let's create a new function GetAttachment and define it as follows:

Private Function GetAttachment(ByVal sFilePathAndName As String, _
      Optional ByVal bHideAttachment As Boolean = False) _
      As GWWS.AttachmentItemInfo

End Function

Our function will take a file path, retrieve the file located there, create an attachment and return the attachment. We also have a parameter to make the attachment hidden (as we should for html messages). (Bonus, you can use this function to create an attachment for ANYTHING, not just our images). Now we need to create a FileInfo object to hold our file (thus the System.IO import above). We also need to create an empty AttachmentItemInfo that we can build up. So add the following variables to your function:

Dim thefile As FileInfo = New FileInfo(sFilePathAndName)
Dim gwAttachItemInfo(0) As GWWS.AttachmentItemInfo

Now that we've got our file and our AttachmentItemInfo, we can get started with building up our attachment. Let's start with the easy stuff first:

gwAttachItemInfo(0) = New GWWS.AttachmentItemInfo

' Name
gwAttachItemInfo(0).name = thefile.Name

'Date
gwAttachItemInfo(0).date = _
      Format(thefile.CreationTimeUtc, "yyyy-MM-ddTHH:mm:ddZ")
gwAttachItemInfo(0).dateSpecified = True

'Hide attachment?
If bHideAttachment = True Then
   gwAttachItemInfo(0).hidden = True
   gwAttachItemInfo(0).hiddenSpecified = True
End If

Ok, that's the end of the EASY stuff, we instantiated the AttachmentItemInfo object, assigned a name, a date and marked it hidden, if requested. The rest of it gets a little more complicated. Now we need to import the binary data for the image file and add it to the data of the attachment. If you remember back at the very start of the post, you'll remember that I mentioned 2 things didn't work. Well this is #1 that didn't work. I originally started with the code from the example application included in the SDK for GroupWise Soap. Unfortunately the GetAttachment code in their example used a string array to receive the filestream. This resulted in only the information up to the first space being retrieved (in the case of my gif file, "GIF89Aw"). Nothing else. For the longest time I couldn't figure out what was going on. Once I finally realized that the code pretty much only works for text attachments, I had to find something different. Gotta love the internet and copy and paste inheritance though. I found some code, an made some heavy modifications that resulted in some code that DOES work to import the image's data to the AttachmentItemInfo's data property. Add the following:

'Data
Dim fsData As FileStream = thefile.OpenRead()
Dim gwByte(fsData.Length) As Byte
fsData.Read(gwByte, 0, fsData.Length)
fsData.Close()
gwAttachItemInfo(0).data = gwByte

Return gwAttachItemInfo(0)

First we create a FileStream object and assigned it to our image's file stream. Next, we create an array of bytes that is as long as our FileStream, and then read the FileStream into the byte array (starting at 0 and going to the end). Next, we close the FileStream object, add our byte array as the data property of the attachment. And finally, we return our AttachmentItemInfo object so we can continue to manipulate it.

Ok, so now we've created our attachment and returned it to our ConvertHtmlMessageToAttachmentArray (CHMTAA, I'll use this from now on, it's shorter...) function. Now we want to do some processing to the attachment so that it conforms to the standards we need it to. If you remember from the beginning, the Novell documentation states, "Each of the accompanying attachments has a contendId element.." This means that we need to assign an contentID to each attachment. This contentID needs to be unique (or it will pickup the first attachment with the correct contentID and use that for all pictures). We also need to replace the filepath in our HTML code with our new contentID so that it can be located. And we'll also set the contentType so that it can be treated as it should. Add the following inside your For Each loop, after the 'theAttachment' declaration:

theAttachment.contentId = iCount & "." & Path.GetFileName(sSrc)
theAttachment.contentType = "image/" & sSrc.Substring(sSrc.Length - 3)
sMessageToConvert = sMessageToConvert.Replace(matchSrc.Value, _ 
            "src=" & Chr(34) & "cid:" & theAttachment.contentId & Chr(34))

First we assign a contentID. We can't just use the filename because we may have it in our message more than once. So we use a counter variable to make a differentiation. Next, we tell it that our content is an image of type X (whatever the extension is i.e. gif, jpg). Finally, we take our full HTML message and replace the src="xyz" statement with a custom one. This will specify that our src should use the contentID instead of a file path. We do that by adding 'cid:' before our contentId so our final src should look like src="cid:1.xyz.gif". Remember our previous src had the full filepath, now we shorten it to just the filename and the cid directive. Now we can add our attachment to our list of attachments and increment our counter. Add the following to finish out our For Each loop:

theAttachments.Add(theAttachment)
iCount += 1

Basically our For Each loop will cycle through each of the embedded images, create it as an attachment, pack it up as we like, modify our HTML to reflect the new src and then add the attachment to our attachment list. The last thing we need to do is use our modified HTML to create an attachment and create it as the first attachment in our message's attachments array. We're going to create another helper function, one that will take our HTML text, create an attachment with the correct name and return that. So jump out of our CHMTAA function and create a new function as follows:

Private Function ConvertHtmlMessageToAttachment _
      (ByVal sMessageToConvert As String) As GWWS.AttachmentItemInfo

End Function

Our function will take the text of our HTML message, and return an AttachmentItemInfo object that we can add to our attachments list in our CHMTAA function. We need to create two objects to start with, an AttachmentItemInfo that we can pack up and send back, and an encoding object so we can easily convert our string to a byte array. Add the following variable declarations to your code:

Dim gwAttachmentItemInfo As New GWWS.AttachmentItemInfo
Dim encoding As System.Text.Encoding = System.Text.Encoding.UTF8

If you want to know more about the encoding object, you should check out Part 3 of this series. Now we just need to pack up our attachment object and send it back. Add the following code to your subroutine to finish it out:

gwAttachmentItemInfo.name = "text.htm"
gwAttachmentItemInfo.date = Format _
        (Date.Now(), "yyyy-MM-ddTHH:mm:ddZ")
gwAttachmentItemInfo.dateSpecified = True
gwAttachmentItemInfo.data = encoding.GetBytes(sMessageToConvert)
gwAttachmentItemInfo.hidden = True
gwAttachmentItemInfo.contentType = "text/html"

Return gwAttachmentItemInfo

According to our Novell documentation: "HTML messages are returned as the first attachment and are named text.htm." So we name it accordingly. Next we assign a date and let it know that a date has been specified. Then we use our encoding object to convert our HTML text to a byte array and assign that as the attachment's data. We should also mark our attachment as hidden, and finally we should mark the attachment's contentType as being html (why it doesn't recognize that and open in the correct compose mode I don't know but...). And finally we return our object to the calling procedure.

Ok, let's go back to our CHMTAA function one last time and finish it out. We need to get our HTML attachment and add it as the first item in our list, and then return our list as an array to the CreateMessage subroutine. Easily done with the very cool generic list. Add the following two lines to your CHMTAA function AFTER the For Each loop:

theAttachments.Insert _
     (0, ConvertHtmlMessageToAttachment(sMessageToConvert))
Return theAttachments.ToArray()

Using the built-in features or our list of attachments, we request an HTML attachment and insert it in the first position of the list. Next we call the ToArray() function of the list and return the resulting array.

OK, we've covered a lot of ground, but we're still not quite done. We still need to modify our CreateMessage subroutine so that it can handle the HTML. We will leave all the existing code (since it is recommended that we assign a text version of our HTML to the message body). All we need to do is add the attachment array. In your CreateMessage subroutine that we created in Part 3, add the following right after the gwMail.message = gwMessageParts statement:

If chkHtmlMessage.Checked = True Then
    gwMail.attachments = _
     ConvertHtmlMessageToAttachmentArray(txtMessage.Text)
End If

If our HTML checkbox is checked, then we simply pass the HTML text to our CHMTAA function and assign the attachment array that is returned to the attachments property of our message.

Go ahead and run your application and try to send and HTML message with and embedded image. Here's the HTML code that I used to create my HTML message for testing purposes:

<html>
   <head>
      <title></title>
   </head>
   <body>
      <table border=1>
         <tr><td><p align=center>Cell1</p></td></tr>
         <tr>
            <td>
               <p align=center>
                  <img src="C:\My Pictures\xyz.jpg" />
               </p>
            </td>
         </tr>
      </table>
      <img src="C:\My Pictures\abc.gif" />
      <br>
      <img src="C:\My Pictures\subfolder\abc.gif" />
   </body>
</html>

You'll notice that it has 3 images, xyz.jpg and two labeled abc.gif. In my test, the two abc.gif files had the same name but were different images and were in different folders. This is how I determined I needed to use the counter variable in the contentID creation above.

Hopefully your program will work as you want it to. If you're having difficulty with it open as a text file, then check out the beginning paragraph's lament about item #2.

Epilogue

Phew. We covered a bunch of stuff in this post. Now I can create my templates to my hearts content and have them added automatically to people's work in progress folders and they can have full formatting because I can use HTML code to do it (I wonder if I can embed JavaScript this way?) (NOTE: you can embed JavaScript this way, but GWclient doesn't process it).

One of the diagnostic tools that you can use to find problems (and trust me I had lots of problems I had to find) is a trace file. This will give you the xml that the soap functionality sent to the server and the response that it sent back. I had a heck of time getting tracing to work, so I may write a part 5 that goes over what I did to get it to work, however, this concludes what we set out to do. We created a draft item, put it in the users work in progress folder and even added HTML functionality. Go take a well earned break.

As a side note, I'm trying beef up on my bloggins skills, so...