Why Windows Azure

As a consultant, I am always trying to see where new tools may fit so that I can help my clients. So I really need to know where the cloud fits into application development recommendations. Really, it’s just another platform that expands the options for a client. This is my 10,000 foot view of criteria for when Azure applies to solution (given what I know today):

Advantages

  • Lower cost of maintenance. You don’t own the hardware, you don’t fix it, update it, etc.
  • Pay as you go. A short term app (say for a marketing campaign) is very cheap to run.
  • The app is abstracted from the infrastructure. You don’t have access to it, so now you ignore it.
  • Scalability is easy. Apps that are seasonal or have know traffic spikes benefit from this greatly. Also apps that become more or less popular are easier to scale up or down in a very short period of time.
  • Cost. See the Azure Pricing
    • The base cost for Azure is about ~ $1000/yr just to have one active app instance ($0.12/hr). Bandwidth and disk space are extra charges on top of the base cost, but seem inexpensive to me.
    • Since bandwidth and storage are cheap in Azure, so if you have a high need for either Azure is a good idea.
  • Reliability. Azure has Service Level Agreements (three nines or better) with penalties for Microsoft if they fail.
  • There is alerting/API integration with SCOM, so if you want to monitor an Azure app with your in-house tools you can.
  • Geo-location. You can have multiple instances of your Azure app in geographically disparate locations. Of course there are issues with data sharing, authentication, etc. But it’s significantly easier with Azure than other options available today.
  • You can use Azure as a storage-only option and split your app between self hosting and Azure, or vice-versa.

Disadvantages

  • Having an app in the cloud may not be compliant with some regulatory guidelines
  • Cloud applications are web apps only. Not all solutions use web apps.
  • Identity and authentication are a challenge. The app is no longer in your domain, so you will need to look at federation or other options.
  • Most out-of-the-box apps are not hosted on Azure. Think CRM, ERP, financial, etc.
  • Apps that need access to the infrastructure aren’t going to work or need to be re-designed for the new paradigm.
  • It’s a new platform, even if it seems just like ASP.NET. This means you design differently, you develop a little differently as well. So there is a learning curve.
  • You are still responsible for data backups, I’m still not clear on how this works for the non-SQL storage options.

Overall, I think this is a significant new option for all sizes of companies. I’m not sure many companies are ready to give up the security of having mission-critical apps in-house, but I think for smaller companies Azure is a huge step forward. If you are already paying for hosting an app of some kind, give Azure a look.

Posted: Friday, September 18, 2009 4:00:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET | Azure | Cloud

Creating an Extraction Rule for VSTS 2008 Web Tests

Extraction rules are essentially for finding data in the HTTP response and placing it in the output context of the web test. There are a few built-in tests, but they mostly focus on the HTML tags themselves and the attributes. In my case I really needed the data between span tags. I think this could probably be done with the existing rules and some regular expressions, but I couldn’t resist the chance to write some code and learn something new.

All you need to do is place the class file in the Test Project and compile it. The rule automatically becomes available to the tests in the project.  Here is my class that finds a span for a given ClientId. It overrides the Execute method of the ExtractionRule base class and attempts to find a span for the given ID. If the span is found, it parses the HTML string to find the content of the span tag.

namespace WebTestDemo

{

    [System.ComponentModel.DisplayName("Span Extractor")]

    public class ExtractSpan : ExtractionRule

    {

        // The name of the desired input field

        private string nameValue;

        public string ClientId

        {

            get { return nameValue; }

            set { nameValue = value; }

        }

 

        public override void Extract(object sender, ExtractionEventArgs e)

        {

            string[] tagTypeFilter = new string[] { "span" };

 

            //Fail the test if nothing is found (this may need to be modified)

            e.Success = false;

 

            if (e.Response.HtmlDocument != null && e.Response.IsHtml)

            {

                try

                {

                    //Find the span tag based on ID. Exception if none found

                    HtmlTag result = e.Response.HtmlDocument.GetFilteredHtmlTags(tagTypeFilter).First(t => string.Equals(t.GetAttributeValueAsString("ID"), this.nameValue, StringComparison.OrdinalIgnoreCase));

 

                    //The span was found (no exception), now find the data

 

                    //Get the location of the ID in the span tag

                    int startPosition = e.Response.BodyString.IndexOf(this.nameValue);

 

                    //Find the position of the data immediately following the closing angle bracket of the span,

                    //  accounting for the > character as well

                    startPosition = e.Response.BodyString.IndexOf(">", startPosition) + 1;

 

                    //Get the position of the closing tag for the span

                    int endPosition = e.Response.BodyString.IndexOf("</span>", startPosition);

 

                    //Fetch the content

                    string content = e.Response.BodyString.Substring(startPosition, endPosition - startPosition);

 

                    //Add the value to the context output. This could just as easily go to a file or a DB

                    // this step is not necessary for the extraction to succeed

                    e.WebTest.Context.Add(this.ContextParameterName, content);

 

                    //Mark the extraction as successful

                    e.Success = true;

                }

                catch (Exception)

                {

                    e.Success = false;

                    e.WebTest.Context.Add(this.ContextParameterName, string.Format("span tag id={0} not found", this.nameValue));

                }

            }

        }

    }

}

Now that I have the class, I need to wire it up to a URL in a web test. Right-click the URL and choose Add Extraction Rule…

AddRuleToTest[1]

I need to set two properties:

  1. Context Parameter Name. This comes from the base ExtractionRule class, and is the name for the data that ends up in the output.
  2. ClientId. This is a custom property from my class. It is the ClientId of the rendered control in the HTML output. The class finds the span with this name and returns the data.

Now when I run the test, if the ClientId I specified was found, it shows up in the Context output after running the test. The Context Parameter Name was “SpanData” in this case:

ExtractionRuleResults[1]

This could be made more robust by not coding specifically for spans. Certainly there could be issues if the tag ID is used more than once or if there is significant nesting of spans within the tag you are trying to find. This code is intended to prove out the concept, it could certainly be made stronger.

One thing I want to mention is that all the code samples I have run across (MSDN included) show the RuleName property as the way to display the extraction rule name in the Visual Studio UI. But under compilation this property comes up as obsolete. I found the answer on Ed Glas’s blog. The obsolete message mentioned using attributes, but this info was not discoverable, so I was quite grateful for that posting to get the syntax correct.

Helpful Links

Must Read VSTS - Testing Related Blogs and Introductory Articles

How to: Create a Custom Extraction Rule

Custom Extraction Rule and Generating a Code Test from VSTS

Posted: Wednesday, August 26, 2009 8:29:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 3.5 | Performance | Visual Studio | VSTS

Adding A Relationship to the User Profile in Commerce Server 2009

For this post I am assuming you are familiar with Commerce Server 2009 and operation sequences, which in my opinion are the main reason to use the upgraded Commerce Server product. You can find some background on operation sequences on MSDN.

 

The problem we are trying to solve here is the addition of new properties to the Commerce Server Profile system, but we want a one-to-many relationship between the user profile and the new property. As an example, we will be adding a number of Tests to the user’s profile. In this case the user needs to participate in a number of tests that we can track, because these tests determine what products a user will be able to purchase. The more tests and test categories the user participates in, the more products they are eligible for.

 

The basic steps for this are:

  • Create the underlying SQL Server data store
  • Map the store to the user profile in Commerce Server manager
  • Create further mappings for Commerce Entities in the MetadataDefinitions.xml file
  • Create C# classes
  • Add the classes to the operation sequences in ChannelConfiguration.config

See How to Use the Profiles Schema Manager on MSDN for documentation on the Profiles editor in Commerce Server Manager.

  1. Create the underlying data table and fields in SQL Server in the profile database where the data will be stored. This is very important to do first, even though Commerce Server will let you create definitions before the underlying data store exists. (I’m assuming you are familiar with SQL Server). In my example I am going to add a number of tests per each person in the profile. The tests have and ID, name, and category.
  2. Assuming that we are adding our relationship to the User Profile, add a new field to the UserObject table that is nvarchar(255) and named after your custom profile object. Commerce Server uses this field to store the IDs (primary key) of the related items. You may need to make the field size larger if your key is large like a GUID or you expect to have many child records. 255 is the default number from the Commerce Server SDK examples.
  3. Save the database change scripts (for source control)
  4. Open Commerce Server Manager, expand the tree:
    Commerce Server Manager –> Global Resources –> Profiles –>Profile Catalog –> Data Sources –> SQL Data source (name varies) –> Data objects
  5. Create a new object: Right click the Data Objects node and choose New Data Object. Choose the database table name from the list box, the Display Name and Members will fill out automatically, even though the members are not propagated further.
  6. Add Members to your new object: For each field you want to add, right click the new node and choose New Data Member.
    1. Choose the Member Name by clicking the database field name in the list.
    2. Click the Add button.
    3. Repeat for other new members
    4. Click the Finish button.
  7. Move to the Profile Definitions node in Commerce Server Manager.
  8. Right-click the Profile Definitions node and choose New Profile Definition.
  9. Give the new profile definition a name and display name. Click the Finish button, ignoring the Custom Attributes section.
  10. Update the new node by first clicking the node and allowing the editor to load.
  11. Create a new Group by clicking the Add button and selecting Add a New Group.
    1. Name the first group “GeneralInfo” with a display name of “General Information”. This is a common Commerce Server pattern.
    2. Click the Apply button.
  12. For each property you want to add, click the Add button and choose to add a new property
  13. Give the property a name and display name. Case matters later, so pay attention to the details here.
  14. Choose the correct type
    1. If you are adding a relationship to a profile definition type:
      1. Choose Profile.
      2. Choose the definition type for the Type Reference field.
    2. If you are adding a new field
      1. Choose the appropriate data type
  15. Expand the advanced attributes node
    1. Choose the Map To Data and use the Data Source Picker to map the database field
    2. Click the Apply button, or the change won’t update the Properties panel.
  16. Be sure to define the primary key
  17. Save the Profile
  18. Now we will build the relationship between the User Object and the new Profile Item we created.
    Click the User Object node and allow the editor to load.
  19. Expand the General Information group
  20. Add a new property to the General Information group
    1. Name the property after your relationship
    2. Choose Profile as the Type
    3. In the Type Reference field navigate to your new Profile type
    4. In the Map To data field choose the database field you created in the UserObject table in step 2.
    5. Click the Apply button.
  21. Save the Profile
  22. Export the schema as an XML file (to put under source control)
  23. Now we need to map the changes we made in Commerce Server Manager into a CS2009 definition. In your web project in Visual Studio, edit MetadataDefinitions.xml
    1. Find he <CommerceEntities> node under the <DefaultChannel> node
    2. Create a new <CommerceEntity> node for the new item, Test in my example.

      <CommerceEntity name="Test">

          <DisplayName value="Test"/>

          <EntityMappings>

              <EntityMapping

                csType="Microsoft.CommerceServer.Runtime.Profiles.Profile"

                csAssembly="Microsoft.CommerceServer.Runtime, Version=6.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"

                csDefinitionName="Test"

                csArea="Profiles">

                  <PropertyMappings>

                      <PropertyMapping property="TestId" csProperty="GeneralInfo.TestId" />

                      <PropertyMapping property="TestName" csProperty="GeneralInfo.TestName" />

                      <PropertyMapping property="TestCategory" csProperty="GeneralInfo.TestCategory" />

                  </PropertyMappings>

              </EntityMapping>

          </EntityMappings>

          <Properties>

              <Property name="TestId" dataType="String" />

              <Property name="TestName" dataType="String" />

              <Property name="TestCategory" dataType="String" />

          </Properties>

          <Relationships/>

      </CommerceEntity>
       

    3. Find the <CommerceEntity> node named UserProfile.
    4. Add a new PropertyMapping node in the ProperyMappings section.
    5. Add a property attribute to name the property after the name defined in step 20.
    6. Add a csProperty attribute that is the name you chose in step 20a. Case matters. The format is PropertyGroup.PropertyName (ex: GeneralInfo.Gender)
    7. Find the <relationships> node under UserProfile.
      1. Add a relationship:
        ex:

        <Relationships>

            <Relationship name="Tests" type="Relationship" modelName="Test" isMultipleItems="true">

                <DisplayName value="Tests" />

                <Description value="User's Tests" />

            </Relationship>

        </Relationships>

  24. Create the following types of classes for Commerce Server:
    1. A class to hold your new data


      namespace
      CommerceDemo

      {

          [Serializable]

          public class Test

          {

              public string TestId { get; set; }

       

              public string TestName { get; set; }

       

              public string TestCategory { get; set; }

          }

      }

    2. A processor class that inherits RelatedProfileProcessorBase

      namespace CommerceDemo

      {

          public class UserProfileTestProcessor : RelatedProfileProcessorBase

          {

              // Methods

              protected override bool CanDeleteRelatedItem(string relatedItemId)

              {

                  return true;

              }

       

              // Properties

              protected override string CommercePreferredIdKeyName

              {

                  get

                  {

                      return "GeneralInfo.Test";

                  }

              }

       

              protected override string CommerceRelatedIdsKeyName

              {

                  get

                  {

                      return "GeneralInfo.Test";

                  }

              }

       

              protected override string ParentProfileModelName

              {

                  get

                  {

                      return "UserProfile";

                  }

              }

       

              protected override string PreferredItemRelationshipName

              {

                  get

                  {

                      return "TestId";

                  }

              }

       

              protected override string ProfileModelName

              {

                  get

                  {

                      return "Test";

                  }

              }

       

              protected override string RelationshipName

              {

                  get

                  {

                      return "Tests";

                  }

              }

       

          }

      }

    3. A committer class that inherits ProfileCommitterBase

      namespace CommerceDemo

      {

          class UserProfileTestCommitter: ProfileCommitterBase

          {

              protected override string ProfileModelName

              {

                  get { return "Test"; }

              }

          }

      }

    4. A response builder class that inherits RelatedTestResponseBuilderBase

      namespace CommerceDemo

      {

           class TestResponseBuilder : ProfileResponseBuilderBase

          {

       

              protected override string ProfileModelName

              {

                  get

                  {

                      return "Test";

                  }

              }

          }

      }


  25. Open ChannelConfiguration.config
    1. Find the <MessageHandler> node for “CommerceQueryOperation_UserProfile”
      1. Add <Component> nodes for the processor and response builder classes created above:

        <MessageHandler name="CommerceQueryOperation_UserProfile" responseType="Microsoft.Commerce.Contracts.Messages.CommerceQueryOperationResponse, Microsoft.Commerce.Contracts, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35">

            <OperationSequence>

                <Component name="User Profile Loader" type="Microsoft.Commerce.Providers.Components.UserProfileLoader, Microsoft.Commerce.Providers, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35"/>

                <Component name="User Profile Test Processor" type="CommerceDemo.UserProfileTestProcessor, CommerceDemo, Version=1.0.0.0, Culture=neutral,PublicKeyToken=fbeb08efdb2d0b73"/>

                <Component name="User Profile Response Builder" type="Microsoft.Commerce.Providers.Components.UserProfileResponseBuilder, Microsoft.Commerce.Providers, Version=1.0.0.0, Culture=neutral,PublicKeyToken=31bf3856ad364e35" />

                <Component name="User Profile Test Response Builder" type="CommerceDemo.UserProfileTestResponseBuilder, CommerceDemo, Version=1.0.0.0, Culture=neutral,PublicKeyToken=fbeb08efdb2d0b73" />

            </OperationSequence>

        </MessageHandler>

    2. Find the <MessageHandler> node for “CommerceCreateOperation_UserProfile”
      1. Add <Component> nodes for the processor, Committer, and response builder classes created above, similar to the previous step.
    3. Find the <MessageHandler> node for “CommerceDeleteOperation_UserProfile”
      1. Add <Component> nodes for the processor, Committer, and response builder classes created above, similar to step 25a1.
  26. Create any converter classes which converts the collection of Commerce Server properties from a CommerceEntity into a typed C# object.

            public static Test ConvertTest(CommerceEntity testEntity)

            {

                Test result = new Test();

     

                result.Id = testEntity.Id;

                result.TestName = testEntity.Properties[“TestName”].ToString();

                result.TestCategory = testEntity.Properties[“TestCategory”].ToString();

     

                return result;

            }

  27. Redeploy the project
  28. Reset IIS so the Commerce Server cache is emptied.

So how does this look when wired up? That’s a complex question to be sure, but here is a simplified example, assuming we have a User object class with simple common properties:

public User GetUser(string userId)

{

    User user = null;

    OperationServiceAgent svc= new OperationServiceAgent();

 

    CommerceQuery<CommerceEntity> query = new CommerceQuery<CommerceEntity>("UserProfile");

    query.SearchCriteria.Model.Properties["Email"] = userId;

 

    CommercePropertyCollection properties = new CommercePropertyCollection();

    properties.Add("Email");

    properties.Add("FirstName");

    properties.Add("LastName");

    query.Model.Properties = properties;

 

    CommerceQueryRelatedItem<CommerceEntity> test= new CommerceQueryRelatedItem<CommerceEntity>("Tests", "Test");

    test.Model.Properties.Add("TestId");

    test.Model.Properties.Add("TestNumber");

    test.Model.Properties.Add("TestCategory");

    query.RelatedOperations.Add(test);

 

 

    CommerceResponse response = svc.ProcessRequest(GetRequestContext(), query.ToRequest());

   

    CommerceQueryOperationResponse  queryResponse = response.OperationResponses[0] as ommerceQueryOperationResponse;

    if (queryResponse !=null && queryResponse.CommerceEntities.Count > 0)

    {

        Convert(queryResponse.CommerceEntities[0]);

    }

    return user;

}

 

public static User Convert(CommerceEntity userEntity)

{

    User result = new User();

    result.Id = userEntity.Properties["Id"] as string;

    result.Email = userEntity.Properties["Email"] as string;

    result.LastName = userEntity.Properties["FirstName"] as string;

    result.FirstName = userEntity.Properties["LastName"] as string;

 

    CommerceRelationshipList relationships = userEntity.Properties["Tests"] as CommerceRelationshipList;

    result.Tests = ConvertTest(relationships); //Function above in post

 

     return output;

}

Posted: Monday, August 10, 2009 8:48:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET | Commerce Server

Useful LogParser Queries

If you deal with production servers and you don’t use LogParser, you should. It gives SQL-like abilities to query web server logs an other log types (like Event Logs). Coding Horror has a great article about the tool as well, and a link to a good article about forensic log parsing. Here are my most used web queries:

Find Pages with 500 errors
logparser "SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status >= 500) GROUP BY cs-uri-stem, code ORDER BY Hits DESC" -o:CSV >> C:\Data\ErrorPages.csv

Find 404 Requests
logparser "SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status = 404) GROUP BY cs-uri-stem, code ORDER BY Hits DESC" -o:CSV >> C:\Data\404Pages.csv

Find the Slowest Pages
logparser "SELECT TOP 100 cs-uri-stem AS Url, MIN(time-taken) as [Min], AVG(time-taken) AS [Avg], max(time-taken) AS [Max], count(time-taken) AS Hits FROM C:\ProdLogFiles\ex*.log GROUP BY Url ORDER BY [Avg] DESC" -o:CSV >> C:\Data\SlowPages.csv

Posted: Thursday, August 06, 2009 7:41:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET | IIS | Performance | Tools

Another “Why didn’t someone tell me this earlier” Post

90% of the projects I work on need to send emails, and most clients don’t allow developer SMTP servers floating around their network (for good reason). So a big headache ensues trying to get access to an Exchange or SMTP server somewhere. I have been doing this same dance for years. But today I find a blog post about sending emails without an SMTP server. The .Net calls to SMTP just write the emails out on the file system. It’s just configuration even. So much to know, so little time.

Posted: Wednesday, July 29, 2009 7:12:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 2.0 | .Net 3.5 | ASP.NET

Casting a Null

Found an interesting tidbit today. There is an ASP.NET page that is using session variables. The user has visited the page, and populated the session. But then the user lingers, and the session times out on a page that is not protected. The user then clicks a button, and the following line of C# code executes:

User foo = (User)Session[“bar”];

So what should happen here? An Exception? That’s what I would have thought. But not true. Since the session has expired, the session variable is now null, and you can cast a null into another data type having a value of null. Just as long as that type is nullable. So the line above is the same as:

User foo = (User)null;

which is perfectly valid.

I got the null reference exception down the line when trying to access a property of foo. So once again, defensive programming is the rule. Check your session variables for null!

Posted: Thursday, July 16, 2009 9:14:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 2.0 | .Net 3.5 | Strange Problems

LINQ: Selecting in Nested Collections

This took me longer than it should have to figure it out, so I am documenting it for later reuse. The case here is described as such:

I have a collection of person objects, which themselves contain a collection of address objects. I want to find all the addresses in a given city:

public class Address

{

     public string Street { get; set; }

     public string City { get; set; }

     public string Region { get; set; }

     public string Postal { get; set; }

     public AddressType Location { get; set; }

}

 

public class Person

{

     public string Name { get; set; }

     public string ID { get; set; }

     public string Title { get; set; }

     public List<Address> Addresses { get; set; }

}

....

IEnumerable<Address> results = from guy in allPersons

                               from addressResult in guy.Addresses

                               where addressResult City == "London"

                               select addressResult;

I understand this is similar to a join in a SQL statement, but in my case the data does not come from a database but a service where I can't control data selection in that way.

Posted: Tuesday, June 23, 2009 3:07:00 AM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 3.5 | Linq

Anti-Simplicity

Josh Holmes recently posted his presentation “The Lost Art of Simplicity”. As a consultant, I am frequently striving for simplicity for one main reason: I leave behind the code I create. Most of the time I leave the code behind to be maintained by some other developer that works for the client. So I am right there with what Josh is advocating. I wouldn’t want to be the guy trying to figure out some overly-complicated code 6 months down the line (I have been that guy before).

<RANT>

So here on my current project is anti-simplicity in action. The app is an ASP.NET app. There is a “toolbar” of sorts specific for the page. One of the toolbar “buttons” is simply a link to find more products. The “toolbar” is hosted in a user control. The developer coded this link as an ASP Linkbutton. The linkbutton has a server-side event handler to capture the click, which it then propagates outside the user control so the container control can get the click event. The hosting control then captures the event and does a redirect to a hard-coded URL. There is zero conditional logic in all the code. What happened to an HTML <A> tag? There is absolutely no reason for any server processing on this action at all! I can only fathom the reasons the developer did this:

  • There are other buttons that act that way, so all buttons should act that way
  • He’s proud of complex code and solutions
  • Because he can
  • He thinks it’s more secure, hiding the url?

In the end, the developer has forgotten why he is on the project. We are creating software for the client. This includes leaving them with something that has a lower maintenance cost by achieving simplicity.

On a related note, the UI/designers on this project are 100% anti-tables. I understand using CSS for layouts and not tables, I’m all on board with that. But using UL lists stacked next to one another to display tabular data is just wrong. Tables are part of the HTML spec for a reason, mainly tabular data. Because developers/designers perverted the use of tables in the past does not preclude their ultimate use in a solution. Zealotry is another anti-simplicity pattern in my world.

</RANT>

Posted: Wednesday, May 06, 2009 2:13:00 AM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET