Useful LogParser Queries

If you deal with production servers and you don’t use LogParser, you should. It gives SQL-like abilities to query web server logs an other log types (like Event Logs). Coding Horror has a great article about the tool as well, and a link to a good article about forensic log parsing. Here are my most used web queries:

Find Pages with 500 errors
logparser "SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status >= 500) GROUP BY cs-uri-stem, code ORDER BY Hits DESC" -o:CSV >> C:\Data\ErrorPages.csv

Find 404 Requests
logparser "SELECT cs-uri-stem as Url, sc-status as code, COUNT(cs-uri-stem) AS Hits FROM C:\ProdLogFiles\ex*.log WHERE (sc-status = 404) GROUP BY cs-uri-stem, code ORDER BY Hits DESC" -o:CSV >> C:\Data\404Pages.csv

Find the Slowest Pages
logparser "SELECT TOP 100 cs-uri-stem AS Url, MIN(time-taken) as [Min], AVG(time-taken) AS [Avg], max(time-taken) AS [Max], count(time-taken) AS Hits FROM C:\ProdLogFiles\ex*.log GROUP BY Url ORDER BY [Avg] DESC" -o:CSV >> C:\Data\SlowPages.csv

Posted: Thursday, August 06, 2009 7:41:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET | IIS | Performance | Tools

Another “Why didn’t someone tell me this earlier” Post

90% of the projects I work on need to send emails, and most clients don’t allow developer SMTP servers floating around their network (for good reason). So a big headache ensues trying to get access to an Exchange or SMTP server somewhere. I have been doing this same dance for years. But today I find a blog post about sending emails without an SMTP server. The .Net calls to SMTP just write the emails out on the file system. It’s just configuration even. So much to know, so little time.

Posted: Wednesday, July 29, 2009 7:12:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 2.0 | .Net 3.5 | ASP.NET

Casting a Null

Found an interesting tidbit today. There is an ASP.NET page that is using session variables. The user has visited the page, and populated the session. But then the user lingers, and the session times out on a page that is not protected. The user then clicks a button, and the following line of C# code executes:

User foo = (User)Session[“bar”];

So what should happen here? An Exception? That’s what I would have thought. But not true. Since the session has expired, the session variable is now null, and you can cast a null into another data type having a value of null. Just as long as that type is nullable. So the line above is the same as:

User foo = (User)null;

which is perfectly valid.

I got the null reference exception down the line when trying to access a property of foo. So once again, defensive programming is the rule. Check your session variables for null!

Posted: Thursday, July 16, 2009 9:14:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 2.0 | .Net 3.5 | Strange Problems

LINQ: Selecting in Nested Collections

This took me longer than it should have to figure it out, so I am documenting it for later reuse. The case here is described as such:

I have a collection of person objects, which themselves contain a collection of address objects. I want to find all the addresses in a given city:

public class Address

{

     public string Street { get; set; }

     public string City { get; set; }

     public string Region { get; set; }

     public string Postal { get; set; }

     public AddressType Location { get; set; }

}

 

public class Person

{

     public string Name { get; set; }

     public string ID { get; set; }

     public string Title { get; set; }

     public List<Address> Addresses { get; set; }

}

....

IEnumerable<Address> results = from guy in allPersons

                               from addressResult in guy.Addresses

                               where addressResult City == "London"

                               select addressResult;

I understand this is similar to a join in a SQL statement, but in my case the data does not come from a database but a service where I can't control data selection in that way.

Posted: Tuesday, June 23, 2009 3:07:00 AM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 3.5 | Linq

Anti-Simplicity

Josh Holmes recently posted his presentation “The Lost Art of Simplicity”. As a consultant, I am frequently striving for simplicity for one main reason: I leave behind the code I create. Most of the time I leave the code behind to be maintained by some other developer that works for the client. So I am right there with what Josh is advocating. I wouldn’t want to be the guy trying to figure out some overly-complicated code 6 months down the line (I have been that guy before).

<RANT>

So here on my current project is anti-simplicity in action. The app is an ASP.NET app. There is a “toolbar” of sorts specific for the page. One of the toolbar “buttons” is simply a link to find more products. The “toolbar” is hosted in a user control. The developer coded this link as an ASP Linkbutton. The linkbutton has a server-side event handler to capture the click, which it then propagates outside the user control so the container control can get the click event. The hosting control then captures the event and does a redirect to a hard-coded URL. There is zero conditional logic in all the code. What happened to an HTML <A> tag? There is absolutely no reason for any server processing on this action at all! I can only fathom the reasons the developer did this:

  • There are other buttons that act that way, so all buttons should act that way
  • He’s proud of complex code and solutions
  • Because he can
  • He thinks it’s more secure, hiding the url?

In the end, the developer has forgotten why he is on the project. We are creating software for the client. This includes leaving them with something that has a lower maintenance cost by achieving simplicity.

On a related note, the UI/designers on this project are 100% anti-tables. I understand using CSS for layouts and not tables, I’m all on board with that. But using UL lists stacked next to one another to display tabular data is just wrong. Tables are part of the HTML spec for a reason, mainly tabular data. Because developers/designers perverted the use of tables in the past does not preclude their ultimate use in a solution. Zealotry is another anti-simplicity pattern in my world.

</RANT>

Posted: Wednesday, May 06, 2009 2:13:00 AM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
ASP.NET

Invalid postback or callback argument - AJAX Control Toolkit

I was working to add some Ajax interactivity to a page that needed some asynchronous workings. Unfortunately, after displaying a ModalPopupExtender and the user clicking the Close button on the “popup”, the page would throw the following exception:

System.ArgumentException occurred
  Message="Invalid postback or callback argument.  Event validation is enabled using <pages enableEventValidation=\"true\"/> in configuration or <%@ Page EnableEventValidation=\"true\" %> in a page.  For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them.  If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation."
  Source="System.Web"
  StackTrace:
       at System.Web.UI.ClientScriptManager.ValidateEvent(String uniqueId, String argument)
  InnerException:

There are lots of explanations out there for this kind of problem, but none of the common answers really fit. The most common answer was to disable event validation. That wasn’t a good answer for me, I don’t think disabling an important security feature because you have complex code is a good solution. The other common answer was to override the render event to register the control posting back with the ScriptManager. This did not work at all for me.

After finding an reading through this post and the comments, I realized the problem was indeed in the code. After the call to ModalPopupExtender.Show(), the original code was re-binding a grid on the page. This order of events was causing the problem. I changed the binding to occur before the call to the ModalPopupExtender by changing which event containing the rebinding call, and the error went away.

Posted: Monday, March 23, 2009 8:46:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 3.0 | .Net 3.5 | ASP.NET | JavaScript/AJAX

Linq Foray

I’m working on a website with lots of user controls interacting with one another using ASP.NET Ajax using events. So I end up needing to find other controls often, especially within repeaters. Nested controls all over the place. I found this wonderful post about using an extension method to the Controls Collection that flattens the tree and allows for much simpler Linq queries. The extension method adds and All() function which is seriously useful.

Posted: Sunday, March 22, 2009 9:27:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 3.5 | ASP.NET | JavaScript/AJAX | Linq

Don’t Do This

In ASP.NET, the DataList control renders as a table. Opening a new table in the header and trying to get rows to render just won’t end well. Lots of strange browser behavior should be the tip off that this isn't right.

Posted: Sunday, March 22, 2009 7:48:00 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
.Net 2.0 | .Net 3.5 | ASP.NET

Moving Commerce Server 2007 Databases

I needed to move the set of Commerce Server databases from one server to another, so I developed and tested this procedure.

  1. Script the database users to file. Our site is using domain users, so I performed this step first to keep the users in sync with a backup/restore of the databases.
  2. Backup each of the Commerce Server databases:
    • Marketing
    • Marketing_Lists
    • ProductCatalog
    • Profiles
    • TransactionConfig
    • Transactions
    • MSCS_Admin
    • MSCS_CatalogScratch
  3. Transfer the scripts and backup files to the new server.
  4. Run user scripts against the new server
  5. Restore each database on the new server
  6. On the web server, open Commerce Server Manager to change connection strings:
    1. Admin Database:
      1. Click the Commerce Server Manager node
      2. Click the Administration Database link in the right panel
      3. Change the Connection String, click OK
    2. Profiles
      1. Expand this node:
        Global Resources/Profiles
      2. Right-click and change the following 3 connection strings:
        s_BizDataStoreConnectionString
        s_CommerceProviderConnectionString
        s_ProfileServiceConnectionString
    3. Profile Database
      1. Expand this node:
        Global Resources/Profiles/ProfileCatalog/DataSources/ProfileService_SQLSource/Partitions
      2. Right-click SQLSource in the right pane and choose properties
      3. Change the Connection String, click OK
    4. Inventory Database
      1. Expand this node:
        Commerce Sites/CSharpSite/Site Resources/Inventory
      2. Right-click Inventory Database in the right pane and choose properties
      3. Change the Connection String, click OK
    5. Marketing Database
      1. Expand this node:
        Commerce Sites/CSharpSite/Site Resources/Marketing
      2. Right-click Mailing List Connection String in the right pane and choose properties
      3. Change the Connection String, click OK
      4. iv. Right-click Marketing Database Connection String in the right pane and choose properties
      5. Change the Connection String, click OK
    6. Product Catalog
      1. Expand this node:
        Commerce Sites/CSharpSite/Site Resources/Product Catalog
      2. Right-click Catalog Database in the right pane and choose properties
      3. Change the Connection String, click OK
    7. Transaction Config
      1. Expand this node:
        Commerce Sites/CSharpSite/Site Resources/Transaction Config
      2. Right-click Transactions Config Database in the right pane and choose properties
      3. Change the Connection String, click OK
    8. Transactions
      1. Expand this node:
        Commerce Sites/CSharpSite/Site Resources/Transaction Config
      2. Right-click Transactions Database in the right pane and choose properties
      3. Change the Connection String, click OK
  7. Change connection strings in the web.config for site (if any using API calls)
  8. Restart IIS
Posted: Sunday, February 08, 2009 8:28:53 PM (Eastern Standard Time, UTC-05:00)  #    Comments - Trackback
Commerce Server | SQL Server