SteveX Compiled

Google's introduced a thing called Sitemaps, where you can describe your site using a simple XML syntax, and they'll pull this and use it as hints to their crawler to influence how and when it crawls your site.  They probably could have used RSS for this, but I guess that's a point I'll let Dave Winer make (I haven't checked to see if he has, yet, but I'm sure he will).

The Ottawa Events site is somewhat unique in that events are posted, they're valid until the date that the event occurs, and then there's really no point for a search engine to hang onto them anymore.  I don't expire events from the database (yet), so the URLs still work, and Google will still have it indexed, but really, if you're looking for "glebe garage sale" you probably care about the next one coming, not one from 2 years ago.

I cooked up a sitemap.aspx page that generates a sitemap on the fly for the main pages of the site, and the current events.  If Google indexes only these, and not other URLs it might find, then the results will be more useful than if they indexed everything they could find.

Creating the sitemap was simple:  I did the same DB query that I use to generate the front page, and I dump all that into a canned XML document that describes the static pages on the site.  I don't ping Google when a new event is submitted, yet, but I'll add that shortly.

The code is simple enough:

  _doc = new XmlDocument();
  _doc.Load("");
  _urlset = _doc.SelectSingleNode("/urlset");

Then do the DB query to find all the rows I want to add entries for.  Once I've got that:

  foreach (DataRow row in ds.Tables[0].Rows)
  {
    addUrl("EventDetails.aspx?id=" + row["ID"].ToString(), "daily");
  }

The addUrl function just creates the node:

  void addUrl(string url, string changeFreq)
  {
    XmlNode urlNode = _doc.CreateElement("url");
    appendValue(urlNode, "loc", url);
    appendValue(urlNode, "changefreq", changeFreq);
    _urlset.AppendChild(urlNode);
  }

Then at the end of Page_Load, I send the contents of the XML doc to the output stream and end the request:

  _doc.Save(Response.OutputStream);
  Response.End();

Google hasn't pulled it yet, so I can't say if it works yet... but it should.. :)

 

This entry was posted on Thursday, June 2nd, 2005 at 5:43 am.