Henderson_book.

fm Page 959 Thursday, September 25, 2003 5:23 AM

P A R T

I V

Undocumented SQL Server

Henderson_book.fm Page 960 Thursday, September 25, 2003 5:23 AM

Henderson_book.fm Page 961 Thursday, September 25, 2003 5:23 AM

C H A P T E R

2 4

Finding Undocumented Features

To a great designer, not applying knowledge is tantamount to not having obtained the knowledge in the rst place. Steve McConnell1

I dedicated a chapter in each of my last two SQL Server books to exposing undocumented features in the product. In each book, I provided lists of undocumented stored procedures, extended procedures, functions, trace ags, command syntax, and product features. In keeping with the spirit of this bookwhich is that how something works is at least as important as putting it to practical useIm not going to do that in this book. Instead, Im going to show you how to nd these undocumented features for yourself. Its my hope that by pointing out these features, particularly those in Transact-SQLthe facility users go through to reach the serverI will encourage these features either to be documented or to be made completely inaccessible to users. Its my belief that SQL Server has an inordinate number of these hidden goodies. It appears to have far more of these types of things than similar products. Perhaps thats because it has so many more features than other similar products; perhaps thats because it relies too much on undocumented functionality. Many of these undocumented features are robust enough for important parts of the product such as replication or the Index Tuning Wizard to depend on them, so the argument that they arent suitable for use by end users just doesnt ring true. Some are used to implement fringe features of the product and very well might not be suitable for more general use, so I can understand why theyre undocumented. Some trace ags can be downright

1. McConnell, Steve. After the Gold Rush. Redmond, WA: Microsoft Press, 1999, p. 27.

961

Henderson_book.fm Page 962 Thursday, September 25, 2003 5:23 AM

962

Chapter 24

Finding Undocumented Features

dangerous if used improperly, for example, so Ive always been wary of listing many of them. Generally, I list only those undocumented features I consider to be safe to use or so generally applicable that I feel compelled to mention them. I think some undocumented features are undocumented simply because they fell through the cracks. There are so many features in SQL Server in general and in Transact-SQL in particular that I wonder if perhaps some of them were inadvertently left out of the product documentation. Take the TriggerInsertOrder, TriggerDeleteOrder, and TriggerUpdateOrder OBJECTPROPERTY property names, for example. I dont know why they arent documented. For that matter, I dont know why they exist. Replication triggers use them to ensure that a trigger is the rst one executed for a particular type of DML operation. The documented ExecIsFirstInsertTrigger, ExecIsFirstUpdateTrigger, and ExecIsFirstDeleteTrigger property names could be used to return the same informationtheres really no need for these undocumented property names in the rst place. And, even if there were, what would be the harm in documenting them so long as users understood that, other than the rst or last trigger for a particular DML operation, trigger order execution is undocumented and cant be depended on in any fashion? Regardless of the reason a feature may be undocumented, knowing about it and knowing what it does tells us more about the product. I think thats an especially worthwhile endeavor in a book whose whole purpose is to explore how the product works. Knowing about a products undocumented features and why they exist gives you better insight into how it is designed. You get a better feel for where its limitations are and how its designers intended it to be used. These are all good things, irrespective of whether you ever actually make use of any undocumented features. So, all that said, let me repeat the disclaimer from my previous books: Use undocumented features at your own risk. By leaving product features undocumented, a vendor reserves the right to change them at any time. A hot x release, security patch, service pack, or new product version could change or eliminate undocumented functionality that you or your code has come to depend on. Moreover, an undocumented feature you make use of may not have been tested as thoroughly as the rest of the product (its unusual for test teams to develop test suites for undocumented features) and may not work reliably in all situations. Many times, using an undocumented feature isnt even the best tool for the job and may even be completely superuous (e.g., TriggerInsertOrder). The euphoria that comes from discovering an undocumented xproc and putting it to immediate use in production code quickly subsides when it crashes your server with an access violation. And from a product support perspective, assume undocumented means un-

Henderson_book.fm Page 963 Thursday, September 25, 2003 5:23 AM

The syscomments Gold Mine

963

supportedyou cant expect a vendor to support functionality it has intentionally left out of a products documented feature set. The bottom line is: Its best not to use undocumented features unless absolutely necessary. Now that Ive got that out of my system, lets proceed with the discussion of how to nd undocumented SQL Server features. Finding these is surprisingly easy; you will likely be astonished at just how much hidden functionality is right under your nose.

NOTE: I have it on good authority that Microsoft intends to remove much of the access to undocumented features in the next release of SQL Server. Thats probably a good move; however, for now, those features are still there and readily accessible, so well explore how to get at them in order to better understand how the product works.

The syscomments Gold Mine

A bountiful source of undocumented features, commands, functions, and syntax can be found in the syscomments system table. The syscomments table is where the source code to every procedural objectevery stored procedure, trigger, view, user-dened function, default, and rule objectin a database is stored. Each database has its own copy of this table. Im especially fond of the syscomments tables in master, msdb, and the replication distribution databases. Ive also found undocumented features hiding in replication-generated stored procedures, views, and triggers in publication or subscription databases.

Undocumented DBCC Commands

Ive found a gold mine of undocumented commands by snooping around in master..syscomments. For example, to nd all the DBCC commands (on a case-insensitive server) called by procedural objects in master, we might run this T-SQL command:
SELECT OBJECT_NAME(id), SUBSTRING(text,PATINDEX('%dbcc%',text),50) as text FROM master..syscomments WHERE PATINDEX('%dbcc%',text)<>0

Henderson_book.fm Page 964 Thursday, September 25, 2003 5:23 AM

964

Chapter 24

Finding Undocumented Features

Undocumented Trace Flags

To nd just the DBCC TRACEON calls (and thereby nd references to undocumented trace ags), we might run this query:
SELECT OBJECT_NAME(id), SUBSTRING(text,PATINDEX('%traceon%',text),50) as text FROM master..syscomments WHERE PATINDEX('%traceon%',text)<>0

Other Undocumented Goodies in syscomments

Some of the procedural objects in master, msdb, and distribution helpfully divulge whether they make use of undocumented features by including the word undocumented or the words DO NOT DOCUMENT somewhere in their text. Ive found these to be especially useful. You can nd them by using a query like this:
SELECT OBJECT_NAME(id), SUBSTRING(text,PATINDEX('%document%',text),50) as text FROM master..syscomments WHERE PATINDEX('%document%',text)<>0

Goodies in sysobjects
The sysobjects system table exists in every database and contains a row for every object in the database. If you have some idea of the name or type of an undocumented object, you can check for its existence in Enterprise Managers navigation tree and in Query Analyzers Object Browser. You can also query sysobjects directly to get this information, which is where Enterprise Manager and Query Analyzer ultimately get it. The code below queries sysobjects directly for simplicitys sake.

Undocumented Extended Procedures

Extended procedures must reside in the master database and are usually prexed with xp_. Heres a sysobjects query that returns the name of the extended procedures registered in the master database:

Henderson_book.fm Page 965 Thursday, September 25, 2003 5:23 AM

Goodies in sysobjects

965

SELECT LEFT(name, 30) FROM master..sysobjects WHERE TYPE='X'

Note that some of these procedures have prexes of sp_ rather than the more traditional xp_. Also, some are implemented internally by the server rather than in external DLLs. Extended procedures implemented internally by SQL Server are known as special procedures (spec procs for short). You can list the spec procs registered in master via a query like this:
SELECT OBJECT_NAME(c.id) FROM master..syscomments c JOIN master..sysobjects o ON (c.id=o.id) WHERE o.type='X' AND c.text NOT LIKE '%.dll%' OR c.text IS NULL

The name of the DLL containing a regular xproc will be listed in the xprocs text column in syscomments. Since a spec proc doesnt reside in a DLL, its entry in syscomments will contain something besides a DLL name. The query above lists the xprocs entries in syscomments where this is the case. Knowing about the existence of an extended procedure is one thing; knowing how to use it is another. Many xprocs are called by regular procedural objects such as stored procedures and user-dened functions. Heres a T-SQL query that will list all the extended procedures whose names begin with xp_ that are called by objects in master:
SELECT OBJECT_NAME(id), SUBSTRING(text,PATINDEX('%xp_%',text),50) as text FROM master..syscomments WHERE text LIKE '%xp\_%' ESCAPE '\'

(Ive used an ESCAPE character here to help eliminate false hits due to the fact that _ is a wildcard character.)

Undocumented Functions
Another favorite hiding place for undocumented functionality is in undocumented user-dened functions. Some of these are user objects; some are system functions. System functions begin with the prex fn_ and are owned by the system_function_schema pseudo-user. You can nd out which of these

Henderson_book.fm Page 966 Thursday, September 25, 2003 5:23 AM

966

Chapter 24

Finding Undocumented Features

exist in a particular database via a simple query against the appropriate sysobjects table. Its rare for other objects besides functions to have names that begin with fn_, so its usually safe just to search on the name prex alone. Heres a query that will return all the objects that begin with fn_ from master..sysobjects. Since its possible to create an undened function not owned by system_function_schema, I havent limited the search to a particular owner.
SELECT LEFT(name, 30) FROM master..sysobjects WHERE LEFT(name, 3) = 'fn_'

As with extended procedures, knowing about the existence of a function doesnt imply knowing how to use it. Heres a query that shows all the procedural objects in master that reference objects whose names begin with fn_:
SELECT OBJECT_NAME(id), SUBSTRING(text,PATINDEX('%fn_%',text),50) as text FROM master..syscomments WHERE text LIKE '%fn\_%' ESCAPE '\'

Of course, this query (and all the others in this chapter that work similarly) can yield false hits due to extraneous string matches in the text in syscomments. Obviously, youll have to examine the results you get from these queries and eliminate the false matches they return.

Scripting Undocumented and System Objects

Enterprise Manager disables the Generate SQL Script menu item for objects marked as system objects (those whose system bit has been set via a call to sp_MS_marksystemobject). You can still double-click a system object in Enterprise Manager to display its source, but you cant edit it. The obvious intent here is to prevent people from accidentally modifying important internal objects. The Properties dialog for stored procedures and similar objects is a pain to use, and I rarely subject myself to trying to view or edit the source to procedural objects with it. Its a modal dialog (and, therefore, doesnt have a maximize button), and its also missing a resize grip in its lower right corner, something every self-respecting resizable form should have. So rather than deal with the inadequacies of Enterprise Manager, I usually prefer to script stored procedures to a le, then use another tool to view or edit them. Unfortunately, Enterprise Managers attempt to protect me from myself hampers

Henderson_book.fm Page 967 Thursday, September 25, 2003 5:23 AM

The Proler Treasure Trove

967

this a bit when dealing with system procedures. If I cant easily generate a script and I dont want to use Enterprise Managers limited Stored Procedure Properties dialog, how can I quickly view or edit the text of a system stored procedure? Thankfully, Query Analyzer doesnt have the same concern for my well-being that Enterprise Manager does. Its Object Browser allows me to generate a script for any unencrypted object on my server, including system objects. So, when I need to view or edit procedural objectsespecially system objectsusing a capable editor, I use Query Analyzer. Of course, another way to list the source for a system procedural object is to use the sp_helptext procedure. You can use sp_helptext to return the source for any unencrypted procedural object in a database.

The Proler Treasure Trove

An excellent way to learn about undocumented feature use in SQL Server is to use Proler to record the execution of stored procedures and batches on the server, especially by tools included with SQL Server. A couple of my favorites are Enterprise Manager and the Index Tuning Wizard. The replication agents also occasionally reveal some hidden goodies. Examining a Proler trace captured while these tools run can yield interesting details about the way the server works. For example, while watching a Proler trace during an Index Tuning Wizard run, I discovered that the wizard creates what are known as hypothetical indexesstatistics-only indexes used to determine whether a different indexing strategy would improve the performance of a given query or workload. (You can check to see whether an index is a hypothetical index via the IsHypothetical INDEXPROPERTY property name.) The Index Tuning Wizard builds these by calling CREATE INDEX using an undocumented WITH option: statistics_only. This tells us some things about how the server works. For example, if I didnt already know it, this would have told me that the optimizer doesnt look at data when it decides whether to use an indexit looks only at the statistics for an index when optimizing a query plan. It tells us how the wizard can evaluate so many different kinds of indexes in such a (relatively) short period of time: It creates test indexes without any data. So we learn something about the Index Tuning Wizard as well as SQL Server from the discovery of this undocumented feature. Proler also comes in handy in guring out how undocumented xprocs and functions are used. SQL-DMO and Enterprise Manager use undocumented xprocs and functions extensively; capturing a Proler trace while you

Henderson_book.fm Page 968 Thursday, September 25, 2003 5:23 AM

968

Chapter 24

Finding Undocumented Features

run Enterprise Manager through its paces can turn up all sorts of hidden features. One peculiar thing Ive discovered about Proler traces, though, is that the tool is apparently hard-coded to hide lines that contain the text -- sp_ password. Given that this text would be displayed for SP:StmtStarting/StmtCompleted events for the sp_password system procedure, the intent seems to be to keep Proler traces from recording password changes. That said, the rudimentary way in which this has been implemented would allow a hacker to cover her or his tracks by appending -- sp_password to any line in a T-SQL batch or stored procedure she or he did not want to show up in a trace.

Snooping around in the Installation Scripts

Another good source of undocumented information is the set of installation scripts that ship with SQL Server. You can nd these scripts in the Install folder under your root SQL Server installation. You can conduct the same sorts of searches against these scripts that we used with syscomments to nd undocumented trace ags, DBCC commands, stored procedures, user-dened functions, and the like. For example, I discovered how to create my own INFORMATION_SCHEMA views (and documented the process in my last book) by examining ansiview.sql. I gured out how to create my own system functions by examining instdist.sql. Theres a wealth of undocumented information to be had by examining the scripts that ship with the product, especially with respect to creating objects. We know that the server is somehow creating many of these via Transact-SQL. If we want to use the same functionality or just want to better understand how the server works, the installation scripts are good places to start.

DLL Imports
A nal place to check for undocumented features is the SQL Server executables import table. Every Windows executable has an import table. For each of the DLLs that the executable requires to load, this table lists the functions that it has explicitly referenced. By examining this table, you can learn about undocumented functions the executable may be calling. Once

Henderson_book.fm Page 969 Thursday, September 25, 2003 5:23 AM

Knowledge Measure

969

you have a function name, you can use a debugger to set a breakpoint and see when its called. A good example of this is the OPENDS60.DLL that ships with SQL Server. This DLL implements the Open Data Services API that the server uses internally and that is used to construct extended procedures. If you look at the import table for sqlservr.exe, the SQL Server executable, youll see that it imports a large number of functions from OPENDS60.DLL, many of them undocumented. If you were to attach a debugger to sqlservr.exe and set breakpoints for these functions, you could likely determine in what circumstances they were used. This might help narrow down exactly what they do and how and why the server calls them.

Recap
Dont use undocumented features unless theres absolutely no other way. Your best strategy here is not to use them at all unless instructed to by Microsoft. Use your investigation into the undocumented features in SQL Server as a means of getting to know the product better, not a treasure hunt for nifty routines you can drop untested onto production systems.

Knowledge Measure
1. Cite three reasons you shouldnt use undocumented SQL Server features.

Henderson_book.fm Page 970 Thursday, September 25, 2003 5:23 AM