Professional Documents
Culture Documents
3, MARCH 2013
Abstract—An increasing number of databases have become web accessible through HTML form-based search interfaces. The data
units returned from the underlying database are usually encoded into the result pages dynamically for human browsing. For the
encoded data units to be machine processable, which is essential for many applications such as deep web data collection and Internet
comparison shopping, they need to be extracted out and assigned meaningful labels. In this paper, we present an automatic
annotation approach that first aligns the data units on a result page into different groups such that the data in the same group have the
same semantic. Then, for each group we annotate it from different aspects and aggregate the different annotations to predict a final
annotation label for it. An annotation wrapper for the search site is automatically constructed and can be used to annotate new result
pages from the same web database. Our experiments indicate that the proposed approach is highly effective.
1 INTRODUCTION
domain. The domain ontology is then used to assign labels to (i.e., schema value annotator (SA) and frequency-based
each data unit on result page. After labeling, the data values annotator (FA)) are new (i.e., not used is DeLa), three (table
with the same label are naturally aligned. This method annotator (TA), query-based annotator (QA) and common
is sensitive to the quality and completeness of the ontologies knowledge annotator (CA)) have better implementations
generated. DeLa [30] first uses HTML tags to align data units than the corresponding annotation heuristics in DeLa, and
by filling them into a table through a regular expression- one (in-text prefix/suffix annotator (IA)) is the same as a
based data tree algorithm. Then, it employs four heuristics to heuristic in DeLa. For each of the three annotators that have
select a label for each aligned table column. The approach in different implementations, the specific difference and the
[36] performs attributes extraction and labeling simulta- motivation for using a different implementation will be
neously. However, the label set is predefined and contains given in Section 5.2. Furthermore, it is not clear how DeLa
combines its annotation heuristics to produce a single label
only a small number of values.
for an attribute, while we employ a probabilistic model to
We align data units and annotate the ones within the same
combine the results of different annotators. Finally, DeLa
semantic group holistically. Data alignment is an important
builds wrapper for each WDB just for data unit extraction.
step in achieving accurate annotation and it is also used in
In our approach, we construct an annotation wrapper
[25] and [30]. Most existing automatic data alignment
describing the rules not only for extraction but also for
techniques are based on one or very few features. The most
assigning labels.
frequently used feature is HTML tag paths (TP) [33]. The
To enable fully automatic annotation, the result pages
assumption is that the subtrees corresponding to two data
have to be automatically obtained and the SRRs need to be
units in different SRRs but with the same concept usually
automatically extracted. In a metasearch context, result
have the same tag structure. However, this assumption is not
pages are retrieved by queries submitted by users (some
always correct as the tag tree is very sensitive to even minor
differences, which may be caused by the need to emphasize reformatting may be needed when the queries are dis-
certain data units or erroneous coding. ViDIE [21] uses visual patched to individual WDBs). In the deep web crawling
features on result pages to perform alignment and it also context, result pages are retrieved by queries automatically
generates an alignment wrapper. But its alignment is only at generated by the Deep Web Crawler. We employ ViNTs [34]
text node level, not data unit level. The method in [7] first to extract SRRs from result pages in this work. Each SRR is
splits each SRR into text segments. The most common stored in a tree structure with a single root and each node in
number of segments is determined to be the number of the tree corresponds to an HTML tag or a piece of text in the
aligned columns (attributes). The SRR with more segments original page. With this structure, it becomes easy to locate
are then resplit using the common number. For each SRR each node in the original HTML page. The physical position
with fewer segments than the common number, each information of each node on the rendered page, including its
segment is assigned to the most similar aligned column. coordinates and area size, can also be obtained using ViNTs.
Our data alignment approach differs from the previous This paper is an extension of our previous work [22]. The
works in the following aspects. First, our approach handles following summarizes the main improvements of this paper
all types of relationships between text nodes and data units over [22]. First, a significantly more comprehensive discus-
(see Section 3.1), while existing approaches consider only sion about the relationships between text nodes and data
some of the types (i.e., one-to-one [25] or one-to-many [7], units is provided. Specifically, this paper identifies four
[30]). Second, we use a variety of features together, including relationship types and provides analysis of each type, while
the ones used in existing approaches, while existing only two of the four types (i.e., one-to-one and one-to-many)
approaches use significantly fewer features (e.g., HTML were very briefly mentioned in [22]. Second, the alignment
tag in [30] and [33], visual features in [21]). All the features algorithm is significantly improved. A new step is added to
that we use can be automatically obtained from the result handle the many-to-one relationship between text nodes and
page and do not need any domain specific ontology or data units. In addition, a clustering-shift algorithm is
knowledge. Third, we introduce a new clustering-based introduced in this paper to explicitly handle the one-to-
shifting algorithm to perform alignment. nothing relationship between text nodes and data units
Among all existing researches, DeLa [30] is the most while the previous version has a pure clustering algorithm.
similar to our work. But our approach is significantly With these two improvements, the new alignment algorithm
different from DeLa’s approach. First, DeLa’s alignment takes all four types of relationships into consideration. Third,
method is purely based on HTML tags, while ours uses the experiment section (Section 7) is significantly different
other important features such as data type, text content, and from the previous version. The data set used for experiments
adjacency information. Second, our method handles all has been expanded by one domain (from six to seven) and by
types of relationships between text nodes and data units, 22 WDBs (from 91 to 112). Moreover, the experiments on
whereas DeLa deals with only two of them (i.e., one-to-one alignment and annotation have been redone based on the
and one-to-many). Third, DeLa and our approach utilize new data set and the improved alignment algorithm. Fourth,
different search interfaces of WDBs for annotation. Ours several related papers that were published recently have
uses an IIS of multiple WDBs in the same domain, whereas been reviewed and compared in this paper.
DeLa uses only the local interface schema (LIS) of each
individual WDB. Our analysis shows that utilizing IISs has 3 FUNDAMENTALS
several benefits, including significantly alleviating the local
interface schema inadequacy problem and the inconsistent label 3.1 Data Unit and Text Node
problem (see Section 5.1 for more details). Fourth, we Each SRR extracted by ViNTs has a tag structure that
significantly enhanced DeLa’s annotation method. Specifi- determines how the contents of the SRRs are displayed on a
cally, among the six basic annotators in our method, two web browser. Each node in such a tag structure is either a tag
LU ET AL.: ANNOTATING SEARCH RESULTS FROM WEB DATABASES 517
a composite text node is the concatenation of the data types of The weights in the above formula are obtained using a
all its data units. For example, the data type of the text node genetic algorithm based method [10] and the trained weights
“Premier Press/2002/1931841616/0.06667” in Fig. 1 is are given in Section 7.2. The similarity for each individual
<First-Letter-Capitalized-String> <Symbol> <Integer> feature is defined as follows:
<Symbol> <Integer> <Symbol> <Decimal>. Consecutive
terms with the same data type are treated as a single term . Data content similarity (SimC). It is the Cosine
and only one of them will be kept. Each type except Ordinary similarity [27] between the term frequency vectors of
String has certain pattern(s) so that it can be easily identified. d1 and d2 :
The data units of the same concept or text nodes involving the Vd1 Vd2
same set of concepts usually have the same data type. SimCðd1 ; d2 Þ ¼ ; ð2Þ
kVd1 k kVd2 k
3.2.4 Tag Path (TP) where Vd is the frequency vector of the terms inside
A tag path of a text node is a sequence of tags traversing from data unit d, jjVd jj is the length of Vd , and the
the root of the SRR to the corresponding node in the tag tree. numerator is the inner product of two vectors.
Since we use ViNTs for SRR extraction, we adopt the same . Presentation style similarity (SimP). It is the
tag path expression as in [34]. Each node in the expression average of the style feature scores (F S) over all six
contains two parts, one is the tag name, and the other is the presentation style features (F ) between d1 and d2 :
direction indicating whether the next node is the next sibling
(denoted as “S”) or the first child (denoted as “C”). Text node X
6
SimP ðd1 ; d2 Þ ¼ F Si =6; ð3Þ
is simply represented as <#TEXT>. For example, in Fig. 1b,
i¼1
the tag path of the text node “Springer-Verlag/1999/
0387984135/0.06667” is <FORM>C<A>C<BR>S<#TEXT>S where F Si is the score of the ith style feature and it is
<FONT>C<T>C. An observation is that the tag paths of the defined by F Si ¼ 1 if Fid1 ¼ Fid2 and F Si ¼ 0 other-
text nodes with the same set of concepts have very similar wise, and Fid is the ith style feature of data unit d.
tag paths, though in many cases, not exactly the same. . Data type similarity (SimD). It is determined by the
common sequence of the component data types
3.2.5 Adjacency (AD) between two data units. The longest common
For a given data unit d in an SRR, let dp and ds denote the data sequence (LCS) cannot be longer than the number
units immediately before and after d in the SRR, respectively. of component data types in these two data units.
We refer dp and ds as the preceding and succeeding data Thus, let t1 and t2 be the sequences of the data types of
units of d, respectively. Consider two data units d1 and d2 d1 and d2 , respectively, and TLen(t) represent the
from two separate SRRs. It can be observed that if dp1 and dp2 number of component types of data type t, the data
belong to the same concept and/or ds1 and ds2 belong to the type similarity between data units d1 and d2 is
same concept, then it is more likely that d1 and d2 also belong
to the same concept. LCSðt1 ; t2 Þ
SimDðd1 ; d2 Þ ¼ : ð4Þ
We note that none of the above five features is guaranteed MaxðT lenðt1 Þ; T lenðt2 ÞÞ
to be true for any particular pair of data units (or text nodes)
with the same concept. For example, in Fig. 1, “Springer- . Tag path similarity (SimT). This is the edit distance
Verlag” and “McGraw-Hill” are both publishers but they do (EDT) between the tag paths of two data units. The
not share any content words. However, such data units edit distance here refers to the number of insertions
usually share some other features. As a result, our alignment and deletions of tags needed to transform one tag
algorithm (Section 4) can still work well even in the presence path into the other. It can be seen that the maximum
of some violation of these features. This is confirmed by our number of possible operations needed is the total
experimental results in Section 7. number of tags in the two tag paths. Let p1 and p2 be
the tag paths of d1 and d2 , respectively, and PLen(p)
4 DATA ALIGNMENT denote the number of tags in tag path p, the tag path
similarity between d1 and d2 is
4.1 Data Unit Similarity
The purpose of data alignment is to put the data units of the EDT ðp1 ; p2 Þ
SimT ðd1 ; d2 Þ ¼ 1 : ð5Þ
same concept into one group so that they can be annotated P Lenðp1 Þ þ P Lenðp2 Þ
holistically. Whether two data units belong to the same
concept is determined by how similar they are based on the Note that in our edit distance calculation, a
features described in Section 3.2. In this paper, the similarity substitution is considered as a deletion followed
between two data units (or two text nodes) d1 and d2 is a by an insertion, requiring two operations. The
weighted sum of the similarities of the five features between rationale is that two attributes of the same concept
them, i.e.: tend to be encoded in the same subtree in DOM
(relative to the root of their SRRs) even though
Simðd1 ; d2 Þ ¼ w1 SimCðd1 ; d2 Þ þ w2 SimP ðd1 ; d2 Þ some decorative tags may appear in one SRR but
þ w3 SimDðd1 ; d2 Þ þ w4 SimT ðd1 ; d2 Þ ð1Þ not in the other. For example, consider two pairs
þ w5 SimAðd1 ; d2 Þ: of tag paths (<T1><T2><T3>, <T1> <T3>) and
(<T1><T2><T3>, <T1><T4><T3>). The two tag
LU ET AL.: ANNOTATING SEARCH RESULTS FROM WEB DATABASES 519
As a result, the query-based annotator by itself cannot Example 2. In Fig. 1, during the data alignment step, a group
completely annotate the SRRs. is formed for {“$17.50,” “$18.95,” “$20.50”}. Clearly the
DeLa [30] also uses query terms to match the data unit data units in this group have different values. These
texts and use the name of the queried form element as the values share the same preceding unit “Our Price,” which
label. However, DeLa uses only local schema element names, occurs in all SRRs. Furthermore, “Our Price” does not
not element names in the IIS. have preceding data units because it is the first unit in this
line. Therefore, the frequency-based annotator will assign
5.2.3 Schema Value Annotator (SA) label “Our Price” to this group.
Many attributes on a search interface have predefined values
on the interface. For example, the attribute Publishers may 5.2.5 In-Text Prefix/Suffix Annotator (IA)
have a set of predefined values (i.e., publishers) in its In some cases, a piece of data is encoded with its label to
selection list. More attributes in the IIS tend to have form a single unit without any obvious separator between
predefined values and these attributes are likely to have the label and the value, but it contains both the label and the
more such values than those in LISs, because when attributes value. Such nodes may occur in all or multiple SRRs. After
from multiple interfaces are integrated, their values are also data alignment, all such nodes would be aligned together to
combined [14]. Our schema value annotator utilizes the form a group. For example, in Fig. 1, after alignment, one
combined value set to perform annotation. group may contain three data units, {“You Save $9.50,”
Given a group of data units Gi ¼ fd1 ; . . . ; dn g, the “You Save $11.04,” “You Save $4.45”}.
schema value annotator is to discover the best matched The in-text prefix/suffix annotator checks whether all
attribute to the group from the IIS. Let Aj be an attribute data units in the aligned group share the same prefix or
containing a list of values fv1 ; . . . ; vm g in the IIS. For each suffix. If the same prefix is confirmed and it is not a
data unit dk , this annotator first computes the Cosine delimiter, then it is removed from all the data units in the
similarities between dk and all values in Aj to find the value group and is used as the label to annotate values following
(say vt ) with the highest similarity. Then, the data fusion
it. If the same suffix is identified and if the number of data
function CombMNZ [19] is applied to the similarities for all
units having the same suffix match the number of data units
the data units. More specifically, the annotator sums up the
inside the next group, the suffix is used to annotate the data
similarities and multiplies the sum by the number of
units inside the next group. In the above example, the label
nonzero similarities. This final value is treated as the
“You save” will be assigned to the group of prices. Any
matching score between Gi and Aj .
group whose data unit texts are completely identical is not
The schema value annotator first identifies the attribute
considered by this annotator.
Aj that has the highest matching score among all attributes
and then uses gn(Aj ) to annotate the group Gi . Note that 5.2.6 Common Knowledge Annotator (CA)
multiplying the above sum by the number of nonzero
Some data units on the result page are self-explanatory
similarities is to give preference to attributes that have more
because of the common knowledge shared by human beings.
matches (i.e., having nonzero similarities) over those that
For example, “in stock” and “out of stock” occur in many
have fewer matches. This is found to be very effective in
SRRs from e-commerce sites. Human users understand that
improving the retrieval effectiveness of combination systems
it is about the availability of the product because this is
in information retrieval [4].
common knowledge. So our common knowledge annotator
5.2.4 Frequency-Based Annotator (FA) tries to exploit this situation by using some predefined
In Fig. 1, “Our Price” appears in the three records and the common concepts.
followed price values are all different in these records. In Each common concept contains a label and a set of
other words, the adjacent units have different occurrence patterns or values. For example, a country concept has a
frequencies. As argued in [1], the data units with the higher label “country” and a set of values such as “U.S.A.,”
frequency are likely to be attribute names, as part of the “Canada,” and so on. As another example, the e-mail
template program for generating records, while the data address (assume all lower cases) concept has the pattern
units with the lower frequency most probably come from ½a-z0 9: % þ þ @ð½a-z0 9 þ n:Þ þ ½a-zf2; 4g. Given a
databases as embedded values. Following this argument, group of data units from the alignment step, if all the data
“Our Price” can be recognized as the label of the value units match the pattern or value of a concept, the label of this
immediately following it. The phenomenon described in this concept is assigned to the data units of this group.
example is widely observable on result pages returned by DeLa [30] also uses some conventions to annotate data
many WDBs and our frequency-based annotator is designed units. However, it only considers certain patterns. Our
to exploit this phenomenon. Common knowledge annotator considers both patterns and
Consider a group Gi whose data units have a lower certain value sets such as the set of countries.
frequency. The frequency-based annotator intends to find It should be pointed out that our common concepts are
common preceding units shared by all the data units of the different from the ontologies that are widely used in some
group Gi . This can be easily conducted by following their works in Semantic Web (e.g., [6], [11], [12], [16], [26]). First,
preceding chains recursively until the encountered data units our common concepts are domain independent. Second,
are different. All found preceding units are concatenated to they can be obtained from existing information resources
form the label for the group Gi . with little additional human effort.
LU ET AL.: ANNOTATING SEARCH RESULTS FROM WEB DATABASES 523
6 ANNOTATION WRAPPER
Once the data units on a result page have been annotated,
we use these annotated data units to construct an annota-
tion wrapper for the WDB so that the new SRRs retrieved
from the same WDB can be annotated using this wrapper
quickly without reapplying the entire annotation process.
We now describe our method for constructing such a
5.3 Combining Annotators
wrapper below.
Our analysis indicates that no single annotator is capable of Each annotated group of data units corresponds to an
fully labeling all the data units on different result pages. The attribute in the SRRs. The annotation wrapper is a descrip-
applicability of an annotator is the percentage of the attributes
tion of the annotation rules for all the attributes on the result
to which the annotator can be applied. For example, if out of
page. After the data unit groups are annotated, they are
10 attributes, four appear in tables, then the applicability of
the table annotator is 40 percent. Table 1 shows the average organized based on the order of its data units in the original
applicability of each basic annotator across all testing SRRs. Consider the ith group Gi . Every SRR has a tag-node
domains in our data set. This indicates that the results of sequence like Fig. 1b that consists of only HTML tag names
different basic annotators should be combined in order to and texts. For each data unit in Gi , we scan the sequence both
annotate a higher percentage of data units. Moreover, backward and forward to obtain the prefix and suffix of the
different annotators may produce different labels for a given data unit. The scan stops when an encountered unit is a valid
group of data units. Therefore, we need a method to select the data unit with a meaningful label assigned. Then, we
most suitable one for the group. compare the prefixes of all the data units in Gi to obtain
Our annotators are fairly independent from each other the common prefix shared by these data units. Similarly, the
since each exploits an independent feature. Based on this
common suffix is obtained by comparing all the suffixes of
characteristic, we employ a simple probabilistic method to
these data units. For example, the data unit for book title in
combine different annotators. For a given annotator L, let
P ðLÞ be the probability that L is correct in identifying a Fig. 1b has “<FORM><A>” as its prefix and “</A><BR>” as
correct label for a group of data units when L is applicable. its suffix. If a data unit is generated by splitting from a
P ðLÞ is essentially the success rate of L. Specifically, suppose composite text node, then its prefix and suffix are the same
L is applicable to N cases and among these cases M are as those of its parent data unit. This wrapper is similar to the
annotated correctly, then P ðLÞ ¼ M=N. If k independent LR wrapper in [18]. Here, we use prefix as the left delimiter,
annotators Li , i ¼ 1; . . . ; k, identify the same label for a and suffix as the right delimiter to identify data units.
group of data units, then the combined probability that at However, the LR wrapper has difficulties to extract data
least one of the annotators is correct is units packed inside composite text nodes due to the fact that
Y
k there is no HTML tag within a text node. To overcome this
1 ð1 P ðLi ÞÞ: ð7Þ limitation, besides the prefix and suffix, we also record the
i¼1 separators used for splitting the composite text node as well
To obtain the success rate of an annotator, we use the as its position index in the split unit vector. Thus, the
annotator to annotate every result page in a training data set annotation rule for each attribute consists of five compo-
(DS1 in Section 7.1). The training result is listed in Table 1. It nents, expressed as: attributei ¼ <labeli ; prefixi ; suffixi ;
can be seen that the table annotator is 100 percent correct separatorsi ; unitindexi >. The annotation wrapper for the
when applicable. The query-based annotator also has very site is simply a collection of the annotation rules for all the
high success rate while the schema value annotator is the attributes identified on the result page with order corre-
least accurate. sponding to the ordered data unit groups.
An important issue DeLa did not address is what if To use the wrapper to annotate a new result page, for each
multiple heuristics can be applied to a data unit. In our
data unit in an SRR, the annotation rules are applied on it one
solution, if multiple labels are predicted for a group of data
by one based on the order they appear in the wrapper. If this
units by different annotators, we compute the combined
probability for each label based on the annotators that data unit has the same prefix and suffix as specified in the
identified the label, and select the label with the largest rule, the rule is matched and the unit is labeled with the given
combined probability. label in the rule. If the separators are specified, they are used
One advantage of this model is its high flexibility in the to split the unit, and labeli is assigned to the unit at the
sense that when an existing annotator is modified or a new position unitindexi .
524 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 25, NO. 3, MARCH 2013
TABLE 2 TABLE 3
Performance of Data Alignment Performance of Annotation
TABLE 4
Performance of Annotation with Wrapper
TABLE 5
method to combine the basic annotators. Each of these
Performance Using Local Interface Schema annotators exploits one type of features for annotation and
our experimental results show that each of the annotators is
useful and they together are capable of generating high-
quality annotation. A special feature of our method is that,
when annotating the results retrieved from a web database, it
utilizes both the LIS of the web database and the IIS of
multiple web databases in the same domain. We also
explained how the use of the IIS can help alleviate the local
interface schema inadequacy problem and the inconsistent
label problem.
In this paper, we also studied the automatic data
alignment problem. Accurate alignment is critical to achiev-
ing holistic and accurate annotation. Our method is a
clustering based shifting method utilizing richer yet auto-
matically obtainable features. This method is capable of
handling a variety of relationships between HTML text
are less significant comparatively, but without either of them,
nodes and data units, including one-to-one, one-to-many,
the precision and recall drop more than 15 percentage points.
many-to-one, and one-to-nothing. Our experimental results
We use a similar method as the above to evaluate the
show that the precision and recall of this method are both
significance of each basic annotator. Each time, one
above 98 percent. There is still room for improvement in
annotator is removed and the remaining annotators
several areas as mentioned in Section 7.2. For example, we
are used to annotate the pages in DS2. Fig. 8 shows the
need to enhance our method to split composite text node
results. It shows that omitting any annotator causes both
when there are no explicit separators. We would also like to
precision and recall to drop, i.e., every annotator contributes
try using different machine learning techniques and using
positively to the overall performance. Among the six
more sample pages from each training site to obtain the
annotators considered, the query-based annotator and the
feature weights so that we can identify the best technique to
frequency-based annotator are the most significant. Another
the data alignment problem.
observation is that when an annotator is removed, the recall
decreases more dramatically than precision. This indicates
that each of our annotators is fairly independent in terms of ACKNOWLEDGMENTS
describing the attributes. Each annotator describes one This work is supported in part by the following US National
aspect of the attribute which, to a large extent, is not Science Foundation (NSF) grants: IIS-0414981, IIS-0414939,
applicable to other annotators.
CNS-0454298, and CNS-0958501. The authors would also
Finally, we conducted experiments to study the effect of
like to express their gratitude to the anonymous reviewers
using LISs versus using the IIS in annotation. We run the
for providing very constructive suggestions to improve the
annotation process on DS2 again but this time, instead of
using the IIS built for each domain, we use the LIS of each manuscript.
WDB. The results are shown in Table 5. By comparing the
results in Table 5 with those in Table 3 where the IIS is used, REFERENCES
we can see that using the LIS has a relatively small impact on [1] A. Arasu and H. Garcia-Molina, “Extracting Structured Data from
precision, but significant effect on recall (the overall average Web Pages,” Proc. SIGMOD Int’l Conf. Management of Data, 2003.
recall is reduced by more than 6 percentage points) because [2] L. Arlotta, V. Crescenzi, G. Mecca, and P. Merialdo, “Automatic
of the local interface schema inadequacy problem as Annotation of Data Extracted from Large Web Sites,” Proc. Sixth
Int’l Workshop the Web and Databases (WebDB), 2003.
described in Section 5.1. And it also shows that using the [3] P. Chan and S. Stolfo, “Experiments on Multistrategy Learning by
integrated interface schema can indeed increase the annota- Meta-Learning,” Proc. Second Int’l Conf. Information and Knowledge
tion performance. Management (CIKM), 1993.
As reported in [30], only three domains (Book, Job, and [4] W. Bruce Croft, “Combining Approaches for Information Retrie-
val,” Advances in Information Retrieval: Recent Research from the
Car) were used to evaluate DeLa and for each domain, only Center for Intelligent Information Retrieval, Kluwer Academic, 2000.
nine sites were selected. The overall precision and recall of [5] V. Crescenzi, G. Mecca, and P. Merialdo, “RoadRUNNER:
DeLa’s annotation method using this data set are around Towards Automatic Data Extraction from Large Web Sites,” Proc.
80 percent. In contrast, the precision and recall of our Very Large Data Bases (VLDB) Conf., 2001.
[6] S. Dill et al., “SemTag and Seeker: Bootstrapping the Semantic
annotation method for these three domains are well above Web via Automated Semantic Annotation,” Proc. 12th Int’l Conf.
95 percent (see Table 3), although different sets of sites for World Wide Web (WWW) Conf., 2003.
these domains are used in the two works. [7] H. Elmeleegy, J. Madhavan, and A. Halevy, “Harvesting
Relational Tables from Lists on the Web,” Proc. Very Large
Databases (VLDB) Conf., 2009.
8 CONCLUSION [8] D. Embley, D. Campbell, Y. Jiang, S. Liddle, D. Lonsdale, Y. Ng,
and R. Smith, “Conceptual-Model-Based Data Extraction from
In this paper, we studied the data annotation problem and Multiple-Record Web Pages,” Data and Knowledge Eng., vol. 31,
proposed a multiannotator approach to automatically con- no. 3, pp. 227-251, 1999.
[9] D. Freitag, “Multistrategy Learning for Information Extraction,”
structing an annotation wrapper for annotating the search Proc. 15th Int’l Conf. Machine Learning (ICML), 1998.
result records retrieved from any given web database. This [10] D. Goldberg, Genetic Algorithms in Search, Optimization and Machine
approach consists of six basic annotators and a probabilistic Learning. Addison Wesley, 1989.
LU ET AL.: ANNOTATING SEARCH RESULTS FROM WEB DATABASES 527
[11] S. Handschuh, S. Staab, and R. Volz, “On Deep Annotation,” Proc. Yiyao Lu received the PhD degree in computer
12th Int’l Conf. World Wide Web (WWW), 2003. science from the State University of New York at
[12] S. Handschuh and S. Staab, “Authoring and Annotation of Web Binghamton in 2011. He currently works for
Pages in CREAM,” Proc. 11th Int’l Conf. World Wide Web (WWW), Bloomberg. His research interests include web
2003. information systems, metasearch engine, and
[13] B. He and K. Chang, “Statistical Schema Matching Across Web information extraction.
Query Interfaces,” Proc. SIGMOD Int’l Conf. Management of Data,
2003.
[14] H. He, W. Meng, C. Yu, and Z. Wu, “Automatic Integration of
Web Search Interfaces with WISE-Integrator,” VLDB J., vol. 13,
no. 3, pp. 256-273, Sept. 2004.
[15] H. He, W. Meng, C. Yu, and Z. Wu, “Constructing Interface
Schemas for Search Interfaces of Web Databases,” Proc. Web Hai He received the PhD degree in computer
Information Systems Eng. (WISE) Conf., 2005. science from State University of New York at
[16] J. Heflin and J. Hendler, “Searching the Web with SHOE,” Proc. Binghamton in 2005. He currently works for
AAAI Workshop, 2000. Morningstar. His research interests include web
[17] L. Kaufman and P. Rousseeuw, Finding Groups in Data: An database integration systems, search interface
Introduction to Cluster Analysis. John Wiley & Sons, 1990. extraction and integration.
[18] N. Krushmerick, D. Weld, and R. Doorenbos, “Wrapper Induction
for Information Extraction,” Proc. Int’l Joint Conf. Artificial
Intelligence (IJCAI), 1997.
[19] J. Lee, “Analyses of Multiple Evidence Combination,” Proc. 20th
Ann. Int’l ACM SIGIR Conf. Research and Development in Information
Retrieval, 1997.
[20] L. Liu, C. Pu, and W. Han, “XWRAP: An XML-Enabled Wrapper Hongkun Zhao received the PhD degree in
Construction System for Web Information Sources,” Proc. IEEE computer science from State University of New
16th Int’l Conf. Data Eng. (ICDE), 2001. York at Binghamton in 2007. He currently works
[21] W. Liu, X. Meng, and W. Meng, “ViDE: A Vision-Based Approach for Bloomberg. His research interests include
for Deep Web Data Extraction,” IEEE Trans. Knowledge and Data metasearch engine, web information extraction,
Eng., vol. 22, no. 3, pp. 447-460, Mar. 2010. and automatic wrapper generation.
[22] Y. Lu, H. He, H. Zhao, W. Meng, and C. Yu, “Annotating
Structured Data of the Deep Web,” Proc. IEEE 23rd Int’l Conf. Data
Eng. (ICDE), 2007.
[23] J. Madhavan, D. Ko, L. Lot, V. Ganapathy, A. Rasmussen, and
A.Y. Halevy, “Google’s Deep Web Crawl,” Proc. VLDB Endow-
ment, vol. 1, no. 2, pp. 1241-1252, 2008.
[24] W. Meng, C. Yu, and K. Liu, “Building Efficient and Effective Weiyi Meng received the BS degree in mathe-
Metasearch Engines,” ACM Computing Surveys, vol. 34, no. 1, matics from Sichuan University, China, in 1982,
pp. 48-89, 2002. and the MS and PhD degrees in computer
[25] S. Mukherjee, I.V. Ramakrishnan, and A. Singh, “Bootstrapping science from the University of Illinois at Chicago,
Semantic Annotation for Content-Rich HTML Documents,” Proc. in 1988 and 1992, respectively. He is currently a
IEEE Int’l Conf. Data Eng. (ICDE), 2005. professor in the Department of Computer
[26] B. Popov, A. Kiryakov, A. Kirilov, D. Manov, D. Ognyanoff, and Science at the State University of New York at
M. Goranov, “KIM - Semantic Annotation Platform,” Proc. Int’l Binghamton. His research interests include web-
Semantic Web Conf. (ISWC), 2003. based information retrieval, metasearch engines,
[27] G. Salton and M. McGill, Introduction to Modern Information and web database integration. He is a coauthor
Retrieval. McGraw-Hill, 1983. of three books Principles of Database Query Processing for Advanced
[28] W. Su, J. Wang, and F.H. Lochovsky, “ODE: Ontology-Assisted Applications, Advanced Metasearch Engine Technology, and Deep Web
Data Extraction,” ACM Trans. Database Systems, vol. 34, no. 2, Query Interface Understanding and Integration. He has published more
article 12, June 2009. than 120 technical papers. He is a member of the IEEE.
[29] J. Wang, J. Wen, F. Lochovsky, and W. Ma, “Instance-Based
Schema Matching for Web Databases by Domain-Specific Query
Probing,” Proc. Very Large Databases (VLDB) Conf., 2004. Clement Yu received the BS degree in applied
[30] J. Wang and F.H. Lochovsky, “Data Extraction and Label mathematics from Columbia University in 1970
Assignment for Web Databases,” Proc. 12th Int’l Conf. World Wide and the PhD degree in computer science from
Web (WWW), 2003. Cornell University in 1973. He is a professor in
[31] Z. Wu et al., “Towards Automatic Incorporation of Search Engines the Department of Computer Science at the
into a Large-Scale Metasearch Engine,” Proc. IEEE/WIC Int’l Conf. University of Illinois at Chicago. His areas of
Web Intelligence (WI ’03), 2003. interest include search engines and multimedia
[32] O. Zamir and O. Etzioni, “Web Document Clustering: A retrieval. He has published in various journals
Feasibility Demonstration,” Proc. ACM 21st Int’l SIGIR Conf. such as the IEEE Transactions on Knowledge
Research Information Retrieval, 1998. and Data Engineering, ACM Transactions on
[33] Y. Zhai and B. Liu, “Web Data Extraction Based on Partial Tree Database Systems, and Journal of the ACM and in various conferences
Alignment,” Proc. 14th Int’l Conf. World Wide Web (WWW ’05), such as VLDB, ACM SIGMOD, ACM SIGIR, and ACM Multimedia. He
2005. previously served as chairman of the ACM Special Interest Group on
[34] H. Zhao, W. Meng, Z. Wu, V. Raghavan, and C. Yu, “Fully Information Retrieval and as a member of the advisory committee to the
Automatic Wrapper Generation for Search Engines,” Proc. Int’l US National Science Foundation (NSF). He was/is a member of the
Conf. World Wide Web (WWW), 2005. editorial board of the IEEE Transactions on Knowledge and Data
[35] H. Zhao, W. Meng, and C. Yu, “Mining Templates form Search Engineering, the International Journal of Software Engineering and
Result Records of Search Engines,” Proc. ACM SIGKDD Int’l Conf. Knowledge Engineering, Distributed and Parallel Databases, and World
Knowledge Discovery and Data Mining, 2007. Wide Web Journal. He is a senior member of the IEEE.
[36] J. Zhu, Z. Nie, J. Wen, B. Zhang, and W.-Y. Ma, “Simultaneous
Record Detection and Attribute Labeling in Web Data Extraction,”
Proc. ACM SIGKDD Int’l Conf. Knowledge Discovery and Data . For more information on this or any other computing topic,
Mining, 2006. please visit our Digital Library at www.computer.org/publications/dlib.