Identifying Clones in Dynamic Web Sites Using Similarity thresholds

We propose an approach to automatically detect duplicated pages in dynamic Web sites. Our approach analyzes both the page structure, implemented by specific sequences of HTML tags, and the displayed content. In addition, for each pair of dynamic pages we also consider the similarity degree of their scripting source code. The similarity degree of two pages is computed using different similarity metrics for the different parts of a web page based on the Levenshtein string edit distance. We have implemented a prototype to automate the clone detection process on web applications developed using JSP technology and used it to validate our approach in a case study.