Please wait a minute...
FITEE
Front. Inform. Technol. Electron. Eng.  2015, Vol. 16 Issue (6): 449-456    DOI: 10.1631/FITEE.1400377
    
基于URL聚类的快速无障碍检测抽样方法
Meng-ni Zhang, Can Wang, Jia-jun Bu, Zhi Yu, Yu Zhou, Chun Chen
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
A sampling method based on URL clustering for fast web accessibility evaluation
Meng-ni Zhang, Can Wang, Jia-jun Bu, Zhi Yu, Yu Zhou, Chun Chen
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
 全文: PDF 
摘要： 目的：大多数残疾人士上网都会遇到各种障碍。为减少上网障碍，对网站进行无障碍检测评估是十分必要的。鉴于大部分网站具有海量网页且某些网页需涉及人工检测，通常利用抽样算法对网站进行无障碍检测评估。已有的分层抽样算法I/O开销和计算代价大。为解决这一问题，本文提出一种基于URL聚类的抽样算法。仅利用URL信息进行聚类，然后抽样，最终实现快速的无障碍检测和评估。
创新点：大部分网站的网页内容和URL信息都是由有限数量的模板生成的。因此这些网站的无障碍问题都可以追溯到模板。鉴于同一模板生成的网页具有相似结构和URL模式，可基于URL相似性对网页进行聚类，将同一模板的URL聚到一类中。本文所提抽样算法仅利用网页URL模式信息，无需存储全部网页内容，从而减少I/O开销和计算代价，实现快速的无障碍检测和评估。
方法：利用模板生成的网页具有相似URL模式，将URL进行聚类以实现同一模板生成的网页聚在一类中。具体过程：首先，解析爬取到的URL以获取候选URL分词和模板URL分词；然后利用最小长度描述原则进行URL聚类（算法1）；最后在每类中按照抽样比例进行抽样。
结论：不同于现有的分层抽样算法，本文提出的抽样算法仅利用URL模式信息将网页进行聚类，可减少大量I/O开销和计算代价。
关键词： >;  网页抽样;  URL聚类;  无障碍检测    
Abstract: When evaluating the accessibility of a large website, we rely on sampling methods to reduce the cost of evaluation. This may lead to a biased evaluation when the distribution of checkpoint violations in a website is skewed and the selected samples do not provide a good representation of the entire website. To improve sampling quality, stratified sampling methods first cluster web pages in a site and then draw samples from each cluster. In existing stratified sampling methods, however, all the pages in a website need to be analyzed for clustering, causing huge I/O and computation costs. To address this issue, we propose a novel page sampling method based on URL clustering for web accessibility evaluation, namely URLSamp. Using only the URL information for stratified page sampling, URLSamp can efficiently scale to large websites. Meanwhile, by exploiting similarities in URL patterns, URLSamp cluster pages by their generating scripts and can thus effectively detect accessibility problems from web page templates. We use a data set of 45 web sites to validate our method. Experimental results show that our URLSamp method is both effective and efficient for web accessibility evaluation.
Key words: Page sampling    URL clustering    Web accessibility evaluation
收稿日期: 2014-11-02 出版日期: 2015-06-04
CLC:  TP391.3  
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章　　
Meng-ni Zhang
Can Wang
Jia-jun Bu
Zhi Yu
Yu Zhou
Chun Chen

引用本文:

Meng-ni Zhang, Can Wang, Jia-jun Bu, Zhi Yu, Yu Zhou, Chun Chen. A sampling method based on URL clustering for fast web accessibility evaluation. Front. Inform. Technol. Electron. Eng., 2015, 16(6): 449-456.

链接本文:

http://www.zjujournals.com/xueshu/fitee/CN/10.1631/FITEE.1400377    或    http://www.zjujournals.com/xueshu/fitee/CN/Y2015/V16/I6/449

No related articles found!