ICC-Crawler is operated by Universal Communication Research Institute (UCRI) at NICT. The main goal behind developing the crawler is to collect Web pages for researches related to Web-search and data mining. Recently, we are planning to use it for crawling weblogs too. The crawler is used by the members of UCRI at NICT to crawl Web-pages only for the research purposes. Our crawling policy distinctly respects the general crawling norm. Though we duly understand the concern of the webmasters, we would like to assure that our crawler is only crawling pages for performing researches and not for any business use. Please have a glance at our crawling policy for better understanding. We sincerely appreciate your cooperation and support.
Our crawler always respects the common crawling norm as like following:
User-agent: *
Disallow: /cgi-bin
User-agent: ICC-Crawler
Disallow: /
IP addresses of currently crawling machines are:
202.180.34.186
61.86.246.72
We would like to clarify again that our crawler is collecting pages solely for research purposes. We are interested in crawling large volume of pages for following ongoing researches at our group:
Common Infrastructure Group
General Planning Office
Universal Communication Research Institute
National Institute of Information and Communications Technology
Phone. +81-774-98-6300 Fax. +81-774-98-6955