+ Reply to Thread
Results 1 to 10 of 10

Thread: What is web crawling?

  1. #1

  2. #2
    Hosting Guru Gaurav Joshi is on a distinguished road
    Join Date
    Mar 2019
    Location
    Mumbai
    Posts
    80
    A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner.

  3. #3
    SuperExpert RepriseH is on a distinguished road
    Join Date
    Jul 2017
    Location
    India
    Posts
    1,291
    Web crawling is the process of reading through webpage sources by search engine crawlers.
    Cheap VPS | $1 VPS Hosting
    Cheap Dedicated Servers | Dedicated Servers starting from $29/MO

  4. #4
    HWHT Addict sarita670 is on a distinguished road
    Join Date
    Mar 2019
    Location
    india
    Posts
    32
    Web crawling is done by web bots/spider/crawlers to read your website and index them. web crawlers first checks all the **** tags , sitemaps and robots.txt.

  5. #5
    Hosting Guru Chetna is on a distinguished road
    Join Date
    Apr 2019
    Location
    mumbai
    Posts
    84
    A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a "spider" or a "bot."

  6. #6
    Hosting Guru Saurabh is on a distinguished road
    Join Date
    Mar 2019
    Location
    mumbai
    Posts
    61
    A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.

  7. #7
    HWHT Addict Sneha123 is on a distinguished road
    Join Date
    Jan 2019
    Location
    Bangalore
    Posts
    23
    crawler crawls the content of the website which added unique and fresh content into website is called web crawling.

  8. #8
    Hosting Master kailash is on a distinguished road
    Join Date
    Mar 2019
    Location
    mumbai
    Posts
    101
    Web crawling can be a very complicated and technical subject to understand. Every web page on the Internet is different from the next, which means every web crawler is different (at least in some way) from the next.

  9. #9
    HWHT Starter cool is on a distinguished road
    Join Date
    May 2019
    Location
    Dallas
    Posts
    8
    Internet bot that systematically browses the World Wide Web

  10. #10
    Hosting Master ServersBase is on a distinguished road
    Join Date
    Dec 2018
    Location
    Asia
    Posts
    100
    A Web crawler is an Internet bot which helps in Web indexing. They crawl one page at a time through a website until all pages have been indexed.
    Web crawlers help in collecting information about a website and the links related to them, and also help in validating the HTML code and hyperlinks.
    A Web crawler is also known as a Web spider, automatic indexer or simply crawler.

+ Reply to Thread

Similar Threads

  1. What is Crawling?
    By gitanistone in forum Search Engine Optimization
    Replies: 3
    Last Post: 04-12-2019, 06:45 AM
  2. What is Crawling?
    By sydneyearplug in forum Programming Discussion
    Replies: 1
    Last Post: 02-06-2019, 06:51 AM
  3. What is Crawling?
    By vinecottage in forum Programming Discussion
    Replies: 0
    Last Post: 02-05-2019, 09:30 AM
  4. What is Crawling?
    By gitanistone in forum Search Engine Optimization
    Replies: 3
    Last Post: 12-18-2018, 10:16 AM
  5. What is Crawling?
    By designpluz in forum Search Engine Optimization
    Replies: 0
    Last Post: 07-22-2017, 06:23 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts