Does anyone here know of a crawler software?
 

That will do the identical things that Visual Web Spider will do. I tried to buy VWS but my credit card will not allow me to deal with Advangate which is the processor for the software. It can be free or paid if it is still in the same price range.

I have done a lot of searching and have found free crawlers and paid crawlers but none have the identical features let alone at a similar price.

You should probably list the actual features you want otherwise you are limiting responses to those who are familiar with Visual Web Spider or are willing to do the research to be able to answer your question (which is probably not many).

I have done many spiders both general and targeted but since I am totally unfamiliar with Visual Web Spider and have absolutely no desire to do the research I cannot tell you if I have anything, or could write something to match your requirements.

Just a suggestion.

.

Visual Web Spider will crawl both search engines and a text list of URls. I am not interested in crawling the search engines just the text list of URLs.

These are the required settings that are a must have.

1. depth setting

2. follow external urls or not

3. A URl filter setting to save the URl to the hard drive if it contains the filter world, as it is crawling.

4. More importantly it has a filter setting to save to the hard drive for what is on the page it is crawling. In other words if you set the page filter to "camping" it will save only pages that have the word camping on the page.

5. Setting for saving only Domain names or the actual URL.

6. Setting for the number of simultaneous threads. The crawling speed 0 equals the fastest and 1 would equal 1 second. 2 would equal 2 seconds etc. as well as the time to wait on page to load before moving on to the next URL.

7. Setting for limiting the number of URls/Records to save to the hard drive as it is crawling which also includes an unlimited setting.

8. VWS is very fast.

The trial version is for 14 days but alimit of 100 records for each run. It works very well and is very fast and has additional features other than those I listed and is only $49.00 for a one off buy. If I cannot come up with some thing with the features I listed I am going to talk with the bank that issued my card and see if something cannot be done to use my card to purchase the software.

By the way I have down loaded several very expensive crawlers that do not have the features that VWS does.

get a gift card/ pre paid throw away card...

Might work but I understand that there are alot of processors that will not process the throw away cards but I could talk to the Advangate rep and find out. Your traumatized, damaged brain came up with a good suggestion. Congrats and thanks.

my top-up card gets accepted online much more than my regular credit card...it has more to do with the issuing bank than the type of card...youre welcome daquan...

Paypal a trusted member the $49.99 (plus 10%) to purchase it for you? They would use your details, but their card?

Another idea but I swore not to ever use Paypal ever again after I got ripped off by them to the tune of several hundred dollars. Don't ask me for details as it was so long ago I don't remember details. Thanks anyhow for the input.

OK. So why not an actual physical pal or a family member? Its not like Web Spider is an embarrassing piece of software to purchase,. Its not like you are asking them to purchase a recurring monthly membership to xxx-hot-sluts-dot-com if you get me?

I would forget about the "rip off" because paypal is very useful.
Not having paypal is why you had to make this thread.

:2 cents:

Have you tried asking the software company if you can pay by cheque / transferwise or some other method? Most companies are happy to take your money. It seems like it has a lot of specific features you need, I would focus on trying to buy it not look for an alternative.

What kind of credit card do you have that would restrict your use of Advangate? It's a fairly popular system, particularly for software packages.

Do they not allow foreign transactions?

Visa Card. I bought a prepaid card today but it too will not release money to Advangate. I may be forced to go thru Paypal. If it in fact will work but will try to contact the owner of the software first and see what he may suggest or maybe try a different prepaid card.

It is beginning to seem that Avangate may have a shady past.. By the way it is not actually Avangate but 2checkout I think is the name now.

Build one with Python & Selenium.

Much simpler to buy one that will do everything that I require for the cheap price of $50.00 if I can swing the deal.

You really can’t figure out how to make a $50 purchase ?

I am working the problem. Two different cards have now not worked for me but both have been Visa cards. I am going to acquire another card or try to go thru Paypal. So yes I will find a solution. Thank you for your input.

To let everyone know I did purchase the software with an American Express card. It works great.

The more I use this piece of software the more I love it. If anyone needs a crawler for any reason I highly recommend this software.