Metawebsites

Metawebsites would be web sites about web sites.

Note: In what follows, the example.com domain is used as an illustrative example (see RFC 2606).

General utility

Hurricane Electric’s BGP Toolkit offers insight into the structure of the internet. Find all of the network ranges assigned to Amazon using:

http://bgp.he.net/search?search%5Bsearch%5D=amazon

Robtex Swiss Army Knife Internet Tool

http://www.robtex.com/dns/example.com.html#result

CentralOps.net features the Domain Dossier to investigate domains and IP addresses. Get one report with registrant information, DNS records, and more. Can also scan for FTP (21), SMTP (25), HTTP (80), POP3 (110)  and IMAP (143) services.

T1Shopper (a telecom services comparison resource) has traceroute, ping, NSLookup, Whois, Port Scan, a subnet calculator, a file size conversion calculator, a file transfer time – data transfer speed calculator, a catalog of speedtest sites, an Http header viewer and a dictionary of telecom and internet service terms.

Cisco’s SenderBase.org provides a view into real-time threat intelligence across web and email. SenderBase is powered by Cisco Security Intelligence Operations (SIO), a cloud-based capability which analyzes over 100TB of daily security intelligence across over 1.6 million deployed Web, Email, Firewall and IPS appliances. SIO continuously evolves its defenses by looking across multiple security platforms with a global sensor network – brought together and analyzed in the cloud, then delivered back to Cisco customers every 3-5 minutes for protection that goes beyond blacklists and reputation. SIO’s intelligence in augmented by a network of traps, crawlers, third-party partnerships and threat research.

showsiteinfo.org Check links (out and in), speedtest, keywords, description

http://www.showsiteinfo.org/search?name=example.com

MxToolbox focuses on email-oriented DNS and network configuration information.

DNSTrails (claims to be) the World’s Largest Repository of historical DNS data. Probably is.

DNSqueries.com is a large collection of utilities (Domain Health Check, Ip Neighbors, Check IP on RBLs, Reverse lookup DNS, Perform DNS query, Dns Traversal, Get ip geo location, Server banner check, RegExp Tester, Encrypter, IPv4 converter, Http Headers, Ping tool, Traceroute utility, Googlebot Simulator, Check your SMTP server, Http Gzip Test, Keyword Density Analyzer, Whois Lookup, Live port scanner, and Mx Lookup). The Live port scanner is a noisy scanner (which crashed with a “512” when I used it), but at least its not your IP address that will be blocked if someone notices the scan.

DigWebInterface unix dig (domain information groper), in a web interface, for DNS troubleshooting

myip.ms Whois

dazzlepod.com Whois, Host Services (nmap query), Visual Traceroute

centralops.net/co Reverse DNS

VirusTotal Passive DNS https://www.virustotal.com/en/ip-address/xx.xx.xx.xx/information/

Farsight pDNS passive DNS

Site Dossier passive DNS http://www.sitedossier.com/ip/x.x.x.x

tcpiputils.com is another collection of utilities (DNS Lookup (root servers), Email Test, DNS Blackhole List, Ping, Domain Neighbors, MAC Address Lookup (vendor identification), W3C Validator, Geo Targeting SEO, Reverse TinyURL).

ipvoid.com DNS Blacklist (DNSBL) Lookup

nsZones.com offers a DYN Database IP Check http://www.nszones.com/dyn.ip?xx.xx.xx.xx as well as
Domain Name System Block List (DNSBL) services for the following zones:

• bl.nszones.com combination of sbl.nszones.com and dyn.nszones.com in a single zone.
• sbl.nszones.com (Open Relay, Hijacked PCs, Spam Source)
• dyn.nszones.com (Dynamic, ADSL, Cable, no PTR Networks)
• ubl.nszones.com (Domain Names, PTR of Dynamic Networks)

The Composite Blocking List (CBL) lists IPs exhibiting characteristics which are specific to open proxies of various sorts (HTTP, socks, AnalogX, wingate, Bagle call-back proxies etc) and dedicated Spam BOTs (such as Cutwail, Rustock, Lethic etc) which have been abused to send spam, worms/viruses that do their own direct mail transmission, or some types of trojan-horse or “stealth” spamware, dictionary mail harvesters etc.

gwebtools.com Whois, DNS Tools, Port Scanner, IP Subnet Calculator, Name Server Spy (what domains are hosted on a name server?), Domain Checker, SEO Tools, Bit Calculator (how many bits in a petabyte?), My IP

TechnicalInfo.net is a collection of passive information gathering tools, many of which fail by redirecting to swisscom. Includes some of the Sam Spade tools, which can tell you what others can easily learn about you. The Sam Spade tools look up DNS and domain information. The Sam Spade tools are frequently under revision, but one stable source is petri.co.il.

XSSposed Is the new home of examples of cross-site scripting (XSS) exploits that people have discovered and reported. Keep RSnakes’ XSS Cheat Sheet handy. </xssed> is not being updated.

Project Un1c0rn is a search engine exposing open, vulnerable and weak services (leaking mysql, mongo and heartbleed).

Cookiepedia aims to build a comprehensive knowledge base about website cookies and similar technologies.

BFK DNS Logger As a service to CERTs and incident response teams, BFK uses passive DNS replication to collect public DNS data. Compared to the ordinary domain name system, this database adds further search capabilities.

Search Engine Optimization (SEO)

Expectations. It’s all about expectations. Getting a search engine to refer to your web site appropriately requires an understanding to what the search engine crawler does, what it expects to find, and how to address those expectations.

• Search Engine Optimization for Bing
• Google’s Search Engine Optimization Starter Guide [pdf]
• Fighting the Penguin: Tips for Winning Back Google Traffic

Alexa is the leading provider of free, global web metrics. SEO focused.

http://www.alexa.com/siteinfo/example.com

Google Analytics

siteshakedown.com aggregates from various sources to provide the site’s general company information, internet traffic rank, social popularity, twitter feeds, and more. SEO focused.

http://www.siteshakedown.com/q/www.example.com

SpyOnWeb.com Enter website url, ip address, google adsense or google analytics code and find out what resources belong to the same owner. SEO focused.

http://spyonweb.com/example.com

push2check.com aggregates various services. Many are SEO focused.

http://push2check.com/example.com

HubSpot’s Marketing Grader SEO focused.

Of particular interest are the HTML testing sites. Humans don’t notice a lot of HTML errors; search bots have been trained to correct and understand mistakes. Nonetheless, a valid HTML document has more chance of being correctly displayed in current browsers and updated browsers.

• http://www.onlinewebcheck.com/?url=http://www.example.com
• http://jigsaw.w3.org/css-validator/validator?uri=www.example.com

StatsCrop.com is a web service that lets you explore any website’s information and its history. Understand your website traffic or a competitor’s. SEO focused.

http://www.statscrop.com/www/example.com

Markosweb web site monitoring, SEO focused.

http://www.markosweb.com/www/example.com/

pandastats web site monitoring, SEO focused.

http://example.com.pandastats.net/

hostlogr web site monitoring, SEO focused.

http://example.com.hostlogr.com/

craftkeys web site monitoring, SEO focused.

http://craftkeys.com/site-info/example.com

pageglance web site monitoring, SEO focused.

http://www.pageglance.com/example.com

Reputation

webutation.net aggregates from various reputation rating sites.

http://www.webutation.net/go/review/example.com#

scamvoid.com also aggregates reputation information

http://scamvoid.com/check/example.com

urlvoid.com aggregates reputation information as well.

http://urlvoid.com/scan/example.com

ProjectHoneyPot Project Honey Pot is an open source initiative to track abuse, fraud, and other malicious behavior that occurs online. The Project tracks more than a million IP addresses engaged in suspicious behavior each day and reports on them through our website.

Honeynet Project map

PhishTank.com is a collaborative clearing house for data and information about phishing on the Internet.

Web of Trust Unfortunately, no one checks to see what the community has reported in order to interpret the WoT score. They look no further than the raw score. The WoT raw score is largely useless; if it is low, then why? What is the complaint? Two “very poor” ratings can result in a poor reputation score. If it high, then why? Perhaps only one person bothered to rate it.

Malware reported

These are historical (not on-demand, “check now”) malware detection services.

Google’s Safe Browsing Diagnostic Tool reports if malware was detected while crawling and reports networks (ASNs).

http://www.google.com/safebrowsing/diagnostic?site=example.com

StopBadware.org reports if malware was reported. Does not do detection itself.

scumware.org Was any malware detected? Search by MD5, IP address, hostname or the beginnings of a URL (such as example.com). Add a URL to the list of URLs to crawl.

Sucuri Sitecheck https://sitecheck.sucuri.net/results/example.com

Malware URL checks sites using VirusTotal, Wepawet, Anubis and Threat Expert. Was any malware detected?

VirusTotal Was any malware detected? https://www.virustotal.com/en/ip-address/xxx.xxx.xxx.xxx/information/

Clean-MX Was any malware detected?

http://support.clean-mx.de/clean-mx/viruses.php?sort=firstseen%20desc&domain=example.com

AVG Online Web Page Scanner Was any malware detected?

http://www.avgthreatlabs.com/sitereports/domain/example.com/domain-search-widget/www.avg.com.au

F-Secure Was any malware detected?

Norton Safe Web Was any malware detected?

http://safeweb.norton.com/report/show?url=example.com

Malware Domain List All domains on this website should be considered dangerous. If you do not know what you are doing here, it is recommended you leave right away. This website is a resource for security professionals and enthusiasts.

http://www.malwaredomainlist.com/mdl.php?search=example.com&colsearch=All&quantity=50

McAfee Site Advisor Any malware detected? Do any of the sites this site links to have malware?

http://www.siteadvisor.com/sites/example.com

McAfee TrustedSource Any malware detected? Do any of the sites it links to have malware? I do not see how the roles of Site Advisor and TrustedSource differ.

Trend Micro Web Reputation Any malware detected?

URL Blacklist Check blacklists.

URIBL Check blacklist.

Malware detection

Unmask Parasites Tests in real time for evidence that the web site has been hacked.

http://www.unmaskparasites.com/security-report/?page=http%3A//example.com

urlQuery.net is a (beta) service for detecting and analyzing web-based malware. It provides detailed information about the actions a browser takes while visiting a site and presents the information for further analysis. It uses Intrusion Detection Systems (IDSs) (Suricata with Emerging Threats and Snort with VRT), reports about the ASN, reviews the Java scripts, reports requests and responses.

hosts-file.net maintains net block lists and links to many web site information sources.

http://hosts-file.net/default.asp?s=example.com

Of particular interest is vURL Online. Quickly and safely dissect malicious or suspect websites.

http://vurldissect.co.uk/default.asp?url=http%3A%2F%2Fexample.com&btnvURL=Dissect&selUAStr=1&selServer=1&ref=&cbxLinks=on&cbxSource=on&cbxBlacklist=on

The following message from hpHOSTS (about the IP address not matching the PTR record) should not influence your decision to trust the website or ISP. Consult the IP reverse DNS feature of robtex.com or the Domain Dossier feature of CentralOps.net as a convenient mechanism to review DNS records. I find that the IP address often matches the PTR record, contrary to the message.

WARNING: The IP PTR associated with this record, does not resolve. This is considered very bad practice and contravines (sic) the RFC Standards. Most legit ISP’s will have their PTR’s resolve to an IP.