What types of data and information are considered to be Open Source?

Open source data and information is available in a variety of places, most of which are accessible via the internet. Examples include:

• Public records databases

• Government reports, documents, and websites

• The internet

• Mass media (e.g. newspapers, TV, radio, magazines and websites)

• Social networks and social media sites

• Maps and commercial imagery

• Photos and videos

• The dark web

List of Open Source Intelligence Tools

The tools and techniques used in Open Source Intelligence searching go much further than a simple Google search. Following is a list of helpful, time-saving open-source intelligence tools. Note: most are free, although some have advanced features available for a fee.

Email Breach Lookup

This is one of my favorite open source intelligence tools because anyone can use it. Have I Been Pawned? allows you to find out if a particular email address was affected by one of the many data breaches that have occurred over the years. Just enter an email address and it will tell you which breaches the email was affected by. In addition, it gives a brief overview of the breach.

Fact-Checking Websites

Hoaxy – Hoaxy is an open-source search tool that visualizes the spread of articles online

Media Bugs – A media bug is something in the media (e.g. newspaper, magazine, radio, etc.) that is wrong and is correctable. Mediabugs.org allows anyone to report a bug and they will work with the media outlet to get it corrected. The site can be used in identifying fake or incorrect news, and to also look for corrected versions.

PolitiFact – Politifact is focused on fact-checking journalism. Politifact uses a helpful rating scale to rate statement made by journalists, political figures and others. The scale runs from True to Full Flop and is a helpful way to know who is telling the truth and who is lying. Read through some of the statements Politifact has investigated and you’ll quickly realize that a lot of what we hear from politicians just isn’t true. The company won a Pulitzer Prize for its work.

SciCheck – SciCheck is a feature of FactCheck.org that focuses on evaluating false and misleading scientific claims made to influence public policy. For example, when a political figure says that “global warming isn’t real”, they are making a false claim to sway public opinion. SciCheck evaluates such claims and explains whether or not they are true.

Snopes – Snopes.com is one of the oldest, best and most well-known fact-checking sites. The free site researches everything from urban legends, rumors, myths, questionable photos and videos, articles and claims made by public figures.

Verification Junkie – Verification Junkie is a collection of tools designed to help verify and fact check information and assess the validity of eyewitness reports. This site has a collection of very cool tools that are worth the time to dig into.

Hacking and Threat Assessment

Norse – Norse maintains the world’s largest dedicated threat intelligence network. With over eight million sensors that emulate over six thousand applications. Their network gathers data on who the attackers are and what they’re after. Their home page shows a live attack map with real-time information on attacks.

OSINT Image Search

The following tools go beyond the typical Image Search option that is offered by most major search engines.

Current Location – Current Location is a neat map search tool that allows you to select a position on the map anywhere in the world. Then, you can view photos within a certain radius of your location. It checks photos posted to Instagram, Flickr, and 500px.

Image Identification Project – The Wolfman Image Identification Project is a search tool that uses algorithms to identify images. Simply drag and drop, or upload, an image into the search form and it will attempt to tell you what it is.

TinEye – TinEye is a reverse image search that allows you to search by image and find out where that image is online. You can upload an image, drag and drop to the search box, or search using a URL for an image. The index has over 30 billion images.

Public Records (Property)

I assume that everyone is familiar with the information you can get from sites like Zillow, Trulia, Realtor.com, etc. Those sites are always useful and should be a part of your investigative toolbox. The following tools provide various twists on property records searching and are definitely worth checking out.

Melissa Data Property Viewer – This open source intelligence tool allows you to view property information for almost any property. Just enter a zip code to start. Then, you can zoom in using a map or satellite view all the way down to a specific property. Click on a specific property to get public records information such as the full address, owner name, resident’s name, value, year built, building and lot square footage and more. This can be helpful when you’re planning to do surveillance and want to make sure you have the right home.

Emporis Building Search – The Emporis site allows you to search for buildings, companies and design/construction images all over the world. Their free building search engine allows you to search by building name to find the exact location of a building. This can be useful if all you have is the building name and you need to reverse match it back to an address.

OSINT Search Engines

Google Correlate – Google Correlate can help you identify search patterns that correlate with real-world trends. It can be used to identify search patterns that are similar to one another. For example, you could use the tool to find out if there is a link between new legislation and gun sales (at least in terms of the search volume related to the two topics).

Google Search Operator Guide

Million Short – Million Short has an interesting approach to searching for information online. The interface is generally similar to a typical search engine. However, you can sort and filter the results in a number of ways. Examples include popularity, eCommerce, live chat, date, location, and others. Also, it automatically pulls out the top sites that usually occupy the top spots in any search result (e.g. Amazon.com, eBay, YouTube, etc.)

Shodan – Shodan is a search engine and network security tool for the internet of things. It finds devices on the internet like web servers, webcams, appliances, traffic lights, and even power plants.

TalkWalkerAlerts – TalkWalkerAlerts is a great alternative to Google Alerts because, in addition to monitoring the web for certain keywords, it monitors social media, blogs, and forums as well. This can be helpful if you need to find mentions of someone’s name or business across the web.

OSINT Social Media Search Tools

Facebook Search Tools – A collection of helpful links to Facebook online help resources and various open source search forms. Learn how to run searches and queries that go far deeper than the standard website or app search.

TweetBeaver – If you’re doing research that involves a Twitter user, TweetBeaver is a great resource. Use it for things like download a user’s friends list, download a users followers list, check to see if two accounts follow one another, and much more.

OSINT Tool Websites

There are a number of websites dedicated to aggregating tools and links, similar to this page. Some are excellent resources that are worth bookmarking, while others are nothing more than a bunch of dead links. Following are some of the better resources:

• IntelTechniques – This is probably the best site you’ll find in the industry. It has a huge collection of resources, links and online search forms. In addition, the author Michael Bazzell, provides training, news and industry information.

Software

Hunchly – Hunchly is an online evidence collection tool for investigative professionals. The software records all of your online activity to speed the research and discovery process.

Maltego – Maltego is a software tool developed by Paterva. It is used by law enforcement, forensic investigators, and security professionals to analyze open source intel. It runs on Windows, Linux, and OSX. Investigators use the software to collect data and information from various sources and display them graphically. This helps reduce analysis time, make connections and uncover leads.

Surveillance Cameras

Earth Cam – Earth Cam is the gold standard when it comes to aggregating the world’s public live streaming webcams and surveillance cameras in a way that makes sense.

Insecam – Insecam aggregate access to hundreds of surveillance cameras and public web cameras around the world. The cams are not “hacked” and do not include cams on to computers, laptops, tablets, etc. The site has cams organized by a variety of categories, but it is tedious to find exactly the area you’re looking for.

Transportation – Vehicles, Aircraft, Boats

Flight Radar 24 – Created by two Swedish aviation experts, this OSINT site allows you to view live flight tracking information around the globe. The free service tracks over 180,000 flights per day. The graphical interface shows an interactive map with moving plane icons. Hover over an individual plane icon to see the flight number. Click on the plane icon to see flight information and details on the aircraft itself.

Marine Traffic – This website provides global ship tracking intelligence. It is another great example of the visualization of large amounts of data. Visit the site and you’ll see a map of the world, with a million little arrows. Each individual arrow represents a marine vessel. Click on any individual arrow to see what type of boat it is, such as a tanker, cruise ship, etc. You can filter the map in a number of ways, such as by ship type, capacity, voyage and more.

Nationwide Aircraft Registry – Search a registry of all planes registered with the Federal Aviation Administration (FAA). The registry contains information related to the aircraft, the owner(s), airworthiness and more. The only challenge is that you need to know the registration number (n-number) to run a query.

VINCheck – VINCheck is an online tool provided by the National Insurance Crime Bureau. The tool helps determine if a vehicle is reported as stolen, but not recovered. To run a search, just enter the full VIN number and click on the Search VIN button.

Username Check

CheckUserNames – CheckUserNames is a handy open source intelligence tool that will check the availability of user names on more than 500 social networks. Their new site is at Knowem.com, which adds additional services to the mix, including searching the USPTO trademark database.

Namech_k – This OSINT tool is a search engine allows you to search domain names to see what is available, and then go through the process of registering it or making an offer. In addition, it will check to see if a username is available on hundreds of social media networking sites. You can even have them signup for you.

Virus Scanner

VirusTotal – VirusTotal is an online tool that allows you to scan a file or URL to see if it has malware. VirusTotal uses virus scanners and URL/domain blacklisting services. The site allows users to upload a file or provide a URL. Then, it will run a scan and provide results to the user. In addition, VirusTotal shares the file/URL with the security community.

Visual / Clustering Search Engines

These search engines take searching a step further by sorting and organizing results. This helps users narrow down particular areas to focus on. And, some allow users to use visualization tools to analyze data and information in different ways.

Carrot2.org – Carrot2 offers a unique approach to searching as it organizes search results into topics, and visually. They categorize results into topics in three ways. One as a series of folders by topic. Second, it displays the topics in a color-coded circle diagram (very cool). The size of the individual pieces indicates the number of results within each topic. The third method is “Foamtree”, which arranges the topics into a diagram by topic.

Yippy – Yippy searches multiple sources and provides a set of results, similar to any search engine. However, it also groups the results by topic for easier sorting and viewing. It is not as visual as Carrot2, but still a helpful method of culling down the results.

Website Analysis

BuiltWith.com – If you need to gather the technical details on a particular website, try BuiltWith.com. This OSINT tool is a website profiler, competitive analysis, and business intelligence tool. You can use their free search engine to analyze the technology profile of a website. Just enter a domain name and click search to find out which content management system (CMS) or content framework a site is using, including widgets and plugins.

Note: Most of these OSINT tools on this list will work on standard web browsers such as Google Chrome, Internet Explorer, Firefox and Safari. However, they may have small differences in the way the site displays on the screen, or on your mobile device. Some of the solutions are designed specifically to search the dark web. In those cases, you’ll need to download and install the TOR browser.