APU Careers Careers & Learning Original

How to Search for Open Source Information Using Online Resources

Start an Intelligence Studies degree at American Public University.

By Charles M. Russo
Faculty Member, Intelligence Studies at American Military University

Note: This article was originally published on In Public Safety.

Intelligence analysts are thought to be commensurate experts in writing, research and analysis, but does the next generation of analysts have the skills necessary to be successful in the intelligence field? One of the greatest challenges for an analyst today is that the amount of information — as well as the means in which it’s shared — is growing exponentially.

Intelligence analysts must be able to gather, correlate, analyze and evaluate information from a wide variety of sources. These sources can include law enforcement portals and databases, surveillance systems, intelligence networks (various disciplines), geographic information systems (GIS) and private data-mining databases (subscription-based).

[Related: What Will it Take to be Tomorrow’s Intelligence Leader?]

In addition to searching these various restricted access databases available to those within the U.S. Intelligence Community, another important source of information for analysts comes from open sources. There are several skills that analysts must master to effectively search and exploit open source material in this age of information.

Gathering Open Source Information

Open source material can be retrieved from a myriad of public sources including radio, television, press agencies, newspapers and other periodicals, as well as books, blogs, social media platforms and other websites on the Internet. Information from these sources is produced very quickly and is buried just as fast as it is disseminated, thus quickly becoming irrelevant.

Open source material also includes gray literature, such as academic sources like dissertations, research papers, technical papers and information from conference and seminar presentations. Information can also come from government agencies (local, state, and federal), corporations, research centers, associations and societies, as well as professional organizations.

[Related: Why Overt Intelligence Is Important But Often Undervalued]

Many open source websites are free and have open access, while others require an account to be set up or may require you to have a paid subscription. Many higher learning institutions and state and federal government entities provide students and employees with ready access to many of the paid sites.

Mastering the Search

As an intelligence analyst, you are expected to be the master of search and research. You must think about how to find nuggets of information that may not be apparent to all.

Before you begin a search, you should be aware of what information you are looking for. Conducting a general topic search just to see what you can find by throwing a wide net may be useful initially, but you will eventually need to focus in on the information you need or is usable.

[Related: Aspiring Intelligence Analysts: Are You Ready?]

Analysts can go about gathering information in different ways, but it all comes down to their ability to manipulate the search function of the database or portal. All too often, a novice researcher simply plugs in a few key terms — ones they think define what they are looking for — and only investigates the top results of that search. Not often enough does the researcher work to refine the search terms or consider possible abstract terms that may lead to the information they’re looking for.

Beyond choosing the right search terms, how do you connect those terms to further refine your search? Boolean operators are connectors used to define the relationship between search terms and assist in either narrowing or broadening records or results. The most commonly used Boolean operators are AND, OR and NOT. Search engines in portals or websites may have their additional logic and operators to help you in refining your research.

Searching Social Media

If you are building a dossier on a specific individual, you are going to want to start your search with the most popular social media (SM) platforms such as LinkedIn, Facebook, Instagram, Twitter and Snapchat. These are just the start of SM and each one alone is not necessarily going produce the desired results.

People upload personal information across different SM sites, so you need to know what you are and are not looking for. Yes, you can say you don’t know what you are looking for until you find it, but this is not a good mindset to start your search with.

[Related: How Social Media and Volunteered Mapping Has Changed Emergency Response]

Basic information analysts commonly search for on SM include residential or business addresses, email addresses, phone numbers, IP addresses, domain names, property/assets, marital status…the list can go on. You can also conduct more in-depth searches for company ownership, social activities, family or friends, political and/or spiritual leanings, financial status, places they go or other personal details.

Protecting Your Identity

Intelligence analysts must be cautious and may need to hide their identity while searching. For example, you may want to set up new social media accounts that you use only during investigative inquiry. But be aware that some social media networks prohibit fake accounts. You should read their terms and rules carefully before you sign up and create a fake account.

[Related: An Introduction to the Darknet and Bitcoin]

There are several other ways to hide your identity when searching online and sometimes it’s a good idea to do so. Using the Tor browser is one of several reliable ways to search anonymously.

Anonymouseis is another service that allows you to surf the web without revealing any personal information. Then there are proxies. 250 Working Proxies is perhaps the biggest list of anonymous proxies. There are also search engines that don’t track users

Create a Directory of Open Source Sites

Intelligence analysts should have a directory of open source websites at the ready. A directory provides a good starting point to begin your research and should include sites like:

Places to start a general search

  • Ask
  • Bing
  • Cluuz
  • Deeperweb
  • Dogpile
  • DuckDuckGo
  • Exalead
  • Google
  • Ixquick
  • Mozbot
  • Oscobo (UK-based websearch site)
  • Qwant
  • Soovle
  • Sputtr
  • Yahoo
  • Yand ex
  • Yippy

People Searches

  • Black Book
  • Canada411
  • Intellius
  • Peekyou
  • Phonebook of the World
  • Pipl
  • Public Records
  • Rootsweb
  • Name
  • Spokeo
  • Webmii
  • Zaba
  • Zoominfo

Geolocation Searches

Social Media and Dating Sites

  • AshleyMadison
  • Bebo
  • Blogspot
  • Classmates
  • Facebook
  • Flikr
  • Google Plus
  • Hi5
  • Instagram
  • LinkedIn
  • com
  • Meerkcat
  • Meetup
  • MyLife-Reunion
  • MySpace (yes, it is still around)
  • Ourtime
  • Pinterest
  • Reverbnation
  • Snapchat
  • Social Mention
  • Sportstats
  • Tagged
  • Trendsmap
  • Twitter
  • Wayn
  • Webtsta
  • WordPress
  • YikYak
  • YouNow
  • YouTube

Background Checks

  • Accurint
  • BRBPublications
  • CourtLink
  • LexisNexis
  • PACER
  • Tracers
  • US Tax Court

Business Searches

  • Better Business Bureau
  • BizNar
  • Bloomberg Businessweek
  • Cntrl & E European Business Directory
  • Corporate Information
  • Dun & Bradstreet
  • Guidestar
  • Hoovers
  • Industry Canada
  • Mint Business Information
  • Open Corporates
  • SEC
  • SEC Company Search

It is strongly suggested you bookmark the sites you find here or on your own and keep track as new sites are created and others are deactivated. Creating a document with a list of the hyperlinked sites on your desktop, laptop or mobile devices is also advisable for quick and easy access.

Log Research and Store Sources

It is always a good idea to keep a log of your research by using Excel. The Internet is a vast space of information and that information is very fluid. Information that appears one day may not be there the next, so you need to ensure you are capturing your URLs to refer back to.

All too often, I hear analysts and students complain that they did not properly set up their research. By creating a digital case file or case manager folder, you have the ability to recall websites, photos and other documents through hyperlinking and embedding, which is never used enough. This is just another part of a well-thought-out research plan for any competent research analyst.

If you need to return to a website or page that is no longer available, you can make use of the “Wayback Machine” on archive.org. You should also be capturing screenshots of webpages while you are on them. There are many tools available at your fingertips including SnipIt, SnagIt, and other video or screen capturing software. Smartphones also have the ability to snap a photo of its screen as it appears.

Remember to Verify Information

The last point I want to make is that as with any research, there are limitations. Information may be incorrect, incomplete or inaccurate. You should always double-check your information and results.

Anyone and everyone can place information and misinformation out into the ether, so it’s important to know your source and its credibility. If you have any doubts, you should corroborate the information prior to presenting it as fact.

While open source techniques and tools are valuable, they are only as good as the analyst conducting the research. Searching and exploiting open source material are only a starting point and not necessarily the only skills you should be using and honing.

It’s important to network with others to find new and useful research methods, tools and techniques. An analyst should be constantly seeking to improve their skills and gain new ones.

Susan Hoffman is a Managing Editor at Edge, whose articles have appeared in multiple publications. Susan is known for her expertise in blogging, social media, SEO, and content analytics, and she is also a book reviewer for Military History magazine. She has a B.A. cum laude in English from James Madison University and an undergraduate certificate in electronic commerce from American Public University.

Comments are closed.