In a world where information is abundant and easily accessible, OSINT emerges as a vital resource. But what exactly is OSINT?
Open-source intelligence (OSINT) is the term for collecting and analyzing publicly available data to generate actionable intelligence. This data can come from various sources, such as:
For security professionals, OSINT offers a large amount of information that can help to preempt threats, understand adversaries, and protect digital assets more effectively.
Why should security professionals care about OSINT? The answer lies in its ability to provide comprehensive insights with minimal cost and effort.
In the digital world, information is often said to be equal to power. Having access to a wealth of open data means you can:
Here are some of the main benefits of adopting OSINT approaches:
Enhanced threat detection. One of the primary benefits of OSINT is its capacity for early threat detection and threat hunting. With the added information gathered from OSINT sources, security teams can identify suspicious activities or emerging threats before they escalate.
For example, detecting chatter on forums about a new exploit targeting specific software can prompt preemptive measures to protect systems.
Cost-effective solution. Compared to proprietary intelligence services, OSINT is incredibly cost-effective. Since it leverages publicly available information, organizations can gather a wealth of data without significant financial investment.
Comprehensive coverage. OSINT provides a wide array of data sources, offering a more holistic view of the threat landscape. This comprehensive coverage allows security teams to build a more complete picture of potential risks.
(Related reading: threat modeling & threat analysis.)
OSINT experts tend to look for information from a few key sources. Here are some of them:
Social media is a goldmine for OSINT. Threat actors often share information unwittingly on platforms like Twitter, LinkedIn, and Facebook.
(Related reading: data mining & text mining.)
Public databases, such as WHOIS records, offer detailed information about domain registrations. This can be crucial for identifying malicious domains or tracking the digital footprint of threat actors. Additionally, government and academic databases can provide context-specific intelligence.
Here is a table of common sources as originally published in our Workflow Actions & OSINT for Threat Hunting blog:
Type
|
Site
|
IOCs
|
Description
|
IP/Domain/
Shared Domains on IP Address
|
|
IPs, Domains
|
One of the best of breed tools to investigate Domains, IP addresses and more.
|
IP/Domain Information
|
|
IPs, Domains
|
Investigate Domains and IP addresses.
|
Geolocate IPs/Domains
|
|
IPs, Domains
|
Quick way to find the most up-to-date location of a IP from several different vendors.
|
Geolocate IPs/Domains
|
|
IPs, Domains
|
Shows location and provides a nice map.
|
PassiveDNS, SSL Certificates, Shared Domains on IP address
|
|
IPs, Domains
|
Research Domains, IPs, passive DNS sources, SSL certs, and more. Sign up for a free license.
|
SSL Certificates
|
|
SSL Certificate Hashes
|
Scans the internet on a daily basis and allows researchers to search their library for information on SSL certs and more.
|
Historical Whois information
|
|
Domains, Emails, Keywords
|
Search historical whois information.
|
Passive DNS
|
|
IPs, Domains,
|
Look up domains and IPs and recent resolutions without performing an actual DNS query.
|
Malware
|
|
File Hashes
|
Free malware analysis service that allows you to submit files to an open source malware sandbox and search results with an account.
|
Malware
|
|
File Hashes
|
Free malware analysis service that allows you to submit files to an open source malware sandbox and search results
|
Malware (and more)
|
|
File Hashes, IP addresses, Domains
|
Best of breed free malware analysis service that allows you to submit files to an open source malware sandbox and search results. Users can submit URLs and files TO virustotal but this may result in tipping off adversaries to your action… Usually I recommend just passive research on VT.
|
Domain
|
|
File Hashes, IP address, Domains
|
Search engine for threat data and open source intelligence reports and other cyber security sources
|
URLs
|
|
URLs
|
Submit an URL and it will visit the site, take a snapshot, and analysis it to see if it is malicious. Beware of using this to analyze a link unless you are ok with tipping your hand to the adversary
|
Search engine
|
|
Any field
|
Google. No discussion needed. However, I’d recommend disabling pre-fetch https://www.technipages.com/google-chrome-prefetch
|
Code
|
|
Any field
|
Github is one of the largest code repositories on the internet. Often you can find interesting strings in the logs that may be in adversaries (or tool creators) Github repo.
|
Domains, whois
|
|
IPs, Domains,
|
Best of breed for researching DNS history. For a fee, you can setup DNS branding detection and registration history of domains.
|
BGP/ASN
|
|
IPs
|
Often adversaries utilize the same ASN but different IP addresses. It can be worthwhile to find “malicious” ASNs and alert on them.
|
PassiveDNS and more
|
|
IPs, Domains, Names
|
Provides several different DNS research tools. Can find out registrant histories of domains.
|
Malware
|
|
IPs, Domains, File Hashes
|
One of the largest collections of malware on the internet. Great searching capabilities.
|
APT reports
|
|
Any IOC or key word
|
Threatminer combines different threat feeds and a searchable repository of APT reports.
|
IP
|
|
IPs
|
Lightweight site that can quickly find out basic info regarding an IP address.
|
Monitoring news outlets and online forums can reveal emerging threats and trends. Cybersecurity forums can offer insider perspectives on vulnerabilities and exploits. Staying updated with these sources ensures that security teams are aware of the latest developments in the cyber threat landscape.
Here are some examples:
Let’s look at some tools that help gather and harness OSINT.
Maltego is a powerful tool for visualizing the relationships between different data points in a network graph. It enables security professionals to map out connections between domains, IP addresses, and social media profiles, making it easier to identify patterns and potential threats.
Shodan is a search engine for internet-connected devices. It allows users to discover vulnerable devices within their network and assess the potential risk. By using Shodan, security teams can proactively secure exposed systems before they are exploited.
Google Dorking involves using advanced search operators to find specific information on the web. This technique can help to discover information not typically shown on search pages through regular search queries. Some common operators used in Google are:
theHarvester is an OSINT tool designed for gathering email addresses, subdomains, hosts, employee names, open ports and banners from different public sources like search engines, PGP key servers, and SHODAN computer database.
With our understanding of OSINT and how and where we can source this information, let’s now look at how you can use OSINT within the enterprise.
In the aftermath of a cyber incident, OSINT can play a crucial role in incident response.
Through information from open sources, security teams can rapidly piece together the attacker's methods, motives, and potential next steps. This accelerates the response process and minimizes damage.
OSINT is invaluable for conducting vulnerability assessments as well. This works by identifying any weaknesses in their organization's digital footprint. This proactive approach allows for timely remediation of vulnerabilities before they are exploited.
(Related reading: vulnerability types.)
Cybersecurity isn’t the only area to use OSINT. In fact, OSINT can provide competitive intelligence. Monitoring public information about competitors' activities, strategies, and market trends can offer valuable insights for strategic decision-making. This dual application of OSINT makes it a versatile tool for business intelligence.
Adopting OSINT isn't a perfect solution, so you should be expecting some of the following challenges:
With the vast amount of information available, one of the biggest challenges in OSINT is data overload. Security teams must sift through mountains of data to find relevant and actionable intelligence. This is especially tough for investigating sophisticated threats with a large amount of historical data to look at.
Effective data management and prioritization are crucial to overcoming this challenge. In this situation, try to use a combination of tools to narrow down the correct and relevant information.
For example, use Google dorking to filter out irrelevant results and then use Maltego to visualize the remaining data for better analysis.
Not all open-source data is accurate or reliable. This is one of the main concerns of open-source data — that anyone can upload and edit information. Disinformation and misinformation can skew analysis and lead to incorrect conclusions.
This makes it crucial to verify any information gathered from OSINT sources. One way to combat this challenge is by cross-referencing information from multiple sources and comparing data for inconsistencies or discrepancies.
Using OSINT also raises legal and ethical considerations. While the information is publicly available, its use must comply with privacy laws and ethical guidelines. Organizations must establish clear policies to ensure responsible and lawful use of OSINT.
According to the Public-Private Analytic Exchange Program (AEP), intelligence gathered from open sources must not violate existing privacy laws, must not be used maliciously, and must be done only when necessary.
(Related reading: data privacy.)
Lastly, let’s look at a few ways OSINT will increasingly be harnessed in coming months and years.
Ethical hacking, or penetration testing, often incorporates OSINT to identify potential entry points for attacks. Through the use of OSINT, ethical hackers can uncover vulnerabilities before they are exploited. This proactive approach enhances an organization's security posture.
This practice also aligns with ethical guidelines set forth by organizations such as the International Association of Certified Ethical Hackers (IACEH). These guidelines emphasize the importance of obtaining proper authorization and consent before conducting any OSINT investigations.
Furthermore, ethical hacking with OSINT can also assist in identifying potential insider threats within an organization. By monitoring public information, such as social media
The integration of AI and machine learning with OSINT is poised to revolutionize cybersecurity. These technologies can automate data collection and data analysis, providing faster and more accurate intelligence. Predictive analytics can also anticipate future threats based on historical data.
Large language models (LLMs) like ChatGPT also make it easier for the common public to access a large amount of information across the web. They can analyze vast amounts of data quickly based on a highly specific prompt (when required), making OSINT more efficient. However, usage of such open-source tools comes with their own terms of use and, therefore, some limitations.
(Related reading: can LLMs be secure?)
Advanced visualization tools are making it easier to interpret complex data. Interactive dashboards and graphical representations can highlight patterns and correlations that might be missed in raw data. These tools enable more effective decision-making based on OSINT.
Open source intelligence (OSINT) is a powerful tool that can be used in the right manner. For organizations that stand to benefit from valuable insights into the cyber threat landscape, incorporating OSINT into their security strategy is a must. However, do consider the challenges and be cautious about the ethical guidelines for responsible and effective use of OSINT.
See an error or have a suggestion? Please let us know by emailing ssg-blogs@splunk.com.
This posting does not necessarily represent Splunk's position, strategies or opinion.
The Splunk platform removes the barriers between data and action, empowering observability, IT and security teams to ensure their organizations are secure, resilient and innovative.
Founded in 2003, Splunk is a global company — with over 7,500 employees, Splunkers have received over 1,020 patents to date and availability in 21 regions around the world — and offers an open, extensible data platform that supports shared data across any environment so that all teams in an organization can get end-to-end visibility, with context, for every interaction and business process. Build a strong data foundation with Splunk.