Recently, while writing an internal report on business plans, I found myself halfway through the draft without a clear understanding of the actual questions I was trying to answer. After a ‘gentle’ nudge from a colleague, I needed to step back and clarify my purpose. It was a great reminder - it's so crucial to clearly identify the right questions upfront.
Asking good questions isn’t just important in OSINT—it’s critical in almost any field. If we don't ask the right questions upfront, we risk chasing irrelevant data, wasting time, and missing crucial insights. Havard Business Review talk about the "Surprising Power of Questions" in a great article. It won’t be a surprise for experienced analysts, but I would argue a very helpful reminder.
In this blog, we will cover:
The importance of asking good questions.
Strategies for asking better questions of AI systems, such as ChatGPT.
Evolving questions as new information emerges.
Techniques for refining intelligence questions during the planning phase.
Methods for using questions to analyse and identify key points.
By the end, you'll have a solid framework for asking precise, targeted questions that will improve your OSINT investigations and help you better leverage AI systems.
Navigating OSINT Headwinds by Framing the Right Questions
At the Australian OSINT Symposium, there was a lot of discussion around the growing challenges facing OSINT professionals. These challenges (and the opportunities) are significant, but they also reinforce the need for asking good questions. In the intelligence process, it’s not enough to simply collect information. One of the key skills lies in framing the problem correctly from the beginning. This ensures that:
You drive effective information collection.
You challenge assumptions that may be baked into your analysis.
You frame the analysis in a way that highlights the most relevant findings.
And finally, you communicate that analysis clearly to decision-makers.
By focusing on the right questions, we can ensure that we deliver relevant, high-quality intelligence and produce actionable insights.
Asking Questions of AI Systems
Just like with human analysts, asking good questions is absolutely crucial when working with AI systems, and maybe even more important. The same principles that guide effective information collection and analysis apply when interacting with AI, but there’s an added layer: asking good questions transforms your relationship with AI from a passive recipient to an active collaborator. This allows you to maximise AI's strengths and, more importantly, mitigate its limitations. So, what are the key tactics to use when going back and forth with AI systems?
Clarity and Specificity. Always aim for clear and specific questions. Just like you wouldn’t ask a colleague, “Tell me about cybersecurity,” you shouldn’t be vague with AI either. For instance, asking, “What are the most significant cybersecurity trends expected to shape the industry in 2024, particularly in areas like threat detection?” will give you much more focused and actionable information. The clearer you are, the better the AI can understand and deliver a relevant response.
Avoid Leading Questions. Leading questions are a trap for both humans and AI. They can push the conversation toward confirming a specific viewpoint, which can reinforce bias. For example, asking, “AI always enhances productivity, doesn’t it?” narrows the scope of the response. Instead, go for something open-ended like, “What are the potential advantages and disadvantages of using AI for productivity?” This encourages the AI to present a balanced view, helping you avoid confirmation bias.
Be Conscious of Context. AI performs best when it has a clear understanding of the context. The more context you provide, the better tailored the response will be. For example, instead of simply asking, “Can you compile a list of at least 10 unique terms related to X-based cyber actors?” you could ask:
"Can you compile a list of at least 10 unique terms related to X-based cyber actors, with a focus on both state-sponsored groups and underground criminal hacking organisations? The terms should include a mix of slang, codenames, tools, and methods, particularly those relevant to ransomware attacks in 2024. All of the terms should be in Russian, with explanations provided in English. Each term should include a brief explanation of its relevance to X cyber operations, including notable tools, ransomware campaigns, or tactics from 2024."
This gives the AI a clear scenario to work with. Without that context, the response might be too generic or miss the nuances specific to your needs.
Refining Intelligence Questions in Planning
Ambiguity is a common challenge in intelligence analysis, and clear questions help cut through that uncertainty. Ambiguity often appears as unclear or complex intelligence tasks, which can lead to confusion about what’s really being asked. As an analyst, you may need to seek additional information from your customers to clarify the task. This can be tricky, especially when you don't have direct access to those customers, but it's essential to avoid wasted effort and ensure the analysis stays focused.
Refining our intelligence questions helps ensure that our research, collection, and analysis are relevant to the task at hand. This not only prevents us from falling down unnecessary rabbit holes but also reduces the risk of over-collection, which can clutter the intelligence picture. Let's look at an example, in this case, we'll consider questions an intelligence analyst or manager might ask when investigating insider threats:
Is there an insider threat in our organisation?
becomes (after refining)
What external indicators suggest the presence of an insider threat, and are there patterns of suspicious behaviour or external associations that align with known threat profiles?
How do we identify insider threats?
becomes
What behavioural patterns, external affiliations, or online activities could indicate the presence of a potential insider threat within our organisation?
What damage can insider threats cause
becomes
What types of sensitive information related to our organisation have been exposed or discussed on open or restricted-access platforms, and what financial, operational, or reputational damage could result from these disclosures?
These refined questions are more focused and add specificity, giving you a clearer path to collect better information, leading to actionable intelligence. They should also prompt you to think about the scope of your inquiry—should you seek clarity on timeframes, parameters, or other considerations? Does the language in the original question feel too vague or open-ended? These are important factors to consider when refining intelligence questions in the planning phase.
By refining the questions early on, you set yourself up for more efficient data collection and analysis, ensuring that your work is aligned with the customer's needs and avoiding unnecessary detours.
Of course, work with your team (or AI tools) to brainstorm a range of questions. These can start broad (so long as they are clear, specific, and contextual) to help you see the big picture, but it is important to then narrow down, depending on your requirements.
Evolve as new information emerges
In open-source intelligence analysis, it’s easy to get locked into the questions we initially frame. While starting with well-structured questions is important, we also need to stay flexible as new information comes in. Being rigid with your questions can cause you to miss key insights. So, how do you do this?
Revisit and evolve your questions as you gather more data. After each major phase of data collection or analysis, take a step back and look at your original questions. Has new information shifted your perspective? Don’t be afraid to adjust your questions as you go along, so you don’t end up chasing irrelevant details.
Example: Maybe you started with a focused question like, “What are the supply chain risks for our main drone component supplier in Asia?” But then you uncover an emerging geopolitical issue affecting trade routes. It’s important to step back and ask a broader question like, “What other factors, beyond the supplier, could disrupt our supply chain in Asia?” This way, you’re not caught off guard by developments outside your initial scope.
Always ask, “What are we not seeing?" A key part of analysis is questioning what’s missing. Regularly ask yourself, your team, and maybe even your AI system, “What questions aren’t we asking?” or “What are other questions related to this topic”. This prevents you from getting tunnel vision on a single line of inquiry.
Using Questions to Analyse and Identify Key Points
The raw data collected to answer an intelligence question is just that—raw data. To effectively address an intelligence issue, that data needs to be processed and analysed. This requires a mix of skills, experience, and knowledge. At a fundamental level, though, it all starts with asking the right questions—not just about the issue but also about the sources you've gathered. A great way to do this is to apply the 5W1H framework.
The 5W1H questions (Who, What, Where, When, Why, and How) are a great starting point for looking at both the broader intelligence issue and the specific data you’ve collected. By systematically working through these, you can turn a large amount of raw information into clear insights.
Once you’ve reviewed your collected information, the next step is to sort, prioritise, and discard what isn’t relevant. This is where you narrow your focus, distill your research, and get ready to draw conclusions and make recommendations. Let's look at another example, in this case we’re investigating the supply chain risks for a company that builds and distributes drones. The intelligence question we are trying to answer in this scenario is:
"What are the key supply chain risks that could disrupt the production and global distribution of unmanned fixed-wing aircraft?"
We’ve gathered data from a range of sources—reports on key suppliers, logistical pathways, potential vulnerabilities, and geopolitical risks that could affect the supply chain. After an initial review, we’ve identified that the information is relevant to our intelligence question. Now it’s time to dig deeper. And, a great way to dig deeper is to answer a series of 5W1H questions to address supply chain risks, such as:
What critical components are required for the manufacturing of unmanned fixed-wing aircraft, and what vulnerabilities exist in sourcing these parts?
What potential supply chain disruptions have been flagged in media reports regarding the global drone industry?
Who are the key suppliers for essential components, and are there concerns about their stability or reliability?
Who could exploit vulnerabilities in the supply chain, whether competitors, nation-states, or cybercriminal groups?
Where are the primary suppliers and manufacturers located, and do their regions face geopolitical or environmental risks that could disrupt production?
Where are the company’s logistical bottlenecks that could delay the delivery of critical materials?
When have previous supply chain disruptions occurred in this industry, and what triggered them (e.g., regulatory changes, natural disasters, conflicts)?
When do key suppliers face seasonal or cyclical challenges that might impact material availability or delivery?
Why might suppliers be at risk of cyberattacks or physical disruptions, and how would this impact production?
How could external factors—like trade policies, tariffs, or sanctions—affect the availability or cost of materials?
How are components transported from suppliers to manufacturing sites, and where are the vulnerabilities in these logistical chains (e.g., ports, borders)?
The answers to these questions will help identify the key risks in the supply chain. Once you’ve worked through them, it’s important to distil your findings into 2-3 key points. Writing these down clarifies your thinking and ensures you stay focused on the intelligence question.
Asking these questions may also reveal gaps in your understanding or missing information. For instance, you might notice that there's limited data on alternative suppliers for critical components or a lack of clarity around regulatory changes that could affect transportation. Identifying these gaps early helps you refine your intelligence question and direct further research or data collection where it’s needed most.
Summary
At the end of the day, intelligence analysis is all about asking the right questions. Whether you're assessing supply chain risks or investigating ransomware groups, the process of refining your questions and digging into the data is what leads to real, actionable insights. The 5W1H framework is a great tool to keep your analysis focused and relevant, helping you avoid getting bogged down in irrelevant details.
Remember, intelligence work doesn’t stop at collecting information. It’s about understanding what that information means, how it impacts the bigger picture, and what steps need to be taken next. Refining your questions, identifying key points, and spotting gaps in your knowledge will make sure you're always staying on track and delivering valuable insights.
So, as you tackle your next intelligence challenge, keep this in mind: the better the questions, the better the answers.