The Incident Response Team Episode

What is generative AI?: Generative AI is a subset of artificial intelligence focused on creating new content based on existing content. The core technology behind it is Large Language Models (LLMs), which are sophisticated software systems that predict a sequence of words based on a given prompt. These are not magic but rather complex tools that require proper management.

How do you keep up with evolving AI?: It’s a significant challenge as security teams often lag behind business units in adopting new technologies. A common, but ineffective, reaction is to ban AI tools like ChatGPT. This often leads to “shadow IT,” where employees use unmanaged tools anyway, putting company data at risk. The better approach is to establish a consolidated and logical framework for using these technologies safely.

Pros and cons of AI for businesses?:

• Benefits: The primary advantage is the ability to process large amounts of unstructured data (such as emails, Slack messages, and documents) and automate tasks that were previously manual and time-consuming, leading to significant productivity gains.

• Disadvantages (Old Challenges): Using third-party AI tools (from OpenAI, Google, Microsoft, etc.) introduces supply chain security risks. Companies must rely on these vendors to protect their data, which requires proper vetting, understanding their data retention policies, and ensuring adequate security measures are in place.

• Disadvantages (New Challenges): By default, many AI models use the data they are fed for training purposes. This can lead to the AI learning and potentially exposing sensitive company information, intellectual property, or even internal processes like interview questions, as was reportedly the case with Amazon.

What is StackAware’s AI protocol?:

• The protocol is a comprehensive checklist and standard operating procedure (SOP) that provides a structured approach for companies to manage AI risks.

• It emphasizes creating a specific, detailed policy for generative AI rather than just giving vague guidelines.

• It includes steps for vendor management, risk registration, business analysis (cost-effectiveness), and privacy reviews.

• Aggregation Risk: The protocol specifically calls out the need to analyze for “aggregation risk,” which is when an AI tool combines multiple, seemingly innocuous pieces of public information to infer sensitive, non-public details about a company.