How to Discover Shadow AI in Your Organization

Tactical Guide

How to Discover Shadow AI

Practical methods to inventory unauthorized AI usage across your healthcare organization

The Discovery Challenge

Shadow AI is designed to be invisible. These are web-based SaaS tools accessed through personal accounts, personal credit cards, and consumer-grade services.

Traditional IT discovery methods (network monitoring, procurement records, endpoint management) won’t catch them. You need a different approach.

5 Discovery Methods

Combine multiple approaches to get a complete picture of shadow AI usage

Anonymous Staff Surveys

Easy

1-2 weeks

Send organization-wide surveys asking staff to self-report AI tool usage in a non-punitive, anonymous way

How to do it:

Frame it as ‘helping us enable AI safely’ not ‘catching violations’
Ask: What AI tools do you use? How often? For what tasks?
Promise no individual consequences—focus on organizational learning
Offer small incentive (gift card raffle) for completion

Effectiveness:

70-80% of usage discovered

Pros:

Fast, cheap, builds trust

Cons:

Self-reported data may be incomplete

Department Interviews

Medium

2-4 weeks

Conduct structured interviews with department leaders and frontline staff across clinical, administrative, and revenue cycle teams

How to do it:

Interview 2-3 people from each major department
Ask about productivity pain points and workarounds
Listen for AI tool mentions (ChatGPT, Claude, Grammarly, transcription services)
Document workflows where AI could be or is being used

Effectiveness:

60-70% of usage discovered

Pros:

Deep qualitative insights, relationship building

Cons:

Time-intensive, requires skilled interviewer

Network Traffic Analysis

Hard

1 week

Analyze DNS logs and firewall traffic to identify connections to known AI service domains

How to do it:

Pull 30 days of DNS logs from your firewall/proxy
Search for domains: openai.com, anthropic.com, claude.ai, gemini.google.com, etc.
Look for unusual traffic spikes to AI service providers
Correlate by department, time of day, user segments

Effectiveness:

40-50% of usage discovered

Pros:

Objective data, hard evidence

Cons:

Misses personal devices, VPNs, encrypted traffic

Browser Extension Audit

Medium

1 week

If you use endpoint management, audit installed browser extensions for AI writing assistants and productivity tools

How to do it:

Export list of all Chrome/Edge extensions from endpoint management
Flag AI-related extensions: Grammarly, Jasper, Copy.ai, Notion AI, etc.
Check for ChatGPT desktop apps, Claude desktop apps
Document which departments have highest adoption

Effectiveness:

30-40% of usage discovered

Pros:

Specific tool identification

Cons:

Only catches managed devices, misses web-only usage

Credit Card & Expense Review

Easy

1 week

Review corporate credit card statements and expense reports for AI tool subscriptions

How to do it:

Pull 6 months of expense data
Search for merchant names: OpenAI, Anthropic, Jasper, Copy.ai, etc.
Look for recurring monthly charges ($20-50 range)
Note: Most shadow AI is on personal cards, so this catches <10%

Effectiveness:

10-20% of usage discovered

Pros:

Easy to run, identifies paid subscriptions

Cons:

Misses majority of personal-account usage

Recommended Approach

Combine three methods for maximum coverage

1

Start with Anonymous Survey (Week 1)

Fastest way to get broad visibility. Most staff will self-report if framed correctly.

2

Run Network Traffic Analysis (Week 1-2)

Validates survey data and catches usage staff forgot to mention or didn’t realize counted as “AI”.

3

Follow Up with Department Interviews (Week 2-3)

Deep dive into high-risk or high-usage departments to understand workflows and PHI exposure.

Result: 80-90% coverage of shadow AI usage in 2-3 weeks, with both quantitative data and qualitative context.

What to Document

Not just a compliance issue. This is an existential risk for all organizations

Field 1

AI Tool Name

Example: ChatGPT, Claude, Gemini, Grammarly