The Pentest That Found Nothing (And Why That's Bad)
A startup proudly shared their penetration test results: zero findings. Clean bill of health. But when I looked at the scope, the testers only checked their marketing website— not the application where customer data lives. They paid $15K for security theater.
Penetration testing is one of the most valuable security investments you can make. It's also one of the easiest to get wrong. A good pentest finds real vulnerabilities. A bad one gives false confidence.
This guide shows you how to buy, scope, and use penetration testing effectively—so you get actual security value, not just a report for your compliance folder.
Types of Penetration Testing
Most startups should start with web application testing. That's where your customer data lives and where most attacks target SaaS companies. External network testing is useful but often finds less in cloud-native architectures.
Scoping Your Pentest
What to Include
- Production Application — The actual app your customers use (or a staging mirror)
- All User Roles — Test as admin, regular user, and unauthenticated
- APIs — Both documented and undocumented endpoints
- Authentication Flows — Login, password reset, MFA, SSO
- Third-Party Integrations — OAuth connections, webhooks, file uploads
- Admin Interfaces — Internal dashboards, admin consoles
Common Scoping Mistakes
- Marketing Site Only — Testing your WordPress blog while ignoring your app
- Single User Role — Only testing as unauthenticated misses privilege escalation
- Ignoring APIs — API vulnerabilities are the #1 SaaS attack vector
- Excluding Staging — If it mirrors production, it needs testing
- Time-Boxing Too Tight — 2 days isn't enough for a complex application
Your scope should explicitly list: URLs/IPs in scope, user accounts provided (with roles), testing methodology, out-of-scope items, testing window, and point of contact for issues. Ambiguity in scope leads to missed vulnerabilities.
Choosing a Pentest Provider
Types of Providers
Questions to Ask Providers
- Who will actually do the testing? — Ask for tester bios and certifications.
- What methodology do you use? — OWASP, PTES, or custom?
- What's included in the report? — Executive summary, technical details, remediation guidance?
- Do you offer retest? — Verification that fixes work is valuable.
- What's your communication process? — How will critical findings be reported?
- Can you share sample reports? — Quality varies widely.
Ask who specifically will test your application. Firms often sell senior expertise but send junior testers. Get names and backgrounds. A senior tester at a boutique firm often beats a junior at a Big 4.
During the Pentest
Preparation
- Create Test Accounts — Accounts for each user role, clearly labeled
- Provide Documentation — API docs, architecture diagrams help testers go deeper
- Whitelist Tester IPs — So they don't get blocked by WAF/rate limiting
- Notify Your Team — SOC/IT should know testing is happening
- Identify Point of Contact — Someone available for questions during testing
Communication
- Daily Standups — Brief check-ins help redirect effort if needed
- Critical Finding Protocol — How to handle urgent vulnerabilities (call immediately?)
- Questions Channel — Slack/email for tester questions about functionality
Understanding the Report
Severity Ratings
Report Red Flags
- All Automated Findings — Copy/paste from scanner output isn't a pentest
- No Business Context — Generic severity without impact analysis
- Missing Reproduction Steps — You can't fix what you can't reproduce
- No False Positive Review — Scanners have false positives; testers should verify
- Zero Findings — Either scope was too narrow or testing was superficial
After the Pentest
Remediation Process
Triage Findings
Review with your team. Validate findings. Assess actual business impact. Prioritize remediation.
Create Tickets
Convert findings to actionable work items. Assign owners. Set due dates based on severity.
Fix and Verify
Implement fixes. Have another engineer verify. Document the change.
Request Retest
Have the pentest firm verify fixes work. This catches incomplete remediation.
Update Risk Register
Document accepted risks if any findings won't be fixed. Know your residual risk.
Pentest Frequency
Annual Minimum:
- Comprehensive application test
- External network assessment
- Required by SOC 2, many customers
Consider More Frequent:
- After major releases or architecture changes
- High-risk industries (finance, health)
- Continuous testing programs
Common Pentest Mistakes
Mistake 1: Cheapest Option
A $2K pentest is probably automated scanning with a nice cover page. Real manual testing by skilled professionals costs more. You get what you pay for.
Mistake 2: Testing Once, Never Again
Your application changes constantly. A pentest from 18 months ago doesn't reflect current risk. Annual testing is minimum; after major changes is better.
Mistake 3: Report Goes in a Drawer
A pentest only provides value if you fix the findings. Track remediation, verify fixes, close the loop. An unaddressed critical finding is worse than not knowing.
Mistake 4: No Retest
Developers think they fixed it. Did they? Retest verification catches incomplete fixes and regressions. Budget for it.
Quick Start: Your First Pentest
Week 1: Define Scope
What needs testing? List applications, APIs, user roles. Define what's out of scope.
Week 2: Select Provider
Get 2-3 quotes. Ask about testers, methodology, and sample reports. Check references.
Week 3: Prepare
Create test accounts, gather documentation, whitelist IPs, notify your team.
Week 4-5: Testing
Testing executes. Daily check-ins. Address critical findings immediately.
Week 6+: Remediation
Review report, prioritize fixes, remediate, retest.
Next Steps
Penetration testing is one of the highest-value security investments—when done right. Proper scoping, qualified testers, and actual remediation turn a compliance checkbox into real security improvement.
Start with your web application. That's where the risk is for most SaaS companies. Scope it properly, hire qualified testers, and fix what they find.
Planning your pentest program? vCISO Lite helps you track pentest findings, manage remediation, and demonstrate testing history to customers and auditors—a key SOC 2 requirement.