Shadow Copies: Black Box vs. White Box
A skeleton thief wants to open a sealed stone crypt inside the cemetery. In a White Box scenario, the thief has the crypt’s architectural blueprints: he sees every internal gear and gear ratio. He calculates the exact angle to turn the key and opens it instantly. In a Black Box scenario, the blueprints are locked away; the thief can only touch the outside door. To crack it, the thief goes home, builds a cheap wooden replica of the lock, and finds a sequence that opens it. When he applies the same sequence to the real stone crypt, the heavy door swings open.
Because both locks follow the same mechanics, the key for the replica fits the original.
The Scenario
When developers build proprietary AI systems, they hide the model behind an API. They think that because attackers cannot see the weights or architecture, the model is perfectly secure.
But this security boundary is mostly an illusion.
The Reality
This is the divide between White Box and Black Box attacks.
In a White Box attack, the hacker has full access to the model’s parameters and gradients. They use this data to calculate the exact, optimized noise in seconds. In a Black Box attack, the hacker has no access. They use Transferability: they train a local “surrogate model” on similar data, perform a white-box attack on their surrogate, and send the resulting adversarial examples to your API. Because different neural networks learn similar mathematical boundaries, the disguise optimized for the replica easily fools the target. Alternatively, they use Query-based attacks, sending thousands of slightly different inputs to estimate the model’s gradients from the outside.
The Defense
To prevent hackers from using surrogate models or queries to break your API, deploy these defenses:
- Rate Limiting and Detection: Monitor incoming API traffic. Block users who send thousands of highly similar inputs in short succession, which is a signature of query-based gradient estimation.
- Defensive Ensembling: Route queries through an ensemble of different model architectures. A perturbation optimized to fool a ResNet surrogate is much less likely to transfer successfully to a ViT or a ConvNeXt.
The Takeaway
Hiding your model’s code does not make it safe. If an attacker can query your system, they can build a shadow replica and find its blind spots from the comfort of their own lab.
AI specialists call it: White Box vs. Black Box Attacks & Adversarial Transferability White box attacks assume full access to model architecture and weights. Black box attacks assume no access, relying either on transferability (adversarial examples generated on a surrogate model transfer to the target) or query-based gradient estimation. Defenses include rate limiting API queries and deploying diverse ensembles of model architectures.
💬 Have you ever figured out how a hidden system worked just by watching how it responded to different inputs? What was it?
Part 7 (Black Box vs. White Box) of 8 | #AdversarialRobustnessForHumans #ai_edu Based on industry security research