Expanding ZeroFox’s Computer Vision Capabilities with Credit Card and Weapon Detection

Investing in AI and Computer Vision 

Over the last 12 months, ZeroFox has invested heavily in Artificial Intelligence, Computer Vision and Machine-Learning tools to accurately identify a broad range of digital threats at scale. Consuming the internet is no small feat and cannot be done by human analysis alone. Tools like Optical Character Recognition (OCR) to identify harmful text within images, negative sentiment analysis to more accurately understand if posts are threatening, and malicious link detection increase the speed and accuracy at which the ZeroFox Platform and our analysts are able to identify and take action against top digital threats. ZeroFox has expanded those capabilities further, going from text analysis, to text in image analysis, to now detecting threatening objects within images.

ZeroFox Object Detection identifies credit cards and weapons within images. Continuing our investment in building industry-leading AI and computer vision technology, weapon detection is a natural fit to support an important use case for corporate security, violence and physical threats, and location protection.  Expanding on the ability to detect generic objects like cars or books within images as some vendors do, ZeroFox is acutely focused on detecting objects which are more relevant to security use cases, beginning with threatening objects - guns and knives, and credit cards that put consumer financial information at risk. To bring these new capabilities into plain view, we’ve included visualizations within the Platform’s Alert View, overlaying the images with bounding boxes and labels to accurately show what was detected alongside any text content, so you can view and determine what is a directed threat vs. a harmless picture of a hunting trip.

Identifying Stolen and Leaked Credit Cards

One of the top threats facing industries like financial services includes the illegal buying and selling of credit cards and money-flipping scams across social media, surface, deep and dark web. ZeroFox's Credit Card Detection supports financial clients, where use of credit cards in images tend to be a sign of inadvertent information exposure or carding scams.

There are several ways that an image of a credit card may end up on social and digital platforms that threaten consumers and financial organizations, alike. First, a naive consumer may unwittingly post an image of their new credit card. While unlikely, it does happen - young, first-time credit cardholders, for example, may not realize the implications of sharing such an image on social media. One of the more common examples is an impersonating domain meant to look like the website of a legitimate financial institution.

Fraudulent, impersonating websites can be detected through object detection tools that identify images of credit cards used on those sites, as shown in the example above. Finally, images of credit cards can also be found on the deep and dark web as a means of selling stolen financial information.

Protecting Executives and Locations

There’s no question a threat of violence made online alongside images of guns or knives can be highly suggestive of a real attack, and something to take seriously.  Particularly when there are mentions of high profile executives and VIPs, it is worth taking a careful look at any comments to see if there may be an implied threat that may have otherwise gone undetected through traditional text analysis. Alternatively, it could be a harmless social post of a hunting trip with a protected exec, however, most security teams would err on the side of caution.

Weapon detection can be very useful when it comes to protecting locations as well.  For both corporate security and event protection, knowing if there are social posts including images of weapons near stadiums, conference centers or other site locations, which could be posted live or near live shots, is critical to informing and adjusting security posture for physical security teams. Understanding potential threats before they are actualized helps corporate security teams gain situational awareness and early warning.

In the ZeroFox Platform

The platform experience is designed to capture the full nature of the threat, giving users clear, actionable context for why ZeroFox generated the alert and why it is important. For each Weapon Detection alert, for example, the offending content is displayed to immediately show the threat posed. Bounding boxes overlayed on analyzed images to show the type of object detected by our AI model with its associated confidence level, the score outputted by our AI model indicating the likelihood that the associated object is labeled correctly. Security teams can configure the threshold to alert within the policies page, giving them more control over when the rule alerts.  Content text is displayed alongside the image to help users identify if there is a serious threat within the post associated with the image.

For alerts associated with a specific protected location, ZeroFox prominently displays the associated location so corporate security teams have necessary context. Location alerts display an interactive map, showing the protected location with radius, and specifically pinpoint where the content originated. This is particularly useful context during live events with large groups of people where the threat of violence or disturbance may be high.

Like all the capabilities within ZeroFox’s robust AI toolkit, weapon and credit card detection are services available for use in other rules, including custom rules, and in conjunction with other AI services such as our negative sentiment classifier and optical character recognition (OCR).  This gives businesses full flexibility to add the object detection tools to their larger security program or customize logic to suit individualized needs.

See the capabilities in action

Want to see these Computer Vision capabilities in the platform? We created a quick video that shows two object detection examples focused on identifying a weapon and credit card.