Introduction

When faced with a suspicious file, a systematic and cautious approach is crucial to preventing potential system compromise. Here’s an expanded guide, drawing from expert techniques and industry-standard tools, to help you analyze potentially malicious files safely and effectively.

Step 1: Absolute Isolation – The Golden Rule

The first and most important rule is to not execute the file directly on your primary operating system. This seemingly obvious step is paramount in preventing the compromise of your Windows 11 system. Despite having defences you cannot be assured that all malware will be detected by your system.

  • Quarantine the file: Immediately move the suspicious file to a dedicated, isolated folder. This prevents accidental execution and contains any potential initial infection.
  • Disconnect from the network: Disconnecting your system from the internet is critical to prevent the file from performing malicious activities like data exfiltration, downloading additional payloads, or communicating with command-and-control servers. If you believe you are infected, then disconnect from all networks.
  • Stay Online: If you wish to investigate the file further, you can stay online. If you have the computing resources, consider building a specific virtual machine environment for investigating malware.

Step 2: Unmasking the File Extension

Attackers often use double or deceptive file extensions to trick users into executing malicious files.

  • Double extensions: Pay close attention to filenames with double extensions, such as “invoice.pdf.exe”. In this case, the file is an executable (.exe) disguised as a PDF.
  • Uncommon extensions: Be wary of less common file extensions, especially those associated with scripting languages (.vbs, .js, .ps1) or archive formats (.zip, .rar, .7z). These file types can contain malicious code or executables.
  • .docm files: These are macro-enabled Word documents, and macros can be exploited to run malicious code. Enable macros only if you trust the source and content of the document.

Step 3: Leveraging VirusTotal’s Collective Intelligence

VirusTotal is an invaluable resource for quickly assessing the potential threat of a file.

  • Upload and scan: Upload the suspicious file to VirusTotal (www.virustotal.com) and allow it to be checked by over 90 antivirus engines.
  • Analyze the results:
    • High detection ratio: Many antivirus engines flagging the file indicate a strong likelihood of malware.
    • Low detection, reputable vendors: Even with a low overall detection rate, if reputable antivirus vendors like Microsoft, Kaspersky, or CrowdStrike flag the file, exercise extreme caution. These vendors often have advanced detection capabilities.
    • Zero-day malware: VirusTotal may not detect the latest zero-day malware or files using advanced obfuscation techniques. If the file’s behavior seems suspicious, even with a clean VirusTotal report, proceed with further analysis.

Step 4: Diving into Metadata and Static Analysis

Examining a file’s metadata and performing static analysis can reveal valuable clues without executing the file.

  • ExifTool: This versatile tool inspects metadata in various file formats.
    • Download ExifTool from Phil Harvey
    • It supports various metadata formats, including EXIF, GPS, IPTC, XMP, and JFIF.
    • Download the appropriate package for your OS. Remember to move the “exiftool_files” folder to the same location as the “exiftool.exe” file for Windows.
    • Use ExifTool to identify suspicious author names, unusual creation dates, or embedded links.
  • Strings: This utility extracts ASCII and Unicode strings from a file.
  • PEStudio: A tool designed for static analysis of Windows executable files (.exe, .dll).
    • Download PEStudio and drag the executable file into the program interface.
    • PEStudio displays information about the file’s structure, imported and exported functions, libraries used, and other relevant details.
    • It can also grab information from VirusTotal, allowing users to quickly determine whether any antivirus applications have flagged the file.
  • Red flags to look for:
    • Suspicious author names: Names like “Hacker1337” or other malicious handles are strong indicators.
    • Weird embedded links: URLs that point to suspicious domains or unexpected locations.
    • Obfuscated code: Code that has been intentionally scrambled to make it difficult to understand, often using techniques like base64 encoding or ROT13.

Step 5: Controlled Execution in a Virtual Machine

Running the suspicious file in a virtual machine (VM) lets you observe its behavior in a safe, isolated environment.

  • VM Setup:
    • To create a virtual machine, you can use virtualization software such as Hyper-V, VirtualBox, or VMware. My preference is Hyper-V since it is already built into Windows.
    • Configure the VM with an operating system similar to the intended target (e.g., Windows 10 for analyzing Windows malware).
    • Snapshot: Take a snapshot of the VM before running the file. This allows you to revert to a clean state if the file is malicious.
    • Network Isolation: Disconnect the VM from the internet to prevent the malware from communicating with external servers or spreading to other systems.
  • Specialized VMs:
    • FlareVM: A Windows-based VM pre-configured with a comprehensive suite of malware analysis tools.
    • REMnux: A Linux-based VM designed explicitly for malware analysis.
  • Monitoring VM activity:
    • Observe the file’s behavior closely after execution.
    • Look for attempts to modify system files, create new processes, modify the registry, or establish network connections.
    • Use tools within the VM to monitor system activity and network traffic.

Step 6: Automated Analysis with Sandboxes

Online sandboxes provide an automated way to analyze a file’s behavior without requiring manual setup.

  • Popular Sandboxes:
    • Any.Run: An interactive online sandbox that allows you to control the file’s execution and observe its behavior in real time.
    • Hybrid Analysis: A detailed static and dynamic analysis platform providing comprehensive file behavior reports.
    • Joe Sandbox: A comprehensive malware analysis service that provides detailed behavioral reports and threat intelligence.
  • Analyzing Sandbox Reports:
    • Review the sandbox report for suspicious activities, such as:
      • Attempts to modify system files or registry keys.
      • Connections to unknown or malicious IPs/domains.
      • Creation of new processes or scheduled tasks.
      • Download of additional files.
      • Installation of software.

Step 7: Deep Dive – Monitoring Network Connections

If the file exhibits network activity, monitoring these connections is crucial to understand its purpose and potential impact.

  • Wireshark: A powerful network packet analyzer that captures and analyzes network traffic.
    • Use Wireshark to examine the file’s network connections, identify the destination IPs/domains, and analyze the transmitted data.
    • Look for traffic to known malicious IPs or domains.
  • Process Monitor (Procmon): A Windows Sysinternals tool that monitors file system, registry, and process activity in real-time.
    • Use Procmon to track system activity and identify any suspicious file system or registry modifications.

Step 8: Post-Analysis and Remediation

Once you have completed your analysis, take appropriate action based on your findings.

  • If the file is deemed malicious:
    • Delete the file securely.
    • Run a full system scan with your antivirus software.
    • If the analysis was performed on a VM, revert to the clean snapshot or delete the VM.
    • Report the file to VirusTotal or other threat intelligence platforms to help protect others.
  • If the file is deemed safe:
    • Exercise caution when opening the file, even if it appears benign.
    • Keep your antivirus software up to date.
    • Be wary of any unexpected behavior after opening the file.

Consider Windows Sandbox

Windows Sandbox provides a safe, isolated desktop environment in which to run applications. You can execute and monitor applications while removing them by simply closing the Sandbox window.

  • Windows Sandbox is included in Windows 10 Pro, Enterprise, and Education editions.
  • It offers a lightweight, disposable environment for testing applications and exploring unknown files.
  • Any software installed in the sandbox remains isolated from the host machine.
  • Closing the sandbox deletes all software and files, providing a clean instance each time.
  • Windows Sandbox enables network connections by default. However, the Windows Sandbox configuration file can disable them. Enabling networking can expose untrusted applications to the internal network.

Final Reminder

Analyzing suspicious files requires caution, technical skill, and the right tools. By following these steps, you can significantly reduce the risk of infection and gain valuable insights into the behavior of potentially malicious software. Remember, when in doubt, it’s always best to err on the side of caution and delete the file.

Resources:

TheZoo Malware Archive: GitHub – ytisf/theZoo: A repository of LIVE malwares for your own joy and pleasure. theZoo is a project created to make the possibility of malware analysis open and available to the public.

Briefing Document: Briefing Document: Malware Analysis and Sandboxing Techniques | Blog | PortalFuse