Imagine: there’s a new security threat. How do you find out if your organization is affected? You might research the CVE to gauge the severity and impact of the vulnerability. You might perform a vuln scan — if there’s a vuln check available. At some point, you’ll eventually end up with a list of devices that you need to update.

What are your next steps?

The cost of not tracking asset ownership

In an ideal world, your asset inventory would be the first place you would look for information. However, the reality is: most organizations have their asset inventory data distributed across multiple solutions and maintained by different teams. So instead of being able to focus on mitigating issues, your security team spends an inordinate amount of time doing detective work. And for security practitioners, time is of the essence.

Asset inventory is the first step to getting context around a device: the hardware, OS, software, etc. But what about who owns it? More and more, knowing who is responsible for an asset is as important as knowing what an asset is. Without clear asset ownership tracking, you waste a lot of time going from team to team, person to person, trying to find out who is responsible for an asset.

Let’s take a look at three reasons why a lack of asset ownership can adversely impact your business.

Reason #1: Forgotten assets can be costly

One of the biggest obstacles to tracking asset ownership is humans. Humans are dynamic, often upgrading to new equipment, changing roles, or even leaving organizations entirely. As a result, assets are often left abandoned, unmanaged, and unowned. Documenting asset ownership manually, like in a spreadsheet, means that the data becomes outdated very quickly. Effective asset ownership tracking requires regular updates and attention. Without a major investment of time and resources to maintain asset ownership tracking, stale data will continue to plague your organization. For example, consider infrastructure that no longer has an owner, but is still racking up recurring expenses. These forgotten assets can be costly over time.

Reason #2: Lack of asset ownership can lead to service outages

Your business relies on having systems that are working efficiently. Systems need to be updated, upgraded, and maintained regularly to ensure that everything runs smoothly and outages do not occur. However, what would happen if a specific system needed a configuration update to continue to operate? How would you know who to go to?

Oftentimes, it’s a goose chase. You start with one person (or team) and hope they can point you in the right direction. While you’re chasing down the appropriate person to help you, access to the systems you need may be shuttered or months may have passed by. These consequences can be detrimental to business – especially if these systems directly impact revenue.

Reason #3: Wasted time slows down remediation

9 years ago: Shellshock.

5 years ago: Apache Struts.

1 year ago: Log4Shell.

Nearly a decade has come and gone between these major vulnerabilities, and yet, building comprehensive asset inventory and tracking asset ownership continues to be a challenge. One of the biggest challenges faced by security teams is that they often need to rely on asset owners to take action to update and secure their devices. However, tracking down the right asset owner can be a bit of a journey through a myriad of data sources – from CMDBs to VMs to EDRs to device logs to spreadsheets. The amount of time that security teams spend hunting for information is a hindrance to fast response and remediation times.

Tracking asset ownership with runZero

runZero 3.5 introduces the ability to track asset owners in your inventory. Asset owners can be anyone in your organization who can help you remediate issues. For most organizations, assets will likely have multiple owners, such as an individual, team, and business unit. For example, a laptop might have an assigned device user, business owner, IT owner, and security owner. Each of these assignments will help you zero in on the right person who can take action on the device, based on the situation. Let’s take a look at how runZero can help you track different types of owners within your organization.

What are ownership types?

In runZero, ownership types help you classify and assign ownership to assets. There is a default ownership type, called Asset Owner, which automatically pulls owner data from integrations you have configured. Otherwise, you can add up to nine custom ownership types based on what your organization needs. For example, you might want to have ownership types for the security owner, IT owner, and business owner.

When you create an ownership type, you will need to specify the following:

• Name – The name of the asset ownership type, such as IT owner.
• Reference – You can set the reference to user, group, or none. If set, you will be able to easily search within the user or group inventories for owners that match the display name.
• Visibility – You can set the visibility to hidden or visible. This setting controls the ability to view the asset owner from the asset inventory and asset details page.

After you have created your ownership types, you’re ready to start assigning owners within your asset inventory. Let’s take a look at how you can do this in runZero.

How to assign ownership to assets in runZero

There are a couple of ways to assign asset owners: manually or automatically through rules and the API. However, the most efficient way to apply ownership is through rules, which allows you to set up specific conditions and automate the assignment of asset ownership after each scan. For example, let’s say you want to assign an IT owner for all firewalls. Here’s how you can do it with rules:

1. From the Rules page, create a rule using the asset-query-results event type. Based on this event type, the query will run against the asset inventory after a scan completes.
2. Give the rule a descriptive name, like Automate IT ownership for firewalls.
3. Configure the rule with the following conditions:
   • Run the following query after a scan completes: type:firewall and the number of matches is greater than 0.
   • If there is a match on the query, take the following action: modify the asset and set the ownership of the matching assets. This value for the owner can be any name. For our example, we will assign the IT owner to someone on the team named Tim.
   • Make sure the rule is enabled. If it is not, it will not run.
4. Save the rule.

Each time a scan completes, this rule will check for matching conditions and perform the configured actions.

Viewing ownership data for an asset

Now that you’ve set up ownership types and automated ownership assignment, let’s take a look at how you can view this data in runZero. You can view ownership information from two areas of the console: the asset inventory and the asset details page.

There’s a new column in the asset inventory called Owners, which will list the owners for the asset. If there are multiple owners, there will be a plus (+) sign to indicate that there are more for you to view. The owner name that gets displayed in the inventory table depends on the order you have them ranked on the ownership types page. The highest ranked ownership type will take precedence. In our example, we have our IT owner ranked first, so we will see our IT owners displayed in the inventory table. Other owners will be viewable by hovering over the plus (+) sign. From the asset inventory page, you can select some assets then use the Manage asset ownership button to manually update the owner for those devices.

From the asset details page, there is a new ownership section that lists all the visible owners assigned to that asset. If the ownership type has a reference set (to user or group), you’ll be able to click on the magnifying glass next to the owner name to search within those inventories for matching results. From the asset details page, you can go to Manage > Asset ownership to manually update the owner for that specific device.

Searching the inventory for assets based on owners

Now that you have asset ownership data in your inventory, you can search for assets that match specific ownership criteria. To enable searching based on ownership attributes, the following new keyword terms have been added:

• owner – Filter by asset owner name, such as Tim.
• has_owner – Filter assets by whether or not they have an owner. Use t or f as your input.
• owner_count – Use a comparison operator (>, >=, <, <=, =)to filter assets by count.
• ownership_type – Filter by ownership type, such as IT owner.

Here are a few useful queries (based on some common use cases):

• has_owner:f – Searches for assets that don’t have an owner assigned.
• ownership_type:"IT owner" – Searches for assets by ownership type.
• owner_count:>1 – Searches for assets that have more than one owner.

For example, if you need to gauge the number of unowned (and likely unmanaged) assets in your inventory, the query has_owner:f would help identify assets that don’t have an owner. Inversely, you can use has_owner:t to see all the ones that do have an owner. Between these two results, you can discern how well you’ve got your asset ownership data covered. To see how well your organization is tracking asset owners, you can also check out the asset ownership goal from the dashboard.

Zero in on unowned assets on your network

Imagine: there’s a new security threat. Thankfully, you have an asset inventory that includes asset ownership data. With a solid program and solution in place to track asset owners, you’ve eliminated unnecessary time spent chasing down people. You can focus on remediation.

If you’re a runZero Enterprise customer, you can check out the ownership capabilities by going to the new Ownership page in your console. You’ll notice a new menu item for it under Global Settings. Otherwise, if you’re new to runZero, sign up for a free trial to test out this new feature for 21 days.

Popular hypervisor ESXi has been in the news recently due to fresh targeting by a new strain of ransomware. Known as ESXiArgs, this ransomware leverages a 2-year old heap overflow issue in the OpenSLP service that can be leveraged to gain remote code execution on exploitable targets (CVE-2021-21974). Many vulnerable public-facing ESXi servers have already been affected by this malware (currently over 1,900 via Censys search results).

What is the impact?

Targets of this new ransomware campaign are older ESXi servers running certain versions of 6.5, 6.7, or 7 releases and also have the OpenSLP service enabled (it has not been enabled by default in ESXi releases since 2021). Upon successful exploitation of CVE-2021-21974, the ESXiArgs ransomware will encrypt a number of file types on the target system, including VM-related files with extensions .vmxf, .vmx, .vmdk, .vmsd, and .nvram. Ransom notes are saved as HTML files on compromised systems for admins and users to subsequently discover. While some of these ransom notes claim to have stolen data from vulnerable targets, no data exfiltration has been observed at this time.

Are updates available?

VMware made patches available when the OpenSLP heap-overflow vulnerability was initially reported in 2021. The following ESXi releases have been patched against this attack vector currently being exploited by the ESXiArgs campaign:

• ESXi version 7+ (ESXi70U1c-17325551 and later)
• ESXi version 6.7+ (ESXi670-202102401-SG and later)
• ESXi version 6.5+ (ESXi650-202102101-SG and later)

VMware also offers patched releases for Cloud Foundation (ESXi), which includes an ESXi component:

• Cloud Foundation (ESXi) version 4.2+
• Patching instructions for Cloud Foundation (ESXi) version 3.x can be found here

Patching (and also ensuring that your ESXi servers are running a supported, not end-of-life/end-of-support version) is the best course of action. If patching is not a near-term option, VMware has a recommended mitigation via disabling the OpenSLP service.

How do I find potentially vulnerable VMware ESXi assets with runZero?

From the Asset Inventory, use the following pre-built query to locate ESXi assets which may need remediation:

os.product:"ESX" and (os.version:="1.%" or os.version:="2.%" or os.version:="3.%" or os.version:="4.%" or os.version:="5.%" or os.version:="6.0%" or os.version:="6.5.0 build-4564106" or os.version:="6.5.0 build-4887370" or os.version:="6.5.0 build-5146843" or os.version:="6.5.0 build-5146846" or os.version:="6.5.0 build-5224529" or os.version:="6.5.0 build-5310538" or os.version:="6.5.0 build-5969300" or os.version:="6.5.0 build-5969303" or os.version:="6.5.0 build-6765664" or os.version:="6.5.0 build-7273056" or os.version:="6.5.0 build-7388607" or os.version:="6.5.0 build-7967591" or os.version:="6.5.0 build-8285314" or os.version:="6.5.0 build-8294253" or os.version:="6.5.0 build-8935087" or os.version:="6.5.0 build-9298722" or os.version:="6.5.0 build-10175896" or os.version:="6.5.0 build-10390116" or os.version:="6.5.0 build-10719125" or os.version:="6.5.0 build-10868328" or os.version:="6.5.0 build-10884925" or os.version:="6.5.0 build-11925212" or os.version:="6.5.0 build-13004031" or os.version:="6.5.0 build-13635690" or os.version:="6.5.0 build-13873656" or os.version:="6.5.0 build-13932383" or os.version:="6.5.0 build-14320405" or os.version:="6.5.0 build-14874964" or os.version:="6.5.0 build-14990892" or os.version:="6.5.0 build-15256468" or os.version:="6.5.0 build-15177306" or os.version:="6.5.0 build-15256549" or os.version:="6.5.0 build-16207673" or os.version:="6.5.0 build-16389870" or os.version:="6.5.0 build-16576879" or os.version:="6.5.0 build-16576891" or os.version:="6.5.0 build-16901156" or os.version:="6.5.0 build-17097218" or os.version:="6.5.0 build-17167537" or os.version:="6.7.0 build-8169922" or os.version:="6.7.0 build-8941472" or os.version:="6.7.0 build-9214924" or os.version:="6.7.0 build-9484548" or os.version:="6.7.0 build-10176752" or os.version:="6.7.0 build-10176879" or os.version:="6.7.0 build-10302608" or os.version:="6.7.0 build-10764712" or os.version:="6.7.0 build-11675023" or os.version:="6.7.0 build-13004448" or os.version:="6.7.0 build-12986307" or os.version:="6.7.0 build-13006603" or os.version:="6.7.0 build-13473784" or os.version:="6.7.0 build-13644319" or os.version:="6.7.0 build-13981272" or os.version:="6.7.0 build-14141615" or os.version:="6.7.0 build-14320388" or os.version:="6.7.0 build-15018017" or os.version:="6.7.0 build-15160134" or os.version:="6.7.0 build-15160138" or os.version:="6.7.0 build-15999342" or os.version:="6.7.0 build-15820472" or os.version:="6.7.0 build-16075168" or os.version:="6.7.0 build-16316930" or os.version:="6.7.0 build-16701467" or os.version:="6.7.0 build-16713306" or os.version:="6.7.0 build-16773714" or os.version:="6.7.0 build-17167699" or os.version:="6.7.0 build-17098360" or os.version:="6.7.0 build-17167734" or os.version:="7.0.0%" or os.version:="7.0.1 build-16850804" or os.version:="7.0.1 build-17119627" or os.version:="7.0.1 build-17168206" or os.version:="7.0.1 build-17325020")

Each ESXi asset returned in the query results should be checked if the OpenSLP service is enabled. If OpenSLP is enabled, then the asset is vulnerable to exploitation.

As always, any prebuilt queries are available from our Queries Library. Check out the library for other useful inventory queries.

Many OT engineers still believe that active scanning is not safe in OT environments. However, their assumptions don’t have a legitimate basis. 

Yes, regular network and vulnerability scanners can cause devices to act erratically. Printers start spewing out pages. Embedded systems freeze up or reboot. But it doesn’t have to be this way. If you observe a few key aspects and use a purpose-built scanner, actively detecting ICS and IoT equipment is entirely safe. runZero has proven that active scanning is safe, and it’s evident across numerous industries.

Digging into issues with legacy scanners

To better understand the challenges of active scanning, we analyzed why legacy vulnerability and network scanners destabilize systems. We found four different root causes:

• Malformed IP traffic
• Security probes
• Heavy scan traffic per device
• Snowflake devices

Let’s dig into each issue.

Malformed IP traffic

Legacy scanners often send intentionally malformed IP traffic to identify different flavors of operating systems. A robust TCP/IP stack on a Windows or Linux system will process the malformed traffic and respond in a specific manner that helps the scanner identify the flavor of the operating system.

Embedded systems often use legacy or custom TCP/IP stacks. When scanned with malformed IP traffic, these devices can freeze up or reboot because the unexpected traffic causes errors that are handled incorrectly by the stack.

Security probes

Vulnerability scanners send security probes, such as SQL injection exploits, to detect vulnerabilities in target systems. Embedded systems are often written without enough error handling built in, so the problem is similar as with malformed IP traffic: receiving unexpected network traffic can cause the devices to react erratically.

Heavy scan traffic per device

Legacy vulnerability and network scanners scan a large number of ports and can send several probes per port. This traffic is all sent to the end node in rapid succession. When all ports and probes are completed, the scanner moves on to the next host.

Enterprise IT hardware and mainstream operating systems can handle a lot of network traffic at once. OT equipment often doesn’t have a lot of processing power. Heavy scan traffic can overload the device, causing it to slow down or freeze up. In many industrial control applications, response times are critical. Even a slow down can have adverse effects on the overall environment.

Snowflake devices

When scanners avoid malformed IP traffic, security probes, and heavy scan traffic, most of the issues on OT networks can be resolved. However, there are a handful of particularly flakey devices that become unstable with even the most regular scan traffic. Serial-ethernet connectors, also known as print servers, tend to be among the worst “snowflake” devices.

Passive monitoring is expensive and lacks accuracy

That’s why by sticking with passive monitoring solutions instead of active scanning, OT engineers are inviting these issues into their projects:

• Longer deployment cycles – Connecting to SPAN ports or TAP appliances is more complex than deploying a software scanner in the environment.
• Higher cost – Requires lots of disk space and processing power, usually in the form of costly hardware appliances.
• Missing assets – You can’t inventory assets that are not communicating.
• Missing detail – Missing ports that are not communicating.
• Low accuracy – Spotty accuracy because passive monitoring is limited to analyzing existing traffic.
• Not future proof – The increasing amount of encrypted traffic makes passive monitoring solutions less viable over time.

Let’s take a look at the flip side and run through the key gains of leveraging an active scanning approach.

How to safely scan ICS environments

While legacy scanners cannot be used safely on OT assets, modern purpose-built scanners can safely scan ICS environments by following a few basic rules:

• Use only standard-conforming IP traffic – All traffic sent from the scanner must be completely RFC compliant.
• No security probes – Very easy. Just don’t use them.
• Throttle traffic per host – Limit the number of packets sent to each node. A good starting point is 40 packets per second. The best scanners keep overall scan times short by sending all traffic round-robin on the network when the threshold is reached.
• Probe for snowflakes – Detect snowflake devices before running a full port scan and adapt the scan for the particular model.

Now, let’s take a look at how these rules have been applied across different industries and what organizations have been able to uncover as a result.

Active scanning is a proven methodology across industries

Doing research in a lab is one thing, but proving a methodology in the field is another. This approach has been tested and deployed in production environments across many industries, including:

• Building automation
• Consumer and B2B electronics manufacturing
• Biomedical device manufacturing
• Telecommunications
• Broadcasting
• Universities (e.g., research instrumentation)
• Data center technology
• Transportation (e.g., train signals)
• City and state infrastructure (e.g., street signs, surveillance cameras)
• National labs
• Apparel manufacturing
• Car manufacturing
• Aerospace manufacturing
• Building material manufacturing
• Retail stores (e.g., POS systems, HVAC)
• Cattle and fish farms
• Utilities
• Saw mills
• Hospitals
• ICS equipment manufacturers

Some examples of equipment found in these environments include the following device types:

• PLCs
• Industrial control systems
• Serial-Ethernet converters
• HMI/HMI controllers/HDI
• BACNET devices
• Device servers
• Surveillance cameras
• Terminal servers
• Access control systems
• Intercoms
• KVMs
• Rugged WAP

Get started with active scanning of industrial control systems

You wouldn’t deploy a new piece of software across all of your devices without testing it first. The same is true for active scanning in ICS environments. As you’re considering rolling out active scanning technology, here are some tips to get you started:

1. Pick a purpose-built modern scanner – It’s unlikely that you will be successful with legacy network or vulnerability scanners as they send unsafe traffic. Pick a modern, purpose-built solution, such as runZero.
2. Start small and slow – If you have a small handful of devices in a lab, start there. Otherwise, pick a handful of devices to scan during a maintenance window and check their operational status afterwards. If you know you have snowflake devices, include them in your first scan. If it doesn’t work for them, it won’t work for the full network. Start with a very low network scan frequency, such as 1,000 packets per second from the scanner and 20 packets per second per host.
3. Try a bigger segment – Once you are comfortable with a handful of devices, scan a larger network segment during a maintenance window.
4. Plan your deployment – Deploy one scanner per network segment. Don’t scan through any network devices that filter traffic, otherwise the accuracy of your results will be impacted. Don’t scan through stateful devices because each IP/port connection will create another session and you may overload the device. Deploy the scanners on appropriate hardware or virtual machines. For a large network segment, you may want a dedicated host. For a medium-sized network, you can use an existing host. For small environments, you can even use a Raspberry Pi.

Hopefully, these tips will help you eradicate outdated and inaccurate perceptions against active scanning. Utilize these recommended best practices and you’ll be able to safely detect ICS and IoT devices via active scanning. runZero continues to prove this over and over again across multiple industries.

Printer manufacturer Lexmark recently published details on a vulnerability that affects over 100 of their printer models. Discovered by researcher Peter Geissler, this vulnerability can be leveraged to achieve unauthenticated remote code execution for an attacker. Firmware across devices in Lexmark’s small/medium business product line and also their enterprise product line have been found to contain this vulnerability.

What is the impact?

Lexmark assigned a CVSS score of 9.0 (“critical” severity rating) to this vulnerability (tracked as CVE-2023-23560), which allows server-side request forgery (SSRF) via the Web Services feature listening on port 65002 of affected printers. A successful attacker can exploit this vuln in a chain to gain code execution as root on vulnerable devices. Lexmark’s advisory states that, as of last week, they are not aware of anyone currently exploiting this vulnerability, but proof-of-concept exploit code is publicly available.

Are updates available?

All firmware versions (release numbers 081.233 and prior) for affected printer models contain this vulnerability (CVE-2023-23560). Lexmark has made firmware updates available for each affected device, via release numbers 081.234 and later (see Lexmark’s advisory for specific release version details per affected printer).

If updating firmware isn’t a near-term option for admins/owners of affected printers, Lexmark does offer a straightforward mitigation:

Disabling the Web-Services service on the printer (TCP port 65002) blocks the ability to exploit this
vulnerability. The port can be blocked by following process: “Settings”->“Network/Ports”- > “TCP/IP”- > “TCP/IP Port Access” then uncheck “TCP 65002 (WSD Print Service )” and save.

How do I find potentially vulnerable Lexmark printer assets with runZero?

Please note that the following query relies on you having already performed a scan with our latest Explorer/scanner release (v3.4.22), which now includes the scanning of port 65002. Alternatively, you can perform a new scan using an older Explorer/scanner, just add port 65002 to the Included TCP ports list under the Advanced tab of your task settings prior to running the scan.

From the Asset Inventory, use the following pre-built query to locate Lexmark printer assets which may need remediation:

type:printer AND vendor:Lexmark AND tcp_port:65002

Query results can then be checked against Lexmark’s list of vulnerable models and firmware versions.

As always, any prebuilt queries are available from our Queries Library. Check out the library for other useful inventory queries.

Printer manufacturer Lexmark recently published details on a vulnerability that affects over 100 of their printer models. Discovered by researcher Peter Geissler, this vulnerability can be leveraged to achieve unauthenticated remote code execution for an attacker. Firmware across devices in Lexmark’s small/medium business product line and also their enterprise product line have been found to contain this vulnerability.

What is the impact?

Lexmark assigned a CVSS score of 9.0 (“critical” severity rating) to this vulnerability (tracked as CVE-2023-23560), which allows server-side request forgery (SSRF) via the Web Services feature listening on port 65002 of affected printers. A successful attacker can exploit this vuln in a chain to gain code execution as root on vulnerable devices. Lexmark’s advisory states that, as of last week, they are not aware of anyone currently exploiting this vulnerability, but proof-of-concept exploit code is publicly available.

Are updates available? 

All firmware versions (release numbers 081.233 and prior) for affected printer models contain this vulnerability (CVE-2023-23560). Lexmark has made firmware updates available for each affected device, via release numbers 081.234 and later (see Lexmark’s advisory for specific release version details per affected printer).

If updating firmware isn’t a near-term option for admins/owners of affected printers, Lexmark does offer a straightforward mitigation:

Disabling the Web-Services service on the printer (TCP port 65002) blocks the ability to exploit this
vulnerability. The port can be blocked by following process: “Settings”->“Network/Ports”- > “TCP/IP”- > “TCP/IP Port Access” then uncheck “TCP 65002 (WSD Print Service )” and save.

How do I find potentially vulnerable Lexmark printer assets with runZero?

Please note that the following query relies on you having already performed a scan with our latest Explorer/scanner release (v3.4.22), which now includes the scanning of port 65002. Alternatively, you can perform a new scan using an older Explorer/scanner, just add port 65002 to the Included TCP ports list under the Advanced tab of your task settings prior to running the scan.

From the Asset Inventory, use the following pre-built query to locate Lexmark printer assets which may need remediation:

type:printer AND vendor:Lexmark AND tcp_port:65002

Query results can then be checked against Lexmark’s list of vulnerable models and firmware versions.

As always, any prebuilt queries are available from our Queries Library. Check out the library for other useful inventory queries.