Data added to packets to help route them through the network and reassemble the original message

Preprocessors

In Snort Intrusion Detection and Prevention Toolkit, 2007

The frag2 Preprocessor

Before we get too deep, it's important to note a couple of things. First, frag3 replaces frag2, which we will discuss later, and second, frag2 will be removed from Snort in the near future, so you should not use this preprocessor anymore. That being said, let's cover what fragmentation is and how it is used both legitimately and maliciously.

Fragmentation is a normal part of the Internet Protocol (IP). In essence, each type of network potentially has a different Maximum Transfer Unit (MTU), a number that quantifies how much data can be transferred in a single “chunk” on the medium. For example, Ethernet's MTU is 1,500 bytes, and it calls its data chunks frames. The sending IP stack in a communication generally puts as much data in a packet as it can, basically using the MTU of the outgoing network as a maximum size for the outgoing chunk. If the packet is too big to travel in between two routing devices, it gets broken into fragments. These fragments look like IP packets in their own right and can traverse the network. They are reassembled when they reach their destination. It is up to the host receiving the fragmented packets to put the packets back together in the right order to make sense of the traffic it's receiving. The problem is that different operating systems reassemble fragments in different orders! (We'll discuss this issue in greater detail in our discussion of frag3, later in the chapter.)

In the meantime, fragmented packets can pose a difficulty to many network IDSes (NIDSes). Remember, IDSes that are based on signature matching work by matching individual packets, not collections of them, against attack patterns. An attacker can use a tool such as Dug Song's fragroute (http://naughty.monkey.org/~dugsong/fragroute) to break a packet into multiple fragment packets in the hope that no single fragment packet will match the pattern for his attack. Snort's frag2 preprocessor, in spp_frag2.c, addresses this type of attack by reassembling fragmented packets before they go through the detection engine. In essence, frag2 rebuilds each packet from the pieces and passes the full packet through for detection once the process is finished.

frag2 is also useful in detecting fragment-based denial-of-service (DoS) attacks. These attacks will often send a series of well-designed fragments to take advantage of a host's particular IP stack vulnerabilities. For example, some machines will reboot, halt, or otherwise react negatively when they receive a fragment that has its offset configured to overwrite a previous fragment's data. Remember, fragments are supposed to be nonoverlapping parts of the packet—an overlapping fragment is just the type of seemingly impossible condition that causes a host to hang.

Configuring frag2

You can configure frag2 by adding parameters after a colon on the preprocessor frag2 directive:

Data added to packets to help route them through the network and reassemble the original message

Let's review the parameters that frag2 accepts:

timeout. The timeout parameter instructs frag2 to stop trying to rebuild a fragmented packet if it hasn't received a fragment in the set number of seconds. The default of 30 seconds is almost certainly overly aggressive. A better default is probably 60 to 90 seconds. Sites that expect that an attacker might either use high-latency links or intentionally slow down the attack should consider setting this number a bit higher.

memcap. The memcap parameter limits the amount of memory that Snort can use to store partially rebuilt packets. When frag2 has used all of this memory, it will begin to aggressively prune partially rebuilt packets out of its fragment table. The 4 MB default might be overly aggressive, especially on a heavily loaded external network interface. It's probably extremely overaggressive for a host on the other end of a low-MTU link.

min_ttl. The min_ttl parameter sets a minimal IP Time-to-Live (TTL) that packets must have in order to be reassembled by Snort. If the TTL of a packet is too low to make it to its destination, you generally don't have to worry about it carrying a payload-based attack. The destination host won't receive the packet; thus, a payload-based attack won't harm that host. That's not to say that packets that don't reach the host can't have a negative effect! If an attacker sends a huge number of packets that die on the router just before they reach the destination host, that destination host will almost certainly find the associated network connection oversaturated and thus useless. Attackers have often used fragment-based attacks to perform DoS attacks. The min_ttl parameter simply prevents frag2 from devoting resources to packets that won't reach their destination. You should set this parameter to the minimum number of hops between the IDS's network and the hosts you're monitoring.

ttl_limit. The ttl_limit parameter sets a maximum difference that will be tolerated between fragments of the same packet. Fragments of the same packet should generally have about the same number of routers to traverse on their way between the two hosts. Even when they take different paths, they should have about the same number of hops to go through. If the number of hops changes too drastically, it might be a sign of someone trying to evade detection. For example, an attacker might insert fragments into the stream that will make it to the IDS, but will expire before they reach the destination. This causes the IDS to see a different picture of the rebuilt packet than the destination host does. It's difficult to choose a safe value for this parameter, although 10 is probably a safe bet. Much of this will depend on how dynamic your ISP's routing is and how dynamic the routing is to your standard destinations. The best rule of thumb is to figure out the maximum number of hops required to reach any host in your environment and then to set the value to be slightly higher than that number.

detect_state_problems. The detect_state_problems parameter activates alerting on anomalies detected in reassembling fragments. This will trigger on several conditions. If a packet has more than one fragment identifying itself as the first fragment (via a fragment offset of zero and the more fragments flag set), this will trigger. It will also trigger if fragments overlap or if a fragment arrives for a packet that is already fully rebuilt. Finally, it will trigger if a nonfirst fragment has IP options set. IP options should be set only in the first fragment. This option does not control whether frag2 alerts on rebuilt packets that are too large, as in the Ping of Death—this alerting is always active.

frag2 Output

frag2 rebuilds a packet from all the fragments it receives and then pushes the rebuilt packet through the normal path taken by a packet that has just left the decoder. The packet is logged and/or run through the preprocessor and detection mechanisms. Does this mean that fragmented traffic is analyzed twice? Yes—once in the fragmented state as it's passing through the engine, and once again when all the fragments have been received.

Data added to packets to help route them through the network and reassemble the original message

Some people tend to think that Snort buffers all fragmented packets until they are reassembled and then passes them through the engine. Essentially creating a bottleneck in the IDS. This misconception is exacerbated when Snort is in IPS (or inline) mode. This is not true. Packets are passed as they are received.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781597490993500112

Layer 3: The Network Layer

In Hack the Stack, 2006

Fragmentation

Next is fragmentation. Its purpose was simply to address the fact that not all networks share the same MTU. For example, Ethernet has an MTU of 1500, while the idiosyncrasies of DSL reduce its MTU to 1472. Issues such as these mean that fragmentation plays an important role in ensuring the successful transmission of data. Concerns arise when we take into account that many routers, firewalls, and IDS systems don’t adequately address the security concerns of fragmentation. These shortcomings in processing mean that fragmentation can be used as a method of obfuscating an attack or bypass detection. One method is the insertion attack. An insertion attack sends packets to an IDS and target device that will be accepted by the IDS and rejected by the target. An evasion attack sends packets to an IDS and target that will be rejected by the IDS and accepted by the target. The idea behind either attack is to send different data streams to each device.

As an example of how evasion attacks are carried out, consider the following. An attacker sends in the first fragment of a fragmented packet to an IDS that has a fragmentation timeout of 15 seconds, while the target system has a timeout of 30 seconds. The attacker simply has to wait more than 15 seconds, but less than 30 seconds, to send the second fragment. When entering the network, the IDS discards the second fragment since the timeout parameter has already triggered the disposal of the first fragment. Upon delivery of the second fragment to the target, it accepts the second fragment since the first fragment is still held in scratch memory. The result is that the attack is successfully delivered to the targeted system and that the IDS has no record of the attack.

Warning

When working with Snort, it’s advisable to keep in mind that it has a default fragmentation reassemble timeout of 60 seconds. After this time limit has passed, the captured fragments will be discarded.

Overlapping fragments can also offer an attacker a means of slipping packets past an IDS and firewall. This is due to the way that different systems reassemble packets or handle them when one or more of the same fragments arrive. To better expand on the concept, consider the fact that Windows-based systems give preferential treatment to original fragments, whereas Cisco devices favor the last fragments. A result of this process order means it’s possible for the following attack to occur. First, the attacker breaks an attack into three fragments and sends fragments one and two to both the Cisco router and the Windows host, which they both accept. Next, the attacker sends fragments two and three. The retransmitted fragment number two is of the same size and offset as the original fragment but contains a different payload. While both systems now have all three fragments and can reassemble them, the Windows systems favored the original packet, whereas the Cisco device processed the last retransmitted packet and, therefore, reassembled something somewhat different than did the Windows system.

Tip

The best way to prevent these types of fragmentation attacks is by ensuring that the parameters are the same on your IDS as on your host. This will allow you to avoid attacks that target this kind of vulnerability.

Fragmentation can also be used to facilitate a Denial of Service. The teardrop attack is an example of this. This fragmentation attack sends packets that are malformed with the fragmentation offset value tweaked so that the receiving packets overlap, as seen in Figure 4.14. The victim does not know how to process these overlapping fragments and so they crash or lockup the receiving system, thereby causing a Denial of Service. The attack affects Windows 3.1x, Windows 95, and Windows NT operating systems, as well as Linux 2.0.32 and 2.1.63.

Data added to packets to help route them through the network and reassemble the original message

Figure 4.14. The Teardrop Attack

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781597491099500083

Developing Ethereal

In Ethereal Packet Sniffing, 2004

Packet Retransmissions

Packet retransmissions are common on busy networks. Your dissector should be able to handle such an occurrence if it is going to attempt to handle fragmented packets. If your dissector or protocol does not process fragmented packets, your dissector can treat these packets as normal packets. In most cases a simple conversation list can check for the occurrence of a request packet but if nothing triggers your dissector, it might be a duplicate entry. You may also find that you need to manage another memory table to track a retransmitted packet. However, in other cases, the lower-level protocol might have already detected the retransmissions.

The packet information structure pinfo can provide information for the status of the current packet being decoded. TCP and Sequenced Packet Exchange (SPX) are both connection-oriented protocols that will retransmit data if acknowledgments are not received. Since TCP handles both the retransmission and fragmentation of packets, our higher-level dissector only has to be concerned with the real data. However, in the case of SPX, the higher-level dissector has to trap for retransmissions if it is handling packet fragmentation. The next section discusses how the SPX dissector passes this information on to the higher-level dissectors. It is important for you to understand that a retransmission is a common occurrence on many networks. Your dissector should be able to determine if a packet is a retransmission or a normal packet. In some cases you may choose not to dissect retransmitted packets and just identify the original packet. Just remember that a retransmission can occur at any time. It may consist of a complete packet or an individual fragment.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781932266825500155

Preprocessors

Jay Beale, ... Brian Caswell, in Snort Intrusion Detection 2.0, 2003

Configuring frag2

You can configure frag2 by adding parameters after a colon on the preprocessor frag2 directive:

preprocessor frag2: timeout 60, memcap 4194304

Let's review the parameters that frag2 accepts. You'll notice that some of the parameters listed here are also in stream4 and have basically the same meaning.

timeout The timeout parameter instructs frag2 to stop trying to rebuild a fragmented packet if it hasn't received a fragment in the set number of seconds. The default 30 seconds is almost certainly overly aggressive. A better default is probably 60 to 90 seconds. Sites that expect an attacker might either use high-latency links or intentionally slow down his attack should consider setting a number even a bit higher.

memcap The memcap parameter limits the amount of memory that Snort can use to store partially rebuilt packets. When frag2 has used all of this memory, it will begin to aggressively prune partially rebuilt packets out of its fragment table. The 4MB default might be overly aggressive, especially on a heavily loaded external network interface. It's probably extremely over-aggressive for a host on the other end of a low-MTU link. You can determine a good setting for this as you did when setting memcap on the stream4 preprocessor. Send Snort a SIGUSR1 signal and read the number of “frag2 memory faults” under the “Fragmentation Stats” heading.

min_ttl The min_ttl parameter sets a minimal IP Time-To-Live (TTL) that packets must have in order to be reassembled by Snort. If the TTL of a packet is too low to make it to its destination, you generally don't have to worry about it carrying a payload-based attack. The destination host won't receive the packet; thus, a payload-based attack won't harm that host. That's not to say that packets that don't reach the host can't have a negative effect! If an attacker sends a huge number of packets that die on the router just before the destination host, that destination host will almost certainly find the associated network connection over-saturated and thus useless. Attackers have often used fragment-based attacks to perform DoS attacks. The min_ttl parameter simply prevents frag2 from devoting resources to packets that won't reach their destination. You should set this parameter to the minimum number of hops between the IDS's network and the hosts you're monitoring.

ttl_limit The ttl_limit parameter sets a maximum difference that will be tolerated between fragments of the same packet. Fragments of the same packet should generally have about the same number of routers to traverse on their way between the two hosts. Even when they take different paths, they should intuitively have about the same number of hops to go through. If the number of hops changes too drastically, it might be a sign of someone trying to evade detection. For example, an attacker might insert fragments into the stream that will make it to the IDS, but will expire before they reach the destination. This causes the IDS to see a different picture of the rebuilt packet than the destination host does. It's difficult to choose a safe value for this parameter, although 10 is probably a safe bet. Much of this will depend on how dynamic your ISP's routing is and how dynamic the routing is to your standard destinations.

detect_state_problems The detect_state_problems parameter activates alerting on anomalies detected in reassembling fragments. This will trigger on several conditions. If a packet has more than one fragment identifying itself as the first fragment (via a fragment offset of zero and the more fragments flag set), this will trigger. It will also trigger if fragments overlap or if a fragment arrives for a packet that is already fully rebuilt. Finally, it will trigger if a nonfirst fragment has IP options set. IP options should only be set in the first fragment. As of Snort 1.9.1, this option does not control whether frag2 alerts on rebuilt packets that are too large, as in the Ping of Death—this alerting is always active.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781931836746500117

Nmap Scanning in the Real World

Angela Orebaugh, Becky Pinkard, in Nmap in the Enterprise, 2008

Frequently Asked Questions

Q: If I am seeing scanning activity on my network how can I tell if it is Nmap?

A: Snort has some rules to detect Nmap scans, but it is very difficult to tell by passively monitoring network traffic what tool is being used. You can sometimes detect Nmap because some scan types send a TCP ACK packet to port 80 by default.

Q: I have never seen any fragmented packets on my network. Should I worry about looking for them?

A: In modern networks, fragmented packets can indeed be uncommon to find. However, that actually provides more reason to watch for them. If you ever actually find fragmented traffic, this could indicate some serious networking problems or as we saw in our examples, IDS or IPS evasion attempts. This certainly isn’t to say that every fragmented packet is indicative of a malicious event, but rather that it should be investigated and understood in your infrastructure.

Q: It sounds like I need to be a TCP/IP guru in order to fully understand Nmap?

A: You can reap a multitude of benefits from Nmap without becoming a TCP/IP top gun. However, learning more about the intricacies of TCP/IP will also help you become a better security analyst and all-around network troubleshooting genius. The authors always like to recommend TCP/IP Illustrated, Volume 1: The Protocols by W. Richard Stevens. Even though the book was published in 1994, it is still one of the best references you will find.

Q: I’m tired of only being able to run Nmap scans and tests in my lab or home network. Do you think I’ll get caught if I start scanning valid IP address space?

A: As common sense-sounding as the answer should be for most, the authors have actually been asked this question on numerous occasions. Will you get caught? No one knows for sure. Could you get caught? You bet. And if you do, there could be any number of consequences depending on whether or not your scanning caused any downtime or damage. Simply put, don’t chance it. They don’t get a lab at all in jail.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B978159749241600008X

Domain 4: Communication and Network Security (Designing and Protecting Network Security)

Eric Conrad, ... Joshua Feldman, in CISSP Study Guide (Third Edition), 2016

IP Fragmentation

If a packet exceeds the Maximum Transmission Unit (MTU) of a network, a router along the path may fragment it. An MTU is the maximum PDU size on a network. Fragmentation breaks a large packet into multiple smaller packets. A typical MTU size for an IP packet is 1500 bytes. The IP Identification field (IPID) is used to re-associate fragmented packets (they will have the same IPID). The flags are used to determine if fragmentation is allowed, and whether more fragments are coming. The fragment offset gives the data offset the current fragment carries: “Copy this data beginning at offset 1480.”

Path MTU discovery uses fragmentation to discover the largest size packet allowed across a network path. A large packet is sent with the DF (do not fragment) flag sent. A router with a smaller MTU than the packet size will seek to fragment, see that it cannot, and then drop it, sending a “Fragmentation needed and DF set” ICMP message. The sending node then sends increasingly smaller packets with the DF flag set, until they pass cleanly across the network path.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780128024379000059

IPv6 for Smart Object Networks — A Technology Refresher

Jean-Philippe Vasseur, Adam Dunkels, in Interconnecting Smart Objects with IP, 2010

15.2.5 The Fragment Header

In contrast with IPv4, the routers along the data path never perform any form of fragmentation. IPv6 mandates that each link must be able to carry 1280-byte packets, which is not always the case in LLN. In particular, the MTU of IEEE 802.15.4 links is equal to 127 bytes. In this case, it is required to handle packet fragmentation and reassembly at the link layer. This is specified in [176] and [124] as a result of a work item from the 6LoWPAN Working Group. These mechanisms are described in Chapter 16.

This implies that IPv6 should support mechanisms to discover the minimum MTU supported on each link along the path to the destination. This is performed using a procedure called path maximum transmission discovery (PMTU) defined in [171]. It uses a sequence of ICMP packets along the path until it discovers the minimum MTU along the path. This value is then cached on the host in a table on a per-destination basis and must be rediscovered on a regular basis since IP paths may change due to rerouting from network element failures. An implementation not supporting PMTU may simply decide to send packets no larger than 1280 octets.

An IPv6 source node fragments a packet each time its size is larger than the minimum MTU along the path to the destination.

The format of the fragment header is shown in Figure 15.5. The fragment header is identified by the value 44 present in the next header field of the previous header (which could either be the IPv6 fixed header or the routing header, if present). The next header value is identical to the original next header type of the fragmented packet. The fragment offset simply indicates the offset of the fragment (in 8-octet units) relative to the start of the fragmentable part of the original packet. The identification field is a 32-bit encoded value chosen by the source node to identify the fragmented packet that will be reassembled by the destination node. Each time a source node fragments a packet it uses a different identification number for each fragmented packet destined to a specific node. The source is expected to use an identification number different from any already sent packet for the expected lifetime of a packet. A simple wraparound counter considering the 32-bit encoding scheme for the identification number is assumed to be perfectly reasonable. The 2-bit “reserved” field is set to 0 and the M-bit is used to indicate whether the fragment is the last one (1: more fragment, 0: last fragment).

Data added to packets to help route them through the network and reassemble the original message

Figure 15.5. Fragment headers.

Now let’s illustrate the fragmentation process of a packet. The original packet has an unfragmentable part made of the original header and any extended header that must be processed by the nodes along the path to the destination (all headers up to and including the routing header if present). The rest of the packet makes the fragmentable part of the packet.

The format of each fragment is shown in Figure 15.5. Each fragment is made of the unfragmentable part of the original packet, the fragment header, and the payload of the fragment. Note that the unfragmentable part of each packet has a payload size equal to the size of the fragment excluding the length of the IPv6 header. The next header field of the last header of the unfragmentable part is set to 44. Now looking at the fragment header, it contains a next header value that identifies the first header of the fragmentable part of the original packet.

Fragments may be lost, especially in LLNs where bit error ratios (BERs) are fairly high and links are potentially quite unstable. IPv6 mandates that all fragments are to be received within 60 seconds after the reception of the first fragment (which may or may not be fragment number 1). After the time expires, and not all fragments have been received, the procedure is simply stopped and all fragments are discarded. An ICMP error message is then sent to the source of the packet. Other error cases (e.g., incorrect packet length, etc.) are also covered and illustrated in [53].

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780123751652000156

Introducing Network Scanning

Angela Orebaugh, Becky Pinkard, in Nmap in the Enterprise, 2008

IP

Internet Protocol (IP) is a connectionless protocol that manages addressing data from one point to another, and fragments large amounts of data into smaller, transmittable packets. The major components of Internet Protocol datagrams are:

IP Identification (IPID) Used to uniquely identify IP datagrams and for reassembly of fragmented packets.

Protocol Describes the higher-level protocol embedded within the datagram.

Time-to-live (TTL) Attempts to keep datagrams and packets from routing in circles. When TTL reaches 0, the datagram is dropped. The TTL allows traceroute to function, identifying each router in a network by sending out datagrams with successively increasing TTLs, and tracking when those TTLs are exceeded.

Source IP Address The IP address of the host where the datagram was created.

Destination IP Address The destination where the datagram should be sent.

Notes from the Underground …

IP Address Source Spoofing

It is possible to spoof any part of an IP datagram; however, the most commonly spoofed IP component is the source IP address. Also, not all protocols function completely with a spoofed source IP address (e.g., connection-oriented protocols such as TCP require handshaking before data can be transmitted, thereby reducing the ease and effectiveness of spoofing-based attacks).

Spoofing can also be used as part of a DoS attack. If Network A sends a datagram to Network B, with a spoofed source IP host address on Network C, Network C will see traffic going to it that originates from Network B, perhaps without any indication that Network A is involved at all. This type of spoofing is common in Smurf and Fraggle attacks.

The best practice for network administrators is to ensure that the network can only originate packets with a proper Source IP address (i.e., an IP address in the network itself). It is also common practice for network administrators to deny inbound packets with source IP addresses matching those of their internal networks.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781597492416000017

Snort: The Inner Workings

Jay Beale, ... Brian Caswell, in Snort Intrusion Detection 2.0, 2003

Preprocessors

Before the packets are sent to the detection engine, they are first sent to the preprocessors. The preprocessor concept originated in Snort v1.5. The main idea behind the introduction of preprocessors was to provide a framework to allow for alerting, dropping, and modification of the packets before they reached Snort's main detection engine

OINK!

For a more detailed look at preprocessors, please refer to Chapter 6, “Preprocessors.”

Referring back to Figure 4.4, the desktop machine pc-2 decides to connect to the Windows 2000 Web server. The user opens the browser and enters the following:

http://10.1.1.251/%73%63%72%69%70%74%73/%68%61%63%6B%6D%65.%65%78%65

The HTTP protocol specifies that binary characters can be passed within the Universal Resource Identifier (URI) by using %xx notation, where xx is the hex value of the character. When the URI arrives at the Web server, it converts it to http://10.1.1.251/scripts/hackme.exe.

Within Snort, the pattern matcher is looking for a pattern of /scripts/hackme.exe, but we have effectively mangled the URI so it does not trigger an alert.

The _decode Family of Preprocessors

One task of the preprocessor is to normalize packets before sending them to the detection engine. In the case of our HTTP request, the http_decode preprocessor changes the URI before it reaches the detection engine. We can still write signatures as we normally do; the preprocessor normalizes them into a standard format.

OINK!

An excellent white paper on the IDS evasion of Web attacks can be found at www.wiretrip.net/rfp/pages/whitepapers/whiskerids.html. The author, Rain Forest Puppy, was one of the key developers of the http_decode preprocessor.

The telnet_decode,ftp_decode, and rpc_decode preprocessors follow the same lines as http_decode. However, instead of normalizing URIs, these preprocessors normalize Telnet, FTP, and RPC traffic before it is sent to the detection engine.

The frag2 Preprocessor

Let's take this concept one step further. What if we fragmented the packets? Fragmentation is a necessary function of the TCP/IP suite, as different routers on the Internet (or internally) have different sized network links. Fragmentation dissects the packets into smaller pieces so that they can fit the smaller links as they travel the network. However, as helpful as fragmentation is, fragmenting packets can also be an extremely effective way of bypassing pattern-based IDSs. One of the most famous tools for doing this is fragrouter.

OINK!

The existence of fragmented packets is normal on a network, especially if it's connected to the Internet. However, a large number of fragmented packets on a network would be classified as suspicious. You can block fragmented packets on most modern firewalls or routers, but in doing so, you might block access to systems for certain users who are traveling over a number of router hops or small links. It's definitely a trade-off between security and usability that needs careful review.

The fragrouter program takes network traffic and fragments it into small pieces before putting it onto the network, thus effectively evading pattern-matching systems. The pattern-based IDSs would only see portions of the packets as they travel across the network. At the far side, the TCP/IP stack on the Web server would reassemble the packets and interpret the results. One downside of fragmentation is that it can be used as an effective denial-of-service (DoS) attack. If your fragmented packets are exceptionally small, the Web server is using valuable system resources to reassemble them, thus clogging up the bandwidth available to your network. An IDS also has to reassemble the fragmented packets, which also requires a significant amount of memory and resources. Thus, if traffic volume is high enough and many fragmented packets are on the network, the IDS might be too busy reassembling fragmented packets to notice an attack on the Web server. Moreover, if the IDS is under load just watching normal traffic, it might miss a single packet that would make the signature invalid.

This brings us to the frag2 preprocessor. Using the frag2 preprocessor, fragmented packets are reassembled before they get to the detection engine, so signatures can be applied to the full sessions, not just the individual small packets. The frag2 preprocessor will also write alerts when fragmentation thresholds are reached.

Finally, what if we could trigger thousands of false-positives, thereby overloading the IDS, and as in a fragmentation attack make the IDS blind due to lack of resources? The stick (www.eurocompton.net/stick) program was released with this in mind. stick takes all of the signatures files in Snort and generates packets that trigger alerts on the IDS. In our previous HTTP example, stick would generate a packet that contained the “/scripts/hackme.exe” string on the HTTP port and then place it on the network. The IDS would see the traffic containing the pattern and alert.

The stream4 Preprocessor

The stream4 preprocessor was designed to make Snort stateful. At over 2000 lines of code, and able to handle up to 64,0000 connections, stream4 is one of the largest components of Snort. By making Snort stateful, it can avoid the problems created by stick and provide better signature-matching capabilities. In addition, as a byproduct, the stateful properties allow Snort to detect operating system fingerprinting techniques and scans using out of state packets, typically used by the NMAP program (www.insecure.org/nmap).

When a TCP connection (Layer 3 of the TCP/IP) from a client to a server is made, a number of events take place. An initial conversation is carried out; this is known as the three-way handshake. Once the connection is set-up, data is transferred. When the data has been transferred, the conversation ends. Using the stream4 preprocessor, during these conversations Snort builds internal tables to represent these sessions and tears them down after each session ends.

By making Snort capable of keeping its own state table, it becomes aware of a full session, not just individual SYN, ACK, and FIN flags to a particular server. This is important when we try to apply signatures, as in the example of a user opening a connection to a Web server. In this case, Snort monitors the conversation between the client (the user's browser) and the server (Web server), and will build internal tables for each session. When the detection engine matches a packet, it checks to see if it is part of an established session, rather than blindly matching packets against signatures (which the stick program exploited). Snort is now capable of only alerting on matched signatures to established sessions.

Another advantage of Snort being stateful is that out-of-sequence scanning techniques can also be detected (for example, a stealth FIN scan, a popular scanning method using Nmap). According to the TCP protocol, FIN packets are only seen during the closing sequence of a connection (see Figure 4.6). If a FIN packet is sent to a closed TCP port, the server should send back an RST packet to the other side. The stream4 preprocessor will alert on the event of a FIN packet sent for a session that was never established.

Data added to packets to help route them through the network and reassemble the original message

Figure 4.6. TCP Session

In the following output from the stream4 preprocessor, the final line shows the SF (SYN FIN) flag set on the packet.

[**] [111:13:1] (spp_stream4) STEALTH ACTIVITY (SYN FIN scan) detection [**]

07/09-19:49:16.000000 10.1.1.236:1034 -> 10.1.1.220:6000

TCP TTL:255 TOS:0x0 ID:31377 IpLen:20 DgmLen:40

******SF Seq: 0xFFFFFFFF Ack: 0x0 Win: 0x1000 TcpLen: 20

The portscan Family of Preprocessors

An important part of an IDS is the ability to detect portscans. Portscans are a regular occurrence on any network connected to the Internet, and are used by potential intruders to identify servers and the ports that are open. Once the attacker discovers which ports are open, he attempts to exploit the services provided on those ports. Referring back to Figure 4.6, a typical TCP portscan works by sending a initial SYN to a server. The server replies with a SYN/ACK if the port is open, and a SYN/RST if the port is not listening. By sending SYNs to multiple ports and watching for the return SYN-ACK, we can map which ports are open on a server. Snort detects portscans through the use of preprocessors. The current 2.0.0 version of Snort, has two portscan preprocessors available:

portscan preprocessor

portscan2 preprocessor

There are some differences in the way in which each preprocessor works. The portscan preprocessor is watching for a single machine (client) connecting to multiple ports on a single server over a period of time. It also detects UDP portscans the same way and Stealth scans (see the section Stealth Portscanning).

The portscan2 preprocessor is effectively the bigger brother of portscan. It has the same functionality as the original portscan preprocessor, but is now a complete rewrite of code. At its core it uses the conversation plug-in, similar to stream4, which tracks connection states. Therefore, portscan2 can determine if a SYN-ACK (see Figure 4.6) is part of a legitimate current connection or an actual scan. portscan2 also has a new output format that provides more information on the portscan, and supports separate thresholds for ports and machines hit in a given time. Going forward, the portscan2 preprocessor will be the standard portscan detector for Snort.

OINK!

The limitation in portscan detectors is, of course, the concept of “slow scans.” If you scan a machine over the course of a week—say, one port per hour—it won't trigger the portscan detectors, as they would have to store details on every connection made throughout the day. On a busy or large network, it's a daunting task.

So, which preprocessor should you run? The portscan preprocessor is a simple and stable portscan detector. The portscan2 preprocessor is new, but does have some additional tuning options, thresholds, and better stealth scan detection. If your Snort server has the system resources in terms of memory and CPU processing power, run both over a period of time to see which gives the best results. In addition, you might run a few portscans yourself and see the results (with permission from your system administrator, of course).

OINK!

The default configuration of both portscan preprocessors will always throw up false positives; the usual suspects are DNS servers and Web proxy servers. The portscan preprocessors can be configured to ignore traffic from certain hosts, and in the case of portscan2, certain ports to and from a server. We show the configuration of the portscan preprocessors later in Chapter 6.

Stealth Portscanning

Nowadays, the “bad guys” are trying to be stealthy in their port scanning attempts. Figure 4.7 is a representation of the TCP header.

Data added to packets to help route them through the network and reassemble the original message

Figure 4.7. TCP Header

Within the TCP header are flags that denote the state of the packet. These flags can be URG, ACK, PSH, RST, SYN, and FIN, and represent different states of the session (see Figure 4.6). By mixing up the TCP flags, the attacker is hoping to elicit a response from the server to allow him to map open services. More importantly, however, the attacker is trying to avoid detection by portscan detection programs.

OINK!

You can also reference the earlier section Storage of Packets; the TCP Header structure is a representation of Figure 4.7.

A number of scans rely on setting different TCP flags; for example:

Full XMAS scan Sets the TCP flags to FIN, URG, PSH. The target server should send back an RST on all closed ports.

TCP FIN scan vSets the TCP flag to be FIN. As with the XMAS scan, the target should send back an RST.

NULL scan Sets the TCP flag to have no options. Again, the target should send back an RST.

portscan Output

The portscan and portscan2 preprocessors both send alerts using the standard Snort output alerting mechanism (see the section Output and Logs later in the chapter). However, they do have their own in-built logging system that's separate from the main Snort output plug-ins. The logging system is configured in snort.conf under the relevant portscan or portscan2 preprocessor section.

The following alerts are generated from portscan and portscan2, and would be sent to the Snort output facility:

[**] [100:1:1] spp_portscan: PORTSCAN DETECTED to port 1 from 192.168.1.53 (STEALTH)

[**] [117:1:1] (spp_portscan2) Portscan detected from 192.168.1.53: 6 targets 14 ports in 11 seconds

The following are sample contents of portscan.log and scan.log:

Portscan.log Sample data is from a Full XMAS scan.Oct 6 20:35:47 192.168.1.53:52645 -> 192.168.1.1:40936 XMAS **U*P**F

Scan.log A full XMAS scan, but contains more data than portscan.log does.10/06-20:35:47.000000 TCP src: 192.168.1.53 dst: 192.168.1.1 sport: 52645 dport: 40936 tgts: 3 ports: 59 flags: **U*P**F event_id: 1298

OINK!

Because the portscan preprocessors only send alerts to Snort, if you are configuring Snort to “Log to Database,” none of the events will be written. No logs will be written either, as the preprocessors will be using their own log format.

Other Preprocessors

A number of other preprocessors ship with Snort version 2.0.0. Some of these are in the experimental stage of development. Table 4.1 lists the preprocessors that are available and a brief description of their functionality.

Table 4.1. Other Snort Preprocessors

PreprocessorsFunction
rpc_decode Decodes RPC, similar to the HTTP decoder
telnet_decode Decodes Telnet and FTP, similar to the HTTP decoder
conversation Provides basic conversation status on protocols (used by the portscan2 preprocessor)
Back Orifice detector arpspoof Decodes Back Orifice network traffic
Experimental ARP misuse detection code
asn1_decode fnord Experimental ASN1 detection code
Polymorphic shellcode analyzer and detector

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781931836746500099

Advanced Snort

In Snort Intrusion Detection and Prevention Toolkit, 2007

Preprocessor Plug-Ins

Preprocessor plug-ins are analysis modules that run after the packet has been captured and decoded. These plug-ins are run on every packet that is received; the snort rules do not have any influence on them. These preprocessors can be used for various purposes, but they are used typically in a context that is too complex to readily express in terms of a rule. Standard Snort includes several preprocessors, such as Frag3, Stream4, Flow, Portscan, and several for processing specific protocols (such as HTTP). Frag3 is for reconstructing fragmented packets before further analysis. Packets can become fragmented as a natural consequence of traveling across the network, but they can also be fragmented deliberately in an attempt to evade notice by simpler IDSes. Stream4 and Flow are for tracking a connection and following it through all the changes in state as it goes from open to established to closed.

Writing your own preprocessor plug-in is actually quite straightforward. For the dynamic plug-ins, there are three functions to write, and one of those is rather well fixed in its form. The first one registers the initialization function for the plug-in to Snort. Let's walk through the example from the Snort manual.

Data added to packets to help route them through the network and reassemble the original message

Snort knows to call this particular function because the compiled module's symbol table equates DYNAMIC_PREPROC_SETUP to ExampleSetup. Snort will invoke ExampleSetup() and load these dynamic modules from the file system when it starts up. It is informed where to look by the dynamicpreprocessor keyword in the configuration file. For example

Data added to packets to help route them through the network and reassemble the original message

will cause all the modules in the directory /usr/local/lib/snort_modules to be loaded.

When Snort encounters a preprocessor setup line in the configuration file (e.g., preprocessor dynamic_example: port 123), it will run the function passed as the second argument of the registration function, in this case ExampleInit () . This function should parse any keywords that go with your preprocessor from the configuration file, and if it is satisfied with the configuration settings, it should also register the primary processing function.

Data added to packets to help route them through the network and reassemble the original message

Data added to packets to help route them through the network and reassemble the original message

Note that this function finally registered ExampleProcess() as the processing function. This function will get called for each packet that has been decoded. It is up to the plug-in to decide if the packet is to be processed or not. In this simple example, you process only TCP packets and will generate an alert whenever the port matches the one that was specified in the configuration file.

Data added to packets to help route them through the network and reassemble the original message

Data added to packets to help route them through the network and reassemble the original message

This alert is directly analogous to the registration function in the dynamic version. The initialization function for the static version works the same way as well: it parses the arguments that may be passed and registers the main processing function. The static version of the initialization function also needs to register two additional functions–one that is invoked when a restart is initiated and the other that is invoked when Snort exits.

Data added to packets to help route them through the network and reassemble the original message

Data added to packets to help route them through the network and reassemble the original message

Data added to packets to help route them through the network and reassemble the original message

As we see here, the coding difference between the dynamic and static preprocessor plug-ins is minimal. The real difference is how they are “installed.” As we saw earlier, to install the dynamic plug-in, you just point to where the compiled object file is located in the Snort configuration file and then restart Snort. For the static plug-in, you need to follow a much more invasive process. First, the primary functions must be declared in a header file (e.g., spp_example.h).

Data added to packets to help route them through the network and reassemble the original message

Data added to packets to help route them through the network and reassemble the original message

This file is then added to the list of includes in the file plugbase.c in the Snort source directory, src.

Data added to packets to help route them through the network and reassemble the original message

Next, you need to add an invocation of the registration function, ExampleSetup(), to the list of similar function calls in the function InitPreprocessors().

Finally, you need to edit the Makefile in the preprocessors source file so that your new preprocessor source file is in the list of preprocessor files.

Then, of course, you need to recompile Snort and install the new binary. Clearly, this is a lot of work; hence, the motivation to develop a technique for loading plug-ins dynamically. Currently, the choice of approaches is available only for preprocessor plug-ins.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781597490993500173

How are data packets sent across a network?

The Internet works by chopping data into chunks called packets. Each packet then moves through the network in a series of hops. Each packet hops to a local Internet service provider (ISP), a company that offers access to the network – usually for a fee.

What is a chunk of data and its metadata used to route and reassemble information on the Internet?

Packets contain a chunk of data and metadata used for routing the packet between the origin and the destination on the Internet, as well as for data reassembly.

What happens when packets of data arrive together at their destination?

What happens when the packets arrive? When the packets arrive at their destination, they are put back together again in the right order. Remember earlier on we told you that each packet was given a number? This makes it possible to correctly reorder them.

Which of the following manages how data packets are transmitted and delivered in a network?

TCP defines how applications can create channels of communication across a network. It also manages how a message is assembled into smaller packets before they are then transmitted over the internet and reassembled in the right order at the destination address.