'The nmap script bitcoin-getaddr often does not return a response
For a project I need to write a scraper that sends a GetAddr request to all Bitcoin nodes. I determine the reachable nodes using the bitnodes.io API. Then I go through all the IPs and send a request using the python-nmap library and the bitcoin-getaddr script from nmap.
The code to do this looks like the following:
def send_get_addr_requests(ip, port):
ip_addr = ipaddress.ip_address(ip)
# check if it is an ipv6 address and fix arguments in this case
arguments_get_addr = '--script bitcoin-getaddr -d -Pn'
if isinstance(ip_addr, ipaddress.IPv6Address):
arguments_get_addr += ' -6'
# send get_addr request to the node
nm = nmap.PortScanner()
response_get_addr = nm.scan(ip, port, arguments=arguments_get_addr)
return response_get_addr
However, as I have found, the nmap script (especially the bitcoin.lua file) is relatively unreliable and in most cases does not return a result on the request. I have already modified the GetNodes function as follows:
getNodes = function(self)
local req = Request.GetAddr:new(
self.host, self.port, self.lhost, self.lport
)
local status, err = self.socket:send(tostring(req))
if ( not(status) ) then
return false, "Failed to send \"GetAddr\" request to server"
end
local status, response = Response.recvPacket(self.socket, self.version)
local all_addrs = {}
-- limit is not usable because it is unclear how many packets are returned
-- thats why we take a timer
-- local limit = 20
local startTime = os.time()
local endTime = startTime + 60 -- wait 60 seconds max
-- Usually sends an addr response with 1 address,
-- then some other stuff like getheaders or ping,
-- then one with hundreds of addrs.
while #all_addrs <= 1 and os.time() < endTime and status do
-- limit = limit - 1
status, response = Response.recvPacket(self.socket, self.version)
if status and response.cmd == "addr" then
for _, addr in ipairs(response.addresses) do
all_addrs[#all_addrs+1] = addr
end
end
end
return #all_addrs > 0, all_addrs
end,
I added a timer and removed the limit. Since the requested nodes send many packets before the GetAddr response, this approach works much better. I also set the replaced version to 70015 and adjusted the ping/pong messages of the script to the newest implementation.
However, in many cases (about 2000 out of 7000) I still do not receive a GetAddr response.
The function Response.recvPacket(self.socket, self.version) from the bitcoin.lua file gets at some point only a status = false and a response = EOF ...
So my question is if this is normal or how I can further customize the script to get a more reliable response? When I manually type in the terminal "nmap -Pn -d -script bitcoin-getaddr -p 8333 <IP>" I get a response from time to time, but not reliable and rather rare. So the requested nodes don't block the GetAddr requests, which makes the whole thing even more complicated to understand.
Thanks in advance for any help!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
