Proud Programmers Blog

The user interface of the database is nothing fancy;

It contains a couple of agents, where the first do all the work of analyzing the log files, updating the documents in the database and finally create the IP blocker file. IP Blocker itself work on a schedule and reload the file every hour if it has changed.

Below you see one IP address;

As you see from the screen shot above, the database will remember all the occurences the IP address has been involved in. Along with the timestamps from the log-database itself, you also see a list of unique, attempted login names.

I was astonished to see that some IP addresses had several thousand attempts just over the last 7 days (remember, the default number of days that the log keep track);

Clearly, it would nice to completely block those IP addresses out of my server, at least for some time.

The main logic in the Authentication Failed Extrator database are;

• Scan specified log files for "Authentication Failed"-messages. Extract the timestamp, IP-address and attempted user name. All "hits" are stored in an internal LotusScript list of CIPAddress-objects. Why use objects? This make it much easier (in my eyes that is) to work IP-ranges later.
• Analyze whether the IP-address already has been registered or not. If it has been registered already, I check whether I have already registered this incident. If not, it is added to the list of attempted login users. New IP addresses are registered.
• Update the Notes-documents in the database. Note that the documents is just storage, and the Analyze-method loads the existing IP addresses. If we find any changes, then the documents are updated.
• Create the IP Blocker file. Since the IP-address lists can use IP-address ranges, this part also analyze whether I have real ranges of trailing IP addresses. This means that for example 185.169.229.34, 185.169.229.35, 185.169.229.36 and 185.169.229.37 will become the range 185.169.229.34 - 185.169.229.37.
• You have the ability to manually register your own IP addresses and IP ranges. For example, it turns out that IP addresses in the range 91.200.12.* Seems to be very active. I have thus blocked everything between 91.200.12.0 – 91.200.12.255. Below you see the 91.200.12.* Activity …
  
  … And below you see the document where I block the whole range;
  
  When ranges are found by the code, it will also check whether a current IP address are within any of the ranges.
   

Some details about the code

As stated before, the database isn't anything fancy UI-wise But it contain some cool code that you might use or reuse yourself. Below you see some of the main code-points;

• I love LotusScript lists! The are super fast and super easy to program when you get the hang of it. My code uses lots of lists, and very seldom arrays.
  
• I love classes! Primarily I love the encapsulation-part of the object oriented development. Sometimes I also use the extendibility and create sub-classes.
  
• The database contain some cool script libraries that you might use.
  
• class:CList. A class to wrap LotusScript lists. It has built in feature to save the list to a file, enumerate the list and what not.
  
• class:CRC32. A list to create CRC32 values from string-content.
  
• class:CEnumerateDatabasesOnServer. A class to quickly enumerate databases and templates on a server. A small overkill in this database, as I often run for only a single database (log.nsf) – but it's there primarily for the idea of using the same database on multiple Domino servers in one go.
  
• class:CCommonUtility. A little bit of every thing, with typical methods needed everywhere. One of the cooler methods developed for this database is ConvertIPv4AddrToDouble which converts an IP-address on the IPv4-format aaa.bbb.ccc.ddd to a Decimal. Trailing IP addresses get trailing decimal numbers, and this is what makes make my code easily detect whether an IP address is within an existing IP range or not, or whether I suddenly have a need to expand a previously single IP address to a new IP range. The method ConvertDoubleToIPv4 does the reverse operation and return the dottet IPv4 format again.
  
• Since log-databases can be quite large, the code also may stamp already processed log-documents, so they won't be scanned again. This make the analyzer pretty fast, especially after the first run   Note that you can force re-analysis if you need to.
  
• Since the internal list of IP-address ranges works all in memory, nothing is written unless we have actually changes. When the IP Blocker file is written, it is very simple to dump the IP-range list to a file.

Ideas for future development

• Run Whois against those bad IP addresses to know more about the perpetrators. Include the CWebContentConsumer-classes presented in Teach the old dog some new tricks: Let LotusScript consume REST Services the easy way and Bark-bark!! The LotusScript CWebContentConsumer now support REST Service POSTs with files++ and query a REST service based Whois-service such as https://jsonwhois.com/ or https://jsonwhoisapi.com/.
  
• Have some fun with the data! Export the attempted login names and get an idea of what names are most common. I spotted some addresses going specifically for just one login name, and that is perhaps more dangerous compared to those addresses just pumping a list of usual network names. Make a wordle from the names A good wordle generator is by the way Wordaizer.
  
• Create a similar scanner for all those spam-mails you receive, and block out the most frequest spammer IP adresses. By the way, IP Blocker itself has the ability to download (also completely automatic) lists of known spammer IP addresses. Perhaps other sources of logs can be uses too, such as the Domino Web Log (domlog.nsf) where you could perhaps spot perpetrators on HTTP? Anybody trying to inject code on your pages? Not anymore
  
• Create a REST Service with SmartNSF (see some of my work with SmartNSF here) and report back to your own main status screen application. How many blocked IP adresses, the top 5 perpetrators or similar.
  
• Create an agent which removes the blocked IP address after a while. This would make it possible for the IP address to get "white" again. Perhaps even have some kind of rules analyzing how aggressive an IP address are. Is it a "one time event", then give the man a chance, but if things keep repeating, block him out.
  
• Create the ability to control white IP addresses or IP address ranges. The complete opposite of the blacklist logic of today, which means that you can ensure that certain addresses always get access, even though the might throw an "Authentication failed" here and there. Please know that IP Blocker itself has a concept of rules, meaning that you can have granular control over what an IP address can do or not. So instead of blocking an address completely (like I do so far), you can tell IP Blocker that an address can access HTTP, but never use FTP for example. Since IP Blocker let you specify the ports, it is a breeze to block SMTP hacking too.

Can there be any disadvantages with this kind of logic? Of course – a lot of spam-machines and bots around the world, are themselves hacked machines. This means that the owner is unaware that her or his machine is part of a bot-network, just pumping out attacks. This means that you may block out perfectly good and "white" IP addresses. Perhaps this elevate the need for some logic to differentiate between types of attacks  

Finally, download and have fun with the database that can be downloaded here.